Learning, Design, and Technology: An International Compendium of Theory, Research, Practice, and Policy 3319174606, 9783319174600

Citation preview

J. Michael Spector Barbara B. Lockee Marcus D. Childress Editors

Learning, Design, and Technology An International Compendium of Theory, Research, Practice, and Policy

Learning, Design, and Technology

J. Michael Spector • Barbara B. Lockee • Marcus D. Childress Editors

Learning, Design, and Technology An International Compendium of Theory, Research, Practice, and Policy

With 568 Figures and 358 Tables

Editors J. Michael Spector Department of Learning Technologies University of North Texas Denton, TX, USA

Barbara B. Lockee Instructional Design and Technology Virginia Tech School of Education Blacksburg, VA, USA

Marcus D. Childress Department of Learning Technologies University of North Texas Denton, TX, USA

ISBN 978-3-319-17460-0 ISBN 978-3-319-17461-7 (eBook) https://doi.org/10.1007/978-3-319-17461-7 © Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG. The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Paper in this product is recyclable.

In Memoriam It was with great sadness that we learned of the passing of Dr. Patricia Cranton, faculty of education professor at the University of New Brunswick (UNB) and member of the Order of Canada. We interacted with Patricia in connection with her contribution on Transformative Learning in this publication. During our correspondence, we found her to be a gifted writer with a deep understanding of adult education. During her distinguished career, Patricia taught at many universities, including McGill, Brock, Penn State, St. Francis Xavier and Teachers College, Columbia University. Patricia was one of North America’s foremost thinkers in the area of Transformative Learning. She and Ed Taylor edited the Handbook of Transformative Learning in 2012. In addition, the third edition of her book, Understanding and Promoting Transformative Learning, was released by Stylus on June 2, 2016. Dr. Cranton’s publications and her thinking and ideas about transformative learning and educating adults in numerous settings have been a bedrock of

the field of adult education for the past 25 years. She was a sought-after mentor who supervised over 100 doctoral dissertations and a keen photographer who loved nature and her dogs Cookie and Foxy. As UNB’s president Eddy Campbell stated Cranton was one of the finest scholars in adult education and she will be greatly missed.

Preface

This volume has taken a couple of years to plan and develop. In that time, learning technologies have changed. New ones have arrived, and older ones have fallen into disuse and disrepute. However, learning itself has not changed. Learning is still marked by significant and sustained changes in what an individual or group knows and can do. While the nature of learning remains unchanged, improvements in learning have also remained relatively stable. In spite of new technologies and enthusiasm about the potential of those technologies to dramatically impact learning, such an impact has only been evident in relatively small and isolated cases. The so-called digital divide has only grown larger (Ritzhaupt, Cheng, Wenjing, & Holfeld, 2020; Vogels, 2021). The underlying purpose of this volume is to emphasize the growth and potential of technologies to improve learning and instruction while learning outcomes, which are all too often not seriously assessed, remain relatively static and unchanged. It has been impressive to see the growth and changes in technology and the potential to improve learning and instruction while support for educators has generally declined in so many places and contexts. Perhaps the growth in revenues has exceeded interest in the growth in learning and instruction, along with little interest in citizenship, civility, understanding, and wisdom. Perhaps that is simply the disappointment of a lifetime of supporting introducing new technologies for adults, college students, and adolescents. Our desire in providing these chapters is to emphasize the potential and create a sense of unattained potential. We would like to see the same kind of innovation and support provided for educational technologies also provided to innovation and support for educators in all sectors all across the world. Our closing thought is that there is so much more that can be done, that should be done, for us to do. Denton, USA Blacksburg, USA Denton, USA October 2023

J. Michael Spector Barbara B. Lockee Marcus D. Childress

vii

viii

Preface

References Ritzhaupt, A., Cheng, L., Wenjing, L., & Holfeld, T. N. (2020). The digital divide in formal education settings: The past, present and future relevance. In M. J. Bishop, E. Boling, J. Elen, & V. Svihla (Eds.), Handbook of research in educational communications and technology: Learning design (pp. 483–504). Springer International Publishing. Vogels, E. A. (2021). Digital divide persists even as Americans with lower incomes make gains in tech adoption. Pew Research Center. Retrieved from https://www. pewresearch.org/short-reads/2021/06/22/digital-divide-persists-even-as-ameri cans-with-lower-incomes-make-gains-in-tech-adoption/

Contents

Volume 1 Section I Learning Theory and the Learning Sciences . . . . . . . . . . . 1

Learning Theory and the Learning Sciences: A Section Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jan Elen

1

3

2

Designing Digital Technologies for Deeper Learning . . . . . . . . . . Jürgen Buder and Friedrich W. Hesse

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cliff Zintgraff and Atsusi Hirumi

33

Learning Theories: The Role of Epistemology, Science, and Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Linda Harasim

75

4

5

6

7

8

Twenty-First-Century Learning, Rhizome Theory, and Integrating Opposing Paradigms in the Design of Personal Learning Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Johannes C. Cronje

9

115

Cognitive Load Theory: What We Learn and How We Learn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . John Sweller

137

Toward a Cognitive Theory of Multimedia Assessment (CTMMA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P. A. Kirschner, B. Park, S. Malone, and H. Jarodzka

153

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs, and Learning Strategies on Help Seeking . . . . Silke Schworm and Hans Gruber

177 ix

x

Contents

9

10

11

Networked Societies for Learning: Emergent Learning Activity in Connected and Participatory Meshworks . . . . . . . . . . . . . . . . Lucila Carvalho

199

Technology-Enhanced Learning: A Learning Sciences Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Eleni A. Kyza

221

Self-Determined Learning: Designing for Heutagogic Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lisa Marie Blaschke

245

12

Redefining Learning: A Neurocognitive Approach Phillip Harris and Donovan R. Walling

...........

267

13

Psychological Framework for Quality Technical and Vocational Education and Training in the Twenty-First Century . . . . . . . . . F. K. Sarfo

319

Future Trends in the Design Strategies and Technological Affordances of E-learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Begoña Gros and Francisco J. García-Peñalvo

345

14

15

The Theory of Totally Integrated Education (TIE) Theodore W. Frick

...........

369

16

Roles and Competencies of Educational Design Researchers: One Framework and Seven Guidelines . . . . . . . . . . . . . . . . . . . . . Susan McKenney and Saskia Brand-Gruwel

403

Board Games as Part of Effective Game-Based Learning Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Antonio Santos

429

17

Section II Impact of Educational Policies and Research on Educational Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

19

20

449

Impact of Educational Policies and Research on Educational Practice: A Section Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . Robert G. Doyle and Drew Polly

451

Insight into MOOCs Research: A Meta-trend Analysis of Publications (2009–2018) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gwendolyn M. Morel and Heather L. Keahey

453

Design-Centric Research-Practice Partnerships: Three Key Lenses for Building Productive Bridges Between Theory and Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yael Kali, Bat-Sheva Eylon, Susan McKenney, and Adi Kidron

481

Contents

21

22

xi

The Confluence Effect of Policy, Mental Models, and Ethics on Individual Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shirley A. Dawson and Vicki S. Napper Teacher Professional Development for Online Teaching: An Update of Insights Stemming from Contemporary Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Brent Philipsen, Jo Tondeur, Yves Blieck, and Silke Vanslambrouck

Section III Technologies for Learning, Instruction, and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

24

25

26

27

513

527

555

Technologies for Learning, Instruction, and Performance: A Section Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dirk Ifenthaler

557

The Cognitive Theory of Multimedia Learning: The Impact of Social Cues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sara West Bechtold

561

An Instrumentalized Framework for Supporting Learners’ Self-Regulation in Blended Learning Environments . . . . . . . . . . Stijn Van Laer and Jan Elen

575

Digital Technologies and Adults: Social Networking, Holding Environments, and Intellectual Development . . . . . . . . . . . . . . . . Smith M. Cecil and Lindstrom Denise L.

613

Concept Maps as Versatile Learning, Teaching, and Assessment Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Beat Adrian Schwendimann

631

28

Technology and Feedback Design . . . . . . . . . . . . . . . . . . . . . . . . . Phillip Dawson, Michael Henderson, Tracii Ryan, Paige Mahoney, David Boud, Michael Phillips, and Elizabeth Molloy

29

Linking Assessment and Learning Analytics to Support Learning Processes in Higher Education . . . . . . . . . . . . . . . . . . . Clara Schumacher

695

741

Volume 2 Section IV 30

Innovative Design and Development Approaches . . . .

Innovative Design and Development Approaches: A Section Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lin Lin-Lipsmeyer and Bernadette Sibuma

781

783

xii

Contents

31

Design Thinking: Towards the Construction of Knowledge . . . . . Brad Hokanson and Jody Nyboer

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder Ecosystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Steven J. Zuiker, Niels Piepgrass, and Mathew D. Evans

809

Design Thinking, Designerly Ways of Knowing, and Engaged Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jonan Phillip Donaldson and Brian K. Smith

837

Considerations for the Design of Gesture-Augmented Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Robert C. Wallon and Robb Lindgren

861

Personalizing Flipped Instruction to Enhance EFL Learners’ Idiomatic Knowledge and Oral Proficiency . . . . . . . . . . . . . . . . . Wen-Chi Vivian Wu, Jun Chen Hsieh, and Jie Chi Yang

883

Designing for Collaborative Creativity in STEM Education with Computational Media . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Florence R. Sullivan and Roberto G. Barbosa

907

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort Degree Program . . . . . . . . . . . . . . . . . Zain Ali, Nanxi Meng, Scott Warren, and Lin Lin-Lipsmeyer

933

33

34

35

36

37

38

39

Toward a Systematic and Model-Based Approach to Design Learning Environments for Critical Thinking . . . . . . . . . . . . . . . Dawit T. Tiruneh, Mieke De Cock, J. Michael Spector, Xiaoqing Gu, and Jan Elen Design of Innovative Learning Environment: An Activity System Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . Juhong Christie Liu

789

955

993

40

A Process Method Approach to Study the Development of Virtual Research Environments: A Theoretical Framework . . . . 1019 Iftekhar Ahmed and Marshall Scott Poole

41

Exploring Immersive Language Learning Using Virtual Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1041 Gary Ka Wai Wong and Michele Notari

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential Learning and the Design of Virtual Patient Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1063 Atsusi Hirumi, Benjamin Chak Lum Lok, Teresa R. Johnson, Kyle Johnsen, Diego de Jesus Rivera-Gutierrez, Ramsamooj Javier Reyes, Tom Atkinson, Christopher Stapleton, and Juan C. Cendán

Contents

xiii

43

Instructional Design as a Moral Ecology of Practice: Implications for Competency Standards and Professional Identity . . . . . . . . . 1111 Stephen C. Yanchar

44

Narrative or Expository Video Cases: Exploring the Influence of Video Cases on Junior Staff’s Attitude and Reflection . . . . . . . 1131 Jingbo Huang

45

Learning Environments for Academics: Reintroducing Scientists to the Power of Creative Environment . . . . . . . . . . . . . 1153 Julia Figliotti, Maggie Dugan, and Donnalyn Roxey

Section V

Transformative Learning . . . . . . . . . . . . . . . . . . . . . . . . .

1171

46

Transformative Learning: A Section Introduction . . . . . . . . . . . . 1173 Kaushal Kumar Bhagat, C. Halupa, Demetria L. Ennis-Cole, Princess M. Cullum, and Konrad Morgan

47

Transformative Learning: A Narrative Patricia A. Cranton

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with Autism Spectrum Disorders (ASD), and Video Game Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1191 Demetria L. Ennis-Cole and Princess M. Cullum

49

Are Students and Faculty Ready for Transformative Learning? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1213 C. Halupa

50

Clicker Interventions in Large Lectures in Higher Education . . . 1237 Kjetil Egelandsdal, Kristine Ludvigsen, and Ingunn Johanne Ness

51

Transformative Learning in an Online Doctoral Programme: Autoethnography as a Pedagogical Method . . . . . . . . . . . . . . . . . 1259 Kyungmee Lee

52

Operationalizing Transformative Learning: A Case Study Demonstrating Replicability and Scaling . . . . . . . . . . . . . . . . . . . 1281 Jeff King and Brenton Wimmer

53

Transformative Learning and the Affordance of Flexible Habits of Mind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1317 Michelle L. Maiese

54

Achieving Education for Sustainable Development (ESD) in Early Childhood Education Through Critical Reflection in Transformative Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1339 Şebnem Feriver, Refika Olgan, and Gaye Teksöz

. . . . . . . . . . . . . . . . . . . . 1177

xiv

Contents

55

Transformative Experience: A Critical Review and Investigation of Individual Factors . . . . . . . . . . . . . . . . . . . . . . . . 1381 Kevin J. Pugh, Cassendra M. Bergstrom, Leah Wilson, Sarah Geiger, Jacqueline Goldman, Benjamin C. Heddy, Simon Cropp, and Dylan P. J. Kriescher

56

Transformative Learning for Sustainability in a Business School Through the Analysis of Students’ Critical Reflection . . . . . . . . . 1417 Janette Brunstein, Marta Fabiano Sambiase, Claudine Brunnquell, Denise Pereira Curi, and Carlos Jonathan Santos

57

A Systematic Mapping of Literature on Transformative Learning Theory in Educational Technology . . . . . . . . . . . . . . . . 1459 Susan L. Stansberry

Volume 3 Section VI

Systems Thinking and Change

...................

1479

58

Systems Thinking and Change: A Section Introduction . . . . . . . . 1481 Eugene G. Kowch

59

What Is Systems Thinking? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1495 Derek Cabrera and Laura Cabrera

60

The Promise of Systemic Designing: Giving Form to Water . . . . 1523 Harold G. Nelson

61

A Change Model Mashup to Guide Educational System Change Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1573 Kyle L. Peck

62

Importance of Educology for Improving Education Systems . . . . 1597 Theodore W. Frick

63

Axiomatic Theory of Intentional Systems (ATIS) and Options-Set Analyses for Education . . . . . . . . . . . . . . . . . . . . . . . 1647 Kenneth Thompson

64

The School System Transformation Process: Guidance for Paradigm Change in School Districts . . . . . . . . . . . . . . . . . . . 1695 Charles M. Reigeluth and Francis M. Duffy

65

Complexity and Systems Thinking Models in Education: Applications for Leaders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1727 Derek Cabrera and Laura Cabrera

Contents

xv

66

Applying Systems Thinking to Learner-Centered User Design for Game and Cyber School Learning Contexts . . . . . . . . . . . . . . 1757 Jason Alphonso Engerman, Victoria Rose Raish, and Alison Carr-Chellman

67

Chaos Theory and the Sciences of Complexity: Foundations for Transforming Educational Systems . . . . . . . . . . . . . . . . . . . . 1797 Charles M. Reigeluth

68

Technology Affordance in Online Learning: A Systems Thinking and System Dynamics Theoretical Framework Jin Mao and Rick L. Shearer

. . . . . . 1809

69

A Systems Thinking Approach to a Story-Centered Curriculum Design and Application in Japanese Higher Education . . . . . . . . 1833 Junko Nemoto and Katsuaki Suzuki

70

Improving Completion in Online Education Systems: An Application of Systems Thinking . . . . . . . . . . . . . . . . . . . . . . 1853 Hoyet Hemphill, Leaunda Hemphill, and Roger Runquist

71

Systems Thinking as a Heuristic for the Implementation of Service Learning in a University . . . . . . . . . . . . . . . . . . . . . . . . . . 1891 Beth Rajan Sockman, Laurene Clossey, Olivia M. Carducci, LuAnn Batson-Magnuson, David Mazure, Gene White, Adrian Wehmeyer, Bonnie A. Green, and Holly Wells

72

The Minnesota New Country School: Systemic Change Thinking in Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1919 Sinem Aslan and Charles M. Reigeluth

73

An Investigation into State-Level Paradigm Change and Politics in Education: Ohio’s Transformational Dialogue for Public Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1949 Eulho Jung, Charles M. Reigeluth, Minkyoung Kim, and Scott Trepper

Section VII

Assessment, Testing, and Evaluation . . . . . . . . . . . . . .

1983

74

Assessment, Testing, and Evaluation: A Section Introduction . . . 1985 Minhong Wang, Cher Ping Lim, and Tzy-Ling Chen

75

Evaluations of Educational Practice, Programs, Projects, Products, and Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1989 J. Michael Spector

76

The Conceptual Model of Formative Assessment of Structural Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2007 Alla Anohina-Naumeca

xvi

Contents

77

Customizable Tool for Online Training Evaluation . . . . . . . . . . . 2049 Cheryl A. Murphy, Elizabeth A. Keiffer, Jack A. Neal, and Jessica Howton

78

Structural Assessment of Knowledge as, of, and for Learning . . . 2077 David L. Trumpower and Arun S. Vanapalli

79

Digital Forms of Assessment in Schools: Supporting the Processes to Improve Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . 2099 C. Paul Newhouse

80

Learning Analytics: Negotiating the Intersection of Measurement Technology and Information Technology . . . . . . . . . . . . . . . . . . . 2127 Mark Wilson and Kathleen Scalise

81

Benchmarking: A Method for Quality Assessment and Enhancement in Higher Education . . . . . . . . . . . . . . . . . . . . . . . . 2149 Ebba S. I. Ossiannilsson

82

Assessment for Twenty-First-Century Learning: The Challenges Ahead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2169 Patricia Broadfoot

83

The Future of Assessment in Technology-Rich Environments: Psychometric Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2191 Kristen E. DiCerbo, Valerie Shute, and Yoon Jeon Kim

84

E-portfolios as Digital Assessment Tools in Higher Education . . . 2213 Min Yang, Tianchong Wang, and Cher Ping Lim

Volume 4 Section VIII Case Studies in Learning Design and Instructional Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2237

85

Case Studies in Learning Design and Instructional Technology: A Section Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2239 Stephanie Moore and Heather Leary

86

Pre-service Teachers’ Perceptions Towards Using Games for Learning About Socio-scientific Topics: A Case Study . . . . . 2243 Shamila Janakiraman, Sunnie Lee Watson, and William R. Watson

87

Fostering Visuospatial Skills in Children Through Inquiry-Based Learning with Origami: The Case Study of VisMO Lessons . . . . 2273 Yaoran Li, Perla Myers, David C. Geary, Taryn Robertson, and Vitaliy Popov

Contents

xvii

88

Apprenticeship Programs in a Blended Design Case . . . . . . . . . . 2305 Douglas E. Archibald, Emily Pulham, and Danny Young

89

Data-Informed Learning Design in a Computer Science Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2323 Daron Williams, Larry Cox II, Margaret Ellis, Bob Edmison, Taha Hassan, M. Aaron Bond, Quinn Warnick, Virginia Clark, Daniel Yaffe, Molly Domino, and Derek Haqq

90

Using Design-Based Research to Develop a Professional Development Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2347 Chandra Hawley Orrill and Rachael Eriksen Brown

91

Designing Educative Tools for Scientific Argumentation: A Case Study of DBR Before and During the Pandemic . . . . . . . 2375 Kathleen M. Easley, Steven McGee, Randi McGee-Tekula, Anne Britt, Kathryn E. Rupp, and Karyn Higgs

92

Combining User Experience and Learning Efficacy in Design and Redesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2405 Isa Jahnke, Shangman Li, Kanupriya Singh, Fan Yu, and Nathan Riedel

93

Integrating Computational Thinking in Humanistic Subjects in Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2431 Inger-Marie F. Christensen

94

Utilizing Constructivist-Based Multimedia Principles in Blended Course Design Supports Greater Learner Autonomy: A Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2477 Kuang-Chen Hsu and Judith Lewandowski

95

Deploying Gamification Design to Promote Student Interaction in an Online Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2509 Jiaming Cheng, Ye Chen, and Lili Zhang

96

Using Backward Design for Flipped Learning Environments . . . 2543 Olgun Sadik and Funda Ergulec

97

Dashboard Applications to Support Motivation: A Design Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2563 Natercia Valle, Pavlo Antonenko, and Denis Valle

98

Designing a Motivation Intervention for Students Learning Algebra Online . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2601 Nikki G. Lobczowski, Michael W. Asher, J. Elizabeth Richey, Yun Huang, Cameron Hecht, Shailaja Bhardwaj, Vincent Aleven, Kenneth Koedinger, and Judith Harackiewicz

xviii

Contents

99

Parent Learning Through Complementary Online Social Collaboration: A Case Study of Parentopia . . . . . . . . . . . . . . . . . 2635 Susan K. Walker

100

Building a Performance-Based Assessment of Graph Construction Using Evidence-Centered Design . . . . . . . . . . . . . . 2663 Eli Meir, Stephanie M. Gardner, Susan Maruca, Elizabeth Suazo-Flores, and Joel K. Abraham

101

Designing Interactive Virtual Manipulatives: The Case of Puzzle Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2699 Seungoh Paek and Daniel L. Hoffman

102

Transfer of STEM Research for Designing Contextually Relevant Curriculum in Pakistan: A Case Study . . . . . . . . . . . . . 2727 Tasneem Anwar and Umber Siddiqi

103

EdTech for “Littles”: Using a Learning Engineering Approach to Create a Digital Math Readiness Program for 2- and 3-Year-Old Children . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2759 Khanh-Phuong Thai, Sarah Buchan, Amanda Kates, Elana Blinder, Carrie Zierath, and Anastasia Betts

104

The Ethical Choices with Educational Technology Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2793 Scott Warren and Dennis Beck

105

Designing Avatar Personas for an Implicit Bias Simulation Through Empathic Design Approaches . . . . . . . . . . . . . . . . . . . . 2821 Irene A. Bal and Mia L. Knowles-Davis

Section IX

Technologies Influencing Educational Futures . . . . . . .

2853

106

Technologies Influencing Educational Futures: A Section Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2855 Guangtao Xu, Qiyun Wang, and Youqun Ren

107

Technology Laboratories for Learners with Autism Spectrum Disorder (ASD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2859 Demetria L. Ennis-Cole

108

Students’ Motivation to Learning with Information Technology in Statistics Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2879 Ken W. Li

109

The Innovative Influence of Technologies on Education in China: Ongoing and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2897 Youqun Ren, Xudong Zheng, and Guangtao Xu

Contents

Section X

xix

Adaptive Technologies and Augmented Realities . . . . .

2913

110

Adaptive Technologies and Augmented Realities: A Section Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2915 Gwo-Jen Hwang and Nian-Shing Chen

111

The Essential Components of Game Design in 3D Virtual Worlds: From a Language Learning Perspective . . . . . . . . . . . . . . . . . . . . 2919 Yu-Ju Lan

112

Exploring Ubiquitous Geometry Learning in Real Situation . . . . 2937 Wu-Yuin Hwang, Ankhtuya Ochirbat, and Li-Kai Lin

113

Design and Analysis of Recommendation Learning System Based on Multiple Intelligences Theory . . . . . . . . . . . . . . . . . . . . 2955 Hong-Ren Chen and Yu-Hsuan Chang

114

The Application and Evaluation of Augmented Reality-Integrated E-books in Living Technology Education . . . . . . . . . . . . . . . . . . . 2975 Ting-Chia Hsu

115

The Application of Augmented Reality in English Vocabulary Learning for Elementary School Students . . . . . . . . . . . . . . . . . . 2997 Ting-Chia Hsu and Gwo-Jen Hwang

116

An Adaptive and Personalized English Reading Recommendation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3015 Ting-Ting Wu and Shu-Hsien Huang

117

Using Non-player Characters to Scaffold Non-gamer Students in Serious Gaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3035 Morris S. Y. Jong, Junjie Shang, and Vincent W. L. Tam

118

Building the Virtual Experiment Learning Activities to Facilitate Self-Adaptive Learning in IPv6 Subject . . . . . . . . . . . . 3053 Jun-Ming Su and Shian-Shyong Tseng

119

Applying a Repertory Grid-Oriented Mindtool to Developing a Knowledge Construction Augmented Reality Mobile Learning System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3077 Hui-Chun Chu

Section XI 120

E-Learning in Formal and Informal Settings . . . . . . . . .

3099

E-learning in Formal and Informal Settings: A Section Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3101 Insook Lee

xx

Contents

121

Blended Learning Research in Higher Education and K-12 Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3107 Lisa R. Halverson, Kristian J. Spring, Sabrina Huyett, Curtis R. Henrie, and Charles R. Graham

122

From Distance Education to Massive Open Online Courses in Taiwan: Progressing with a Global Perspective and Local Commitments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3137 Chun-Yi Lin and Chien-Han Chen

123

What Motivates Exemplary Online Teachers? A Multiple-Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3161 Evrim Baran and Ana-Paula Correia

124

Formal E-learning in Arab Countries: Challenges and Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3179 Gihan Osman

125

Do You Have a SOLE? Research on Informal and Self-Directed Online Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . 3205 Curtis J. Bonk, Minkyoung Kim, and Shuya Xu

126

Sociability of Online Learning Environments: Examining Discussion Group Sizes and Social Network Sites . . . . . . . . . . . . 3237 Eunbae Lee and Mete Akcaoglu

127

Informal Learning Through Social Media: Opportunities for Learning Professionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3253 Susan N. Genden

Volume 5 Section XII Smart Learning Environments and Learning Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3277

128

Smart Learning Environments and Learning Analytics: A Section Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3279 Kinshuk

129

Analyzing Learner and Instructor Interactions within Learning Management Systems: Approaches and Examples . . . . . . . . . . . . 3285 Mimi Recker and Ji Eun Lee

130

Beyond Cognitive and Affective Issues: Designing Smart Learning Environments for Psychomotor Personalized Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3309 Olga C. Santos

Contents

xxi

131

Competency-Based Personalization Process for Smart Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3333 Gilbert Paquette

132

Context-Aware Ubiquitous Learning in Science Museum with iBeacon Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3369 Guang Chen, Yuanjing Zhang, Nian-Shing Chen, and Zhengcheng Fan

133

Creation of Cognitive Conflict by Error-Visualization: Error-Based Simulation and Its Practical Use in Science Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3393 Tsukasa Hirashima and Tomoya Horiguchi

134

Developing the Petal E-learning Platform for Facial Analytics and Personalized Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3427 Vincent W. L. Tam, Edmund Y. Lam, Y. Huang, Kelly Liu, Victoria Tam, and Phoebe Tse

135

From Reflective Practitioner to Active Researcher: Towards a Role for Learning Analytics in Higher Education Scholarship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3445 Lorenzo Vigentini, Negin Mirriahi, and Giedre Kligyte

136

Game Learning Analytics: Learning Analytics for Serious Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3475 Manuel Freire, Ángel Serrano-Laguna, Borja Manero Iglesias, Iván Martínez-Ortiz, Pablo Moreno-Ger, and Baltasar Fernández-Manjón

137

Learning Analytics and Learning Objects Repositories: Overview and Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . 3503 S. Yassine, S. Kadry, and M. A. Sicilia

138

Learning Analytics for Smart Learning Environments: A Meta-analysis of Empirical Research Results from 2009 to 2015 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3533 Zacharoula Papamitsiou and Anastasios A. Economides

139

Learning Model of Recorded Lectures: Implications to Learning Analytics Theory and Practice . . . . . . . . . . . . . . . . . . . 3557 Ben Kei Daniel

140

Pedagogical Framework for Developing Thinking Skills Using Smart Learning Environments . . . . . . . . . . . . . . . . . . . . . . 3581 Sahana Murthy, Sridhar Iyer, and Madhuri Mavinkurve

xxii

Contents

141

Smart (but Also) Challenging Learning Environments: The Case of Conversational Agents That Foster Productive Peer Dialogue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3629 Stavros N. Demetriadis and Stergios D. Tegos

142

Smart Learning Environments in School: Design Principles and Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3659 Jinbao Zhang, Qianxia Jing, Yue Liang, Hongyan Jiang, and Nannan Li

143

Students’ Potential as Active Participants in Peer Content Analysis Through Asynchronous Discussions . . . . . . . . . . . . . . . . 3687 Maria Tzelepi, Kyparisia Papanikolaou, and Petros Roussos

144

Utilizing Real-Time Descriptive Learning Analytics to Enhance Learning Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3709 Hallvard Trætteberg, Anna Mavroudi, Kshitij Sharma, and Michail Giannakos

145

What Massive Open Online Course (MOOC) Stakeholders Can Learn from Learning Analytics? . . . . . . . . . . . . . . . . . . . . . . 3731 Mohammad Khalil and Martin Ebner

Section XIII

Cultural and Regional Perspectives . . . . . . . . . . . . . . .

3761

146

Cultural and Regional Perspectives: A Section Introduction . . . . 3763 Patricia A. Young and Tutaleni I. Asino

147

The Theory of Virtuality Culture and Technology-Mediated Human Presence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3767 Jennifer Camille Dempsey

148

Cultural and Historical Influences on a Project-Based Learning Training Program in Medellín, Colombia . . . . . . . . . . . . . . . . . . . 3787 Cliff Zintgraff, Miguel F. Daza, António Rodriguez Vides, Carol Fletcher, Jennifer J. Kaszuba, and Joules M. Webb

149

Emergent Bilinguals Self-Affecting Their Self-Efficacy Through Bilingual Digital Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . 3823 Julian Viera

150

Supporting the Development of Information Communication Technology Education in Ghana . . . . . . . . . . . . . . . . . . . . . . . . . . 3851 Princess Allotey and Sarah Murray

Contents

xxiii

151

Educational Technologists in Latin America and the Caribbean: Perceived Importance of Competencies for Practice . . . . . . . . . . 3875 Enilda Romero-Hall, Leonor Adams, Erika Petersen, and Adriana Vianna

152

Community of Inquiry in a Collectivist/Feminine Society: An Examination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3901 Ana-Paula Correia

Section XIV Literature Reviews and Systematic Reviews of Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3923

153

Literature Reviews and Systematic Reviews of Research: The Roles and Importance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3925 Hale Ilgaz, Gloria Natividad, and Arif Altun

154

Gauging the Effectiveness of Educational Technology Integration in Education: What the Best-Quality Meta-analyses Tell Us . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3929 Robert M. Bernard, Eugene Borokhovski, Richard F. Schmid, and Rana M. Tamim

155

Gamification in Open and Distance Learning: A Systematic Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3953 Murat Sümer and Cengiz Hakan Aydın

156

What Do Studies of Learning Analytics Reveal About Learning and Instruction? A Systematic Literature Review . . . . 3969 Ji Eun Lee and Mimi Recker

157

Developing Critical Thinking: A Review of Past Efforts as a Framework for a New Approach for Childhood Learning . . . . . 4007 Shanshan Ma, Kaushal Kumar Bhagat, J. Michael Spector, Lin Lin-Lipsmeyer, Dejian Liu, Jing Leng, Dawit T. Tiruneh, and Jonah Mancini

158

Universal Design in Postsecondary Education: A Systematic Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4035 Soonhwa Seok, Boaventura DaCosta, and Linda S. Heitzman-Powell

159

Examining Technology Use and Evaluation in Computer-Supported Collaborative Learning: A Systematic Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4059 Dhvani Toprani, Mona AlQahtani, and Marcela Borge

xxiv

Contents

160

Integrative Literature Review of Interactions in Online Courses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4085 Victoria Abramenka-Lachheb

161

Visual Literacy (VL) Assessment: A Review of Critical Voices and Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4111 Mingyu Li and Kenneth Potter

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4139

About the Editors

J. Michael Spector Department of Learning Technologies University of North Texas Denton, TX, USA Michael Spector, Regents Professor and former Chair and Doctoral Program Director in the Learning Technologies Department at the University of North Texas, was previously Professor of Educational Psychology and Doctoral Program Coordinator at the University of Georgia, Professor and Associate Director of the Learning Systems Institute at Florida State University, Chair of Instructional Design, Development and Evaluation at Syracuse University, and Director of the Educational Information Science and Technology Research Program at the University of Bergen. He earned a Ph.D. from The University of Texas. He is a Visiting Research Professor at Beijing Normal University, at East China Normal University, and the Indian Institute of TechnologyKharagpur. His research focuses on assessing learning in complex domains, developing inquiry and critical thinking skills in children, and program evaluation. He was Executive Director and Treasurer of the International Board of Standards for Training, Performance and Instruction and a Past President of the Association for Educational and Communications Technology. He is Editor Emeritus and Featured Papers Editor of Educational Technology Research and Development. He edited two editions of the Handbook of Research on Educational Communications and Technology and the SAGE Encyclopedia of Educational Technology. He is

xxv

xxvi

About the Editors

currently lead editor of Learning, Design and Technology: An International Compendium of theory, Research, Practice and Policy and section editor for educational technology in the Routledge Encyclopedia of Education. He has more than 200 academic publications to his credit and has recently been awarded an NSF-IUSE grant focusing on STEM education in small and minority serving colleges and universities. Barbara B. Lockee Instructional Design and Technology Virginia Tech Blacksburg, VA, USA Barbara B. Lockee is Associate Vice Provost of Faculty Affairs and Professor of Instructional Design and Technology at Virginia Tech. Since 1996, Dr. Lockee has engaged in teaching and research related to instructional design and distance education and has advised the research of more than 50 doctoral students. Her scholarly inquiry is focused on mediated and online education and has been funded by various federal agencies, including the National Science Foundation, the US Department of Agriculture, and the US Agency for International Development, among others. She has also consulted for a variety of organizations, including the NASA Jet Propulsion Laboratory, the US Army Training and Doctrine Command, and the University of Southern California’s Institute for Creative Technologies. Her recent co-authored book, Streamlined ID: A Practical Guide for Instructional Design, strives to make the design of learning solutions accessible and pragmatic for those who develop educational courses and programs in workplace contexts. Dr. Lockee is Past President of the Association for Educational Communications and Technology, a professional society for educational technology researchers and practitioners. She currently serves as Vice-President of the International Academic Forum, a global scholarly community based in Nagoya, Japan. She is also Vice-Chair of the Board of Directors for CMR Institute, a healthcare education provider for the life sciences industry. Dr. Lockee earned her B.A. in

About the Editors

xxvii

1986 from Appalachian State University in Communication Arts, M.A. in 1991 from Appalachian State University in Curriculum and Instruction (Educational Media), and Ph.D. in 1996 from Virginia Tech in Curriculum and Instruction (Instructional Technology). Marcus D. Childress Department of Learning Technologies University of North Texas Denton, TX, USA Marcus Childress is Associate Graduate Faculty Advisor/Mentor for the Learning Technologies Ph.D. program at the University of North Texas. Dr. Childress is the Founder and Owner of Course Jockey, LLC, a company dedicated to delivering quality professional development for professors, teachers, and aspiring instructional designers. Dr. Childress earned his Ph.D. in Instructional Design and Technology from Virginia Tech and his M.M. and B.M. degrees in Music from Appalachian State University. He began his teaching career as a high school band director (8 years) in the North Carolina public schools. Dr. Childress served on the faculty at Old Dominion University (VA), Emporia State University (KS), and Baker University (KS), where he served as Dean of the School of Education and Vice President for Academic Affairs/Chief Academic Officer. Dr. Childress’ research interests include online teaching and learning, instructional design, virtual worlds for training and education, and using technology as a catalyst for educational reform. His research has been documented in publications such as Distance Education, Journal of Research on Computers in Education, International Journal of Educational Telecommunications, Globalized e-Learning Cultural Challenges, EDUCAUSE Quarterly, Academic Leadership Journal, and the Encyclopedia of Distance Learning, Teaching, Technologies, and Applications. In addition to his higher education experience, his training involvement includes consulting with the Intel Corporation (Senior Trainer, Intel Teach to the Future), the People’s Bank of China Training Center, the Virginia Modeling, Analysis and Simulation Center, and the United States Joint Training Analysis and Simulation Center. Dr. Childress has shared his expertise in the

xxviii

About the Editors

USA and across the globe, including Malaysia, Indonesia, China, Japan, Taiwan, Hong Kong, Korea, and the Czech Republic. He is a past President (2012–2013) of the Association for Educational Communications and Technology (AECT). He served on the AECT executive committee and board of directors and was the 2012 AECT International Convention Chair.

Section Editors

Section I: Learning Theory and the Learning Sciences Jan Elen KU Leuven, Leuven, Belgium Geraldine Clarebout KU Leuven, Leuven, Belgium Section II: Impact of Educational Policies and Research on Educational Practice Drew Polly College of Education, The University of North Carolina at Charlotte, Charlotte, NC, USA Robert G. Doyle Harvard University, Cambridge, MA, USA Section III: Technologies for Learning, Instruction, and Performance Dirk Ifenthaler University of Mannheim, Mannheim, Germany Curtin University, Perth, WA, Australia Section IV: Innovative Design and Development Approaches Lin Lin-Lipsmeyer Department of Teaching and Learning, Southern Methodist University, Dallas, TX, USA Bernadette Sibuma Massachusetts Bay Community College, Wellesley Hills, MA, USA Section V: Transformative Learning Konrad Morgan Eduvate, Havant, Hampshire, UK Kaushal Kumar Bhagat Advanced Technology Development Centre, Indian Institute of Technology Kharagpur, Kharagpur, India Şebnem Feriver Systems Thinking Association, Ankara, Turkey Section VI: Systems Thinking and Change Eugene G. Kowch Werklund School of Education, University of Calgary, Calgary, AB, Canada xxix

xxx

Section Editors

Section VII: Assessment, Testing, and Evaluation Minhong Wang Faculty of Education, The University of Hong Kong, Hong Kong, China Cher Ping Lim Faculty of Education and Human Development, Education University of Hong Kong, Hong Kong, SAR, China Tzy-Ling Chen Graduate Institute of Bio-Industry Management, National Chung Hsing University, Taichung, Taiwan Section VIII: Case Studies in Learning Design and Instructional Technology Stephanie Moore Organization, Information, and Learning Sciences, University of New Mexico, Albuquerque, NM, USA Heather Leary Instructional Psychology and Technology, Brigham Young University, Provo, UT, USA Section IX: Technologies Influencing Educational Futures Youqun Ren East China Normal University, Shanghai, China Qiyun Wang Learning Sciences and Assessment, National Institute of Education, Nanyang Technological University, Singapore, Singapore Guangtao Xu Department of Educational Technology, School of Education, Hangzhou Normal University, Hangzhou, China Section X: Adaptive Technologies and Augmented Realities Nian-Shing Chen Program of Learning Sciences, National Taiwan Normal University, Taipei, Taiwan Gwo-Jen Hwang Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology, Taipei, Taiwan Section XI: E-Learning in Formal and Informal Settings Insook Lee Department of Education, Sejong University, Seoul, South Korea Mimi Miyoung Lee Department of Curriculum and Instruction, University of Houston, Houston, TX, USA Section XII: Smart Learning Environments and Learning Analytics Kinshuk College of Information, The University of North Texas, Denton, TX, USA Section XIII: Cultural and Regional Perspectives Patricia A. Young Department of Education, University of Maryland Baltimore County, Baltimore, MD, USA

Section Editors

xxxi

Tutaleni I. Asino College of Education, Health and Aviation, Oklahoma State University, Stillwater, OK, USA Section XIV: Literature Reviews and Systematic Reviews of Research Hale Ilgaz Ankara University, Ankara, Turkey Arif Altun Hacettepe University, Ankara, Turkey Gloria Natividad Instituto Tecnológico de Saltillo, Tecnológico Nacional de México, Saltillo, México

Contributors

M. Aaron Bond Virginia Tech, Blacksburg, VA, USA Joel K. Abraham California State University Fullerton, Fullerton, CA, USA Victoria Abramenka-Lachheb Indiana University Bloomington, Bloomington, IN, USA Leonor Adams Old Dominion University, Norfolk, VA, USA Iftekhar Ahmed Department of Communication Studies, University of North Texas, Denton, TX, USA Mete Akcaoglu Department of Leadership, Technology, and Human Development, Georgia Southern University, Statesboro, GA, USA Vincent Aleven Carnegie Mellon University, Pittsburgh, PA, USA Zain Ali Learning Technologies Department, University of North Texas, Denton, TX, USA Princess Allotey Centre College, Danville, KY, USA Mona AlQahtani The Pennsylvania State University, State College, PA, USA Arif Altun Hacettepe University, Ankara, Turkey Alla Anohina-Naumeca Riga Technical University, Riga, Latvia University of Latvia, Riga, Latvia Pavlo Antonenko University of Florida, Gainesville, FL, USA Tasneem Anwar Institute for Educational Development, The Aga Khan University, Karachi, Pakistan Douglas E. Archibald Mountainland Technical College, Orem, UT, USA Michael W. Asher University of Wisconsin, Madison, WI, USA Tutaleni I. Asino College of Education, Health and Aviation, Oklahoma State University, Stillwater, OK, USA xxxiii

xxxiv

Contributors

Sinem Aslan Intel Labs, Intel Corporation, Hillsboro, OR, USA Tom Atkinson Ashford University, San Diego, CA, USA Cengiz Hakan Aydın Department of Distance Education, Anadolu University, Tepebaşı, Eskişehir, Turkey Irene A. Bal Old Dominion University, Norfolk, VA, USA Loyola University Maryland, Baltimore, MD, USA Evrim Baran Department of Educational Sciences, Middle East Technical University, Ankara, Turkey Roberto G. Barbosa Federal University of Paraná – UFPR (Littoral Sector), Matinhos, PR, Brazil Federal University of Paraná – UFPR (Littoral Sector), Curitiba, PR, Brazil LuAnn Batson-Magnuson East Stroudsburg University of Pennsylvania, East Stroudsburg, PA, USA Sara West Bechtold Pima Community College, Tucson, AZ, USA Dennis Beck University of Arkansas, Fayetteville, AR, USA Cassendra M. Bergstrom School of Psychological Sciences, University of Northern Colorado, Greeley, CO, USA Robert M. Bernard Department of Education, Centre for the Study of Learning and Performance (CSLP), Concordia University, Montreal, QC, Canada Anastasia Betts Age of Learning, Glendale, CA, USA Kaushal Kumar Bhagat Advanced Technology Development Centre, Indian Institute of Technology Kharagpur, Kharagpur, India Shailaja Bhardwaj Carnegie Mellon University, Pittsburgh, PA, USA Lisa Marie Blaschke Center for Lifelong Learning, Carl von Ossietzky University of Oldenburg, Oldenburg, Germany Yves Blieck Vrije Universiteit Brussel, Brussels, Belgium Open University Hasselt, Hasselt, Belgium Elana Blinder Age of Learning, Glendale, CA, USA Curtis J. Bonk Indiana University, Bloomington, IN, USA Department of Instructional Systems Technology, Indiana University School of Education, Bloomington, IN, USA Marcela Borge The Pennsylvania State University, State College, PA, USA

Contributors

xxxv

Eugene Borokhovski Centre for the Study of Learning and Performance (CSLP), Concordia University, Montreal, QC, Canada David Boud Centre for Research in Assessment and Digital Learning, Deakin University, Geelong, VIC, Australia Saskia Brand-Gruwel Faculty of Psychology and Educational Sciences, Open University of the Netherlands, Heerlen, The Netherlands Anne Britt Northern Illinois University, DeKalb, IL, USA Patricia Broadfoot School of Education, University of Bristol, Bristol, UK Rachael Eriksen Brown Pennsylvania State University Abington, Dartmouth, MA, USA Claudine Brunnquell Universidade Presbiteriana Mackenzie, São Paulo, SP, Brazil Centro Universitário Senac, São Paulo, SP, Brazil Janette Brunstein Universidade Presbiteriana Mackenzie, São Paulo, SP, Brazil Sarah Buchan Age of Learning, Glendale, CA, USA Jürgen Buder Leibniz-Institut für Wissensmedien, Tübingen, Germany Derek Cabrera College of Human Ecology, Cabrera Research Lab, Cornell University, Ithaca, NY, USA Laura Cabrera College of Human Ecology, Cabrera Research Lab, Cornell University, Ithaca, NY, USA Olivia M. Carducci East Stroudsburg University of Pennsylvania, East Stroudsburg, PA, USA Alison Carr-Chellman University of Idaho, Moscow, ID, USA Lucila Carvalho Institute of Education, College of Humanities and Social Sciences, Massey University, Auckland, New Zealand Juan C. Cendán College of Medicine, University of Central Florida, Orlando, FL, USA Yu-Hsuan Chang Department of Digital Content and Technology, National Taichung University of Education, Taichung, Taiwan Chien-Han Chen Graduate Institute of Curriculum and Instruction, Tamkang University, Taipei, Taiwan Guang Chen School of Educational Technology (SET), Faculty of Education, Beijing Normal University, Beijing, China Beijing Key Laboratory of Education Technology, Beijing Normal University, Beijing, China

xxxvi

Contributors

Hong-Ren Chen Department of Digital Content and Technology, National Taichung University of Education, Taichung, Taiwan Nian-Shing Chen Program of Learning Sciences, National Taiwan Normal University, Taipei, Taiwan Tzy-Ling Chen Graduate Institute of Bio-Industry Management, National Chung Hsing University, Taichung, Taiwan Ye Chen Department of Educational Technology and Foundations, University of West Georgia, Carrollton, GA, USA Jun Chen Hsieh Graduate Institute of Network Learning Technology, National Central University, Taoyuan, Taiwan Jiaming Cheng School of Education, Liaoning Normal University, Dalian, Liaoning, China Inger-Marie F. Christensen Department of Design and Communication and Center for Learning Computational Thinking, University of Southern Denmark, Odense, Denmark Hui-Chun Chu Department of Computer Science and Information Management, Soochow University, Taipei, Taiwan Virginia Clark Virginia Tech, Blacksburg, VA, USA Laurene Clossey East Stroudsburg University of Pennsylvania, East Stroudsburg, PA, USA Ana-Paula Correia Educational Studies Department, Center on Education and Training for Employment, College of Education and Human Ecology, The Ohio State University, Columbus, OH, USA Larry Cox II Virginia Tech, Blacksburg, VA, USA Patricia A. Cranton Adult Education, University of New Brunswick, Fredericton, PA, Canada Johannes C. Cronje Cape Peninsula University of Technology, Cape Town, South Africa Simon Cropp School of Psychological Sciences, University of Northern Colorado, Greeley, CO, USA Princess M. Cullum Cancer Treatment Center of America, Newnan, GA, USA Denise Pereira Curi Universidade de Aveiro, Aveiro, Portugal Boaventura DaCosta Solers Research Group, Orlando, FL, USA Patricia A. Cranton died in 2002.

Contributors

xxxvii

Ben Kei Daniel Educational Technology Research Group, Higher Education Development Centre, University of Otago, Dunedin, New Zealand Phillip Dawson Centre for Research in Assessment and Digital Learning, Deakin University, Geelong, VIC, Australia Shirley A. Dawson College of Education, Department of Teacher Education, Weber State University, Ogden, UT, USA Miguel F. Daza The Magellan International School, Austin, TX, USA Mieke De Cock Department of Physics and Astronomy & LESEC, KU Leuven, Leuven, Belgium Stavros N. Demetriadis School of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece Jennifer Camille Dempsey Northwest Institute of Research, Erie, PA, USA Lindstrom Denise L. Department of Curriculum and Instruction/Literacy Studies, College of Education and Human Services, West Virginia University, Morgantown, WV, USA Kristen E. DiCerbo Pearson, Phoenix, AZ, USA Molly Domino Virginia Tech, Blacksburg, VA, USA Jonan Phillip Donaldson School of Education, Drexel University, Philadelphia, PA, USA Robert G. Doyle Harvard University, Cambridge, MA, USA Francis M. Duffy Gallaudet University, Washington, DC, USA Maggie Dugan Inclusive Innovation, Barcelona, Spain Kathleen M. Easley The Learning Partnership, Chicago, IL, USA Martin Ebner Educational Technology, Graz University of Technology, Graz, Austria Anastasios A. Economides Interdepartmental Programme of Postgraduate Studies in Information Systems, University of Macedonia, Thessaloniki, Greece Bob Edmison Department of Computer Science, Virginia Tech, Blacksburg, VA, USA Kjetil Egelandsdal Faculty of Psychology, Centre for the Sciences of Learning and Technology (SLATE), University of Bergen, Bergen, Norway Jan Elen Centre for Instructional Psychology and Technology, KU Leuven, Leuven, Belgium Margaret Ellis Virginia Tech, Blacksburg, VA, USA

xxxviii

Contributors

Jason Alphonso Engerman Department of Digital Media Technologies, East Stroudsburg University, East Stroudsburg, PA, USA Demetria L. Ennis-Cole Department of Learning Technologies, College of Information, University of North Texas, Denton, TX, USA Funda Ergulec Eskisehir Osmangazi University, Eskisehir, Turkey Mathew D. Evans Mary Lou Fulton Teachers College, Arizona State University, Tempe, AZ, USA Bat-Sheva Eylon The Science Teaching Department, The Weizmann Institute of Science, Rehovot, Israel Zhengcheng Fan School of Educational Technology (SET), Faculty of Education, Beijing Normal University, Beijing, China Şebnem Feriver Faculty of Education, Department of Elementary and Early Childhood Education, Middle East Technical University, Ankara, Turkey Baltasar Fernández-Manjón Department of Software Engineering and Artificial Intelligence, Universidad Complutense de Madrid, Madrid, Spain Julia Figliotti Knowinnovation, Portland, OR, USA Carol Fletcher The University of Texas at Austin, Austin, TX, USA Manuel Freire Department of Software Engineering and Artificial Intelligence, Universidad Complutense de Madrid, Madrid, Spain Theodore W. Frick Department of Instructional Systems Technology, School of Education, Indiana University Bloomington, Bloomington, IN, USA Francisco J. García-Peñalvo Universidad de Salamanca, Salamanca, Spain Stephanie M. Gardner Purdue University, West Lafayette, IN, USA David C. Geary University of Missouri, Columbia, MO, USA Sarah Geiger Department of Graduate Psychology and Counseling, Andrews University, Berrien Spring, MI, USA Susan N. Genden Learning Design and Technology, Wayne State University, Detroit, MI, USA Michail Giannakos Department of Computer Science, Norwegian University of Science and Technology (NTNU), Trondheim, Norway Jacqueline Goldman Division of Counselor Education and Psychology, Delta State University, Cleveland, MS, USA Charles R. Graham Instructional Psychology and Technology, Brigham Young University, Provo, UT, USA

Contributors

xxxix

Bonnie A. Green East Stroudsburg University of Pennsylvania, East Stroudsburg, PA, USA Begoña Gros Universidad de Barcelona, Barcelona, Spain Hans Gruber University of Regensburg, Regensburg, Germany University of Turku, Turku, Finland Xiaoqing Gu Department of Educational Information Technology, East China Normal University, Shanghai, China C. Halupa A.T. Still University, Kirksville, MO, USA Dean Online Learning, East Texas Baptist University, Marshall, TX, USA Lisa R. Halverson Instructional Psychology and Technology, Brigham Young University, Provo, UT, USA Derek Haqq Virginia Tech, Blacksburg, VA, USA Judith Harackiewicz University of Wisconsin, Madison, WI, USA Linda Harasim Simon Fraser University, Burnaby, BC, Canada Phillip Harris Association for Educational Communications and Technology, Bloomington, IN, USA Taha Hassan Virginia Tech, Blacksburg, VA, USA Cameron Hecht University of Texas at Austin, Austin, TX, USA Benjamin C. Heddy Department of Educational Psychology, Oklahoma University, Norman, OK, USA Linda S. Heitzman-Powell Pediatrics, University of Kansas Medical Center, Kansas City, KS, USA Hoyet Hemphill Engineering Technology Department, Western Illinois University, Macomb, IL, USA Leaunda Hemphill Engineering Technology Department, Western Illinois University, Macomb, IL, USA Michael Henderson Faculty of Education, Monash University, Melbourne, VIC, Australia Curtis R. Henrie Instructional Psychology and Technology, Brigham Young University, Provo, UT, USA Friedrich W. Hesse Leibniz-Institut für Wissensmedien, Tübingen, Germany Karyn Higgs Northern Illinois University, DeKalb, IL, USA Tsukasa Hirashima Learning Engineering Laboratory, Department of Information Engineering, Hiroshima University, Hiroshima, Japan

xl

Contributors

Atsusi Hirumi University of Central Florida, Orlando, FL, USA Daniel L. Hoffman University of Hawai‘i at Mānoa, Honolulu, HI, USA Brad Hokanson University of Minnesota, Minneapolis, MN, USA Tomoya Horiguchi Graduate School of Maritime Sciences, Kobe University, Kobe, Hyogo, Japan Jessica Howton H-E-B Grocery Company, LP, San Antonio, TX, USA Kuang-Chen Hsu Office of Digital Learning at the University of Notre Dame, Notre Dame, IN, USA Ting-Chia Hsu Department of Technology Application and Human Resource Development, National Taiwan Normal University, Taipei, Taiwan Jingbo Huang United Nations University Research Institute in Macau, Macau, China Shu-Hsien Huang Department of Information and Learning Technology, National University of Tainan, Tainan, Taiwan Y. Huang Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, China Yun Huang Carnegie Mellon University, Pittsburgh, PA, USA Sabrina Huyett Teacher Education, Brigham Young University, Provo, UT, USA Gwo-Jen Hwang Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology, Taipei, Taiwan Wu-Yuin Hwang National Central University, Taoyuan City, Taiwan Dirk Ifenthaler Learning, Design and Technology, University of Mannheim, Mannheim, Germany Curtin University, Perth, WA, Australia Hale Ilgaz Distance Education Center, Ankara University, Ankara, Turkey Sridhar Iyer Department of Computer Science and Engineering, Indian Institute of Technology Bombay, Mumbai, India Isa Jahnke University of Technology Nuremberg, Nuremberg, Germany Shamila Janakiraman Learning Design and Technology, Purdue University, West Lafayette, IN, USA H. Jarodzka Open University of the Netherlands, Heerlen, The Netherlands Lund University, Lund, Sweden Hongyan Jiang Collaborative and Innovative Center for Educational Technology, Beijing, China

Contributors

xli

Qianxia Jing Beijing Normal University, Beijing, People’s Republic of China Kyle Johnsen College of Engineering, University of Georgia, Athens, GA, USA Teresa R. Johnson Johns Hopkins University School of Medicine, Baltimore, MD, USA Carlos Jonathan Santos Universidade Presbiteriana Mackenzie, São Paulo, SP, Brazil Morris S. Y. Jong Department of Curriculum and Instruction and Centre for Learning Sciences and Technologies, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong Eulho Jung Boise State University, Boise, ID, USA S. Kadry American University of the Middle East, Eqaila/Ahmedi, Kuwait Yael Kali University of Haifa, Haifa, Israel Jennifer J. Kaszuba The University of Texas at Austin, Austin, TX, USA Amanda Kates Age of Learning, Glendale, CA, USA Heather L. Keahey Liberty University, School of Education, Lynchburg, VA, USA Elizabeth A. Keiffer Mathematical Sciences, J. William Fulbright College of Arts and Sciences, University of Arkansas, Fayetteville, AR, USA Mohammad Khalil Educational Technology, Graz University of Technology, Graz, Austria Adi Kidron University of Haifa, Haifa, Israel Minkyoung Kim University of West Florida, Pensacola, FL, USA Yoon Jeon Kim MIT, Cambridge, MA, USA Jeff King Center for Excellence in Transformative Teaching and Learning, University of Central Oklahoma, Edmond, OK, USA Kinshuk College of Information, The University of North Texas, Denton, TX, USA P. A. Kirschner Open University of the Netherlands, Heerlen, The Netherlands Oulu University, Oulu, Finland Giedre Kligyte School of Education and Learning and Teaching Unit, University of New South Wales, Sydney, NSW, Australia Mia L. Knowles-Davis Old Dominion University, Norfolk, VA, USA

xlii

Contributors

Kenneth Koedinger Carnegie Mellon University, Pittsburgh, PA, USA Eugene G. Kowch Werklund School of Education, University of Calgary, Calgary, AB, Canada Dylan P. J. Kriescher School of Psychological Sciences, University of Northern Colorado, Greeley, CO, USA Eleni A. Kyza Department of Communication and Internet Studies, Cyprus University of Technology, Limassol, Cyprus Edmund Y. Lam Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, China Yu-Ju Lan NTNU TELL Lab, Department of Applied Chinese Language and Culture, National Taiwan Normal University, Taipei, Taiwan Heather Leary Instructional Psychology and Technology, Brigham Young University, Provo, UT, USA Eunbae Lee The Catholic College, Catholic University of Korea, Bucheon-si, Gyeonggi-do, Republic of Korea Insook Lee Education, Sejong University, Seoul, South Korea Ji Eun Lee Department of Instructional Technology and Learning Sciences, Utah State University, Logan, UT, USA Kyungmee Lee Lancaster University, Lancaster, UK Jing Leng East China Normal University, Shanghai, China Judith Lewandowski Purdue University Global, West Lafayette, IN, USA Ken W. Li Department of Information Technology, Hong Kong Institute of Vocational Education (Tsing Yi), Hong Kong, China Mingyu Li Virginia Polytechnic Institute and State University, Blacksburg, VA, USA Nannan Li College of Education and P. E, Bohai University, Jinzhou, Liaoning Province, People’s Republic of China Shangman Li University of Missouri-Columbia, Columbia, MO, USA Yaoran Li University of San Diego, San Diego, CA, USA Yue Liang School of Educational Technology, Beijing Normal University, Beijing, People’s Republic of China Cher Ping Lim Department of Curriculum and Instruction, The Education University of Hong Kong, Hong Kong, SAR, China Chun-Yi Lin Graduate Institute of Curriculum and Instruction, Tamkang University, Taipei, Taiwan

Contributors

xliii

Li-Kai Lin Institute of Network Learning Technology, National Central University, Taoyuan City, Taiwan Robb Lindgren College of Education, University of Illinois at Urbana-Champaign, Champaign, IL, USA Lin Lin-Lipsmeyer Simmons School of Education and Human Development, Southern Methodist University, Dallas, TX, USA Department of Learning Technologies, University of North Texas, Denton, TX, USA Dejian Liu NetDragon, Fujian, China Juhong Christie Liu James Madison University, Harrisonburg, VA, USA Kelly Liu Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA Nikki G. Lobczowski Carnegie Mellon University, Pittsburgh, PA, USA Benjamin Chak Lum Lok Computer and Information Sciences and Engineering Department, University of Florida, Gainesville, FL, USA Kristine Ludvigsen Faculty of Psychology, Department of Education, University of Bergen, Bergen, Norway Smith M. Cecil College of Education and Human Services, West Virginia University, Morgantown, WV, USA Shanshan Ma Department of Learning Technologies, University of North Texas, Denton, TX, USA Paige Mahoney Centre for Research in Assessment and Digital Learning, Deakin University, Geelong, VIC, Australia Michelle L. Maiese Department of Philosophy, Emmanuel College, Boston, MA, USA S. Malone Saarland University, Saarbrücken, Germany Jonah Mancini Private Consultant, Round Rock, TX, USA Borja Manero Iglesias Department of Software Engineering and Artificial Intelligence, Universidad Complutense de Madrid, Madrid, Spain Jin Mao Wilkes University, Wilkes Barre, PA, USA Iván Martínez-Ortiz Department of Software Engineering and Artificial Intelligence, Universidad Complutense de Madrid, Madrid, Spain Susan Maruca SimBiotic Software, Missoula, MT, USA Madhuri Mavinkurve IDP in Educational Technology, Indian Institute of Technology Bombay, Mumbai, India

xliv

Contributors

Anna Mavroudi Department of Computer Science, Norwegian University of Science and Technology (NTNU), Trondheim, Norway David Mazure East Stroudsburg University of Pennsylvania, East Stroudsburg, PA, USA Steven McGee The Learning Partnership, Chicago, IL, USA Randi McGee-Tekula The Learning Partnership, Chicago, IL, USA Susan McKenney ELAN, Department of Teacher Professional Development, Faculty of Behavioral, Management and Social Sciences, University of Twente, Enschede, The Netherlands Eli Meir SimBiotic Software, Missoula, MT, USA Nanxi Meng Learning Technologies Department, University of North Texas, Denton, TX, USA Negin Mirriahi School of Education and Learning and Teaching Unit, University of New South Wales, Sydney, NSW, Australia Elizabeth Molloy Department of Medical Education, University of Melbourne, Melbourne, VIC, Australia Stephanie Moore Organization, Information, and Learning Sciences, University of New Mexico, Albuquerque, NM, USA Gwendolyn M. Morel Digital Learning, Texas Higher Education Coordinating Board, Austin, TX, USA Pablo Moreno-Ger Department of Software Engineering and Artificial Intelligence, Universidad Complutense de Madrid, Madrid, Spain Konrad Morgan Eduvate, Havant, UK Cheryl A. Murphy Educational Technology, College of Education and Health Professions, Univeristy of Arkansas, Fayetteville, AR, USA Sarah Murray Centre College, Danville, KY, USA Sahana Murthy IDP in Educational Technology, Indian Institute of Technology Bombay, Mumbai, India Perla Myers University of San Diego, San Diego, CA, USA Vicki S. Napper College of Education, Department of Teacher Education, Weber State University, Ogden, UT, USA Gloria Natividad Instituto Tecnológico de Saltillo, Saltillo, Mexico Jack A. Neal Conrad N. Hilton College of Hotel and Restaurant Manegement, University of Houston, Houston, TX, USA Harold G. Nelson Computer Science, University of Montana, Missoula, MT, USA

Contributors

xlv

Junko Nemoto Meijigakuin University, Tokyo, Japan Ingunn Johanne Ness Faculty of Psychology, Centre for the Sciences of Learning and Technology (SLATE), University of Bergen, Bergen, Norway C. Paul Newhouse Centre for Schooling and Learning Technologies (CSaLT), School of Education, Edith Cowan University, Perth, WA, Australia Michele Notari PHBern, University of Teacher Education, Bern, Switzerland Institute of Lower Secondary Education, Pädagogische Hochschule Bern, Bern, Switzerland Jody Nyboer Syracuse University, Syracuse, NY, USA Ankhtuya Ochirbat Department of Computer Science and Information Engineering, National Central University, Taoyuan City, Taiwan Department of Information and Computer Sciences, School of Engineering and Applied Sciences, Ulaanbaatar City, Mongolia Refika Olgan Faculty of Education, Department of Elementary and Early Childhood Education, Middle East Technical University, Ankara, Turkey Chandra Hawley Orrill University of Massachusetts Dartmouth, Dartmouth, MA, USA Gihan Osman Graduate School of Education, and Center of Learning and Teaching, American University in Cairo, New Cairo, Egypt Ebba S. I. Ossiannilsson The Swedish Association for Distance Education, and the Ossiannilsson Quality in Open Online Learning (QOOL) Consultancy, Lund, Sweden Seungoh Paek University of Hawai‘i at Mānoa, Honolulu, HI, USA Zacharoula Papamitsiou Interdepartmental Programme of Postgraduate Studies in Information Systems, University of Macedonia, Thessaloniki, Greece Kyparisia Papanikolaou School of Pedagogical and Technological Education, Athens, Greece Gilbert Paquette LICEF Research Center, Télé-université, Montréal, QC, Canada B. Park Saarland University, Saarbrücken, Germany Kyle L. Peck The Pennsylvania State University, University Park, PA, USA Erika Petersen University of Tampa, Tampa, FL, USA Brent Philipsen Educational Sciences, Vrije Universiteit Brussel, Brussels, Belgium Michael Phillips Faculty of Education, Monash University, Melbourne, VIC, Australia

xlvi

Contributors

Niels Piepgrass Mary Lou Fulton Teachers College, Arizona State University, Tempe, AZ, USA Drew Polly University of North Carolina at Charlotte, Charlotte, NC, USA Marshall Scott Poole Department of Communication, University of Illinois Urbana-Champaign, Urbana, IL, USA Vitaliy Popov University of Michigan, Ann Arbor, MI, USA Kenneth Potter Virginia Polytechnic Institute and State University, Blacksburg, VA, USA Kevin J. Pugh School of Psychological Sciences, University of Northern Colorado, Greeley, CO, USA Emily Pulham Mountainland Technical College, Orem, UT, USA Victoria Rose Raish Pattee and Paterno Libraries, The Pennsylvania State University, State College, PA, USA Mimi Recker Department of Instructional Technology and Learning Sciences, Emma Eccles Jones College of Education and Human Services, Utah State University, Logan, UT, USA Charles M. Reigeluth School of Education, Indiana University, Bloomington, IN, USA Youqun Ren East China Normal University, PO, Shanghai, China Ramsamooj Javier Reyes Indiana State University, Terre Haute, IN, USA J. Elizabeth Richey Carnegie Mellon University, Pittsburgh, PA, USA Nathan Riedel University of Missouri-Columbia, Columbia, MO, USA Diego de Jesus Rivera-Gutierrez Microsoft Corporation, Redmond, WA, USA Taryn Robertson University of San Diego, San Diego, CA, USA Enilda Romero-Hall University of Tampa, Tampa, FL, USA Petros Roussos Department of Psychology, National & Kapodistrian University of Athens, Athens, Greece University Campus, School of Philosophy, Athens, Greece Donnalyn Roxey Knowinnovation, Columbus, OH, USA Roger Runquist Engineering Technology Department, Western Illinois University, Macomb, IL, USA Kathryn E. Rupp Northern Illinois University, DeKalb, IL, USA Tracii Ryan Faculty of Education, Monash University, Melbourne, VIC, Australia

Contributors

xlvii

Olgun Sadik Inonu University, Middle East Technical University, Ankara, Turkey Marta Fabiano Sambiase Universidade Presbiteriana Mackenzie, São Paulo, SP, Brazil Antonio Santos Department of Educational Sciences, Universidad de las Américas Puebla, Cholula, Mexico Olga C. Santos aDeNu Research Group. Artificial Intelligence Department. Computer Science School, UNED, Madrid, Spain F. K. Sarfo Department of Educational Leadership, University of Education, Winneba, Winneba, Ghana Kathleen Scalise University of Oregon, Eugene, OR, USA Richard F. Schmid Department of Education, Centre for the Study of Learning and Performance (CSLP), Concordia University, Montreal, QC, Canada Clara Schumacher University of Mannheim, Mannheim, Germany Beat Adrian Schwendimann University of California, Berkeley, CA, USA Silke Schworm University of Regensburg, Regensburg, Germany Soonhwa Seok ZÖe Center for ABA and Development Services, Columbus, GA, USA Ángel Serrano-Laguna Department of Software Engineering and Artificial Intelligence, Universidad Complutense de Madrid, Madrid, Spain Junjie Shang Learning Science Lab, Department of Educational Technology, Graduate School of Education, Peking University, Beijing, China Kshitij Sharma Department of Operations, Faculty of Business and Economics, University of Lausanne/Computer Human Interaction in Learning and Instruction, École polytechnique Fédérale de Lausanne (EPFL), Renens, Lausanne, Switzerland Rick L. Shearer The Pennsylvania State University, University Park, PA, USA Valerie Shute Florida State University, Tallahassee, FL, USA Bernadette Sibuma Massachusetts Bay Community College, Wellesley Hills, MA, USA M. A. Sicilia University of Alcalá, Madrid, Spain Computer Science Department, University of Alcalá, Alcalá de Henares (Madrid), Spain Umber Siddiqi Freelance Science Educator/Researcher, Karachi, Pakistan Kanupriya Singh University of Missouri-Columbia, Columbia, MO, USA

xlviii

Contributors

Brian K. Smith ExCITe Center and School of Education, Drexel University, Philadelphia, PA, USA Beth Rajan Sockman East Stroudsburg University of Pennsylvania, East Stroudsburg, PA, USA J. Michael Spector Department of Learning Technologies, University of North Texas, Denton, TX, USA Kristian J. Spring Instructional Psychology and Technology, Brigham Young University, Provo, UT, USA Susan L. Stansberry Oklahoma State University, Stillwater, OK, USA Christopher Stapleton Simiosys Real World Laboratory, Orlando, FL, USA Jun-Ming Su Department of Information and Learning Technology, National University of Tainan, Tainan, Taiwan Elizabeth Suazo-Flores Purdue University, West Lafayette, IN, USA Florence R. Sullivan University of Massachusetts, Amherst, MA, USA Murat Sümer Department of Computer Education and Instructional Technology, Uşak University, Merkez, Uşak, Turkey Katsuaki Suzuki Research Center for Instructional Systems, Kumamoto University, Kumamoto, Japan John Sweller School of Education, University of New South Wales, Sydney, NSW, Australia Victoria Tam Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA Vincent W. L. Tam Department of Electrical and Electronic Engineering, Faculty of Engineering, The University of Hong Kong, Hong Kong, China Rana M. Tamim Zayed University, Dubai, UAE Stergios D. Tegos School of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece Gaye Teksöz Faculty of Education, Department of Mathematics and Science Education, Middle East Technical University, Ankara, Turkey Khanh-Phuong Thai Age of Learning, Glendale, CA, USA Kenneth Thompson System-Predictive Technologies, Columbus, OH, USA Dawit T. Tiruneh University of Cambridge, Cambridge, UK Jo Tondeur Vrije Universiteit Brussel, Brussels, Belgium University of Wollongong, Wollongong, Australia

Contributors

xlix

Dhvani Toprani The Pennsylvania State University, State College, PA, USA Hallvard Trætteberg Department of Computer Science, Norwegian University of Science and Technology (NTNU), Trondheim, Norway Scott Trepper Indiana University, Bloomington, IN, USA David L. Trumpower Faculty of Education, University of Ottawa, Ottawa, ON, Canada Phoebe Tse Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA Shian-Shyong Tseng Department of M-Commerce and Multimedia Applications, Asia University, Taichung, Taiwan Maria Tzelepi Department of Psychology, National & Kapodistrian University of Athens, Athens, Greece Denis Valle University of Florida, Gainesville, FL, USA Natercia Valle University of Florida, Gainesville, FL, USA Stijn Van Laer Centre for Instructional Psychology and Technology, KU Leuven, Leuven, Belgium Arun S. Vanapalli Faculty of Education, University of Ottawa, Ottawa, ON, Canada Silke Vanslambrouck Vrije Universiteit Brussel, Brussels, Belgium Adriana Vianna University of Tampa, Tampa, FL, USA António Rodriguez Vides Institución Educativa José Acevedo y Gómez, Medellín, Colombia Julian Viera Department of Mathematics, University of Texas at El Paso, El Paso, TX, USA Lorenzo Vigentini School of Education and Learning and Teaching Unit, University of New South Wales, Sydney, NSW, Australia Susan K. Walker Department of Family Social Science, University of Minnesota, St. Paul, MN, USA Donovan R. Walling Bloomington, IN, USA Robert C. Wallon College of Education, University of Illinois at Urbana-Champaign, Champaign, IL, USA Minhong Wang Faculty of Education, The University of Hong Kong, Hong Kong, China Qiyun Wang Learning Sciences and Assessment, National Institute of Education, Nanyang Technological University, Singapore, Singapore

l

Contributors

Tianchong Wang Department of Curriculum and Instruction, The Education University of Hong Kong, Hong Kong, SAR, China Quinn Warnick Virginia Tech, Blacksburg, VA, USA Scott Warren Learning Technologies Department, University of North Texas, Denton, TX, USA Sunnie Lee Watson Learning Design and Technology, Purdue University, West Lafayette, IN, USA William R. Watson Learning Design and Technology, Purdue University, West Lafayette, IN, USA Joules M. Webb The University of Texas at San Antonio, San Antonio, TX, USA Adrian Wehmeyer East Stroudsburg, PA, USA

Stroudsburg

University

of

Pennsylvania,

East

Holly Wells East Stroudsburg University of Pennsylvania, East Stroudsburg, PA, USA Gene White East Stroudsburg University of Pennsylvania, East Stroudsburg, PA, USA Daron Williams Virginia Tech, Blacksburg, VA, USA Leah Wilson School of Psychological Sciences, University of Northern Colorado, Greeley, CO, USA Mark Wilson University of California, Berkeley, CA, USA Brenton Wimmer Center for Excellence in Transformative Teaching and Learning, University of Central Oklahoma, Edmond, OK, USA Gary Ka Wai Wong Faculty of Education, The University of Hong Kong, Pokfulam, Hong Kong Ting-Ting Wu Graduate School of Technological and Vocational Education, National Yunlin University of Science and Technology, Douliou, Yunlin, Taiwan Wen-Chi Vivian Wu Department of Foreign Languages and Literature, Asia University, Taichung, Taiwan Guangtao Xu Department of Educational Technology, School of Education, Hangzhou Normal University, Hangzhou, China Shuya Xu Indiana University, Bloomington, IN, USA Department of Instructional Systems Technology, Indiana University School of Education, Bloomington, IN, USA Daniel Yaffe Virginia Tech, Blacksburg, VA, USA

Contributors

li

Stephen C. Yanchar Instructional Psychology and Technology, Brigham Young University, Provo, UT, USA Jie Chi Yang Graduate Institute of Network Learning Technology, National Central University, Taoyuan, Taiwan Min Yang Department of Curriculum and Instruction, The Education University of Hong Kong, Hong Kong, SAR, China S. Yassine American University of the Middle East, Eqaila/Ahmedi, Kuwait University of Alcalá, Madrid, Spain Danny Young Mountainland Technical College, Orem, UT, USA Patricia A. Young Department of Education, University of Maryland at Baltimore County, Baltimore, MD, USA Fan Yu University of Missouri-Columbia, Columbia, MO, USA Jinbao Zhang School of Educational Technology, Beijing Normal University, Beijing, People’s Republic of China Lili Zhang College of Computer Science, Sichuan Normal University, Chengdu, China Yuanjing Zhang School of Educational Technology (SET), Faculty of Education, Beijing Normal University, Beijing, China Xudong Zheng East China Normal University, PO, Shanghai, China Carrie Zierath Age of Learning, Glendale, CA, USA Cliff Zintgraff IC2 Institute, The University of Texas at Austin, Austin, TX, USA Steven J. Zuiker Mary Lou Fulton Teachers College, Arizona State University, Tempe, AZ, USA

Section I Learning Theory and the Learning Sciences

1

Learning Theory and the Learning Sciences: A Section Introduction Jan Elen

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 4 5 7 8

Abstract

Learning is a fascinating phenomenon that attracts from very different perspectives ample research attention. This section of the major reference work handles from a myriad of perspectives that fascinating phenomenon. Rather than introducing each of the contributions, this introduction provides an overview of major topics addressed and some observations. Keywords

Learning · Learning sciences · Definition · Methodology

Introduction Learning is a fascinating phenomenon that attracts from very different perspectives ample research attention. This section of the major reference work handles from a myriad of perspectives that fascinating phenomenon. Rather than introducing each of the contributions, this introduction provides an overview of four major topics addressed and some observations. J. Elen (*) Centre for Instructional Psychology and Technology, KU Leuven, Leuven, Belgium e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_128

3

4

J. Elen

Topics The section provides mostly conceptual contributions that help to better understand “learning” and “supporting learning.” Four major topics can be identified: the nature of learning, the learning entity, factors, structures, and processes involved in learning and external factors that affect learning. Not surprisingly, different contributions discuss the nature of learning. Change seems to be important and even more so a change that is not temporary but enduring. While “change” may be the common denominator, “what” changes is sometimes very different. Some stress the importance of behavioral change, whereas other rather point to the changed potential to behave in a particular way, to changes in “knowledge” or to changes in the underlying neurological structure. It seems obvious that a learner learns, and hence the individual is the learning entity. While it is hardly denied that an individual can learn, various contributions point out that restricting learning to individuals is too restrictive. It is pointed out that a group might learn as well. What the group learns exceeds the summation of what has been learned by the individuals. Furthermore, digital evolutions allow for learning in larger networks and researchers point out that also these networks (as well as the systems that allow for learning) learn. A second point of attention therefore is the conceptualization of the learning entity. Factors, structures, and processes that are involved in the change at the level of the learning entity, constitute a third important topic. Mostly from the perspective of the individual as learning entity, cognitive oriented research has proposed a number of factors (e.g., prior knowledge, epistemic beliefs, emotions, intentions), structures (e.g., working memory), and processes (e.g., assimilation, accommodation). The different contributions in this respect reveal the complexity of interrelated structures and processes at different levels of granularity. While some discuss these structures and processes at a more molecular level, others focus on very specific elements at a more atomic level. The importance of prior knowledge, for instance, gets broadly accepted. That seems to be an agreement at the molecular level. As soon as more analytical discussions are initiated different emphases are made. Authors distinguish between, for instance, declarative and procedural knowledge, domain-specific knowledge, and epistemic beliefs. The discussion of what factors, structures, and processes are involved seems affected by what is considered to be the learning entity. Different factors, structures, and processes play a role when the learning entity is considered to be an individual, a group, or a (learning) network. External factors that affect learning constitute a fourth topic. Within this topic, the role of context is discussed both from the perspective of what “context” involves and how it influences learning. The discussions on how learning can be influenced and hence also supported is part of this broader topic. In other words, a lot of the research from “the learning sciences” belong to this topic. A richness of ideas characterizes seemingly the field. This richness is revealed by the various propositions to structure “context” by specifying what major contextual variables are and hence what major elements (e.g., role of feedback, need for authentic tasks, need for epistemic, social, as well as set design).

1

Learning Theory and the Learning Sciences: A Section Introduction

5

All of the above topics are closely interrelated, perspective, positions, answers in one topic affect perspectives, positions, answers on other topics.

Observations Assuming that the different contributions reflect the field as it stands and evolves a number of observations can be made. These observations point to agreements, points of discussion, as well as issues to be addressed. In any case, each of the observations suggest that with respect to learning and supporting learning, easy answers are insufficient. Learning and supporting learning are instantiations of ill-structured problems that require multiple perspectives and balanced decision-making. As indicated, the different contributions reflect a myriad of topics and perspectives. That helps to understand the complexity and to see how difficult it is to grasp the “learning” phenomenon. Despite this diversity of perspectives and ongoing discussions, the different contributions do also reflect some basic agreements especially in case learning is regarded to be a phenomenon of individuals. The following insights seem to be broadly shared: learning is a process that requires constructive activity from the learner; learning processes happen in a context that is primarily a social context, learning environments can support learning but can only increase the probability of learning success by inducing and supporting particular learning activities, in the learning process prior knowledge plays a crucial role, prior knowledge is a broad concept that encompasses domain-related content knowledge as well as knowledge about knowledge and learning. In addition to these agreements with respect to learning processes and factors that play a role, authors also seem to agree that learning is contextualized and that that context has become very complex. Hence, it is essential but no longer sufficient to study learning of facts, concepts or problem-solving for well-structured problems. In addition, attention is paid to “complex learning.” Authors seem to be aware that in a complex world there is a need for learning to deal with that complex world and for environments that adequately support that learning. While there is certainly no agreement on the exact meaning of the terms (competencies, dispositions, twentyfirst century skills,. . .), various types of high-level competencies are highly valued as evidenced by the attention for critical thinking, collaboration, or self-determined learning. The context not only is complex, it is also and primordially technological. Repeatedly, the authors in the different contributions highlight the role of technology. Although some authors ask explicitly not to put technology at the core and/or not to exclude analog technology, in most contributions technology is equated with digital technologies and even more narrowly to the extraordinary technological affordances/tools that are provided by the Internet. Research on learning is done in and on the digital age. The impact of digital technology/Internet on learning seems complex, diverse, and layered. First, the Internet seems to make learning obsolete as well as more demanding. The Internet contains an overwhelming amount of information that can be easily accessed. Hence, the need for factual knowledge seems to

6

J. Elen

decrease. At the same time, however, the information is so abundant that the need for adequate skills to retrieve and select appropriate information is increasing. These skills, however, require a lot of (declarative) knowledge in order to be able to assess the quality of the information and benefit from it. Second, although not uncriticized, the ubiquitous presence of information and communication technology has given rise to learning “theories” that, on the one hand, highlight the connected nature of learning (e.g., connectivism, collaborativism) and, on the other, broaden the scope of learning entity (e.g., a network learns). Third, the role of technology with respect to supporting learning attracts a lot of research attention. It has gradually become clear that supporting learning in a basically digital context requires approaches that to a large extent differ from the ones in analog contexts. Fourth, the availability of digital technologies requires and has allowed for new research tools (e.g., log file analysis, network analysis, learning analytics). They enable to observe interactions of learners with the environment as well as with one another. Three themes of specific importance when it comes to (supporting) learning come to the front. First, as was highlighted complexity is broadly acknowledged. Complexity asks for analysis (with appropriate tools) but also for integration. Different authors argue for integration. That integration may refer to more substantive elements (such as integration of knowing what, knowing how and knowing that one), it may also relate to the integration of cognition, emotions, and intentions. The issue of integration pertains both to what is to be learned as to how it can be done and supported. A second concern pertains to “assessment.” In various contributions, “assessment” is paid attention to. On the one hand, this reveals that not all learning is actually considered. Studying learning in this volume is especially interesting when that learning is goal-directed. That goal may not to be extremely specific but there is one and it can – at least in principle – be assessed. In other words, what primarily gets studied is formal or educational learning. On the other hand, and aligned with new conceptions on learning, new approaches for assessment are proposed. In line with the overall attention of technology, technology both in terms of potential and challenges is addressed (e.g., “learning analytics” or the use of multimedia in assessment). Given the focus on formal learning, it is not surprising that selfregulation is a major point of interest. Self-regulation is not a major problem in informal learning contexts. In informal learning settings with self-determined goals, learners seem pretty well able to engage in activities that (at least from the perspective of the learner) help to reach the goals they aim at. Self-regulation becomes an issue as soon at the self-evident self-determined nature of the goals is doubtful. While somebody might want to be able to play the guitar, it is quite different thing when somebody decides that in order to graduate in high school you have to be able to play the guitar. At that moment self-regulation becomes a problem as the regulation (of the self) has to get oriented towards externally established goals and the question on the quantity and quality of learner control arises. How much learner control and on what dimensions need to be provided in order to ensure that learners keep engaging in learning activities that help them to reach goals (they do not necessarily have established themselves).

1

Learning Theory and the Learning Sciences: A Section Introduction

7

In this section “the learning sciences” are well-represented. Under the heading “the learning sciences,” important groups of researchers have started to study learning from the perspective of how learning can be optimally supported in reallife settings. The focus is on authentic learning environments, activities that engage learners in deep-learning, collaboration among learners, and adequate use of technology. “The learning sciences” movement is largely a reaction to “positivist” mostly cognitive oriented research in (quasi-)lab settings. The approach most commonly advocated in research is design-based research. That methodology is specific as it involves actual interaction with practitioners and the elaboration (in different iterations) of learning environments. That is far from simple as it adds a number of roles to the one of “researcher” and hence there are great consequences for the “training” of design-based researchers. While the learning sciences movement is strong and important insights have been gained (e.g., the role of discourse, different approaches to scaffolding), one needs to remain aware that the movement also has limitations and does not represent “the” sciences of learning (see Seel, 2012). “The” learning sciences seem to represent a specific approach to education: a specific view on what are important educational goals, on what learning entails, and on how it can be best supported. Perhaps a better label for the important movement might be “the supporting learning sciences.” Any reader of this section will be impressed and even overwhelmed or dazzled by the plethora of concepts. This is already the case for the major topic of the section “learning.” While “endured change” seems to be a constant element, authors use a wealth of adjectives to delineate the specific focus of their attention: informal learning, formal learning, e-learning, active learning, deep learning, and collaborative learning. In addition to ample use of terms, consistent use of the terms might also be problematic. “Digital learning” may refer to the acquisition of digital skills as well as to learning in a digital environment. Also in this respect, “balance” is argued for. Becoming rigid in terminology use will probably hamper researchers in their work as they may conceive aspects of the learning phenomenon in a different way than suggested by a rigid definition of a concept. What might help, though, is that authors make very explicit how they conceive a particular concept on the one hand and avoid conceptual confusion on the other (e.g., “the learning sciences”; when describing learning as a “constructive process,” one would better refrain from using the term “constructivist learning environments” as this suggest that only in particular learning environments learning is constructive (see teaching fallacy as pointed at by Mayer (2004))).

Conclusion The section “Learning Theory and the Learning Sciences” offers a privileged outlook on how researchers on learning and supporting learning struggle with one of the most complex and fascinating phenomena: learning. It is regarded to be a meeting place for experienced researchers and scholars as well as a point of

8

J. Elen

departure for new investigators who can bring in their own perspective, discourse, and methodology in order to question, enlarge, and deepen our understanding.

References Mayer, R. E. (2004). Should there be a three strikes rule against pure discovery learning? American Psychologist, 59(1), 14–19. Seel, N. M. (Ed.). (2012). Encyclopedia of the sciences of learning. New York, NY: Springer.

Jan Elen is Full Professor at the KU Leuven, Center for Instructional Psychology and Technology. He was the head of the educational support office of the KU Leuven, coordinator of the expertise network School of Education of the Association KU Leuven, and Vice-Dean Education of the Faculty of Psychology and Educational Sciences. He is currently the academic leading the teacher education program in behavioral sciences. He has been the coordinator of the Special Interest Group on Instructional Design of the European Association for Research on Learning and Instruction and the Editor-in-Chief of “Instructional Science.” His research mainly pertains to instructional design and higher education. He teaches courses on educational technology, the design of learning environments, and didactics for behavioral sciences.

2

Designing Digital Technologies for Deeper Learning Jürgen Buder and Friedrich W. Hesse

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Humans in Complex Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deeper Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Digital Technologies as Cognitive Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design of Cognitive Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Information Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Use of Multiple External Representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Use of Group Awareness Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creation of Cognitive Conflicts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interaction Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Designing for Intuitive Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Designing for Exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Designing for Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

10 11 13 16 18 19 20 21 22 23 24 25 26 27 29

Abstract

Higher education in academic fields is often quite disconnected from professional practice. Deeper learning approaches aim at closing the gap between the way that students learn and the affordances of complex problems in their environment. This chapter deconstructs the term deeper learning, identifying its focus on problems, on declarative knowledge, on scientific inquiry skills, on skills in self-regulation, and on skills in collaboration. Moreover, the role of digital technologies is discussed: how their progress lent them the potential to become “cognitive interfaces” mediating between individuals and their environment and how they can support deeper learning. Based on a distinction between

J. Buder (*) · F. W. Hesse Leibniz-Institut für Wissensmedien, Tübingen, Germany e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_47

9

10

J. Buder and F. W. Hesse

information design and interaction design, six principles are derived that aim at the development of skills in scientific inquiry, self-regulation, and collaboration. Information design can support the development of scientific inquiry skills through the use of multiple external representations, the use of group awareness technologies, and the creation of cognitive conflict. In contrast, interaction design can support the development of self-regulation and collaboration skills through designing for intuitive interaction, designing for exploration, and designing for collaboration. Keywords

Deeper learning · Interface design · Information processing · Problem solving

Introduction The world is becoming increasingly complex. While the global society offers many potentials – just take the Internet as an example – we are also facing global challenges. For instance, armed conflicts, environmental degradation, and mass migrations are global problems that call for global solutions. They require us to deal with the complexities of a globalized world. In order to deal with complexity, people first have to learn how to deal with complexity. In that regard, it is not surprising that scholars in (higher) education and the learning sciences increasingly call for changes in how we should learn (and how we should teach). This has resulted in concepts like twenty-first-century skills (Griffin & Care, 2015), problem-based learning (Hmelo-Silver, 2004), or deeper learning (Hewlett Foundation, 2013). They are all variations of the underlying theme that teaching and learning should lead to an ability to deal with complexity and ultimately to meet the challenges of a globalized world. And yet, humans have not changed very much. Our cognitive apparatus is not exactly hardwired for dealing with complexity. Our ability to process complexity is just as limited as it was hundreds of years ago. So how can we actually learn how to deal with complexity? How can we develop twenty-first-century skills, and how can we become “deeper learners”? There are several answers to this question, for instance, by changing what we teach or by changing how we teach. This chapter explores a third answer: we can learn how to deal with complexity by using tools. In particular, we intend to outline ideas in how far the toolkit that digital technologies offer can foster deeper learning. The chapter begins with a brief description of the “stakeholders” involved in deeper learning: humans and the environments that humans try to make sense of. Then, in order to understand the interplay between humans and their (increasingly complex) environments, we will deconstruct the notion of deeper learning. With that at hand, we will argue why modern digital technologies can be great tools to foster deeper learning, leading us to the notion of “cognitive interfaces.” After that, we will discuss the design of cognitive interfaces and try to arrive at a set of

2

Designing Digital Technologies for Deeper Learning

11

design principles which might foster deeper learning. The chapter closes by looking at the road ahead and discussing some open questions for research.

Humans in Complex Environments Deeper learning, in fact, any kind of learning, involves individuals trying to make sense of their environment. Therefore, it is useful to start out with the two “stakeholders” in this interaction: the humans who try to make sense and the environment that is made sense of. Of course, there are myriad ways to describe humans, and there are myriad ways to describe complex environments. As this chapter is about learning, we will try to describe humans and their environments in terms of cognitive constructs, for instance, the individual mind, knowledge that is stored and processed in individual minds, information that is available in the environment, or flows of information between minds and environments. In describing humans, this chapter is firmly grounded in psychology, a relatively recent academic field that originated in the late nineteenth century. While early versions of psychology tried to describe humans in terms of habits and drives that determine human and animal behavior, most of these earlier ideas were swept away by the so-called cognitive revolution in the 1950s. Ever since, most scholars in psychology regard humans as cognitive systems that process information (Newell & Simon, 1972), thus likening the functioning of the human mind to the functioning of computers. According to the majority of theories in cognitive science, the human mind is made of interlinked but distinguishable cognitive structures, among them are structures in which knowledge is stored over longer stretches of time (long-term memory) and structures in which information is held and processed on a moment-tomoment basis (often referred to as working memory; Baddeley, 2007). Another fairly common distinction that is relevant to learning is between declarative and procedural knowledge (Anderson et al., 2004). Declarative knowledge is knowledge about facts, concepts, and their interrelations, whereas procedural knowledge refers to a person’s ability to get things accomplished (e.g., knowledge on how to manually multiply two numbers or how to drive a car). How does the cognitive system of humans deal with a complex environment? In order to give a rough answer to this broad question, it is useful to highlight two properties of human information processing that are quite consensual among psychologists and cognitive scientists. First, humans have only limited processing capacities. Only a small fraction of the environmental information can be perceived by our sensory systems and attended to and processed in the human mind. Even those pieces of information that do enter the cognitive system are subject to processing limitations. For instance, working memory can hold and process only a relatively small amount of information at a time (Baddeley, 2007). The second important finding about human’s ability to deal with a complex environment originated from social psychology, and it holds that human information processing and reasoning are motivated (Kunda, 1990). This notion entails that the way we process information can be dependent on the current situation, on the way we are motivated,

12

J. Buder and F. W. Hesse

on our emotions, on our attitudes, or on our social surroundings. Another way to put this second finding is that human information processing is not “objective,” but can be heavily biased (Tversky & Kahneman, 1974). We often use heuristics (“mental shortcuts”) when judging a situation, for instance, favoring information that we encountered recently or information that was easy to recall (availability heuristic; Tversky & Kahneman, 1974) or favoring information that supposedly stems from experts. We also sometimes process information in a way that our worldview will be protected, for instance, by interpreting ambiguous information in a way that confirms our prior attitudes and beliefs or by selectively attending to information that is in line with our attitudes (congeniality bias; Hart et al., 2009). Furthermore, human information processing can be biased through others that we want to impress (impression motivation; Chaiken, Liberman, & Eagly, 1989) or, more generally, through others that exert social influence on us (conformity; Asch, 1951). In sum, the limitations of cognitive processing capacity and the strong biases in information processing are evidence that the human mind is highly selective. This should be kept in mind when thinking about how people learn and how they try to face the challenges that a complex environment has to offer. After describing humans in terms of cognitive and informational constructs, the question arises on how to describe environments using related terms. Our suggestion here is to describe environments in terms of tasks, challenges, or problems. To illustrate this, imagine the following three examples. The first example would be about a physician who is faced with suggesting a particular treatment for a particular patient. The second example is not job related, but refers to everyday life: imagine that you want a house to be built. And the third example involves a politician who wants to create an international policy to reduce greenhouse gas emissions. The three examples differ widely in scope: suggesting a treatment as a doctor or buying and building a house only affects a few people, whereas creating an international policy has a global effect. Suggesting a treatment or creating a policy requires a formal education (and thus is strongly related to learning), whereas a house can be bought and built by everybody with sufficient monetary resources. And yet, we suggest, the three examples have a lot in common. First, they all can be regarded as problems to be solved. Problem solving is an area that has a long tradition in cognitive science (Newell & Simon, 1972), so by framing the environment in terms of problems or challenges, it can be addressed with the language of cognitive science. A second commonality among all three problems is that you will probably arrive at a better solution, the more declarative knowledge or information you have at your disposal. Knowing more about the patient or considering different types of treatment will help in prescribing the best treatment. Knowing more about things as different as mortgage plans or building materials will likely help you to come to a better decision where and how to build your house. And having research conducted about greenhouse gas emissions, their causes and consequences, will certainly improve policymaking. Third, all three examples are about so-called ill-defined problems: that is, they are so complex that there generally is no easy and clearly demonstrable best solution for them. In the language of cognitive science, one can conceptualize ill-defined problems as having multiple constraints that have to be accommodated.

2

Designing Digital Technologies for Deeper Learning

13

When prescribing a particular drug for a patient, the physician should also take potential side effects of the medicine or particular intolerances of the patient into account. When buying a house, one has to weigh the pros and cons about the costs, the neighborhood, and the like. Similarly, a policy to reduce greenhouse gases should consider multiple constraints (whether a nation is fully developed, emerging, etc.). Weighing these different pieces of evidence is a key determinant of what scientists do when facing complex problems, so the ill-defined nature of problems calls for a mode of scientific inquiry. Fourth, conceiving of the environment as problems to be solved entails that a person seeks to define goals and tries to advance toward these goals in a self-directed and constructive manner. Prescribing a drug, building a house, or drafting a policy all require action on behalf of a problem solver, and the self-directed approach towards goal attainment is what psychologist typically refer to as self-regulation (Boekaerts, 1999). And fifth, complex problems can rarely be solved single-handedly, but require a diverse range of expertise from many fields. A physician might consult another doctor (or rely on information provided by other doctors) in order to find the best treatment, building a house is best be done in conjunction with experts (bank managers for financial issues, architects for the building itself, etc.), and a global solution, of course, can only be found by a large number of stakeholder who actually cooperate in order to achieve the goal. In other words, complex problems typically require collaborative approaches. In sum, for the current purposes we propose to conceptualize the environment that individuals in real-life, everyday scenarios face as a set of problems to be solved. Having a firm body of declarative knowledge about the problems is likely to lead to a more informed decision. Complex problems are typically ill defined and have multiple constraints that need to be weighed in though inquiry. Solving the problems typically requires self-regulation of stakeholders as well as collaboration among stakeholders. At the same time, we know that humans often have difficulties to properly deal with a complex environment, as their processing capacities are limited and their information processing might be biased. In order to enable individuals to meet the challenges of a complex environment, they must learn to do so. Scholars in the learning sciences have begun to realize this, and as a consequence new paradigms of “deeper learning” have entered the scene.

Deeper Learning There is a growing awareness among educational scientists and practitioners in higher education that we need better teaching and learning methods if we want to enable students to better deal with complexity. The 2016 Higher Education Horizon report has seized on this idea and predicts that by the late 2010s or early 2020s, higher education will shift toward “deeper learning approaches” (Johnson et al., 2016). The term “deeper learning” was coined in a position paper from the Hewlett Foundation in 2013, and it consists of a list of six competencies that should lead to better achievements of students in the classroom and beyond: (1) mastering core

14

J. Buder and F. W. Hesse

academic content, (2) critical thinking and complex problem solving, (3) collaboration, (4) effective communication, (5) learning how to learn, and (6) developing an academic mindset. Our following discussion is inspired by this list of competencies, but proposes a somewhat different organization of skills that also addresses concepts from fields such as problem-based learning (Hmelo-Silver, 2004), inquiry-based learning (National Institute for Health, 2005), or cognitive science research on human expertise (Glaser & Chi, 1988). In particular, in the following paragraphs, we will try to map the six competencies involved in deeper learning to the five characteristics of complex environments that were discussed in the previous section. The first important issue about deeper learning is that it emphasizes complex problem solving. Traditional higher education often puts learning content at the forefront and neglects that, in order to prepare learners for their later career, a good strategy is to challenge learners with real-world problems that they need to solve. Conceptualizing a complex environment as a set of problems involves breaking down a larger problem into smaller ones and to set subgoals on the way to achieving the larger goal (Newell & Simon, 1972). In this way, limitations of the human mind can be overcome. Of course, prioritizing problems over learning content is not entirely new to deeper learning. This has led to the advent of methods like problem-based learning (Hmelo-Silver, 2004) where typically small groups of students try to find a solution to a fairly complex, practical problem (e.g., prescribing a drug based on a patient description). In sum, by trying to match the requirements and affordances of learners in higher education similar to the requirements and affordances of professional work life, deeper learning invites learners to treat their field as a series of problems to be tackled. A second important issue about deeper learning is related to declarative knowledge (facts, concepts, and the relations among them). Of course, having core academic knowledge at one’s disposal is an important requirement to tackle complex problems. This is an area where traditional higher education has a very long-standing tradition. For example, several methods suggest that retention of declarative knowledge improves if learners formulate questions before reading a text, compose summaries of texts, or use mnemonics (Levin, Levin, Glasman, & Nordwall, 1992). Moreover, if learners do not only study isolated facts, but learn relations among facts, retention of material will be fostered. A structured and organized pool of declarative knowledge also serves as a basis for learning transfer (the application of knowledge to a similar task or problem; Bransford, Brown, & Cocking, 2000). Acquiring declarative knowledge has always been a hallmark of higher education. However, a crucial distinction of deeper learning approaches is that they also stress the importance of procedural knowledge or skills, so the remaining three issues about deeper learning all refer to skills. The third important issue about deeper learning refers to thinking skills. The original paper on deeper learning (Hewlett Foundation, 2013) refers to skills in critical thinking (the ability to make well-informed judgments based on sound reasoning about multiple sources) and skills in scientific inquiry (the ability to generate and test plausible hypotheses) as key components. Some features of scientific inquiry skills can be gleaned from expertise research which found that

2

Designing Digital Technologies for Deeper Learning

15

experts are better in recognizing meaningful patterns in complexity (Chase & Simon, 1973) and are capable of mentally representing a problem based on deep structures (e.g., complex relations between concepts) rather than surface structures. The ability to critically evaluate information helps dealing with the complexity of ill-defined problems. Moreover, if learners have the ability to challenge their own view through scientific inquiry, they are better equipped to counteract motivated biases in information processing. The facilitation of such thinking skills is at the heart of inquirybased learning (National Institute for Health, 2005). The fourth issue of deeper learning revolves around the notion of self-regulation, the ability of learners to align their feelings, their thoughts, and their actions with self-set goals. This is captured by the phrase of “learning how to learn” as a constituent of deeper learning (Hewlett Foundation, 2013). Boekaerts (1999) formulated a three-layer model of self-regulation. On the first layer, learners need to choose appropriate cognitive strategies for a task (e.g., elaboration of learning materials rather than rote rehearsal). The second layer captures metacognition, the ability to repeatedly evaluate, monitor, and control one’s learning progress (Fogarty, 1994). The third layer refers to the ability to set goals and select appropriate resources (this overlaps with the notion of “developing an academic mindset” in the original description of deeper learning competencies). Many scholars argue that it is this sense of agency that drives learning forward. There are several techniques of how to foster self-regulated learning, such as posing questions, requiring regular self-assessments, or requiring students to think aloud while performing a task. The fifth and final important issue of deeper learning refers to collaboration skills (“collaboration” and “effective communication,” as the original document on deeper learning puts it). Most complex problems can only be tackled by teams of collaborators, often involving different fields of expertise. Consequently, the use and the benefits of collaborative learning have been emphasized by many scholars (e.g., Johnson & Johnson, 1997). While over the last two decades collaborative learning has become more widespread at schools, it arguably still plays a minor role in higher learning. One of the difficulties involved in establishing collaborative learning may arise from the fact that assessments are typically based on individual performances rather than team accomplishments. Summarizing these issues, we can see that the central issues of deeper learning approaches map quite nicely to the affordances of complex environments. Constructing complex environments as a set of related goals and subgoals is a skill that can be learned through problem-based instruction. On the one hand, solving complex problems often benefits from declarative (factual and conceptual) knowledge, so deeper learning should also aim at mastery of core academic content. However, the power of deeper learning mostly rests in procedural knowledge (skills) rather than declarative knowledge. Methods of inquiry learning help to acquire thinking skills that helps in tackling wicked and ill-defined problems, and they can help to overcome biases in information processing. Solving complex problems requires agency and self-regulation. And finally, most complex problems can only be solved through collaboration, therefore raising the issue for collaborative rather than individual learning methods. The similarity of deeper learning methods and the

16

J. Buder and F. W. Hesse

cognitive structure of complex environments makes perfect sense, as deeper learning approaches (in a nutshell) try to have learners face the same problems that practitioners in a job typically face (viz., solving mostly ill-defined problems). The mastery of core knowledge and facts and concepts about a domain has always taken center stage in education through the centuries. However, it should have become clear by now that the paradigm shift of deeper learning is with regard to the skill sets. How can we improve scientific inquiry skills of learners in higher education? And how can we assist learners in developing self-regulatory skills or collaborative skills? These are the central challenges of deeper learning approaches. We already stated that there are several ways on how to answer this question, for instance, by making curricula more problem centered or by dedicated teaching methods. This chapter takes a somewhat different approach by exploring the ways in which digital technologies are suited to support deeper learning skills. Therefore, the remainder of this chapter focuses on digital technologies, their potentials to support scientific inquiry, self-regulation, and collaboration.

Digital Technologies as Cognitive Interfaces In earlier sections we noted that the cognitive apparatus of individual humans is ill-equipped to deal with a highly complex environment. The processing capabilities of learners are limited, and processing itself is often prone to biases. Of course, this does not imply that humans are incapable of acting intelligently in an environment. The “trick” is that humans do use tools to compensate for physical or intellectual shortcomings: glasses improve eyesight, and books serve as an external memory. In the same vein, digital technologies can be regarded as tools that enable us to do things that would be difficult or impossible to achieve otherwise. A traditional metaphor of digital technologies is that of a repository. The early World Wide Web was like a giant, extended computer hard disk that made millions of files on millions of computers easily accessible from a local machine. The endless possibilities for storage and retrieval transformed how we deal with information. For instance, in the 1980s getting a weather forecast for a foreign country was quite a complex task, whereas getting a phone number from a foreign country was almost impossible. Today, in contrast, these types of information can typically be found in mere seconds. The World Wide Web has taken our memory capabilities to entirely new levels (and made the necessity to be able to mentally retrieve information somewhat obsolete). Consequently, it was not surprising that digital technologies soon were used as tools for learning. Educational technology made educational information accessible to students. Students can enroll and register for courses online, they receive learning materials and assignments via the Internet, and rather than sitting in an auditorium, students might just download the latest lecture on their digital device. With the advent of mobile phones, students could access learning resources whenever they wanted and wherever they were. The idea of unlimited access also led to relatively recent educational technology developments like

2

Designing Digital Technologies for Deeper Learning

17

Fig. 1 Interaction between humans and their environments: (a) unmediated interaction and (b) interaction mediated by digital technologies

massively open online courses (MOOCs) where students can take courses from around the globe. Figure 1 displays how digital technologies have changed the way that humans interacted with their environment. The upper panel of Fig. 1 shows the technologically unmediated interaction between a human being and the physical environment of that person. The lower panel of Fig. 1 shows that with the advent of digital technologies, humans had two environments from which information could be drawn: the physical environment and a digital environment where vast amounts of online information were accessible. However, as humans cannot directly perceive the digital environment, the lower panel of Fig. 1 also introduces an interface that provides access to the digital informational environment. If digital technologies are seen as a giant repository, they enable access to information and certainly improve the declarative knowledge base that is at our disposal; but as we outlined, procedural knowledge or skills are what really matters in deeper learning. However, we argue that digital technologies can be much more than just repositories, particularly in light of two relatively recent developments. A first development refers to the blurring between digital and physical environments. Until a couple of years ago, digital and physical environments were separate entities: the computer did not “know” much about the physical environment that a human was embedded in. But nowadays the digital informational environment is increasingly capable of registering context information from the physical environment: our smartphones do “know” where we are located, and smart sensors can measure a device’s tilt, or the local temperature, and the like. Our digital devices are no longer “blind” to the physical environment, thus becoming context-aware. Digital technologies do not only “know” more about our environment; they also “know” more about us, be it through our browsing history and our social network contacts or, increasingly, through sensors that measure physiological data (heart rate, touch intensity, brain waves, etc.). A second feature of recent advances in digital technologies is the idea of combining and evaluating all the new information sources that they can tap into. We are entering an age of “big data” where computers interpret the amount of information that they have about humans and their environments and

18

J. Buder and F. W. Hesse

make inferences: e.g., automatically muting the ringtone when they detect that a user is in a movie theatre or selecting soothing music when a user’s heart rate goes up. Technological capabilities of combining and (automatically) interpreting data have also arrived in education under the rubric of “learning analytics” (Greller & Drachsler, 2012). The fact that modern digital technologies “know” more about their users and the physical environments the users are embedded in, and the fact that digital technologies can (and do) “intelligently” act on these pieces of information, suggests an entirely new technology metaphor. Rather than being passive repositories, digital technologies have capabilities to become active mediators of our interaction with the environment. As they are no longer blind to “their” environment, they become agentic, i.e., they can adapt to environmental changes, and they can offer help or feedback or provide recommendations. Imagine a smartphone that measures your blood sugar concentration and based on this might recommend a nearby restaurant before you even realize that you are hungry. Or to take an example from education, imagine an interface that measures your brain activity while learning, recognizes when you are cognitively overloaded or frustrated, and automatically presents easier tasks to you. Whether you regard such scenarios as a dream or a nightmare, it becomes evident that digital technologies do have the potential to become an active mediator in the interaction with our environment. As the interfaces mediating between humans and their environment “know” more and “act” better, they ultimately exhibit cognitive features of information processing. In order to accommodate for this new and active role, we propose the term “cognitive interface” to describe recent digital technologies. Not only do cognitive interfaces assist in cognitive endeavors of humans; they also exhibit cognitive properties themselves. This is captured in Fig. 2. Note that the digital environment is now context-aware, meaning that is also connected to the physical environment through sensors. Thus, the loop between humans, their environments, and the interface is now closed. This enables interfaces to become “cognitive interfaces.” Cognitive interfaces are much more than a passive repository. They are capable of diagnosing a situation, and they can vary their “behavior” accordingly. It is this computational power that ultimately lends cognitive interfaces the potential to support deeper learning. A repository can only provide factual information in a passive manner. In contrast, cognitive interfaces are flexible, constantly changing what information is available to a learner, based on situational affordances. They can monitor a learner’s progress, thus assisting in the development of skills in scientific inquiry, self-regulation, and collaboration.

Design of Cognitive Interfaces To address the question of an appropriate design of cognitive interfaces, it is useful to differentiate between two design functions: information design and interaction design. Information design is concerned with the question of what information is

2

Designing Digital Technologies for Deeper Learning

19

Fig. 2 Cognitive interfaces as mediators between humans and their environment

made available to a learner. We already discussed that the information processing capacity of humans is limited, and we can only attend to a small amount of information that is actually available in our environment. Therefore, a cognitive interface should select information from the (physical and digital) environment and only display those pieces of information that are most helpful in supporting a learner or user. Essentially, information design is about the relationship between the cognitive interface and the digital environment depicted in Fig. 2. In contrast, interaction design is about the relationship between the cognitive interface and the individual human, as it addresses the question about what actions a person can perform at any given moment to exert an influence on the environment. Through appropriate information design and interaction design, learner activities are facilitated that help to develop skills in scientific inquiry, in self-regulation, and in collaboration. To answer the questions about proper information design and interaction design in the context of deeper learning, one should take into account principles from educational psychology, cognitive psychology, and social psychology. How these general principles can be incorporated into the design of cognitive interfaces will be explained in the following sections.

Information Design Information design mediates between a cognitive interface and the digital environment (or, by extension, the physical environment). The main question therefore is which type of information from the digital environment is selected by the cognitive interface and how this information should be properly represented in order to foster deeper learning activities. In particular, we argue that by appropriate information selection and information representation, information design can facilitate scientific inquiry skills of learners. Therefore, it is helpful to recap some of the thinking and inquiry skills that we identified in the section on deeper learning: (1) scientific inquiry involves the skills to flexibly perceive a complex environment from multiple viewpoints, (2) having the skills to integrate information from a diverse set of sources is an important prerequisite of scientific inquiry, and (3) scientific inquiry also entails the ability to critically assess not only the viewpoints of others but also

20

J. Buder and F. W. Hesse

one’s own viewpoint. On this basis we propose three principles on how the information design of cognitive interfaces might create windows of opportunity for students to develop deeper learning skills. All principles are concerned with flexibility, variability, and embracing informational diversity. The first principle (use of multiple external representations) deals with the way that information is represented in a cognitive interface. The other two principles (use of group awareness technologies, creation of cognitive conflicts) deal with the way that a cognitive interface selects information from the digital environment. Each principle will be discussed in turn.

Use of Multiple External Representations Experts at complex problem solving are highly flexible when it comes to processing information. They do not take information at face value, but can seamlessly transform it to their needs (e.g., abstracting away from a problem’s surface structure and re-represent it in terms of deep structures). Therefore, a good strategy to develop deeper learning skills is to provide students with an opportunity to practice this fluency and flexibility of how to represent information. Educational psychology and instructional design have a long research tradition on the effectiveness of representational formats (Mayer, 2009). For instance, the so-called multimedia principle holds that people learn better from text plus pictures than from text alone, and this was experimentally confirmed across many studies (Mayer, 2009). The proposed reason for this effect is that textual information will be processed in a verbal channel in working memory, whereas pictorial information can be simultaneously processed in a visual channel of working memory. Taking the multimedia principle one step further, Ainsworth (1999) investigated the use of so-called multiple external representations. For instance, take numbers: you can represent numbers as text (“four”), as an Arabic number (“4”), as a Roman numeral (“IV”), as a picture of four small circles, or as a picture of a hand with four fingers pointing upwards. There is evidence that students are better the more they are able to fluently switch between and integrate different types of external representations. Ainsworth (1999) stated three reasons why multiple external representations are advantageous. First, multiple representations are complementary to each other: a representation might be advantageous for one task, but detrimental for another one; therefore, being able to fluently change or transform representations is helpful. Second, some representations constrain each other in meaningful ways. For instance, the picture of a house needs to be much more specific than just the word “house.” And third, fluency in switching between multiple external representations makes transfer of learning much more likely. It also increases the likelihood that learners will construct a deeper understanding. In sum, learning with multiple external representations might be harder than learning with a single representational format, but it should pay off with regard to the development of deeper learning. Cognitive interfaces can make use of this effect in various ways. First, they can easily provide access to different representational formats. Moreover, they can also

2

Designing Digital Technologies for Deeper Learning

21

support learners in integrating information from different representational formats (Bodemer, 2011). A concrete example that makes heavy use of modern digital technologies was presented by Oestermeier, Mock, Edelmann, and Gerjets (2015). They had children assemble two-dimensional structures with plastic toy bricks. The bricks then could be laid onto an interactive multi-touch tabletop which converted the brick layout into music. Through experimenting with the bricks, the children learned basic principles of musical composition, and though this was not directly tested, it can be hypothesized that children would also learn actual musical notation faster. In sum, the digitally enhanced use of multiple external representations has a high potential to help learners in seeing a problem from multiple perspectives. This should increase the likelihood that learners will develop scientific inquiry skills to discover deep structures which link various representations.

Use of Group Awareness Technologies While novices often get lost in detail, experts tend to seek for the bigger picture in order to discover deep structures. In order to arrive at the bigger picture, it is necessary to integrate information from multiple sources. For instance, experts in complex problem solving take more time analyzing and scrutinizing the wide range of information that is available to them (Glaser & Chi, 1988). Embracing diversity of opinions and variability of patterns of information is another key feature of scientific inquiry. Cognitive interfaces can support this mode of scientific inquiry by trying to preserve or even highlight informational diversity. One way to accomplish this is through the use of so-called group awareness tools (Engelmann, Dehler, Bodemer, & Buder, 2009). Group awareness tools select data from a group (e.g., objective test results, objective amounts of talk, subjective conceptualizations of declarative knowledge, subjective ratings expressing opinions) and feedback these data to the group as a whole. Some group awareness tools aggregate these data from the physical and digital environment, showing how a group “thinks” or has acted about a given state of affair. For example, Buder and Bodemer (2008) explored a group awareness tool in which contributions of an online discussion forum were rated by subjective novelty and subjective agreement. The aggregated quality and agreement ratings for each discussion post were then represented in a visualization, thus providing the bigger picture of how a group as a whole is “thinking” about a range of different discussion contributions. This tool was geared at making minority opinions in a group more salient, and it was shown that individuals and groups actually arrived at better decisions when they were able to see the variability and the bigger picture. Other group awareness tools do not aggregate information over all group members, but just represent the data from all participants in a format that visualizes interindividual variability and diversity. For instance, the partner knowledge awareness tool asks learners to individually indicate how much they understood

22

J. Buder and F. W. Hesse

paragraphs from an online textbook with a simple yes/no judgment (DehlerZufferey, Bodemer, Buder, & Hesse, 2011). Subsequently, dyads of learners were provided with their own and their partner’s yes/no ratings. From these visualizations, it was easy to see which parts of the textbook both partners understood, which parts they both had difficulties with, and which parts provided an opportunity for one learner to learn from the peer. It was shown that this information design improved the interaction of dyadic learners and subsequent performance. The effectiveness of group awareness technologies is well documented, both with regard to their potentials (Engelmann et al., 2009) and to some shortcomings (Ray, Neugebauer, Sassenberg, Buder, & Hesse, 2013). However, it should be noted that group awareness tools are only one way to visualize and thus experience informational diversity. What all these tools have in common is that they want to improve scientific inquiry skills by making informational variability salient: ranges of declarative knowledge, ranges of opinions, and ranges of options. Rather than boiling down the complexity of an informational environment, these tools embrace variability. In this way, they can lay the foundation for a mode of scientific inquiry that is a hallmark of deeper learning.

Creation of Cognitive Conflicts Another key feature of scientific inquiry skills entails having a critical stance: toward information in the environment, toward others, but also toward oneself. Consequently, deeper learning does involve permanently testing and questioning one’s own beliefs. This resonates well with the psychological and educational idea that behavioral change can be brought about by cognitive conflict between the self and the environment (e.g., Festinger, 1957). As learning implies behavioral change, theorists like Piaget suggested that we learn when we are cognitively thrown off-balance (Piaget & Inhelder, 1969). If there is a discrepancy between our current knowledge or worldview and the environment, we need to adapt to the situation, either by extending our existing schemas with the new information in an environment (assimilation) or by completely restructuring and building new schemas in our mind (accommodation). In short, cognitive conflict is viewed as a vital prerequisite for learning and development. As a consequence, collaborative learning methods such as the structured controversy approach try to purposefully create cognitive conflict among students by framing topics as a controversy and by having students argue from different viewpoints (Johnson & Johnson, 1997). The goal is not to show who is right (as in a debate), but by trying to integrate various viewpoints into a consensus position. In organizational psychological literature, too, cognitive conflicts are often regarded as particularly conducive to performance – as long as they occur on a task-related level (van Knippenberg, De Dreu, & Homan, 2004). Finally, the creation of cognitive conflict is also central to scientific progress. Science advances by questioning existing theories and trying to falsify them.

2

Designing Digital Technologies for Deeper Learning

23

How can cognitive interfaces create cognitive conflict that actually challenges our knowledge and our worldviews? One way of creating these challenges is by exposing learners to a variability of viewpoints (which are likely to include dissenting viewpoints) – an approach that was covered in the preceding section on group awareness tools. However, just exposing individuals to pro-attitudinal and counterattitudinal information might not be sufficient, as a wealth of studies have shown that individuals are often motivated to neglect dissenting information (congeniality bias; Hart et al., 2009). Apparently, in order to actually make learners process conflicting information, some more “nudging” in information design is needed. For instance, in our own research (Schwind, Buder, Cress, & Hesse, 2012), we exposed information seekers with arguments for and arguments against neuro-enhancement (the alleged facilitation of cognitive performance through the intake of drugs). When the participants had to select arguments on which they wanted to know more about, they picked preference-consistent information, exhibiting the congeniality bias that is highly typical of motivated processing. However, we also manipulated information design by presenting the information in the fashion of a so-called recommender system. When preference-consistent information was recommended, we again found a congeniality bias. However, if a preference-inconsistent argument was recommended, information seekers selected this conflicting information more often, subsequently arrived at a more balanced view on the topic, were better able to recall dissenting information, and exhibited better critical thinking. Other ways to create cognitive conflict through information design is by framing a topic in a way that generates controversy. For instance, Fischer (2001) developed a 3-D model of a neighborhood with tangible objects (trees, park benches, etc.) located on a digital touch surface. This 3-D model then served as a so-called boundary object, a common frame of reference for an ensuing controversial discussion among stakeholders with partially conflicting interests (architects vs. urban planners) who could move objects around to create a better environment. Similarly, using online discussion forums also appears to be a good strategy for learners to shape their argumentative skills and learn how to question others’ beliefs as well as their own (Buder, Buttliere, & Ballmann, 2015). Apparently, a key to creating cognitive conflict and to fostering scientific inquiry and deeper learning is not only to expose learners to a range of information but to activate learners to process and elaborate dissenting information: through recommending preference-inconsistent arguments (a kind of “nudging”) or by letting them directly interact with people who have a different worldview (as is the case with boundary objects or discussion forums).

Interaction Design While information design is about the ways that information should be selected and presented, interaction design is about the type of activities that should be afforded to users in order to instigate deeper learning. We have already made the point that deeper learning is predominantly about the development of procedural knowledge

24

J. Buder and F. W. Hesse

(skills) of scientific inquiry, self-regulation, and collaboration. In the previous section we suggested that good information design caters to scientific inquiry skills and activities. In contrast, we believe that interaction design is aimed at selfregulation skills and collaboration skills. As for self-regulation, one of the strengths of cognitive interfaces is that they are naturally designed for interactivity. Learners interact with their digital environment, constantly shaping and reshaping what they see and how they can act on it, whether it is through browsing on the Internet, exploring virtual environments, using cognitive simulations, or manipulating digital objects on interactive tabletops. In this regard, cognitive interfaces provide a basis for students to practice self-regulatory skills. However, it should be noted that students to do not become excellent selfregulators just by “thrown into” a digital environment that requires self-regulation. On the contrary, many students are overwhelmed when acting completely on their own (Kirschner, Sweller, & Clark, 2006). Therefore, the development of expertise in self-regulation requires external support. With their capabilities to diagnose learners and environments, cognitive interfaces can provide exactly those types of support and feedback that are conducive to deeper learning. As for collaboration, cognitive interfaces are a perfect means to connect learners. On the one hand, digital technologies like e-mail, chat, or videoconferencing enable learners to communicate with each other over the distance. However, cognitive interfaces can also facilitate face-to-face collaboration among learners. An example of the latter would be small group interaction at large, interactive tabletops where learners can see and interact with digital objects while simultaneously interacting with other group members. Based on this background, we propose three interaction design principles how cognitive interfaces can foster the development of self-regulation and collaboration skills. The first principle (designing for intuitive interaction) is about reducing workload which can then be put into the development of self-regulation and collaboration skills. The second principle (designing for exploration) is about supporting self-regulation. And the third principle (designing for collaboration) addresses interaction design for groups.

Designing for Intuitive Interaction Effective self-regulation and collaboration requires cognitive resources, particularly for students who have little experience with this mode of interaction. Cognitive interfaces could therefore be designed to make interaction as easy and intuitive as possible, in order to overcome limitations of our cognitive system. Touchscreen devices like smartphones, tablets, or interactive tabletops have become widespread over the last few years and are increasingly used for interacting with learning contents. Touch-sensitive displays have the advantage that learners can directly manipulate digital objects by touching them. Moreover, learners can immediately perceive the consequences at the point of touch. Gestures like tapping,

2

Designing Digital Technologies for Deeper Learning

25

dragging, or rotating are regarded as highly intuitive, similar to interacting with physical objects. From a viewpoint of cognitive science, touch displays have a number of potential benefits. For instance, it is known that stimuli near to the hands are given preferential attention by the cognitive apparatus and are processed in more detail (Brockmole, Davoli, Abrams, & Witt, 2013). More generally, haptic displays and gesture-based forms of interaction are examples of so-called embodied cognition (Clark, 2008) which holds that cognitive processes often involve the entire body. For instance, we point at things with our hands, we rotate objects in our hands in order to better understand them, and for some tasks we prefer to count with our fingers. Along the same lines, intuitive interaction has the potential to simplify cognition. For example, by arranging digital objects on a touch display, inference processes are transformed into relatively simple perceptual processes: rather than mentally rotating objects, they can be rotated in the physical environment, thus reducing working memory load. Finally, there is evidence that features of objects are better remembered if a person has previously interacted with the objects (Kirtley & Tatler, 2016). It is not clear whether interacting with digital objects on a touch display has similar, beneficial effects on retention, but recent research results are quite promising that this might indeed be the case (Truong, Chapman, Chisholm, Enns & Handy, 2016). Of course, designing for intuitive interaction is neither a necessary nor a sufficient condition for developing deeper learning skills. However, given that self-regulation skills and collaboration skills need practice and require cognitive resources, simplifying interaction appears to be a useful strategy. In a similar vein, well-established instructional design principles hold that the so-called extraneous cognitive load created through the design of a technology should be minimized (Sweller, 2005).

Designing for Exploration We outlined that dealing with complex environments requires an active and selfregulatory stance (e.g., planning, monitoring one’s progress, evaluating one’s progress, and taking actions if needed). Ideally, deeper learning should try to engender a similar spirit of self-regulatory activity. Self-regulation can be observed on different levels of granularity. For instance, if someone plans in which order to read a number of books, this would entail self-regulation on a very molar level. However, reading a book is not by itself a very interactive endeavor, as the content of a book does not change depending on the reading behavior. In contrast, take an example of an interactive simulation where a learner changes parameters and conducts a number of small experiments in order to try out which set of parameters yields the best results (de Jong & van Joolingen, 1998). In this case, the interactivity cycles are very short. Each change of a parameter will be followed by running the simulation, and learners receive immediate feedback on the results of the parameter change. One of the benefits of cognitive interfaces is that they have very short interactivity cycles: they invite learners to act on the environment and provide immediate feedback. We

26

J. Buder and F. W. Hesse

believe that these technological affordances of tightly knit interactivity cycles provide an ideal playground to develop self-regulatory skills. There are many ways on how cognitive interfaces can provide this high level of interactivity. Exploring virtual environments is a type of interaction where a learner’s activities play out in their navigational patterns. However, an even clearer case of interactivity is afforded by cognitive interfaces in which learners actually manipulate digital objects in their environment: by re-arranging, resizing, deleting, combining, or annotating elements. It can be argued that active exploration and manipulation off-load working memory, as relations among elements can be directly perceived in the environment yet at the same time invite learners to modify their environments and learn through this constant cycle of actions and effects. Moreover, cognitive interfaces should not only enable the manipulation of external representations but also the creation of new digital objects in one’s environment (externalization). In this way, internal mental structures can be translated into one’s environment where they can be inspected and, if necessary, changed by oneself and others. It should be noted, however, that designing for exploration is a double-edged sword. Many scholars have pointed out that merely providing an environment that can be explored does not necessarily foster learning, whether it is with regard to the exploration of virtual environments (Bowman, Wineman, Hodges, & Allison, 1999), interactive simulations (de Jong & van Joolingen, 1998), or, more generally, for technologies with little guidance (Kirschner et al., 2006). Therefore, cognitive interfaces should always try to combine explorations with explicit guidance and feedback. This can be accomplished through the use of rapid assessments and instructional prompts (Renkl, Skuballa, Schwonke, Harr, & Leber, 2015) or through modelling techniques (Hoogerheide, van Wermeskerken, Loyens, & van Gog, 2016) that assist learners in developing self-regulatory skills. As cognitive interfaces become better in diagnosing the state that learners are in, they will also get better in providing adaptive guidance in exploration.

Designing for Collaboration We already emphasized that many real-world problems (e.g., building a car) are just way too complex to be addressed by a single person. Moreover, many real-world problems require the consent and participation of a large number of stakeholders (e.g., developing a fair agenda on how to reduce the emission of greenhouse gases). Finally, it is often through collaboration with others that learners get to know different perspectives and experience cognitive conflict that can overcome biases and is conducive to learning. All these issues point towards a need to collaborate. In fact, in practical, work-related scenarios, collaboration is the norm rather than the exception, with problem solving typically occurring on the level of collaborative teams and committees. Educators have realized this, which has led to a host of learning approaches that try to make use of collaborative learning (Johnson & Johnson, 1997). While there is ample evidence that collaborative groups are more productive than individuals (Johnson & Johnson, 1997), it should be noted that

2

Designing Digital Technologies for Deeper Learning

27

social psychology typically holds that groups do not make use of their full potential. Hinsz, Tindale, and Vollrath (1997) suggested that groups, more than individuals, tend to reduce variability: they produce not as many ideas as the sums of individuals could produce, they prefer socially shared over unshared information, and they can become more extreme after exchanging information. We believe that the interaction design of cognitive interfaces should take into account that collaboration increases productivity, but should also try to maximize the potentials that groups have. For instance, there is reason to believe that the productivity of groups depends on the actual tasks that groups try to accomplish. Following a taxonomy from McGrath (1984), we propose to distinguish between three types of group tasks. The first type refers to tasks where neither diversity nor conflict among group members is an issue (e.g., brainstorming). For these tasks, it is probably best to even prevent direct interaction among group members: ideas should be generated individually and then be combined at a later stage of teamwork. A second type of task involves diversity of knowledge and expertise, but not necessarily conflict (e.g., assigning a team to read 20 scientific articles). In such a case, it is probably best to create a division of labor where members tackle nonoverlapping subtasks. Finally, in the third type of task, cognitive conflict among group members may arise (e.g., group decision making on ill-defined problems or negotiation tasks or any task that requires a commitment from stakeholders). For these tasks it is best to have actual discussions and deliberations among collaborators. The first two task types (parallel work like brainstorming, division of labor) would call for an interaction design where each individual group member has his or her own cognitive interface. Of course, the output of group members must be merged at some stage, and it can probably benefit from the use of group awareness tools which highlight similarities and differences that may exist in the group. However, for tasks with cognitive conflicts, it is probably best to have a shared cognitive interface that visualizes the to-be-discussed issues. Having a shared interface provides a common ground for all group members, and this enables participants to point at particular elements. Moreover, each change that is made to the externalized representation is immediately visible to all stakeholders. In order to get the most out of a group, interaction design principles for collaboration should be married to the information design principles that we identified in this chapter (using multiple external representation, creating group awareness, creating cognitive conflicts among group members).

Conclusions The use of digital technologies has found its integral part in higher education. The ability to enroll for courses online or to watch a video of a lecture at any time and any place is just an example of how digital technologies can make student life easier and more accessible. While these advances have transformed how we learn, they have not changed what we learn. However, there is a growing awareness that is the latter part (what we learn) that needs to be transformed. With its focus on the acquisition of

28

J. Buder and F. W. Hesse

facts and concepts (declarative knowledge), many students in higher education institutions are rather ill-equipped for the daily problems of academic (or non-academic) professional practice. They often lack the skills needed to solve complex problems. Recognizing this deficit in higher education has led to the development of deeper learning approaches (Hewlett Foundation, 2013). They suggest that the best way to achieve a smooth transition between higher education and professional work life is if students are trying to develop and employ the necessary skills as early as possible. In this chapter we tried to deconstruct the somewhat vague notion of “deeper learning” into smaller parts, thereby arriving at five components. First, deeper learning means to frame the environment in terms of concrete problems to be solved. Second, deeper learning requires mastery of declarative knowledge in order to make sense of the environment. Third, deeper learning involves the development of new ways of thinking: integrating information from various sources, embracing diversity, and being spurred by a sense of scientific inquiry. Fourth, deeper learning entails self-regulation to develop the ability to act on a complex environment. And fifth, deeper learning requires collaboration to make use of distributed expertise needed to solve highly complex problems. There is some research on the first issue (e.g., with regard to problem solving or problem-based learning). There is certainly a lot of research on the second issue (i.e., how to improve retention of facts). However, we believe that the biggest challenges for deeper learning are with regard to the third (scientific inquiry), fourth (selfregulation), and fifth issue (collaboration), as they focus more on the actual skills. This chapter therefore focused on how digital technologies (especially if they serve as “cognitive interfaces”) create an environment where inquiry skills and skills of self-regulation and collaboration can be facilitated. We proposed that scientific inquiry skills can best be fostered through an appropriate information design: by using multiple external representations, by using group awareness technologies that make “the bigger picture” salient, and by creating cognitive conflict between a learner and the environment. Moreover, we proposed that self-regulation and collaboration can best be facilitated through an appropriate interaction design: by making interfaces intuitive, by inviting learners to explore (though not without guidance), and by creating opportunities for collaboration. Of course, the distinction between information design and interaction design is more of an academic nature, hinting at the different goals that cognitive interfaces might tackle. From a practitioner’s point of view, information design and interaction design go hand in hand. The trend toward “deeper learning” is relatively recent, and actually many experts in higher education even regard “deeper learning” as a future trend (Johnson et al., 2016). Therefore, it is no wonder that conceptual and empirical work on deeper learning is still in its infancy. Thus, out of necessity the current chapter is built on quite a lot of speculation. In the best sense of our argument from the passage on “collaboration,” this chapter is like an external representation made visible to various stakeholders: it describes the views of two scholars, but is not intended as a definitive “be-all and end-all.” If the views described herein are perceived to have some merit, they will certainly be built upon, refined, or even refuted.

2

Designing Digital Technologies for Deeper Learning

29

That being said, it is highly likely that other experts would deconstruct the notion of “deeper learning” differently than we did. They might arrive at a completely different set of skills that need to be fostered. Or they might modify the list of design principles that we proposed with regard to information design and interaction design. Even if someone fully subscribes to our present conceptualization, the soundness of our arguments needs to be tested empirically. For each of the six design principles we identified, there is some empirical evidence: we know that the use of multiple external representations can deepen understanding, we know that group awareness technologies increase performance, and the like. However, most of the findings that we presented were not explicitly tested in the field of higher education, so they should be replicated in the relevant educational settings. Another area that certainly requires a lot of empirical work has to do with the best balance between challenging and overburdening the cognitive system. For instance, using multiple representations is more cognitively demanding than using just one representation, creating cognitive conflict requires more effort than avoiding conflict, and self-regulation is harder than being told what to do. Following the same logic, it is evident that there are situations where learners cannot handle multiple external representations or experience too much conflict, and this is likely to have detrimental effects. Therefore, in designing for deeper learning, we should be well aware that there is a (potentially small) window between getting a cognitive system involved and getting it overloaded. It certainly needs empirical research to identify these windows of opportunity. If we have a deeper understanding of how students and experts make sense of a complex environment, the full potential of deeper learning can be uncovered. Acknowledgments This work was funded through the Leibniz ScienceCampus Tübingen “Informational Environments.”

References Ainsworth, S. (1999). The functions of multiple representations. Computers & Education, 33, 131–152. Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111, 1036–1060. Asch, S. E. (1951). Effects of group pressure upon the modification and distortion of judgment. In H. Guetzkow (Ed.), Groups, leadership and men. Pittsburgh, PA: Carnegie Press. Baddeley, A. (2007). Working memory, thought, and action. Oxford: Oxford University Press. Bodemer, D. (2011). Tacit guidance for collaborative multimedia learning. Computers in Human Behavior, 27, 1079–1086. Boekaerts, M. (1999). Self-regulated learning: Where are we today? International Journal of Educational Research, 31, 445–457. Bowman, D. A., Wineman, J., Hodges, L., & Allison, D. (1999). The educational value of an information-rich virtual environment. Presence, 8, 317–331. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn. Washington, DC: National Academy Press. Brockmole, J. R., Davoli, C. C., Abrams, R. A., & Witt, J. K. (2013). The world within reach: Effects of hand posture and tool use on visual cognition. Current Directions in Psychological Science, 22, 38–44.

30

J. Buder and F. W. Hesse

Buder, J., & Bodemer, D. (2008). Supporting controversial CSCL discussions with augmented group awareness tools. International Journal of Computer-Supported Collaborative Learning, 3, 123–139. Buder, J., Buttliere, B., & Ballmann, A. (2015). Cognitive conflict in forum discussions on scientific topics. In Work-in-progress poster proceedings of the 23rd International Conference on Computers in Education (ICCE 2015) (pp. 4–6). Hangzhou, China. Chaiken, S., Liberman, A., & Eagly, A. H. (1989). Heuristic and systematic information processing within and beyond the persuasion context. In J. S. Uleman & J. A. Bargh (Eds.), Unintended thought (pp. 212–252). New York: Guilford Press. Chase, W. G., & Simon, H. A. (1973). The mind’s eye in chess. In W. G. Chase (Ed.), Visual information processing (pp. 215–281). New York: Academic Press. Clark, A. (2008). Supersizing the mind: Embodiment, action, and cognitive extension. Oxford: Oxford University Press. de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery with computer simulations of conceptual domains. Review of Educational Research, 68, 179–201. Dehler-Zufferey, J., Bodemer, D., Buder, J., & Hesse, F. W. (2011). Partner knowledge awareness in knowledge communication: Learning by adapting to the partner. Journal of Experimental Education, 79, 102–125. Engelmann, T., Dehler, J., Bodemer, D., & Buder, J. (2009). Knowledge awareness in CSCL: A psychological perspective. Computers in Human Behavior, 25, 949–960. Festinger, L. (1957). A theory of cognitive dissonance. Stanford University Press. Fischer, G. (2001). Articulating the task at hand and making information relevant to it. HumanComputer Interaction, 16, 243–256. Fogarty, R. (1994). How to teach for metacognition. Palatine, IL: IRI/Skylight Publishing. Glaser, R., & Chi, M. T. H. (1988). Overview. In M. T. H. Chi, E. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. xv–xxviii). New York: Psychology Press. Greller, W., & Drachsler, H. (2012). Translating learning into numbers: Toward a generic framework for learning analytics. Educational Technology and Society, 15, 42–57. Griffin, P., & Care, E. (Eds.). (2015). Assessment and teaching of 21st century skills: Methods and approach. Dordrecht: Springer. Hart, W., Albarracín, D., Eagly, A. H., Brechan, I., Lindberg, M. J., & Merrill, L. (2009). Feeling validated versus being correct: A meta-analysis of selective exposure to information. Psychological Bulletin, 135, 555–588. Hewlett Foundation. (2013). Deeper learning competencies. Retrieved from http://www.hewlett. org/uploads/documents/Deeper_Learning_Defined__April_2013.pdf Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997). The emerging conceptualization of groups as information processors. Psychological Bulletin, 121, 43–64. Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16, 235–266. Hoogerheide, V., van Wermeskerken, M., Loyens, S. M. M., & van Gog, T. (2016). Learning from video modeling examples: Content kept equal, adults are more effective models than peers. Learning and Instruction, 44, 22–30. Johnson, D. W., & Johnson, F. P. (1997). Joining together: Group theory and group skills (4th ed.). Englewood Cliffs, NJ: Prentice-Hall. Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Hall, C. (2016). NMC horizon report: 2016 Higher education edition. Austin, Texas: The New Media Consortium. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41, 75–86. Kirtley, C., & Tatler, B. W. (2016). Priorities for representation: Task settings and object interaction both influence object memory. Memory & Cognition, 44, 114–123.

2

Designing Digital Technologies for Deeper Learning

31

Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 108, 480–498. Levin, J., Levin, M., Glasman, L., & Nordwall, M. (1992). Mnemonic vocabulary instruction: Additional effectiveness evidence. Contemporary Educational Psychology, 17, 156–174. Mayer, R. E. (2009). Multimedia learning (2nd ed.). Cambridge, MA: Cambridge University Press. McGrath, J. E. (1984). Groups: Interaction and performance. Inglewood, NJ: Prentice Hall. National Institute for Health. (2005). Doing science: The process of science inquiry. Retrieved from http://science.education.nih.gov/supplements/nih6/inquiry/guide/info_process-a.htm Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall. Oestermeier, U., Mock, P., Edelmann, J., & Gerjets, P. (2015). LEGO music: Learning composition with bricks. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC '15) (pp. 283–286). New York: ACM. Piaget, J., & Inhelder, B. (1969). The psychology of the child. New York: Basic Books. Ray, D. G., Neugebauer, J., Sassenberg, K., Buder, J., & Hesse, F. W. (2013). Motivated shortcomings in explanation: The role of comparative self-evaluation and awareness of explanation recipient’s knowledge. Journal of Experimental Psychology: General, 142, 445–457. Renkl, A., Skuballa, I. T., Schwonke, R., Harr, N., & Leber, J. (2015). The effects of rapid assessments and adaptive restudy prompts in multimedia learning. Journal of Educational Technology & Society, 18, 185–198. Schwind, C., Buder, J., Cress, U., & Hesse, F. W. (2012). Preference-inconsistent recommendations: An effective approach for reducing confirmation bias and stimulating divergent thinking? Computers & Education, 58, 787–796. Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 19–30). New York: Cambridge University Press. Truong, G., Chapman, C. S., Chisholm, J. D., Enns, J. T., & Handy, T. C. (2016). Mine in motion: How physical actions impact the psychological sense of object ownership. Journal of Experimental Psychology: Human Perception and Performance, 42, 375–385. Tversky, A., & Kahneman, D. (1974). Judgement under uncertainty: Heuristics and biases. Science, 185, 1124–1131. van Knippenberg, D., De Dreu, C. W., & Homan, A. C. (2004). Work group diversity and group performance: An integrative model and research agenda. Journal of Applied Psychology, 89, 1008–1022.

Jürgen Buder has been Deputy Head of the Knowledge Exchange Lab at the Leibniz-Institut für Wissensmedien in Tübingen since January 2012. Moreover, he coordinates the scientific development of the Leibniz ScienceCampus Tübingen “Informational Environments.” Within the Leibniz ScienceCampus, he is speaker of a research cluster on “Peer productivity in Web 2.0 environments” in which he has a project on productivity in online discussion forums. His research deals with the question of how people deal with conflicting information on the Net and how digital technologies can be employed to counteract biases in human information processing. Jürgen Buder studied psychology in Göttingen (diploma) and moved to Tübingen in 1995. There, he was working at the German Institute for Research on Distance Education (DIFF; 1995–2000) and at the Department of Applied Cognitive Psychology and Media Psychology of Tübingen University’s Psychology Institute (2000–2008). In 2002, he received a Faculty Award for his Ph.D. thesis on knowledge exchange. Friedrich W. Hesse is founder and Executive Director of the Leibniz-Institut für Wissensmedien and Head of the Knowledge Exchange Lab (since 2001). Moreover, he is the Scientific VicePresident of the German Leibniz Association (since 2010). He also is Head of the Department for Applied Cognitive Psychology and Media Psychology at the University of Tübingen (since 1999)

32

J. Buder and F. W. Hesse

and spokesman of the Leibniz ScienceCampus Tübingen “Informational Environments” (since 2009). Together with his Lab he works on fundamental principles of individual and cooperative knowledge acquisition and knowledge exchange with new media and the practical implementation of concepts of virtual learning and teaching. His research interests are learning with new media, net-based knowledge communication, and computer-supported collaborative learning (CSCL).

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice Cliff Zintgraff and Atsusi Hirumi

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rationale and Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rationale for Surveyed Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Challenges Are Associated with LCD Alignment? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Plethora of Terms and Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems with Alignment: Top-Down from Philosophy to Practice . . . . . . . . . . . . . . . . . . . . . . . . Problems with Alignment: Bottom-Up from Practice to Philosophy . . . . . . . . . . . . . . . . . . . . . . . . Problems with Internal Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems with Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problem: Lacking a Systems View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Proposing and Grounding the LCD Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Framework Proposal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Topic of the Overall Framework: LCD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Framework Core Concepts and How They Relate: Philosophy, Theory, Research, and Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sections of the Framework: Descriptions and Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Applying the Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Personalizing the Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Preliminary Assessment Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LCD Artifact Assessment Tool Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

34 34 36 37 37 37 39 40 41 43 43 43 43 46 47 59 59 59 61 63 65 65 66 67

C. Zintgraff (*) IC2 Institute, The University of Texas at Austin, Austin, TX, USA e-mail: [email protected] A. Hirumi University of Central Florida, Orlando, FL, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_119

33

34

C. Zintgraff and A. Hirumi

Abstract

Learner-centered approaches to teaching and learning are favored by many educators and instructional designers. However, even those who favor learnercentered design (LCD) recognize that there are multiple interpretations and applications, and that some designs work better than others. Why is there such variance in LCD? One reason is the lack of alignment in LCD philosophy, theory, research, and practice. Lack of alignment begins with confusion over the plethora of constructs associated with LCD. Then, as the LCD process unfolds, alignment can suffer in two ways, top-down and bottom-up. Alignment should flow down from philosophies, to theory and research, and into the practice of instructional design, curriculum development, and delivery. Alignment should also flow bottom-up, from the practical constraints of the classroom, to those of curriculum development and instructional design, and then further to inform research, theory, and educational philosophies. Within a course, the learning objectives, strategies, and assessments must also be aligned to ensure the kind of internal consistency that is fundamental to the design of effective, efficient, and engaging learning experiences. Finally, when different researchers examine evidence, multiple overlapping interpretations contribute to differing conclusions about the effectiveness of LCD and resulting implications. To facilitate the alignment of LCD philosophy, theory, research, and practice, the authors pursue a coherent framework that captures the key elements of LCD and their relationships. Building on the framework, a simple model is proposed for assessing the artifacts of LCD, including documentation of LCD techniques, methods, curricula, and implementations in online and classroom settings. The framework and related assessments can guide future LCD research and practice. Keywords

Learner-centered design · LCD · Student-centered learning, Project-based learning · Constructivism,

Introduction Rationale and Purpose Learner-centered design (LCD) is a highly touted approach for facilitating learning in K-12 and higher education (Reigeluth & Carr-Chellman, 2009b). Through LCD, students receive relevant instruction that connects to their current knowledge and skills, directed at many types of learning outcomes (Reigeluth, Myers, & Lee, 2017). The increasing prevalence of STEM-dedicated schools deploying learner-centered designs (e.g., Scott, 2012; Texas Education Agency, 2014), and the proliferation of STEM-focused programs during and outside the school day (e.g., FIRST, 2016; Project Lead the Way, 2014) provide further impetus for the development of this

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

35

chapter. As these trends toward LCD develop, many questions remain. An examination of related literature reveals several issues. For instance: • There is a plethora of terms, constructs, and propositions regarding LCD (Reigeluth & Carr-Chellman, 2009b); • The principles and other associated intentions of LCD often do not get translated into classroom practice (Cuban, 2001; Hall & Hord, 1987; Mishra & Koehler, 2006; Spector, 2014). • Classroom realities are not honored; the limitations of the classroom often are not considered when creating designs and curriculum (Cuban, 2001; Hall & Hord, 2006; Reigeluth, Beatty, & Myers, 2017). • There are debates over what evidence exists regarding LCD and the implications of such findings (Hmelo-Silver, Duncan, & Chinn, 2007; Kirschner, Sweller, & Clark, 2006; Tobias & Duffy, 2009). This chapter presents educators and educational researchers with a framework for aligning and advancing LCD philosophy, theory, research, and practice. A common framework for understanding can help stakeholders overcome challenges that often hold LCD back from reaching its full potential. To demonstrate the efficacy of the framework, an assessment model is proposed and examples are presented of how the framework might be used to evaluate the congruence of LCD artifacts. The chapter begins with exploration of the problems seen with LCD, establishing the need for the proposed framework. Next, the framework is presented, with support for each framework element and relationship from literature, illustrating how the framework may be used to assess the alignment of LCD artifacts. Selected examples are given to demonstrate how such assessments might look when completed. The chapter concludes with recommendations for additional research to advance LCD philosophy, theory, research and practice. In developing the framework, the authors contend that the design of instruction in general, and LCD in particular, should be grounded in research and theory. Grounded design is “the systematic implementation of processes and procedures that are rooted in established theory and research in human learning” (Hannafin, Hannafin, Land, & Oliver, 1997, p. 102). The authors also believe that “. . . effective instructional design is possible only if the developer has reflexive awareness of the theoretical basis underlying the design. . .[it] emerges from the deliberate application of some particular theory of learning” (Bednar, Cunningham, Duffy, & Perry, 1995, pp. 101–102). Likewise, designs must reflect the realities of the classroom and other settings in which learning occurs. The designs must recognize teachers’ roles as professionals who facilitate learning. Instructional designers have an implied responsibility to create effective, efficient, and engaging learning experiences that work in real classrooms with practicing teachers. Cuban (1986), in his review of educational reform from 1920 to 1990 demonstrated the negative outcomes that almost always result when the needs of teachers within their classroom setting are ignored. Ertmer and Ottenbreit-Leftwich (2010) and Harris, Mishra, and Koehler (2009), in their

36

C. Zintgraff and A. Hirumi

contemporary analysis of constructivist methods entering new classrooms, also described the central role of teachers in achieving practical results. Only when learner-centered research, theory, and designs are aligned with classroom realities will LCD achieve its potential in classrooms.

Rationale for Surveyed Literature There is a large knowledge base associated with LCD. This chapter is grounded primarily in four strands, including literature on (a) learner-centered instructional theories, (b) constructivism debates, (c) instructional systems design models, and (d) models created to improve the alignment of LCD constructs in practice. Reigeluth’s Green Books consist of four highly cited volumes on instructional design theories and models. Volume III (Reigeluth & Carr-Chellman, 2009b) and Volume IV (Reigeluth, Beatty, et al., 2017), are focused on learner-centered design. Those volumes help frame questions around LCD constructs, definitions of terms, the progression of thinking in the field, and alignment among the full range of LCD stakeholders. Under the assertion that constructivism is the philosophical foundation of LCD (Hmelo-Silver et al., 2007; Reigeluth & Carr-Chellman, 2009b; Reigeluth, Myers, et al., 2017), a survey was performed of the series of commentaries beginning with Kirschner et al. (2006) that debated constructivist approaches to teaching and learning. The series contains multiple exchanges (e.g., Hmelo-Silver et al., 2007; Schmidt, Loyens, van Gog, & Paas, 2007). An edited volume by Tobias and Duffy (2009) facilitated yet more discussion. Authors in these works debated the evidence for the success of constructivist approaches. Their debates demonstrated basic disagreements over definitions of terms, epistemological perspectives, interpretations of evidence, and the proper course forward based on research to date. The third strand is a collection of literature on instructional systems design models that highlight the importance of following a systematic process to ensure alignment of three fundamental elements of instruction: learning objectives, assessments, and strategies. The collection also defines various strategies for facilitating learning in general, and for learner-centered design in particular, that are based on learning and instructional theories. Some keyworks in this collection are Dick, Carey, and Carey (2015), Morrison, Ross, Kemp, and Kalman (2010), Gustafson and Branch (2002), and Smith and Ragan (1999) on systematic design of instruction. This collection of literature is especially relevant to how the alignment of instructional goals, strategies, design, curriculum development, delivery, and assessment happen when targeting specific learning outcomes. The fourth strand is a set of models used to help teachers align new instructional methods with their own training, beliefs, and classroom goals. The three models included in the fourth strand are the Concerns-Based Adoption Model (CBAM) (Hall & Hord, 1987), Technological Pedagogical Content Knowledge (TPACK) (Mishra & Koehler, 2006), and the Constructivist Learning Environment Survey (CLES) (Taylor, Fraser, & White, 1994). These models and associated research are

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

37

representative of significant prior efforts to achieve alignment in the LCD field with a focus on classroom implementation issues. They help illustrate the challenges and level of effort associated with alignment.

What Challenges Are Associated with LCD Alignment? As a result of the LCD literature survey, several challenges were noted that face educators and instructional designers, challenges centered on the lack of alignment between LCD philosophy, theory, research, and practice. These challenges include (a) formulating a common understanding of terms, (b) establishing a bi-directional alignment between more abstract elements (e.g., philosophy) and more concrete elements (e.g., practice), (c) interpreting evidence for LCD, and (d) accounting for the effects of applying LCD in real-world settings.

Plethora of Terms and Constructs LCD involves a plethora of terms and concepts. Simple misunderstandings about their meanings can lead to miscommunications and misinterpretations that make it challenging for researchers and practitioners to align LCD philosophy, theory, research, and practice. To facilitate alignment among stakeholders, a basic agreement about the meaning of terms is required. In Volume 3 of the Green Books, Reigeluth and Carr-Chellman (2009b) observed, “Instructional theorists often use different terms to refer to the same constructs and the same term to refer to different constructs. This is confusing for researchers, practitioners, and graduate students, and it is the most obvious indicator of the lack of a common knowledge base” (The Nature of Instructional Theories: Constructs and Terms, para. 1). They reported limited success in surveying colleagues and arriving at common terms, with challenges both in response rates and in their colleagues’ beliefs that their desire for common terms was achievable. Nevertheless, they proposed a number of terms, many of which were quite technical in nature. One might note that, consistent with Volume III goals, Reigeluth and CarrChellman did not consider teachers in their attempt to define terms. Later, in Volume IV, Reigeluth, Myers, et al. (2017) concluded that all stakeholders, including teachers, must be included in the thought process of how to advance the LCD field. Table 1 further illustrates the problem. The table includes many – but not all – instructional methods and constructs associated with LCD. It is not difficult to understand how communication problems can easily arise.

Problems with Alignment: Top-Down from Philosophy to Practice At some point in the development and delivery of instructional content, decisions are made – explicitly or implicitly – about the philosophical perspectives (values), theories, and related research that will be used to drive instructional design,

38

C. Zintgraff and A. Hirumi

Table 1 A sample of terms and constructs associated with LCD Term or construct Twenty-first century skills Case-based learning Cognitive apprenticeship Cognitive load Cognitivism Competency-based education Connectivism Constructionism Constructivism Discussion-based learning Empiricism and rationalism Experiential learning Flipped classrooms Game-based learning Inquiry-based learning Just-in-time instruction Learning objects Maker space models Metacognition Mobile learning Problem-based learning Participatory design Project-based learning Scaffolding Simulation learning Situated cognition Social cognitive theory Social constructivism Task-centered instruction User-centered design User-experience design Zone of proximal development

Seen in Reigeluth, Myers, et al. (2017) Ertmer and Newby (1996) Collins, Brown, and Newman (1988) Sweller (1988) Ertmer and Newby (1993) Voorhees and Voorhees (2017) Siemens (2014) Papert and Harel (1991) Ertmer and Newby (1993) Gibson (2009) Ertmer and Newby (1993) Lindsey and Berger (2009) Strayer (2017) Myers and Reigeluth (2017) Prince and Felder (2006) Novak and Beatty (2017) Wiley (2000) McKay and Glazewski (2017) Veenman, Van Hout-Wolters, and Afflerbach (2006) Cochrane and Narayan (2017) Savery (2015) and Hmelo-Silver et al. (2007) Spinuzzi (2005) Markham, Larmer, and Ravitz (2003) Verenikina (2003) Gibbons, Mcconkie, Seo, and Wiley (2009) Brown, Collins, and Duguid (1989) Bandura (1994) Palincsar (1998) Francom (2017) Abras, Maloney-Krichmar, and Preece (2004) Kuniavsky (2010) Vygotsky (1978)

curriculum development, and content delivery. Assuming those choices are sound, then alignment from philosophy to theory and research, and then to instructional design and curriculum development, and then to high-fidelity implementation by classroom teachers, will ultimately benefit student outcomes (Hall & Hord, 1987; Mishra & Koehler, 2006; Reigeluth & Carr-Chellman, 2009b). Top down alignment does not always occur. Spector (2014) observed how new pedagogies are often evaluated “without examining the quality of the implementation” (p. 16). Reigeluth and Carr-Chellman (2009b) made clear the terminology translation errors that can occur among stakeholders. Ertmer & Ottenbreit-Leftwich

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

39

(2010) and Harris et al. (2009) observed how lack of teacher professional development can lead to classroom deployment of new curricula and pedagogies that teachers believe are high fidelity but are actually weak in their implementation. In fact, strong evidence for the importance of top-down alignment can be inferred from the amount of time and effort expended to help teachers understand the philosophies, theories, and designs they are asked to bring into classrooms. Three examples can be seen in models used to help prepare teachers for novel approaches in their classrooms. The Concerns-Based Adoption Model (CBAM) (Hall & Hord, 1987, 2006) was developed to track teacher concerns during the implementation of new educational methods in classrooms. The Technological Pedagogical Content Knowledge model (TPACK) (Mishra & Koehler, 2006) helps teachers integrate technology with pedagogical methods and content goals. The Constructivist Learning Environment Survey (CLES) is an instrument taken primarily by students, but also by teachers, to assess whether constructivist methods are being used in classrooms. All three of these tools have been subjected to rigorous analysis across multiple international settings (Albion, Jamieson-Proctor, & Finger, 2010; Christou, Eliophotou-Menon, & Philippou, 2004; Nix, 2012). In particular, Nix reported on 15 studies using CLES with the instrument taken by 11,632 students. These tools were rigorously developed and are designed to determine if curriculum is being implemented as intended.

Problems with Alignment: Bottom-Up from Practice to Philosophy The problem of alignment from practice to philosophy can be understood from the following hypothetical scenario. Imagine an integrated math, science, and physics curriculum developed for high school students. Imagine an idealistic case where constructivist philosophies have been adopted, LCD theories have been identified, educational programs have been designed based on these theories, and teachers have been trained to apply LCD theory and implement LCD programs, all with high fidelity. Now imagine that this curriculum is mandated in a school with traditionally minded administrators, a traditional culture that values the teacher as the center of epistemological authority, and a physical building layout that is a barrier to block scheduling and co-teacher planning times for coordination among the math, science, and physics teachers. Sadly, the curriculum deployment stands little chance of success. It will not work in the target setting. What is missing in this scenario is bottom-up alignment, a scenario where designers and developers, as well as researchers and theorists, allow practicalities of the classroom to inform their designs. Once again, the authors note how the work of Cuban (2001), Ertmer & Ottenbreit-Leftwich (2010), and Harris et al. (2009), among others, highlighted the professional role of teachers and the importance of cocreating new instructional approaches with the involvement of a representative group of educators. Reigeluth, Myers, and Lee (2017) reached the same conclusion in Green Book Volume IV. Stakeholders must be involved in conversations if LCD is to achieve its potential for wide adoption in the target setting.

40

C. Zintgraff and A. Hirumi

In doing so, designers, developers, researchers, and theorists can learn from other disciplines’ use of design methods to engage front-line stakeholders. Two methods consistent with the principles and practices associated with LCD are user-centered design (also referred to as user-experience design) and participatory design. Both have a long history of practice across multiple fields, recently in software development, and also in the fields of consumer product development, sociology, action research, and even politics (Bannon, Bardzell, & Bødker, 2019; Computer Professionals for Social Responsibility, n.d.; Interaction Design Foundation, n.d.; Kuniavsky, 2010; Norman, 2013; Spinuzzi, 2005). Common themes include: (1) the early and ongoing participation of those served by designs in a test-learn-revise process; (2) a related iterative approach to product development; and (3) an equity lens, making sure bottom-up alignment includes consideration of underserved populations (Bannon et al., 2019). Which challenges from K-12 and higher education might be addressed by bottom-up alignment? Classroom instructors have prior beliefs that will not simply be set aside (Cuban, 2001; Ertmer, 2005), whether due to conscious choice, momentum, or lack of awareness. Meanwhile, teachers work in settings with their own physical layout, structure, culture, demands, and resources (Reigeluth & CarrChellman, 2009b). As noted above, these preexisting conditions can be barriers to the redesign of a school day supporting LCD designs. If teachers cannot co-plan interdisciplinary content, or secure resources for LCD projects, or receive administrator support for their efforts, or teach to the standards required of them by law, or take the time required to absorb a new way of thinking, or generally follow the instructional design they are asked to execute, designers will not see expected results. How can bottom-up concerns of educators be addressed? Designers and curriculum developers can allow flexibility for teachers to incorporate their own beliefs and practices (Cuban, 2001; Ertmer, 2005). It may even be possible for barriers to be flipped or otherwise exploited to create better end products (Norman, 2013). Designers and developers can provide flexibility for the preexisting conditions of schools (Reigeluth & Carr-Chellman, 2009b). They can invite educators to be cocreators during design, curriculum development, and classroom delivery. An iterative design process, among other benefits, can identify professional development needs and provide actionable insights. In the end, a lack of some mix of these attributes – flexibility, cocreation, and iteration – indicates a lack of bottom-up alignment.

Problems with Internal Alignment How many times have students taken a test and rightly wondered where a question or answer came from? How often are students left wondering what is need-to-know versus nice-to-know information?

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

41

These problems reflect poor internal alignment. Once specific learning objectives are introduced, alignment must be maintained between learning objectives, instructional strategy (including the provision of content information), and assessment. While the terms used to refer to these elements may differ between settings, the alignment of these elements is fundamental to high-quality training and educational programs. This is true regardless of the philosophies and strategies adopted and regardless of whether instruction is delivered online, in conventional classrooms, or in hybrid learning environments. For instance, if an objective for learners is to apply an algorithm, assessment should ask learners to apply the algorithm, and instructional strategy should give learners practice in applying the algorithm. Sometimes objectives are externally specified in state or national curriculum standards or by professional organizations. Strategies and assessment should map directly to those externally specified objectives. A primary reason for defining objectives is to help learners focus their attention on what matters most. Ironically, many students have been conditioned to do the opposite – to ignore objectives because subsequent tests or assignments are not aligned. Perhaps worse, students are often presented with fuzzy objectives or receive no assessment criteria at all. When learning objectives are clear and concrete, and learner assessments are aligned with objectives, and instructional strategies lead learners to successful assessment, LCDs are efficient and effective, and they are most likely to be appealing to teachers and students. Internal alignment between objectives, assessment, and strategy must begin in curriculum design, continue through curriculum development, and be sustained with fidelity during classroom delivery.

Problems with Interpretation Fundamental to one’s philosophy of research are beliefs about which evidence matters and what such evidence implies. The Kirschner et al. (2006) initiated series of commentaries illustrate this issue in action. Likewise, the essays in Tobias and Duffy (2009) demonstrated how researchers of different philosophical persuasions vary widely in interpreting past results. Kirschner et al. (2006) and his like-minded colleagues (Sweller, Kirschner, & Clark, 2007) made several arguments against constructivist and related approaches to instruction. They argued: (a) constructivist methods lean heavily toward discovery learning, where students discover content on their own; (b) such methods are not consistent with human cognitive architecture and concerns about cognitive load; and (c) meta-analyses focused on properly controlled studies do not support constructivist methods of instruction. A sampling of authors with competing views, such as Jonassen (1999), HmeloSilver et al. (2007), Schmidt et al. (2007), Schwartz, Lindgren, and Lewis (2009), and Wise and O’Neill (2009), argued that: (a) constructivist teaching methods include strong guidance for students; (b) methods are consistent with human

42

C. Zintgraff and A. Hirumi

cognitive architecture; and (c) alternate meta-analyses support constructivist methods of instruction, as do results from early applications of the methods in medical education. The authors do not seek to quell the debate between constructivism and guidance. Most relevant to the current chapter is the ways in which these disagreements persist. Critics of constructivist teaching methods declined to recognize the guidance inherent in many constructivist methods (e.g., Duffy, 2009; Hmelo-Silver et al., 2007). They also chose to ignore the consistent message about the importance of guidance in instruction, even when selected constructivists went so far as to acknowledge direct instruction as a legitimate method in appropriate circumstances (Duffy, 2009; Schwartz et al., 2009). Critics of constructivist teaching argued philosophical bias on the part of constructivists, but one can also argue that positivists also operate within their biases (Alexander, 2006; Bernstein, 1995). Meanwhile, advocates of constructivism seemed insufficiently interested in considering the information processing mechanisms active during instruction (Duffy, 2009) and their potentially negative implications for constructivist teaching methods. For example, Moos (2011) documented how high-context constructivist environments are more likely to benefit high-performing students and are often problematic for the low-performing students who most need guidance and direction. Constructivism’s advocates could do a better job of counting the cost of context. Furthermore, Duffy (2009) highlighted several deficiencies standing in the way of additional research into constructivist techniques: clear definition of techniques; sharper definitions of scaffolding, guidance, and when to apply certain methods; and identification of hypothesized variables at work, even if those variables are more difficult to measure in the kind of field research demanded by constructivist claims. On these points, Duffy defended constructivism’s critics and put the onus on its supporters to more clearly define its variables and mechanisms and find ways to test its claims. Why do these disconnects persist? Duffy (2009) argued that philosophical differences in how we come to know about the world are one cause. The authors often “talk[ed] past each other” (Tobias & Duffy, 2009, p. 6). Reigeluth and CarrChellman (2009b) might refer to this as a difference in values. As a result, differing interpretations of results occurred, with the two camps highlighting different studies. Ardent critics of constructivist teaching methods wished to declare a lack of evidence over time and close the case, while advocates demonstrated some evidence of success and highlighted the promise available if constructivist techniques can be deployed consistently and brought to full scale. This chapter’s offering of an LCD framework and assessment tool is in the spirit of Tobias and Duffy’s (2009) effort to bridge the gaps in communication and shared understanding of terms like those in Table 1. Lines of communication should be opened to address disagreements in the field (Duffy, 2009). The current authors carry forward the insight that there is a difference between established research and ongoing research. Acknowledging this difference might allow one to think more deeply about how the system of philosophies, theories, research efforts, and practices interact to achieve, or fail to achieve, alignment in given circumstances.

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

43

Problem: Lacking a Systems View It was previously noted that Reigeluth, Myers, et al. (2017), in Green Book Volume IV, moved away from Volume III’s focus on researchers and graduate students and toward a fuller consideration of an LCD system involving all stakeholders, including teachers. Invoking the phrase “systems thinking” (Preface, Why a Volume IV, para. 2), they argued that “decisions about what to teach, how to teach it, and how to assess it must all be dramatically different now compared to those that were appropriate for the Industrial Age, and those decisions should be made together because they are interdependent. That interdependency has not been addressed in Volumes I–III, but it is addressed here in Volume IV” (para. 3). As Duffy (2009) closed the Tobias and Duffy (2009) volume, he shared this citation from Lave and Wenger (1991): “Learning is achieved through changes in a system that includes elements such as people, social norms and expectations, resources such as books or calculators, history, institutional expectations, and individual cognitive architecture—all of which work together in ways that cannot be separated” (Tobias & Duffy, p. 354). Duffy’s goal was to open lines of communication. The goal of this chapter is to recognize and document the system at work, better define the elements of the system, and define their relationships, all toward the goal of facilitating learning by improving alignment.

Proposing and Grounding the LCD Framework Framework Proposal The authors propose the Learner-Centered Design Framework, depicted in Fig. 1, by exploring the terms, concepts, and relationships presented in the illustration that are grounded in the problems described in the surveyed literature, and supported by descriptions of terms and relationships in the same literature.

The Topic of the Overall Framework: LCD Smith and Ragan (1999) defined instructional design as the “systematic and reflective process of translating principles of learning and instruction into plans for instructional materials, activities, information resources, and evaluation” (p. 2). For LCD, the principles referenced by Smith and Ragan are those specific to learnercentered instructional methods. The philosophical foundation of LCD is constructivism (Hmelo-Silver et al., 2007; Reigeluth & Carr-Chellman, 2009b; Reigeluth, Myers, et al., 2017). Constructivism embraces the belief that learners do not simply receive and store knowledge, but that new knowledge is constructed by learners in a manner that builds on their prior knowledge (Ertmer & Newby, 1993; HmeloSilver et al., 2007).

44

C. Zintgraff and A. Hirumi

Fig. 1 Learner-Centered Design (LCD) framework

Consistent with the words learner-centered, LCD is commonly understood to shift attention away from the teacher to the learners (e.g., American Psychological Association, 1993; CTGV, 1992; Holmes Group; 1990). Figure 2 illustrates the difference between the two approaches. In short, for teacher-directed designs, focus is placed on the teacher’s knowledge, experience, and content to be delivered by the teacher. In teacher-directed environments, learning is facilitated as an additive process. Students are viewed as empty vessels and new information is simply added on top of existing knowledge. Training and education is designed for the average learner and large groups of students advance at the same rate. Parents and community members may contribute to student learning, but they are rarely involved in any systematic way. In contrast, learner-centered designs begin with students’ existing knowledge and experience. Research suggests that students are not empty vessels. They come to class with their own perceptual frameworks (Erickson, 1984) and learn in different

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

45

Fig. 2 A comparison of teacher-directed and learner-centered designs. (Note: Figure 1 from “Student-centered, technology-rich, learning environments (SCenTRLE): Operationalizing constructivist approaches to teaching and learning,” by A. Hirumi, 2002, Journal for Technology and Teacher Education, 10(4), 497–537. Reprinted with permission)

ways (Kolb, 1984). Learning is no longer viewed as a passive event; rather, learning is an active and dynamic process in which learners constantly reformat connections between key concepts and ideas and continuously revise their schemas (Cross, 1991). Students derive their own meaning from the content by talking, listening, reading, writing, and reflecting what they learned (Meyers & Jones, 1993). In fully student-centered environments, learners are given direct access to the knowledge-base and work individually and in small groups. Many such environments emphasize the advantage of authentic problems toward driving engagement and deep learning during instruction. Many learner-centered designs deploy parents and community members to interact directly with teachers, students, and the knowledge-base (Markham et al., 2003; Zintgraff, 2016). These adults play an integral role in the learning process to help meet the individual needs and interests of students. While the dichotomy between the two approaches is apparent, it is rare to find cases where instruction is strictly delivered in one sustained manner. In a stringent teachercentered environment, the instructor attempts to dictate what is learned and how it is learned, presenting all learning objectives, strategies, and assessments in a highly prescribed manner (Kirschner et al., 2006). In extreme learner-centered environments, students define their own learning goals, and they take complete responsibility for what they learn, how they learn, and how they are assessed (Bruner, 1961; Lefrancois, 1997). In reality, teaching and learning in today’s schools usually consists of a blend of approaches where instruction is teacher-centered at times, and learner-centered at other times. At their core, learner-centered designs must bring each student’s knowledge to the surface. Learning experiences must engage students in a manner that holds them responsible for their learning. Instructors are present and may direct learning at times, but they serve primarily as facilitators of the learning process.

46

C. Zintgraff and A. Hirumi

The full complexity of LCD-in-practice was highlighted by Reigeluth, Myers, et al. (2017). They wrote that only with proper organization of the school, school system, and state education system can LCD thrive. As previously noted, Cuban (1986) documented a decades-long history of educational reforms in classrooms and how the fate of reform is bound to the practical realities of teachers in classrooms. Ertmer and Newby (1993), Mishra and Koehler (2006), and Gibson (2009), among others, all highlighted the teacher’s role. Student-centered does not mean that these realities have ended. Over the long term, the realities of classrooms may change to better accommodate LCD (Reigeluth, Beatty, et al., 2017). Meanwhile, if LCD is to be practical and find a place in schools, LCD must account for these realities as it attempts to build on the existing knowledge of learners. Collecting the thoughts from this section, the following definition is provided. LCD is the process of translating philosophies, theories, and research of constructivist teaching and learning into designs for instruction, where such designs both (1) emphasize the needs of individual learners, and (2) succeed in practice in the intended settings of learning.

Framework Core Concepts and How They Relate: Philosophy, Theory, Research, and Practice People operate with different explicit or implicit definitions of philosophy, theory, research, and practice. To advance clarity, use of the terms in this chapter is described. In explaining their usage, the authors also establish their understanding of the relationship among these four constructs, relationships reflected in the proposed framework. Philosophy is any statement that represents a deep-held belief about what is real in the world (ontology) or how people come to know (epistemology) (Grayling, 1998; Teichmann & Evans, 1999). Philosophies are often adopted by people as values (Reigeluth & Carr-Chellman, 2009b). When broadly shared among people, philosophies facilitate progress. When the subject of debate, philosophical positions slow progress – which may in fact be healthy and appropriate. Theories are propositions that help explain or predict phenomena that may be tested and supported with empirical research (Hoy & Adams, 2015). While philosophies are not empirically testable, theories might be formulated from a philosophical position for subsequent testing. In return, emerging proven theories can help define new philosophical positions. Theories that are not yet established may emerge from empirical, positivist research, and/or from applied or naturalistic approaches that work to describe phenomena and lay the groundwork for more rigorous study (Guba & Lincoln, 1994). All theories, both developing and well-established ones, are subject to revision based on new research results. Reigeluth and Carr-Chellman (2009b) noted two types of theory important in the field of instructional design. Descriptive learning theories describe how the world works. Theories that explain mechanisms of human cognition (Atkinson & Shiffrin, 1968) exemplify descriptive theory. Instructional design theories, in comparison,

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

47

prescribe effective ways to create instruction to achieve particular results. A rigorous method for developing project-based learning is one example of a design theory. Research places theories and other propositions under rigorous methods of study. Experiential, quasi-experimental, and correlational research examines testable hypotheses using quantitative data. Qualitative and other naturalistic studies are designed to better understand phenomena without necessarily testing cause-andeffect relationships. Often, qualitative research attempts to better understand a phenomenon solely within the context of local conditions. Design research, in comparison uses both quantitative and qualitative data to improve theories, tools, policies, products, or processes (Brown, 1992; Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003; McKenney & Reeves, 2018). Guba and Lincoln (1994) described a continuum of research paradigms (p. 109) from fully positivist to highly context- and observer-dependent. Research methods across this continuum contribute new knowledge and insights about LCD. That said, the strongest support for theory comes from repeated and rigorous empirical results. Often the term research is invoked to lend an idea credibility, but some research results are more established than others. For example, research into the cognitive architecture of short-term and long-term memory (Atkinson & Shiffrin, 1968) is well established. The research evidence for the efficacy of inquiry-driven models of instruction is: (a) established in the view of some researchers; (b) ongoing at best in the view of others; and (c) clearly subject to debate among researchers across the field (Hmelo-Silver et al., 2007; Kirschner et al., 2006; Tobias & Duffy, 2009). In this chapter, the authors highlight the difference between established and ongoing research, believing this tactic enables greater accuracy in describing LCD. Practice happens whenever general ideas are applied toward specific learning objectives. In using the criteria of learning objectives as the line where practice begins, practice is defined more broadly than as simply what happens in a classroom. By this definition, instructional designers are practicing when they bring philosophies, theories, and research to bear when creating designs that target specific content objectives. Curriculum developers practice when they create curriculum, and of course teachers practice when they deliver content to students.

Sections of the Framework: Descriptions and Rationale Building on the core concepts of philosophy, theory, research, and practice, the LCD-specific framework is presented, section by section. All sections build on the four core concepts of philosophy, theory, research, and practice. The framework also expands on those ideas and demonstrates interrelationships. The authors’ goal is to create a framework that can be used across a wide group of stakeholders to align LCD philosophy, theory, and research and practice.

LCD Philosophy How exactly do people come to learn new things? Some argue for positivism, meaning there are facts in the world independent of people’s individual perspectives

48

C. Zintgraff and A. Hirumi

(positivist ontology), and people learn by getting those facts transferred into their mental schemas (positivist epistemology) (Driscoll, 1994). Positivism does not consider the importance of the prior knowledge people build on when they learn, since facts are the same for all people. Others argue for constructivism. In constructivist (also known as interpretist) ontology, all facts one perceives are interpreted through the lens of their existing knowledge. All coming to know (learning) occurs by building on prior knowledge (aka. constructivist or interpretist epistemology) (Driscoll, 1994; Reigeluth & CarrChellman, 2009b). Constructivist philosophy (ontology plus epistemology) is viewed as the foundation of learner-centered design (Reigeluth & Carr-Chellman, 2009b; Reigeluth, Myers, et al., 2017). Constructivism posits that people learn by constructing new knowledge that builds on their existing knowledge. Ertmer and Newby (1993) viewed constructivism as the synthesis of Plato’s rationalism and Aristotle’s empiricism (Ertmer & Newby, 1993). Through empiricism (experience), people are exposed to new things, and through rationalism (mind, thought), people construct new knowledge driven by those experiences. Peirce (1878) and Dewey (1900, 1997) advanced pragmatist ideas consistent with constructivist philosophy. The iteration between thinking and doing is one of the key ways people learn. If learners create their own new knowledge, then what two people learn from the same instruction will be in some respects different. Evaluating such learning is more difficult than in behaviorist or cognitivist paradigms, which consider particular knowledge as being transferred from teacher to student (Ertmer & Newby, 1993). This challenge raises questions about how researchers should properly gather and interpret evidence regarding LCD approaches. In other words, constructivist philosophy also raises questions about research philosophy. Guba and Lincoln’s (1994) research paradigms – positivism, post-positivism, critical theory, and constructivism (the research paradigm, not the learning theory) – provide flexibility in assessing the outcomes of LCD instruction. However, the value of different paradigms is a point of debate among researchers with differing views of LCD (e.g., Hmelo-Silver et al., 2007; Kirschner et al., 2006). An additional philosophical concern can be seen in the evolution of the Green Books. Reigeluth, Beatty, et al. (2017) described the motivation for Volume IV as a need to expand the scope of conversation along multiple dimensions. They described the need for integrated discussions across curriculum, assessment, and instruction. They stated that “decisions about what to teach, how to teach it, and how to assess it. . .should be made together because they are interdependent” (Why a Volume IV, para. 3). Invoking systems thinking, they noted that fully successful learner-centered instruction requires changes not only in design and development of curriculum, but also in school systems at the level of the school, district, and state. They declared the reform of such systems as a priority task to bring effective LCD to students. This theme is adopted, and it is advanced by incorporating the idea of systems (Reigeluth, Beatty, & Myers 2017) in this chapter’s approach. This chapter intentionally considers LCD in the context of school organizations and systems, including

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

49

Table 2 Elements of LCD philosophy Element Constructivism

Summary The core philosophy of how people learn

Research philosophy

Agreements, and debates, over proper approaches to research

Systems

For practical alignment, LCD must be viewed in context of school system

Philosophy as values

Shared philosophy (values) facilitate alignment; competing values hurt alignment

Authors Ertmer and Newby (1993) Peirce (1878) Dewey (1900, 1997) Kirschner et al. (2006) Hmelo-Silver et al. (2007) Guba and Lincoln (1994) Reigeluth, Myers, et al. (2017) Cuban (1986, 2001) Duffy (2009) Reigeluth and CarrChellman (2009b)

their practical realities, arguing that any attempt at alignment that omits such concerns will not have practical application. Cuban’s (2001) review of many decades of educational reform reinforces this perspective. To fully understand LCD philosophy, it is useful to see the overlap between philosophy and values. Reigeluth and Carr-Chellman (2009a) wrote that “the complete set of values underlying a theory of instruction represents a philosophy of instruction” (2.1. Values, para. 2). They highlighted the importance of these values being made explicit for all stakeholders, thereby reinforcing the systems nature of LCD and the importance of alignment across a broad group of stakeholders. Table 2 summarizes elements that, taken together, form the philosophical view of LCD adopted in this chapter.

LCD Foundation: Descriptive Learning Theories and Established Research It was previously noted that some theories are more established than others, and that LCD stakeholders often reference research results as a way to justify their choices in design, development, and delivery of learning experiences. Therefore, established theory and established research are combined in one section, labeled as the LCD foundation, from which LCD stakeholders make decisions about instruction. This section considers only descriptive learning theories, rather than prescriptive design and instructional theories. Descriptive learning theories and established research combine to inform stakeholders’ beliefs about how and why people learn. Reigeluth and Carr-Chellman (2009b) contrasted descriptive learning theories with prescriptive design and instructional theories. Design theories are human-created constructs that prescribe recommended approaches to teaching in specific situations. In other words, design theories prescribe instructional strategies, methods, and models (e.g., project-based learning). The placement of design theories in the

50

C. Zintgraff and A. Hirumi

Table 3 LCD: Established descriptive theories Theory Zone of proximal development Scaffolding Social constructivism Situated cognition Situated learning Cognitive apprenticeship Constructionism

Summary Novices expand their knowledge assisted by experts Learners best advance when helped by more knowledgeable others People create their own knowledge inspired by social interactions Cognition and context cannot be separated The mentorship of leaders enables growth of new members in groups Tasks tacit to experts must be made explicit to novices Learner’s ideas are constructed through sharing visible artifacts

Authors Vygotsky (1978) Van de Pol, Volman, and Beishuizen (2010) Vygotsky (1978) Brown et al. (1989) Brown et al. (1989) Collins et al. (1988) Papert and Harel (1991) Melchior, Burack, Hoover, and Marcus (2016)

framework is described in a subsequent section. Table 3 lists established descriptive learning theories and research related to LCD. Scaffolding and the Zone of Proximal Development The work of Vygotsky (1978) is core to the development of constructivist learning theory, especially Vygotsky’s seminal theory of the Zone of Proximal Development (ZPD). For a learner, ZPD describes three kinds of tasks, those a learner can perform without help, those a learner can perform with help from a more knowledgeable other, and those beyond a learner’s current capabilities even with help. Vygotsky believed that the most effective learning occurs when students attempt tasks near the limit of their ZPD assisted by a more knowledgeable person. Significant exploration and research has built on Vygotsky’s work. Chaiklin (2003) provided substantive overviews of Vygotsky’s original texts. Kozulin, Gindis, Ageyev, and Miller’s (2003) also provided a broad and informative overview of ZPD’s history and relevance. Vygotsky’s work is generally acknowledged as the inspiration for scaffolding (Van de Pol et al., 2010), a teaching approach consistent with ZPD. In their systematic literature review, van de Pol et al. studied scaffolding literature between 1998 and 2009. Despite critiques of the difficulty of measurement, lack of measurement instruments, and that most studies were observational, van de Pol et al. reported that studies generally pointed toward the effectiveness of scaffolding. Van de Pol et al. were especially complementary of the work of Palincsar (1986) and of Palincsar and Brown (1984) in systematically studying scaffolding, studies which also reported the approach to be effective. Van de Pol et al. highlighted three essential elements of scaffolding. Contingency is the teacher’s degree of adaptation to the specific circumstances of learning. Fading is the process of gradually withdrawing assistance from the learner. Transfer of responsibility is the complementary process of the learner assuming ownership for performing the task.

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

51

Social Constructivism Vygotsky’s original work went beyond the one-to-one relationship of learner to teacher. Vygotsky considered learning in groups, and in particular, he explored the process and sequence in which children learn new things. Vygotsky came to believe that social learning – learning in groups – preceded the psychological processes by which children internalized new lessons. Palincsar (1998) wrote about Vygotsky’s and others’ perspectives on social constructivism, including a review of many small studies on group learning that highlighted best practices and positive outcomes, as well as a review of more philosophical perspectives. Palincsar noted the availability of a variety of instruments and approaches to research, with a focus on instruments for dynamic assessment that allow for variety “in terms of the nature of the task, the type of assistance that is provided, and the outcomes that are reported” (p. 367). At a higher level, she wrote that “social constructivist perspectives, which regard schooling as a system rather than as a set of isolated activities, have been extremely useful to understanding and describing the complexities of teaching, learning, and enculturation into schools” (p. 371). Situated Cognition, Situated Learning, and Cognitive Apprenticeship The theory of situated cognition reinforces the premise that learning is influenced by the situation in which it occurs, and that “people’s knowledge is embedded in the activity, context, and culture in which it was learned” (learning-theories.com, 2020b, para. 1). In their seminal article, Brown et al. (1989) cited examples of learning multiplication and learning problem solving to demonstrate how context affects what students learn. Lave and Wenger (1991) developed the similar idea of situated learning. They focused on legitimate peripheral participation, or stated more simply, how new members of a group become accepted members and eventually expert members, but only if embraced by the group’s leaders. Their research showed that the embrace of new members by leaders has strong implications for the new member’s opportunity to grow in the field of interest. Collins et al. (1988) developed the similar idea of cognitive apprenticeship, highlighting the importance of making expert’s tacit knowledge explicit to novices. Jenlink (2013) described instruments and measurements associated with situated cognition, and he described research studies using those instruments. The studies involved science process skills; mathematics learning using situated learning versus a traditional control group; the student process of problem finding; and situated learning in online learning communities. Each study yielded results where situated learning was believed to provide benefits to students. Constructionism Constructionism (Papert & Harel, 1991) is a learning theory that highlights the role of creating artifacts and accepting critique as a method of social learning. When applied in specific curricula, constructionism “make[s] the process of thinking and learning visible and [allows for] a more process-oriented engagement with an idea via construction and deconstruction” (learning-theories.com, 2020a, para. 4). The

52

C. Zintgraff and A. Hirumi

idea of educational robotics as a learning activity was raised by Papert, and its proliferation is an example of constructionist learning. Independent study of longterm impacts of the FIRST Robotics program has reported positive results regarding student interest and pursuit of STEM careers (Melchior et al. 2016).

The Debate Over Cognitive Load In the debate over constructivist learning in Kirschner et al. (2006) and subsequent replies (e.g., Hmelo-Silver et al., 2007), and in Tobias and Duffy’s (2009) volume, cognitive load is a prominent and controversial topic. Cognitive load theory describes how much information a person can process at one time and the resulting implications on teaching and learning (Moos, 2011). Does cognitive load have negative implications for the commonly advanced approaches to LCD? Or do common LCD approaches properly address cognitive load through scaffolding and inclusion of strong guidance? Behaviorists tie cognitive load theory to underlying theories of human cognitive architecture. From that starting point, they argue for minimal context and a focus on direct instruction (Kirschner et al., 2006). Research on cognitive load in context-rich constructivist environments has led to mixed results. Some have argued that minimizing context and using direct instruction techniques are the best ways to honor human limits to tolerate cognitive load – in short, that this approach is the most learner-centered (e.g., Kirschner et al., 2006; Sweller, 1988). Some research indicates that better students benefit most from context, while struggling students often have poor learning outcomes in context-rich environments (Moos, 2011). Others have argued for the benefits of context. Supporting evidence and suggestions on ways to balance cognitive load and context can be seen in Hmelo-Silver et al. (2007) and Schmidt et al. (2007). In particular, Hmelo-Silver et al. (2007) wrote that strong guidance in context-rich environments is common, important, and fundamentally part of LCD. Scaffolding is fundamentally adapted to specific learners and settings and can help address differences in student capabilities. Through scaffolding and strong guidance, the learner’s limits in holding rich context are addressed in the LCD paradigm.

LCD Strategies and Tactics To be applied, philosophy, theory, and research must be made more concrete. In the framework, this next step toward practical application is called strategies and tactics. These fairly accessible terms are selected to represent what theorists describe using more finely detailed definitions. Strategies are human-created design theories for how to develop instruction (Reigeluth et al., 2009). LCD strategies are based on the LCD foundation. LCD strategies, when applied to develop specific curricula, lead to a learner-centered experience that is predesigned and generally lasts a classroom period or more. Tactics lean on the same LCD foundation but are individual actions a teacher executes in a classroom under their professional judgment, actions that last just a few minutes. For example, a teacher might invent a problem on-the-spot to reinforce learning for a particular student.

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

53

While more concrete, strategies and tactics remain abstract relative to specific learning objectives or to any particular content objectives. Selection of a strategy is one element of curriculum design. For example, one might select project-based learning as a strategy to be applied. Likewise, selection of a tactic happens either through teacher planning, or ad hoc in the classroom, for all types of learning objectives and content. For example, a teacher might deploy reflection as a way to test student understanding in the classroom.

Strategies Readers familiar with academic literature may recognize this definition of strategy as overlapping other terms used in the literature. Reigeluth and Carr-Chellman (2009b) listed numerous terms they observed in literature including method, model, technique, and strategy. They argued for adoption of the term design theory as a description of how to design a good learning experience and when the theory itself should be used. Reigeluth and Carr-Chellman (2009b) further defined six kinds of instructional design theory, one related to analysis; another to planning, another to preparing for implementation; another regarding the nature of instructional events and the context for their delivery; and others. To clarify their explanation, they offered “a good analogy. . .the building process that results in homes, offices, skyscrapers, hospitals, and other buildings” (An Analogy, para. 1). They further wrote: “While it is conceptually helpful to understand that all these different kinds of instructional design-theory exist, it is essential to understand that useful guidance for practitioners must integrate all of them” (Instructional Design Theory, para. 11). Using the LCD framework, a broad range of LCD stakeholders can visualize relationships in a single and accessible illustration. Meanwhile, strategy is closest to instructional-event design theory, “theories about. . .the buildings. . .the products. . .[and their] use. . .in the form of everyday structures” (An Analogy, para. 1). Table 4 lists a number of strategies commonly associated with LCD. Among the strategies included as examples in the framework are discussion-based learning, case-based learning, inquiry learning, experiential learning, problem-based learning, and project-based learning. The 5E instructional model is a particularly detailed definition of a strategy. All these strategies share common underlying constructivist learning principles (Markham et al., 2003; Prince & Felder, 2006). The reader will note particular elements in common among these strategies. They demonstrate the LCD principle that children, adolescents, and adults learn best when presented with authentic, real-world challenges. They better accomplish the acquisition of facts and rules. The development of skills and dispositions occurs within the context of addressing such challenges. Also, students better take responsibility for their learning while still receiving guidance from teachers as indicated in curriculum. Within these strategies, students often create literal or figurative artifacts to facilitate and demonstrate learning. The artifacts can be, for example, the content of their spoken statements in discussions, or the concrete artifacts created in projects, all subject to peer critique.

54

C. Zintgraff and A. Hirumi

Table 4 LCD: Strategies (design theories) Strategy/design theory Discussion-based Case-based

Inquiry-based Experiential learning

Problem-based learning

Project-based learning Maker-based instruction

5E instructional model

InterPLAY

WebQuests

Student-Centered, Technology Rich, Learning Environments (SCenTRLE) Historical inquiry

Summary Discussion to drive active student learning, led by instructors Students analyze case studies, solving problems, and/or making decisions Students take the lead in investigating a topic Instruction based on authentic tasks and real-life (or life like) experiences, self-directed, and exposes students to feedback Learning through solving problems; students learn both content and thinking strategies Learning through development of projects and creating artifacts Transdisciplinary approach making artifacts, integrating digital, analog and art, driven by inquiry Learners are engaged, explore content, and explain what they learned, followed by instructorfacilitated elaborations and evaluations Elements of story, play, and game are integrated with experiential learning principles to create engaging and memorable experiences

Authors Gibson (2009) Prince and Felder (2006)

Prince and Felder (2006) Lindsey and Berger (2009)

Savery (2015) and Barrows (1985) Markham et al. (2003) McKay and Glazewski (2017)

Bybee (2002)

Hirumi et al. (2017; ▶ Chap. 42, “NERVE, InterPLAY, and Design-Based Research: Advancing Experiential Learning and the Design of Virtual Patient Simulation”) Dodge (1998)

Learners are presented with challenges and are tasked with exploring online resources to overcome the challenges Students work with the instructor Hirumi (2002) to negotiate learning objectives, strategies, and assessments based on students’ needs and interest Learners engage in primary and Waring (2011) secondary sources to create plausible narratives to answer fundamental questions

Tactics Not all learner-centered experiences are the result of rigorous development and delivery of curriculum by a system of designers, developers, and teachers. Some LCD experiences are the result of teacher planning or improvisation, with teachers applying

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

55

Table 5 Improvisational tactics in Sawyer (2011) Tactic Authentic tasks Collaborative work Peer example Reflection Feedback Posing problems

Summary Approximating an authentic problem Assigning group work A teacher arranges for a student to learn by watching another student Driving students to reflect on work (a form of inquiry) Providing students critique Posing in-class problems

Authors Barker and Borko (2011) DeZutter (2011) Beghetto and Kaufman (2011) Burnard (2011) Beghetto and Kaufman (2011) Martin and Towers (2011)

professional judgment and bringing ad hoc learner-centered tactics into their classrooms (Cuban, 1986; Sawyer, 2011). Teachers can do this whether or not their school, district, state policies, etc., are amenable to at-scale LCD instruction. Table 5 contains improvisational tactics found in Sawyer’s (2011) book on improvisation in teaching. In light of this chapter’s audience and the arguments made about the important role of teachers in the LCD system, the authors adopt the term tactic to describe what teachers do when they bring learner-centered instruction into their classrooms, either through their own planning or on an ad hoc basis. The term contrasts with the term strategy, the latter describing a rigorous approach to designing units of instruction. The power and inevitability of teachers applying tactics, under their professional judgment, was described by Cuban (1986). As an example, he described how a chalkboard provides a powerful affordance to a teacher. Chalkboards are flexible and reliable tools for enriching students’ learning experience that teachers can use at a moment’s notice. Cuban contrasted chalkboards with film, reminding readers of Thomas Edison’s infamous statement that film would replace books in schools. In practice, use of film requires advance planning to secure a projector, secure the film desired, and create a backup plan should the projector fail, a common occurrence. In the end, film was not a replacement for books in the classroom setting. It was an inflexible tool for teachers, whereas books and chalkboards were highly flexible and did a better job of putting professional judgment in the hands of teachers. In recognizing teachers’ ad hoc ability to deploy tactics, the framework is made more powerful. LCD experiences do not fully depend on designers, curriculum developers, administrators, standards set by policymakers, and/or the design of schools. Individual teachers can bring some manner of LCD experiences to students regardless of context. Table 5 contains examples of tactics a teacher can deploy in their classrooms.

LCD Practice Practice as the Targeting of Content Objectives The Lexico (Oxford) dictionary defines practice as “the actual application or use of an idea, belief, or method, as opposed to theories relating to it” (Lexico, 2019, Noun,

56

C. Zintgraff and A. Hirumi

Table 6 Practice as targeting of content objectives Stakeholder Instructional designer Curriculum developer Classroom teacher

Example of practice Applying philosophies, theories, and research to create a design that teaches to specific content objectives, within the context of classrooms and school systems Considering philosophies, theories, and research as an instructional design is translated into curriculum, within the context of classrooms and school systems Considering philosophies, theories, and research as a curriculum is delivered to students, within the context of their own classroom and school systems

para. 1). Some might consider educational practice to be only what happens in a classroom. However, this fails to meet the need of the current chapter, as between theory and classroom practice, one must explicitly or implicitly consider descriptive theories and apply design theories (strategies) to create instructional designs and develop curriculum. Table 6 provides a summary of what practice looks like for instructional designers, curriculum developers, and classroom teachers. The common theme among those activities is targeting of specific content objectives. Therefore, practice is defined as the application of philosophies, theories, research, and strategies by instructional designers, curriculum developers, and classroom teachers to target specific content objectives. It is when specific content objectives are introduced that abstractions begin to prove themselves true or false. Internal Consistency in Instructional Design and Curriculum Development Conversations about alignment of philosophy, theory, research, and practice are academic until one reaches the point of targeting specific learning objectives. Stated simply, when a topic to teach is chosen, the discussion becomes real. Generalities about alignment must then take form in the curriculum design and development process. Within that process, internal consistency is needed, and it is achieved through internal alignment. An analogy can be drawn to the idea of internal consistency in research designs. Choices of philosophy, theory, and a topic to research can be well chosen, but if the internal design of a study is not self-consistent, then research activities will not be productive and results not valid. Likewise, the best conceived instruction will fail if the artifacts of practice (designs, curriculum, and activities/material during classroom delivery) are not internally consistent. Dick et al. (2015) described the essential steps of any instructional design process. It is the alignment of these steps and their outcomes that drive internal consistency. The steps are: (a) identification of learning objectives; (b) identification of appropriate assessments; and (c) selection of an appropriate instructional strategy, like the LCD strategies described above. Dick et al. (2015) described significant additional details about how this process unfolds. They recommended starting with a needs assessment to ensure training or education is an appropriate response to the situation at hand (e.g., that training is needed to solve a problem in a professional setting). Designers must come to

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

57

understand required performance goals, the nature of the learners, and the context of the situation. If goals have already been specified by accrediting bodies, professional organizations, the state, the district, or other school governing bodies, those goals should be identified. Specific objectives should be defined, and there should be assessments for each objective. Media appropriate to the situation should be selected. Dick et al. described various analysis techniques at length that are useful to instructional designers. With objectives aligned to prescribed standards and/or performance outcomes, Dick et al. (2015) recommend selecting appropriate learner assessments to monitor students’ progress and determine if they can demonstrate achievement of the objectives. The use of conventional multiple-choice, true/false, and fill-in-the-blank assessment are to be considered along with the use of performance or product checklists, or analytic or holistic assessment rubrics. Assessment items and criteria are specified and aligned to each objective to ensure learning has occurred for each objective. For example, if an objective is for students to interpret historical events, then assessment should involve students making historical interpretations. Though described as linear, the design process is almost always iterative in practice. Rethinking, reviews, and increasing understanding of the project and context all cause iteration. Internal consistency exists when, within a specific curriculum development project, there is alignment among learning objectives, student assessments, and the instructional strategy selected. The principles of Design-Based Research (DBR) (Barab & Squire, 2004) formally exemplifies a less linear approach. With DBR in education settings, research, design, and curriculum development occur in naturalistic context, with full exposure to the resources and limits of the instructional setting. Objectives and assessments might be adjusted during the process to better align with the local setting. For example, in a school deeply committed to PBL methods, PBL instructional strategy may be all but required, with objectives and assessment adjusted accordingly. Such a case is, in fact, an example of bottom-up alignment driven from the realities of the classroom setting. The philosophical moorings of a particular environment will indicate the extent to which linear or nonlinear approaches are effective. Regardless, the LCD framework is defined such that objectives, assessment, and instructional strategy must be aligned. Failure to align represents failed internal consistency. Internal consistency is fundamental to optimizing learning, and the alignment of learning objectives, learner assessments, and instructional strategies is the cornerstone of internal consistency.

Fidelity in Delivery and Teachers as Cocreators Finally, education experiences are delivered to students in physical and online experiences frequently but not always led by teachers. During delivery, the intersection of top-down and bottom-up concerns intersect in practical and unavoidable ways. In any case, internal consistency between objectives, strategy, and assessments must be maintained.

58

C. Zintgraff and A. Hirumi

Top-down, there is ample evidence that fidelity to design and curriculum is problematic. Spector (2014) argued that effective evaluation of programs must include robust review of whether programs were executed in accordance with design. Hannafin et al. (1997) also give a reminder of the value of grounding designers of research and theory. The aforementioned prevalence of teacher professional development and the models used to help teachers implement new approaches and curriculum (Hall & Hord, 1987; Mishra & Koehler, 2006; Taylor et al., 1994) indicate that teacher fidelity in intended program designs is a constant concern. Bottom-up, a view of teachers as professionals demands that their own views on education philosophy, theory, research, and practice must be honored (Cuban, 1986; Ertmer & Ottenbreit-Leftwich 2010; Harris et al., 2009), and that no stakeholder controls all factors of the class setting, including readiness of students, time available, structure of class days, prevalent philosophies in their organizations, etc. Cuban made clear that teachers co-create the experience in classrooms whether they are invited to do so or not. Well-considered bottom-up alignment provides room for teachers to make that contribution during delivery.

Ongoing Research and Theory Development The framework illustrates the ongoing research and theory development that happens in the LCD field. The placement in the framework intentionally overlaps philosophy, theory, research, and practice. The varying color shades represent the dynamic nature of ongoing research efforts, efforts that work to connect philosophy to theory, theory to practice, and philosophy to practice, both top-down and bottomup. In highlighting ongoing research and theory development, the authors reemphasize the idea that ongoing work differs categorically from established theory and research. Of course, well-intended researchers will disagree about which theories and research results are well-established. Such differences do not undermine the usefulness of the framework as a representation of what actually happens in the LCD process. Table 7 contains a list of established research results and ongoing research questions as viewed by the current authors. Framework Summary Rationale for Choosing Terms for the Framework There are many different criteria one might choose for selecting terms, each rational, but often in conflict with one another in specific circumstances. One criteria applied was to create as few new terms as possible, implying selection of terms already used in the literature. Another goal was to choose terms that are not so general that a reader could interpret them in many ways. Another goal was to choose terms that a diverse group of stakeholders will naturally understand. Another goal was to use the shortest terms possible. Unavoidably, multiple competing terms were found in the literature for the same construct (e.g., see discussion of design theory in Reigeluth & Carr-Chellman, 2009b, and their own mention of the terms strategy and technique). Final decisions were a judgment call by the authors.

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

59

Table 7 Established and ongoing research in LCD Category Established research

Examples Social constructivist learning theories

Ongoing research (debates in the field)

Guided inquiry and related techniques Role of cognitive load in LCD instruction Differences in philosophy and values Differences in measurement Different conclusions about implications

Questions raised

Supporting Citations Sweller (1988) Vygotsky (1978) Palincsar and Brown (1984) Brown et al. (1989) Collins et al. (1988) Kirschner et al. (2006) Hmelo-Silver et al. (2007) Tobias and Duffy (2009) Tobias and Duffy (2009) and Duffy (2009)

Summary of Framework Terms Table 8 contains a summary of the terms selected and is a comprehensive list of the elements of the framework.

Applying the Framework Rationale Statistician George Box (1976) is credited with the idea that that all models are wrong but some are useful. In the attempt to provide a valuable framework for LCD, the authors acknowledge it must contain imperfections, especially as it is considered within specific circumstances. Two basic ideas are offered about how the LCD framework might be applied in a useful fashion. One idea is for a person or group of collaborators to personalize the framework to reflect their own beliefs about learning theory, research, and practice, and apply the individualized framework to align their own design tasks. A second idea is to develop a tool for assessing an LCD artifact, one separate from the framework but incorporating its principles. These ideas are described below and form the basis for future research recommendations.

Personalizing the Framework For a Person No doubt readers will have noted differences between details of the framework and their own beliefs. For example, the reader may wish to delete or add philosophies,

60

C. Zintgraff and A. Hirumi

Table 8 Constructs and other elements in the LCD framework Term Philosophy Established theory Established research Developing theory Ongoing research Strategy

Tactic Practice Content objectives Prescribed curriculum design Objectives Instructional strategy and resources Assessment (plan/tool)

Classroom Classroom design and delivery Assessment (outcomes)

Description The core philosophies and values that underlie LCD Learning theories established through rigorous research, and especially through empirical research Widely accepted research results Theories that are proposed but not yet widely accepted Research that is ongoing or with results not yet widely accepted A formal and rigorously described approach for developing an instructional design (a design theory per Reigeluth & Carr-Chellman, 2009a) A description of a learner-centered approach a teacher can apply in a classroom, either by choice during teacher planning or ad hoc What instructional designers, curriculum developers, and classroom teachers do when pursuing content goals Specific content targeted by instructional designers, curriculum developers, and classroom teachers Plans teachers can follow in their classroom targeting specific content goals The content objectives as expressed in a prescribed curriculum design The description of the intended classroom experience within a curriculum design, and associated resources The plan and/or tool/s for assessing whether students achieved the objectives set out for them, as described in a prescribed curriculum design The setting for instruction The act of planning and delivering instruction to students, in the setting of delivery, using the prescribed design, targeting content goals The student grades and program evaluations that result from delivery

delete or add theories, exclude or augment certain research results, etc. Readers may also note structural changes that would make the framework more meaningful and useful for their own interests. Whatever an individual’s needs may be, the basic framework identifies and depicts the relationship between philosophical beliefs, theories, and research on human learning and instruction, and educational and instructional design practices that should be aligned to advance teaching and learning. A useful exercise for any LCD professional would be to modify the proposed framework to suit their particular beliefs. Professors and teacher-educators might use the framework as a driver for students to think deeply about their own beliefs and establish their own foundation for teaching and learning. To support this option, and consistent with the capabilities available in this online volume, the framework source file is available. It is in Microsoft PowerPoint format at www.zlearnlab.com/lcd-

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

61

framework. The reader is encouraged to share modified versions with the corresponding author of this work, and at the reader’s option, to give permission for that modified version to be shared in turn.

For a Team The process of customizing the framework could be a powerful team-building exercise that facilitates communications and advances research and development among team members. The process may help explicate each member’s prior knowledge, dispositions, and potential biases regarding LCD which, in turn, would enable the team to leverage individual strengths as well as fill-in gaps in individual knowledge when necessary. The process may also help facilitate communications and teamwork by creating a common vision and understanding of LCD principles, concepts, and terminology. Ultimately, the framework may be used to bring team members’ effort into alignment to facilitate and advance LCD theory, research, and practice.

Preliminary Assessment Tool Description In addition, the framework may be used as a tool for assessing the alignment of educational artifacts with LCD philosophy, theory, research, and practice. For example, one might assess the design of a particular curriculum against defined LCD principles. One might assess a particular curriculum, or the delivery of that curriculum. Table 9 contains a preliminary assessment tool based on the LCD framework. The table identifies seven principles established in the LCD framework, gives examples or additional details about the principle, and provides a simple rubric-based measurement method for assessing the fidelity of each artifact as high, medium, or low for each principle. High is always aimed toward what would be most aligned with LCD principles. The principles provided are: value aligned, theoretically aligned, proven effective (research aligned), structurally aligned, practical to access, and system aligned. Table 9 provides additional definition of each principle and a rubric for determining a high, medium, or low rating for each principle. Use in Traditional Versus LCD Settings Two alternative rating approaches are provided for structural alignment principle. The rationale for alternative ratings is rooted in the difference between a strategy and a tactic. Applying an LCD strategy leads to an LCD curricular unit. The setting for delivery will require basic structural support for the curriculum to be successful. For example, a curriculum requiring block scheduling will not succeed during a traditional structured school day. On the other hand, a teacher wishing to deploy an LCD tactic in an otherwise traditionally structured setting will encounter fewer barriers in a short application of LCD ideas in their traditional setting.

62

C. Zintgraff and A. Hirumi

Table 9 Preliminary LCD artifact assessment tool Principle Philosophically aligned Philosophies consistent with LCD principles Philosophy Non-evidence-based values

Theoretically aligned Instructional events and activities consistent with LCD principles LCD theories of learning

Empirically aligned (proven effectiveness) Instructional events and activities consistent with empirical LCD research findings. Evidence-based learning experiences Existence of research results Rigor of research results Variety of research results

Structurally aligned to traditional classroom Class physical setting Class time blocking Class duration Team teaching Student teams versus individual

Structurally aligned to LC classroom Class physical setting Class time blocking Class duration Team teaching Student teams versus individual

Citations and Rubric Reigeluth and Carr-Chellman (2009a), Reigeluth, Beatty, et al. (2017), and Hall and Hord (1987) Low: Behaviorist, teacher-directed model without considering individual learner state Medium: At least considers state of individual learners High: Strong consideration of individual learners; power of decision shared with students and local educators Ertmer and Newby (1993), Markham et al. (2003), Prince and Felder (2006), Reigeluth and Carr-Chellman (2009a), and Reigeluth, Beatty, et al. (2017) Low: Behaviorist, broadcast models Medium: Batch methods applied to evoke and build on individual learner’s state High: An individual experience builds on each learner’s state; individual meaning is valued and counts toward assessment Reigeluth and Carr-Chellman (2009a), Reigeluth, Beatty, et al. (2017), Hmelo-Silver et al. (2007), and Kirschner et al. (2006) Low: No rigorous, evidence-based claims Medium: Claims are via anecdotal or via less rigorous research methods; moderate volume of studies High: Research results are rigorous, positive, peer reviewed, and widely accepted Hall and Hord (1987), Cuban (1986), Mishra and Koehler (2006), Reigeluth and CarrChellman (2009a), and Reigeluth, Beatty, et al. (2017) Low: Significant barriers in traditional setting Medium: Moderate barriers in traditional setting High: Low barriers in traditional setting Hall and Hord (1987), Cuban (1986), Mishra and Koehler (2006), Reigeluth and CarrChellman (2009a), and Reigeluth, Beatty, et al. (2017) Low: Significant barriers in learner-centered setting Medium: Moderate barriers in learnercentered setting High: Low barriers in learner-centered setting (continued)

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

63

Table 9 (continued) Principle Practical to access Advance preparation required Time scope Materials required Integration required External (e.g.,) community support required

Internally consistent Objectives aligned to performance outcomes Assessments aligned to objectives Instructional strategy (including contents and use of media) aligned to objectives and assessments

System aligned Standards requirements Political mandates Meta-trends Available funding

Citations and Rubric So that teachers can select and deploy whenever they deem it useful in their professional judgment (Cuban, 2001; Hall & Hord, 1987; Harris, 2005; Mishra & Koehler, 2006; Reigeluth & Carr-Chellman, 2009a; Reigeluth, Beatty, et al., 2017) Low: Significant preparation required Medium: Some advance preparation required High: Can be deployed on the spot Dick et al. (2015) and Hirumi (2014a, 2014b, 2014c) Low: Unmeasurable objectives, unclear relationship between objectives, assessments misaligned to objectives, instructional strategy inconsistent with objectives and/or assessments Medium: Assessments aligned to clear and measurable objectives, or instructional strategy consistent with and supports achievement of objectives High: Assessments aligned to clear and measurable objectives, and instructional strategy consistent with and supports achievement of objectives Cuban (1986), Reigeluth and Carr-Chellman (2009a), Reigeluth, Beatty, et al. (2017), and Markham et al. (2003) Low: Does not satisfy standards or benefit from trends Medium: Some benefit to standards or from trends High: Strong motivators from meeting standards or from societal trends

Following Table 9, additional tables demonstrate example ratings for two types of artifacts, one for aligning the tactic of coaching, and the other for aligning the strategy of project-based learning.

LCD Artifact Assessment Tool Examples Tactic: Coaching For this example, coaching is viewed as an instructional tactic, and is defined as a teacher giving one-on-one assistance to a student, or customized assistance to a small team, during formal instruction time. As such, coaching must necessarily take into account the state of the learners to effectively coach learners forward. The coaching

64

C. Zintgraff and A. Hirumi

Table 10 Hypothetical attributes of the coaching tactic Principle Philosophically aligned Theoretically aligned

Rating High High

Empirically aligned

High

Structurally aligned to traditional classroom Structurally aligned to LC classroom Practical to access Internally consistent

Medium

System aligned

High

High High High

Rating rationale Individual attention to students is highly valued in LCD Coaching requires understanding what a student already knows, and delivering customized help Significant research shows the value of coaching (a type of scaffolding) to student learning (Hmelo-Silver et al., 2007) Content, volume, and time constraints may prevent use at scale LC classrooms are organized intentionally to facilitate coaching Teacher simply stops to work with student or team Coaching may greatly facilitate alignments between objectives, assessments, and instructional strategy, clarifying expectations and requirements if/when necessary Coaching improves understanding of traditional content, impacting standardized tests, and the deep understanding associated with LCD

tactic can be used in traditional and novel educational settings. Coaching might have the attributes listed in Table 10. Readers might want to attempt rating the coaching tactic in the settings they operate in, and/or readers might identify other LCD tactics and rate for their individual settings.

Strategy: Project-Based Learning Markham et al. (2003) write about the Buck Institute for Education’s strategy for PBL, among the large volume of literature describing how to deliver PBL. The Buck Institute’s 6 As – authenticity, academic rigor, applied learning, active exploration, adult connections, and assessment practices – are adopted as the characteristics of PBL. PBL might then have the attributes listed in Table 11. Remember then low ratings are not value statements. They simply reflect the characteristics of the method. The ratings might be different if done in the narrower context of a school already prepared for constructivist teaching methods at scale. Readers might use this example as inspiration to rate PBL in their school settings. One might scope the setting as being the whole school, a particular department, a particular class, or toward a particular subject. Low-rated items might be used to identify areas that need work if PBL is to be introduced. Ratings might be used to either argue for or against PBL as an appropriate strategy to bring to the setting in question. Suggestions for Applying the Framework The above examples are intentionally generic, and their purpose is to provide readers a sense for how the LCD Assessment Tool might be used. The tool is not statistically tested, and no claims of causation, correlation, or quantitative rigor are made. The

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

65

Table 11 Hypothetical attributes of the PBL strategy Principle Value-aligned

Rating High

Theoretically aligned

High

Proven effective (research aligned)

Medium

Structurally aligned to traditional classroom Structurally aligned to LC classroom Practical to access

Low

Low

Internally consistent

Variable

System aligned

Medium

High

Rating rationale Authentic experience; especially high if students are given power to help choose project Research and artifact creation exercise mechanisms identified by constructivist theory There is research both in support of the method, and questioning the method. Careful alignment to goals is indicated Multiple barriers exist when adapting a traditional classroom LC classrooms are organized intentionally to facilitate projects Teacher must identify projects, possibly form teams, integrate with teaching goals Based on design of problem and alignment of course, module, or lesson objectives, assessments, and content with PBL strategy Not well suited to high volume of content standards; especially well-suited to deeper learning competencies and deep understanding

Table 12 Steps for applying the LCD assessment tool Step 1. 2. 3. 4.

Description Select an LCD strategy, tactic, artifact, event, or outcome Set the scope for evaluation: Scope might be a particular subject, classroom, teacher, department, school, district, or any combination of the above Perform the assessment using the rubric in Table 9 Optional: Customize the rubric to locally adopted philosophies, theories, and research, and then perform the assessment

assessment tool is, however, a platform for additional research, and also a tool LCD stakeholders can use to give structure to their thinking about LCD in their work. Table 12 contains the simple steps an LCD stakeholder can follow to apply the assessment tool in their setting.

Conclusion Summary The authors have argued that there are problems in the alignment of philosophy, theory, research, and practice in learner-centered design. The problems were summarized as misunderstanding over the plethora of terms and constructs, lack of top-down and bottom-up alignment, absence of internal consistency, and disagreements over evidence and interpretation. These problems limit the effectiveness,

66

C. Zintgraff and A. Hirumi

generalizability, and continuous improvement of individual LCD designs and discourage greater adoption. In response, a framework was proposed for aligning LCD philosophy, theory, research, and practice. The goal was to create a framework that is succinct, thorough, useful, and accessible to all LCD stakeholders, from philosophers to classroom teachers. The framework was presented along with an assessment tool to facilitate the application of the framework. The framework was developed based on a survey of selected literature and incorporates constructs, artifacts, ideas, and relationships observed or clearly implied in that survey. Learner-centered design is framed by philosophy and by foundational theories and research. LCD is based primarily on constructivist learning theory. In LCD, systems thinking is needed to stay true to LCD philosophy while simultaneously achieving practical results. LCD leverages the power of learning with others and learning-in-context as motivational, effective, and leading to qualitatively different learning than with a transmission model from teacher to student. LCD embraces ongoing research. It makes room for developing theories that have the potential to improve learning experiences. It makes a place for research results that self-recognize as tentative. Theory development happens in the interaction between different elements of the framework. Philosophy, theory, and research crystallize into strategies and tactics. Strategies are rigorous descriptions of ways to develop learner-centered experiences. Projectbased learning is one example among the commonly applied strategies. Tactics are individual actions a teacher executes in a classroom, either due to teacher preplanning or ad hoc. Even in a traditionally structured classroom a teacher can, for example, invent a problem on the spot to reinforce learning. Practice happens when abstract ideas get directed at content objectives. When an instructional designer puts a strategy into practice, an instructional design is created. When a curriculum developer implements that design, curriculum is created. When a teacher delivers a curriculum or deploys an LCD tactic in a specific setting, that teacher brings LCD into practice in the classroom. Practice always involves the alignment of objectives, strategy, and assessment in the context of the content objective. Views were shared on which philosophies, theories, research results, strategies, tactics, and best practices represent LCD, but no attempt was made to resolve all issues. Rather, a framework was offered and a preliminary assessment tool developed to help the adherents of LCD philosophies improve their own alignment. The authors wish to encourage better individual understanding; better shared understanding among coworkers; and better mutual understanding between theorists, researchers, and practitioners of different persuasions. Readers are encouraged to take the framework, assessment tools, and ideas in this chapter and make them their own.

Future Research The LCD Framework and LCD Assessment Tool provide robust opportunities for future research. In any setting where LCD is being applied, the framework can be

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

67

customized for that setting. Customizing can be done through the judgment calls of an expert or group, or via surveys and quantitative instruments that determine which philosophies, theories, research, etc., are appropriate to inclusion. Those results can be explained, published, and/or measured in ways appropriate to the local setting. The structure of the framework itself can also be further explored. If one considers the current structure to be a hypothesis, evidence for relationships within the structure can be sought. In particular, the authors argue for separate constructs of established research versus ongoing research. Disagreements about which research falls in which category is at the root of any number of debates. Do these constructs, abstract as they are, help colleagues get to the root of debates? Do they help expose misalignments? The authors judge that there is significant promise in using the framework as a team assessment tool. Both qualitative and quantitative approaches can be used to measure team alignment and to drive improved alignment among team members. Alignment may also be determined among team members along separate elements; for example, for LCD designs, curricula, and delivery. Finally, there can be further development of the LCD Alignment Assessment Tool. The tool can be applied in different settings and to different artifacts. Statistical analyses might be performed in different settings in pursuit of a validated tool for specific purposes and settings. Readers are invited to share their customized or team-oriented adaptations with the corresponding author. Alternative versions of the framework are welcome. Other LCD stakeholders are invited to pursue the same shared goal – improved communication, improved alignment, and improved outcomes from learner-centered designs. Acknowledgments The authors acknowledge the role of Dr. Michael Spector and the National Science Foundation-funded Cyberlearning Early Career Workshop-University of North Texas for organizing the activities that led to this publication.

References Abras, C., Maloney-Krichmar, D., & Preece, J. (2004). User-centered design. In W. Bainbridge (Ed.), Encyclopedia of human-computer interaction. Thousand Oaks, CA: Sage. Albion, P., Jamieson-Proctor, R., & Finger, G. (2010, March). Auditing the TPACK competence and confidence of Australian teachers: The teaching with ICT audit survey (TWICTAS). In Society for Information Technology and Teacher Education Conference (SITE), San Diego, California. Alexander, H. (2006). A view from somewhere: Explaining the paradigms of educational research. Journal of Philosophy of Education, 40(2), 205–221. American Psychological Association. (1993). Learner-centered psychological principles (ERIC Document Reproduction Service No. ED371994). ERIC Clearinghouse). Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control processes. Psychology of Learning and Motivation, 2, 89–195. Bandura, A. (1994). Self-efficacy. In V. S. Ramachaudran (Ed.), Encyclopedia of Human Behavior, 4, 71–81. New York: Academic Press.

68

C. Zintgraff and A. Hirumi

Bannon, L., Bardzell, J., & Bødker, S. (2019). Reimagining participatory design. ACM Interactions, 26(1), 27–32. ACM. Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1–14. Barker, L., & Borko, H. (2011). Conclusion: Presence and the art of improvisational teaching. In R. K. Sawyer (Ed.), Structure and improvisation in creative teaching (pp. 279–298). New York, NY: Cambridge University Press. Barrows, H. S. (1985). How to design a problem based curriculum for the preclinical years. New York, NY: Springer Publishing. Bednar, A., Cunningham, D. J., Duffy, T., & Perry, D. (1995). Theory in practice: How do we link? In G. Anglin (Ed.), Instructional technology: Past, present, and future (2nd ed., pp. 100–112). Englewood, CO: Libraries Unlimited. Beghetto, R. A., & Kaufman, J. C. (2011). Teaching for creativity with disciplined improvisation. In R. K. Sawyer (Ed.), Structure and improvisation in creative teaching (pp. 94–109). New York, NY: Cambridge University Press. Bernstein, R. J. (1995). The new constellation: The ethical-political horizons of modernity/ postmodernity. Cambridge, MA: MIT Press. Box, G. E. P. (1976). Science and statistics. Journal of the American Statistical Association, 71, 791–799. Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions. Journal of the Learning Sciences, 2, 141–178. Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42. Bruner, J. S. (1961). The act of discovery. Harvard Educational Review, 31, 21–32. Burnard, P. (2011). Creativity, pedagogic partnerships, and the improvisatory space of teaching. In R. K. Sawyer (Ed.), Structure and improvisation in creative teaching (pp. 51–72). New York, NY: Cambridge University Press. Bybee, R. W. (2002). Scientific inquiry, student learning, and the science curriculum. In R. W. Bybee (Ed.), Learning science and the science of learning (pp. 25–36). Arlington, VA: NSTA Press. Chaiklin, S. (2003). The zone of proximal development in Vygotsky’s analysis of learning and instruction. In Vygotsky’s educational theory in cultural context (Vol. 1, pp. 39–64). New York, NY: Cambridge University Press. Christou, C., Eliophotou-Menon, M., & Philippou, G. (2004). Teachers’ concerns regarding the adoption of a new mathematics curriculum: An application of CBAM. Educational Studies in Mathematics, 57(2), 157–176. Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. Cochrane, T., & Narayan, V. (2017). Design considerations for mobile learning. In C. M. Reigeluth, B. J. Beatty, & R. D. Myers (Eds.), Instructional-design theories and models, Volume IV: The learner-centered paradigm of education. New York, NY: Routledge. Collins, A., Brown, J. S., & Newman, S. E. (1988). Cognitive apprenticeship: Teaching the craft of reading, writing and mathematics. Thinking: The Journal of Philosophy for Children, 8(1), 2–10. Computer Professionals for Social Responsibility. (n.d.). Participatory design. Retrieved November 1, 2020, from http://cpsr.org/issues/pd/ Cross, K. P. (1991, June 12). Every teacher a researcher, every classroom a laboratory. The Chronicle of Higher Education, p. B2. CTGV (Cognition & Technology Group at Vanderbilt). (1992). The Jasper experiment: An exploration of issues in learning and instructional design. Educational Technology Research and Development, 40(1), 65–80. Cuban, L. (1986). Teachers and machines: The classroom use of technology since 1920. New York, NY: Teachers College Press.

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

69

Cuban, L. (2001). Oversold and underused: Reforming schools through technology, 1980–2000. Cambridge, MA: Harvard University. Dewey, J. (1900). Psychology and social practice. The Psychological Review, VII(2), 105–124. Dewey, J. (1997). My pedagogic creed. In The curriculum studies reader (pp. 17–23). New York, NY: Routledge. DeZutter, S. (2011). Professional improvisation and teacher education: Opening the conversation. In R. K. Sawyer (Ed.), Structure and improvisation in creative teaching (pp. 27–50). New York, NY: Cambridge University Press. Dick, W., Carey, L., & Carey, J. O. (2015). The systematic design of instruction (8th ed.). Upper Saddle River, NJ: Pearson. Dodge, B. (1998). The WebQuest page. Retrieved April 3, 2000, from http://edweb.sdsu.edu/ webquest/webquest.html Driscoll, M. P. (1994). Psychology of learning for instruction. New York, NY: Wiley. Duffy, T. M. (2009). Building lines of communication and a research agenda. In S. Tobias & T. M. Duffy (Eds.), Constructivist instruction: Success or failure? New York, NY: Routledge. Erickson, F. (1984). School literacy, reasoning, and civility: An anthropologist’s perspective. Review of Educational Research, 54(4), 525–546. Ertmer, P. A. (2005). Teacher pedagogical beliefs: The final frontier in our quest for technology integration? Educational Technology Research and Development, 53(4), 25–39. Ertmer, P. A., & Newby, T. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6(4), 50–72. Ertmer, P. A., & Newby, T. (1996). Students’ responses and approaches to case-based instruction: The role of reflective self-regulation. American Educational Research Journal, 33(3), 719–752. Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of Research on Technology and Education, 42(3), 255–284. FIRST. (2016). Vision and mission. Retrieved April 24, 2016, from http://www.firstinspires.org/ about/vision-and-mission Francom, G. M. (2017). Principles for task-centered instruction. In C. M. Reigeluth, B. J. Beatty, & R. D. Myers (Eds.), Instructional-design theories and models, Volume IV: The learner-centered paradigm of education. New York, NY: Routledge. Gibbons, A. S., McConkie, M., Seo, K. K., & Wiley, D. A. (2009). Simulation approach to instruction. In C. M. Reigeluth & A. A. Carr-Chellman (Eds.), Instructional-design theories and models, Volume III: Building a common knowledge base. New York, NY: Routledge. Gibson, J. T. (2009). Discussion approach to instruction. In C. M. Reigeluth & A. A. Carr-Chellman (Eds.), Instructional-design theories and models, Volume III: Building a common knowledge base (Vol. 3) [Kindle DX Version]. New York, NY: Routledge. Grayling, A. C. (1998). Philosophy 1: A guide through the subject. Oxford, UK: Oxford University Press. Guba, E. G., & Lincoln, Y. S. (1994). Competing paradigms in qualitative research. In Handbook of qualitative research (Vol. 2, No. 163–194, p. 105). Thousand Oaks, CA: Sage. Gustafson, K. L., & Branch, R. M. (2002). What is instructional design? In R. A. Reiser & J. V. Dempsey (Eds.), Trends and issues in instructional design and technology (Vol. 1, pp. 16–25). Upper Saddle River, NJ: Prentice-Hall. Hall, G. E., & Hord, S. M. (Eds.). (1987). Change in schools: Facilitating the process. New York, NY: SUNY Press. Hall, G. E., & Hord, S. M. (2006). Implementing change: Patterns, principles, and potholes. Boston, MA: Pearson. Hannafin, M. J., Hannafin, K. M., Land, S., & Oliver, K. (1997). Grounded practice in the design of learning systems. Educational Technology Research and Development, 45(3), 101–117. Harris, K. S. (2005). Teachers’ perceptions of modular technology education laboratories. Journal of Industrial Teacher Education, 42(4), 52–71.

70

C. Zintgraff and A. Hirumi

Harris, J., Mishra, P., & Koehler, M. (2009). Teachers’ technological pedagogical content knowledge and learning activity types: Curriculum-based technology integration reframed. Journal of Research on Technology in Education, 41(4), 393–416. Hirumi, A. (2002). Student-centered, technology-rich, learning environments (SCenTRLE): Operationalizing constructivist approaches to teaching and learning. Journal for Technology and Teacher Education, 10(4), 497–537. Hirumi, A. (Ed.). (2014a). Grounded designs for online and hybrid learning: Practical guidelines for educators and instructional designers. Book I – Design fundamentals. Eugene, WA: International Society for Technology in Education. Hirumi, A. (Ed.). (2014b). Grounded designs for online and hybrid learning: Practical guidelines for educators and instructional designers. Book II – Designs in action. Eugene, WA: International Society for Technology in Education. Hirumi, A. (Ed.). (2014c). Grounded designs for online and hybrid learning: Practical guidelines for educators and instructional designers. Book III – Trends and technology. Eugene, WA: International Society for Technology in Education. Hirumi, A., Johnson, K., Kleinsmith, A., Reyes, R., Rivera-Gutierrez, D., Kubovec, S., . . . Cendan, J. (2017). Advancing virtual patient simulations and experiential learning with InterPLAY: Examining how theory informs design and design informs theory. Journal of Applied Instructional Design, 6(1), 49–65. https://doi.org/10.28990/jaid2017.061005. Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42, 99–107. Holmes Group. (1990). Tomorrow’s schools: Principles for the design of professional development schools. East Lansing, MI: The H.G. Inc. Hoy, W. K., & Adams, C. M. (2015). Quantitative research in education: A primer. Los Angeles, CA: Sage. Interaction Design Foundation. (n.d.). User-centered design. Retrieved November 1, 2020, from https://www.interaction-design.org/literature/topics/user-centered-design#:~:text¼User% 2Dcentered%20design%20(UCD),and%20accessible%20products%20for%20them Jenlink, P. M. (2013). Situated cognition theory. In B. Irby, G. H. Brown, R. Lara-Alecio, & S. A. Jackson (Eds.), The handbook of educational theories (pp. 185–198). Charlotte, NC: Information Age Publishing. Jonassen, D. (1999). Designing constructivist learning environments. In Instructional design theories and models: A new paradigm of instructional theory (Vol. 2, pp. 215–239). Erlbaum. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86. Kolb, D. A. (1984). Experiential learning: Experience as a source of learning and development. Englewood Cliffs, NJ: Prentice-Hall. Kozulin, A., Gindis, B., Ageyev, V. S., & Miller, S. M. (2003). Vygotsky’s educational theory in cultural context. New York, NY: Cambridge University Press. Kuniavsky, M. (2010). Smart things: Ubiquitous computing user experience design. Amsterdam, Netherlands: Elsevier Science. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press. Learning-theories.com (2020a). Educational robotics and constructionism (Papert). Retrieved January 9, 2021 from https://www.learning-theories.com/educational-robotics-andconstructionism.html. Learning-theories.com (2020b). Situated cognition (Brown, Collins, & Duguid). Retrieved January 9, 2021 from https://www.learning-theories.com/situated-cognition-brown-collins-duguid.html. Lefrancois, G. R. (1997). Psychology for teachers (9th ed.). Belmont, CA: Wadsworth. Lexico. (2019). Practice. Retrieved July 30, 2019, from https://www.lexico.com/en/definition/ practice

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

71

Lindsey, L., & Berger, N. (2009). Simulation approach to instruction. In C. M. Reigeluth & A. A. Carr-Chellman (Eds.), Instructional-design theories and models, Volume III: Building a common knowledge base. New York, NY: Routledge. Markham, T., Larmer, J., & Ravitz, J. (2003). Project based learning handbook: A guide to standards-focused project based learning for middle and high school teachers. Novato, CA: Buck Institute for Education. Martin, L. C., & Towers, J. (2011). Improvisational understanding in the mathematics classroom. In R. K. Sawyer (Ed.), Structure and improvisation in creative teaching (pp. 252–278). New York, NY: Cambridge University Press. McKay, C. S., & Glazewski, K. D. (2017). Designing maker-based instruction. In C. M. Reigeluth, B. J. Beatty, & R. D. Myers (Eds.), Instructional-design theories and models, Volume IV: The learner-centered paradigm of education. New York, NY: Routledge. McKenney, S. E., & Reeves, T. (2018). Conducting educational design research: What, why and how. London, England: Taylor & Francis. Melchior, A., Burack, C., Hoover, M., & Marcus, J. (2016). FIRST longitudinal study: Findings at follow-up (year 3 report). Waltham, MA: The Center for Youth and Communities, Heller School for Social Policy and Management, Brandeis University. Meyers, C., & Jones, T. B. (1993). Promoting active learning: Strategies for the college classroom. San Francisco, CA: Jossey-Bass. Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. The Teachers College Record, 108(6), 1017–1054. Moos, D. C. (2011). Self-regulated learning and externally generated feedback with hypermedia. Journal of Educational Computing Research, 44(3), 265–297. Morrison, G. R., Ross, S. M., Kemp, J. E., & Kalman, H. (2010). Designing effective instruction. Hoboken, NJ: Wiley. Myers, R. D., & Reigeluth, C. M. (2017). Designing games for learning. In C. M. Reigeluth, B. J. Beatty, & R. D. Myers (Eds.), Instructional-design theories and models, Volume IV: The learner-centered paradigm of education. New York, NY: Routledge. Nix, R. K. (2012). Cultivating constructivist classrooms through evaluation of an integrated science learning environment. In B. J. Fraser, K. G. Tobin, & C. J. McRobbie (Eds.), Second international handbook of science education (pp. 1291–1303). Dordrecht, The Netherlands: Springer. Norman, D. (2013). The design of everyday things: Revised and expanded edition. New York, NY: Basic Books. Novak, G. M., & Beatty, B. J. (2017). Designing just-in-time instruction. In C. M. Reigeluth, B. J. Beatty, & R. D. Myers (Eds.), Instructional-design theories and models, Volume IV: The learner-centered paradigm of education. New York, NY: Routledge. Palincsar, A. S. (1986). The role of dialogue in providing scaffolded instruction. Educational Psychologist, 21, 261–277. Palincsar, A. S. (1998). Social constructivist perspectives on teaching and learning. Annual Review of Psychology, 49(1), 345–375. Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and monitoring activities. Cognition and Instruction, 1, 117–175. Papert, S., & Harel, I. (1991). Situating constructionism. Constructionism, 36, 1–11. Peirce, C. S. (1878). How to make our ideas clear. Popular Science Monthly, 12, 286–302. Prince, M. J., & Felder, R. M. (2006). Inductive teaching and learning methods: Definitions, comparisons, and research bases. Journal of Engineering Education, 95(2), 123–138. Project Lead the Way. (2014). PLTW. Retrieved November 25, 2014, from http://www.pltw.org Reigeluth, C. M., Beatty, B. J., & Myers, R. D. (Eds.). (2017). Instructional-design theories and models, Volume IV: The learner-centered paradigm of education [Kindle DX Version]. New York, NY: Routledge. Reigeluth, C. M., & Carr-Chellman, A. A. (Eds.). (2009a). Instructional-design theories and models, Volume III: Building a common knowledge base. New York, NY: Routledge.

72

C. Zintgraff and A. Hirumi

Reigeluth, C. M., & Carr-Chellman, A. A. (2009b). Understanding instructional theory. In C. M. Reigeluth & A. A. Carr-Chellman (Eds.), Instructional-design theories and models, Volume III: Building a common knowledge base (Vol. 3, pp. 3–26) [Kindle DX Version]. New York, NY: Routledge. Reigeluth, C. M., Myers, R., & Lee, D. (2017). Chapter 1, The learner-centered paradigm of education. In C. M. Reigeluth, B. J. Beatty, & R. D. Myers. (Eds.), Instructional-design theories and models, Volume IV: The learner-centered paradigm of education (Vol. 4) [Kindle DX Version]. New York, NY: Routledge. Savery, J. R. (2015). Overview of problem-based learning: Definitions and distinctions. In Essential readings in problem-based learning: Exploring and extending the legacy of Howard S. Barrows (pp. 5–15). Purdue University Press. Sawyer, R. K. (Ed.). (2011). Structure and improvisation in creative teaching. New York, NY: Cambridge University Press. Schmidt, H. G., Loyens, S. M. M., van Gog, T., & Paas, F. (2007). Problem-based learning is compatible with human cognitive architecture: Commentary on Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42, 91–97. Schwartz, D. L., Lindgren, R., & Lewis, S. (2009). Constructivism in an age of non-constructivist assessments. In Constructivist instruction (pp. 46–73). New York, NY: Routledge. Scott, C. (2012). An investigation of science, technology, engineering and mathematics (STEM) focused high schools in the US. Journal of STEM Education: Innovations and Research, 13(5), 30–39. Siemens, G. (2014). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2(1). Retrieved May 22, 2017, from http:// www.itdl.org/journal/jan_05/article01.htm Smith, P. L., & Ragan, T. J. (1999). Instructional design (2nd ed.). New York, NY: Wiley. Spector, J. M. (2014). Program and project evaluation. In J. M. Spector (Ed.), Handbook of research on educational communications and technology (pp. 195–201). New York, NY: Springer. Spinuzzi, C. (2005). The methodology of participatory design. Technical Communications, 52(2), 163–174. Strayer, J. F. (2017). Chapter 12, Designing instruction for flipped classrooms. In C. M. Reigeluth, B. J. Beatty, & R. D. Myers (Eds.), Instructional-design theories and models, Volume IV: The learner-centered paradigm of education (Vol. 4). New York, NY: Routledge. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. Sweller, J., Kirschner, P. A., & Clark, R. E. (2007). Why minimally guided teaching techniques do not work: A reply to commentaries. Educational Psychologist, 42(2), 115–121. Taylor, P. C., Fraser, B. J., & White, L. R. (1994). CLES: An instrument for monitoring the development of constructivist learning environments. In Annual meeting of the American Educational Research Association, New Orleans, LA. Teichmann, J., & Evans, K. C. (1999). Philosophy: A beginner’s guide. Oxford, UK: Blackwell Publishing. Texas Education Agency. (2014, August). TEA news releases online. Retrieved December 15, 2014, from http://tea.texas.gov/news_release.aspx?id¼25769815392 Tobias, S., & Duffy, T. M. (Eds.). (2009). Constructivist instruction: Success or failure? New York, NY: Routledge. Van de Pol, J., Volman, M., & Beishuizen, J. (2010). Scaffolding in teacher–student interaction: A decade of research. Educational Psychology Review, 22(3), 271–296. Veenman, M. V., Van Hout-Wolters, B. H., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1(1), 3–14. Verenikina, I. (2003). Understanding scaffolding and the ZPD in educational research. In Proceedings of the international education research conference (AARE – NZARE), 30 November–3 December 2003, Auckland, New Zealand.

3

Aligning Learner-Centered Design Philosophy, Theory, Research, and Practice

73

Voorhees, R. A., & Voorhees, A. B. (2017). Principles for competency-based education. In C. M. Reigeluth, B. J. Beatty, & R. D. Myers (Eds.), Instructional-design theories and models, Volume IV: The learner-centered paradigm of education. New York, NY: Routledge. Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press. Waring, S. M. (2011). Conducting authentic historical investigations in the digital age. In A. Hirumi (Ed.), Grounded designs for online and hybrid learning: Practical guidelines for educators and instructional designers. Eugene, WA: International Society for Technology in Education. Wiley, D. A. (2000). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In D. A. Wiley (Ed.), The instructional use of learning objects. Bloomington, IN: Agency for Instructional Technology and Association for Educational Communications & Technology. Wise, A. F., & O’Neill, K. (2009). Beyond more versus less: A reframing of the debate on instructional guidance. In Constructivist instruction (pp. 94–117). New York, NY: Routledge. Zintgraff, A. C., Jr. (2016). STEM professional volunteers in K-12 competition programs: Educator practices and impact on pedagogy (Doctoral dissertation).

Dr. Cliff Zintgraff spent 10 years as a Program Manager at the IC2 Institute at the University of Texas at Austin. He continues his association with the institute, and is now the Chief Learning Officer at SAMSAT, the San Antonio Museum of Science and Technology, after helping lead a merger between the startup science museum and a STEM nonprofit to form the combined organization. Together, they have served 120,000 students since 2012. His core focus is inquiry learning. Around that focus, he leads programs and conducts research on entrepreneurship, technology commercialization, STEM education, and STEM education’s role in regional economic development. Across 50 international trips, he has led programs and trained innovators in Colombia, Portugal, India, Czechia, Turkey, Hungary, Cyprus, Germany, Taiwan, and the UK. Dr. Zintgraff’s academic publications cover STEM education in economic development, professional volunteers in STEM education, regional cyber security secondary education programs, learning technologies, and educational philosophy. His edited volume STEM in the Technopolis: The Power of STEM Education in Regional Technology Policy was published by Springer in June 2020. He was previously a software developer, manager, and executive in medical imaging and secure telecommunications startups, and he founded an educational technology company building STEM-focused products. He holds a B.S. in Computer Science from Trinity University, an M.S. in Technology Commercialization from the University of Texas at Austin, and a Ph.D. in Learning Technologies from the University of North Texas. Dr. Atsusi Hirumi is a Professor of Instructional Technology at the University of Central Florida. Born in New York, Dr. Hirumi spent most of his formative years growing up in Nairobi, Kenya, East Africa. He received his B.S. in Biology from Purdue University with a secondary teacher certification in biology and general science. He received his M.A. in Educational Technology from San Diego State University and his Ph.D. in Instructional Systems from Florida State University. Dr. Hirumi’s work focuses on developing systems to train and empower K-12, university, and corporate educators on the design, development, and delivery of interactive distance education programs. His research concentrates on the design and sequencing of e-learning interactions. He has published over a dozen articles, several book chapters, and has made over 50 presentations at international, national, and state conference on related topics. Recent awards include the Texas Distance Learning Association award for commitment to excellent and innovation and the Star Faculty Award for outstanding teaching, research, and service, and he is the only two time recipient of the WebCT exemplary online course award.

4

Learning Theories: The Role of Epistemology, Science, and Technology Linda Harasim

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Is Learning Theory? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Why Theory Is Important to Educational Professionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Is Epistemology and What Is Its Role in Learning Theory? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Objectivist and Constructivist Epistemologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Is the Role of Science in Learning Theory? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Twentieth Century Learning Theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behaviorist Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cognitivist Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Constructivist Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Is the Role of Technology in Learning Theory? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Twenty-First Century Learning Theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Connectivist Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Collaborativist Learning Theory (a.k.a. Online Collaborative Learning) . . . . . . . . . . . . . . . . . . Artificial Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Augmented Human Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

76 77 77 79 80 81 81 82 87 88 91 92 93 97 102 105 108 110

Abstract

Scientific theories emerged in the nineteenth century to explain natural and social phenomena and became the basis for the twentieth century theories of learning: Behaviorism, cognitivism, and constructivism. Behaviorism and cognitivism were based on an objectivist epistemology emphasizing an absolute “truth,” efficiency, and the superiority of technology; constructivist learning theory was based on an epistemology of progressive change, knowledge building, and human agency. The rise of educational computing and computer networking in L. Harasim (*) Simon Fraser University, Burnaby, BC, Canada e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_48

75

76

L. Harasim

the late twentieth and early twenty-first centuries sharpened epistemological and pedagogical differences: objectivist epistemologies emphasized the role of technology to replace human teachers. Learning theories based on constructivist epistemologies also emerged, emphasizing the role of technology to augment rather than replace human intelligence. Online collaborative learning theory, later renamed collaborativism, is the major learning theory and pedagogy that argues for technology to advance rather than replace human agency and provides empirical evidence of augmenting human learning in the digital age. Keywords

Behaviourism · Cognitivism · Constructivism · Constructivist Epistemology · Connectionism · Connectivism · Collaborativism · Learning Theory · Technology · Online Learning · Education · Theory · Epistemology · Pedagogy · Discourse · Educators · Science, Scientific Thought or Scientific Epistemology · Metaphysical Thought or Metaphysical Epistemology · Positivism · Conditioning · Classical Conditioning · Operant Conditioning · Artificial Intelligence · Augmented Human Intelligence · Computer Networking · Machine Learning · MOOCs · Objectivist Epistemology · Open AI · Facebook · Google · Apple · Microsoft · Pavlov · Watson · Thorndike · Skinner · Piaget · Vygotsky · Siemens · Downes

Introduction Theories attempt to provide scientific explanations for why something is the way it is. The English word theory emerged in the sixteenth or seventeenth centuries, derived from the philosophical term “theoria” in ancient Greek. Theoria was a term used by the ancient Greeks to mean “looking at,” “observing,” or contemplating natural things. The modern adoption of the term “theory” continued the denotation of a thoughtful consideration of things, but with the additional connotation of a rational explanation or understanding of natural things. Theories of learning seek to explain the processes through which people learn and, like all theories, change or evolve over time as our understanding changes. Theories as a scientific explanation, especially in education, have existed for approximately 100 years. The earliest applications of theory in education came from the field of educational psychology. It has only been in recent decades that learning theory has emerged as a distinct field of study with educational researchers focusing on and pursuing many formulations and ideas about how people learn. There are five theories that encompass much of the field of learning research and practice to date. The three earliest theories emerged from the field of psychology, while the more recent theories emerged from the field of education. Theories reflect and articulate a particular view of knowledge or epistemology that sets the context for the theory. Is knowledge a form of “truth” as objectivists claim, or does knowledge undergo changes over time through scientific debate and

4

Learning Theories: The Role of Epistemology, Science, and Technology

77

new information, as constructivists content? We can understand a theory better once we know its epistemological basis. Moreover, theories reflect not just the epistemology but also the science and the technology of the time. This chapter examines learning theories through three perspectives: epistemology, science, and technology and addresses the opportunities and dangers that technology poses to education in the twenty-first century. Tracing the historical context and development of learning theories illuminates how a particular “theory” or view of learning emerged, how it has shaped the way we teach, what we identify as evidence of learning (assignments, quizzes, examinations), how we organize interactions with and between learners, how we use educational technologies, and how these technologies influence and impact how we teach.

What Is Learning Theory? Learning theories aim to help us understand how people learn. Many theories of learning were generated in the twentieth century, and new theories are emerging in the twenty-first century that take into account the growing role of computer networking and digital media in education. Typically, theory is generated by questions or curiosity and attempts to offer answers or explanations. As Albert Einstein stated, “theory provides the framework or lens for our observations.” This chapter examines five major theories: three from the twentieth century and two formulated in the twenty-first century to see how each provides an explanation and a lens through which education professionals (and others) gain a particular perspective on their field of work. The theory that we employ (consciously or not) determines what we see, what we consider to be important, and thus how we design and implement our practice. By understanding learning theory, educators can reflect on their practice, improve upon, refine, and perhaps redefine their approach to learning and their contribution to advancing the discipline.

Why Theory Is Important to Educational Professionals An understanding of how learning occurs is fundamental to designing and teaching successful education programs. In the modern era, theories of how people learn in complex environments are necessary to inform and design educational curricula within the context of the new and ever-changing realities of our technological age. Therefore, theory should not be viewed as something divorced from professional activities; it is integral to educational practice and vice versa. Understanding the major theories of learning that emerged in the twentieth and twenty-first centuries and how they were shaped by particular world views (epistemologies), contemporary views of science, levels of technological development, and

78

L. Harasim

how these then informed contemporary educational practice assists us in understanding how the field of education has developed and transformed over time. Also we will see how each theory is very much a product of its time, reflecting the level of information, development, and the values of that period. Theories of learning reflect the time in which they emerged and gained precedence. Understanding the context of a learning theory helps us to see it as a product of the discourse and the technological realities of that time, a phenomenon of history rather than an eternal truth. Theory is thus a kind of modus operandi; it influences, shapes, and determines our actions, even unknowingly. The theory of our time is like the air that we breathe – it is all around us, accepted by most or all of our colleagues and seems obvious. It is only when we take a step back and begin to consider theories in their historical, social, and technological context do we begin to see the outlines and characteristics of the theory of our time and our discipline. Whether or not we consciously intend to “operationalize” a particular theory of learning, we are nonetheless operating according to some perspective on how to teach (and concomitantly, even if unconsciously), a perspective on how people learn. In his article “Thoughts on Theory in Educational Technology,” Brent Wilson noted, “Theories shape our world just as surely as physical forces do, albeit in a different way” (1997, p. 23). Theories shape how we make sense of ideas and information and how we then act. For educators, theory provides a lens whereby we can view and understand the context of an educational model, a system, or the pedagogy. A theory of learning is a guide to how we each view the purpose of teaching and learning, how we design and implement our educational activities, and how we define learning in order to facilitate or assess it. How we teach reflects what we value and define as learning. Theoretical discussion can provoke creative thought and ideas. Theory also provides a means to “envision” new worlds and new ways to work. Theories establish a language and discourse whereby we can discuss, agree, disagree, and build new perspectives and ways to become knowledgeable (Table 1).

Table 1 What is a theory? The role of theory Explains Provides Why? A framework or lens How? A guide for practice Where? A means to envision When? change What?

Shapes Understanding Discourse Ideas Technology Methodology Actions

The theory we employ (even unknowingly) shapes how we design and implement our practice

4

Learning Theories: The Role of Epistemology, Science, and Technology

79

Theories of learning also have an important philosophical component. Reflection on human experience and behavior, its causation and implications, is part of human consciousness. Thousands of years of philosophical, social, and religious perspectives on learning preceded the development of learning theories. Ancient philosophers developed many important and illuminating insights into learning that contributed to how we view “epistemology” and “knowledge.”

What Is Epistemology and What Is Its Role in Learning Theory? The term “epistemology” comes from the Greek word episteme, meaning knowledge. In simple terms, epistemology is the philosophy of knowledge or of how we come to know. Epistemology asks: what is knowledge? And this begs the more fundamental question: How do we know? Our preconceived notions and our world view – i.e., our epistemology – help to shape or determine what we accept as knowledge and what we accept as truth. That is why an analytical view of the epistemological foundations of any theory or school of thought is essential to our understanding. Until about the nineteenth century, knowledge was viewed as divine, the product of the word of God or gods, who intervened at will in peoples’ lives. Epistemology in Western society had a relatively simple foundation: we know because God told us (Bruffee, 1999). Historically, formal education was “authorized” by the church, the temple, the synagogue, or the mosque. The teachers in ancient civilizations such as Persia and Athenian Greece were to some degree exceptions given their focus on civic laws and virtues, but even civic knowledge was viewed as having a divine origin. Prior to the sixteenth century, the church held major control over art, science, and knowledge in the Western world. Knowledge was based on the belief in a deity, and this deity was viewed as the source of all knowledge. Such a view of knowledge is known as metaphysical epistemology or deism. Questioning knowledge was not just illegal but heretical: to question was to challenge the word and power of God. There were to be no questions, only answers. And the answer was always: God. The scientific revolution of the sixteenth century challenged metaphysical thinking and deism. Galileo Galilei is identified as the father of the age of science because he introduced empirical evidence, thereby challenging metaphysical beliefs held holy by the Catholic Church. Galileo, scientist, philosopher, mathematician, professor, musician, and optician, may not have been the first to invent the telescope but he did use it to study the universe, and it was this use of the empirical evidence of how the stars actually moved, provided by the telescope, that is often credited with bringing science to the study of knowledge. He demonstrated that “divine laws” of knowledge such as the law that the sun circled the earth were false. Galileo’s telescopes provided evidence and proof that the earth circled the sun. This, and other empirical examples of evidence, weakened religion’s authority over knowledge.

80

L. Harasim

The invention of the printing press in 1543, with the sudden accessibility of relatively cheap informational broadsheets, led to a huge demand for literacy in Europe and an interest in understanding the world. The possibility that the average person could access information and that knowledge was based on what one could see or test gave rise to the age of science and the emergence of scientific epistemology. Scientific epistemology has subsequently subdivided into two distinct positions: objectivist epistemology or the belief that finite truths exist beyond our own minds and constructivist epistemology based on the belief that truth is constructed through shared knowledge and perceptions.

Objectivist and Constructivist Epistemologies Up to the seventeenth century, knowledge was viewed as an infinite truth, existing objectively beyond our own minds. This was metaphysical epistemology, which argued the importance of “truth” that came from the mind of a deity: only god knows. The rise of scientific epistemology continued the focus on “truth,” but sought truth through science rather than through a deity. Objectivist scientists look to science for “truth,” and apply laboratory research and experimentation to cleanse the research of elements or factors that might impact and reduce the science and purity of the study. In contrast to the objectivist view of the authority of external truth, constructivist epistemology holds that knowledge about the world is constructed through perception, interaction, and debate within communities of knowledge. Bates and Poole (2003) note that in North America these two dominant epistemological positions – objectivism and constructivism – dominate higher education today. They observe that objectivists: . . .believe that there exists an objective and reliable set of facts, principles, and theories that either have been or will be discovered and delineated over the course of time. This position is linked to the belief that truth exists outside the human mind, or independently of what an individual may or may not believe. (pp. 27–28)

On the other hand, write Bates and Poole, the constructivist epistemology holds . . .that knowledge is essentially subjective in nature, constructed from our perceptions and usually agreed upon conventions. According to this view, we construct new knowledge rather than simply acquire it via memorization or through transmission of those who know to those who did not. (p. 28)

In other words, in the constructivist perspective, knowledge is constructed by the individual through his or her interactions with the community and the environment. Knowledge is viewed as dynamic and changing, constructed and negotiated socially,

4

Learning Theories: The Role of Epistemology, Science, and Technology

81

rather than something absolute and finite. Knowledge is understood as a series of conversations on a topic and, in general, advancing our understanding and practice.

What Is the Role of Science in Learning Theory? Theory and scientific methods were popularized in the nineteenth century under the influence of positivism, a term articulated in 1847 by the French philosopher, Auguste Comte, who rejected metaphysics and argued that truth should be scientifically or mathematically verifiable. Comte (1798–1857) was the first intellectual to systematically articulate positivism and to present empirical or observable methods as a replacement for metaphysics or deism in the history of thought. Comte believed that empiricism should be at the core of scientific endeavor and that formal experimentation was the key to scientific method. Until then, metaphysics, which emphasized that a divine world lies beyond experience and transcends the physical or natural world, had been the dominant view. The scientific revolution made it requisite that the “sciences” develop theories to explain their work. Theories were not to be guesses, but rather accurate and reliable accounts of the real world. Natural sciences rushed to formulate theories and hypotheses in order to rationalize their field and to distinguish their science from false sciences such as alchemy, astrology, quackery, revelation, and belief in the occult. Formal learning had existed for perhaps 6,000 years – linked to the rise of literacy and numeracy (Harasim, 2017) – yet the science or theory of learning began to emerge only within the last 100 years, with twentieth century marking a period when theories of learning and academic scholarship in education emerged and flourished. Beyond the scientific revolution, a second major factor behind the development of learning theory was the industrial revolution. The rise of industrialism increased the pressure for mass education, to create a labor force of workers with basic literacy and numeracy skills. As a result, initial scientific efforts were undertaken to find empirical evidence of how learning occurs, with particular attention to efficient learning. These efforts resulted in the origins of the first learning theories.

Twentieth Century Learning Theories The earliest learning theories emerged from the field of psychology during the twentieth century as a result of efforts to understand human behavior. Three of the major theories are presented below in their historical and chronological order of development. Each may be understood as emerging in reaction to its predecessor. The three major theories of learning that emerged during the twentieth century were: 1. Behaviorist learning theory 2. Cognitivist learning theory 3. Constructivist learning theory

82

L. Harasim

Behaviorist Learning Theory Positivism and the rise of scientific methodology had a strong influence on the emerging field of education in the early twentieth century. In particular, the discipline of psychology had an impact on education because psychology studied human behavior and established empirical research methods based on a positivist framework. Educational researchers and psychologists sought to better understand learning by collecting and analyzing empirical data generated through clinical experimentation. The nature of learning, how learning occurs, what influences learning (positively and negatively), how to structure and support learning, and what we believe constitutes learning were largely based on interpretations of experiments with laboratory animals. The resultant perspectives were influential, but also subject to significant debate. In the early twentieth century, with the rise of modern science and new communication technologies, the speed of change increased: ideas were more easily communicated, disseminated, and debated. Sigmund Freud, and his contemporaries, contributed significantly to the rise of psychology as an empirical field and discipline. Moreover, to some degree, Freudian theory influenced and led to the first theory of learning: behaviorism. Behaviorism emerged in reaction to Freud’s emphasis on the unconscious mind and the Freudian use of introspective analysis and self-reports to study the mind. Behaviorism was a counterargument to Freud’s position. The motto of behaviorism might well be expressed as “behavior, not mind!” Behaviorism distrusted selfreports as a source of reliable data and instead emphasized that which was strictly observable and, as a result, the definition of learning was reduced to simple conditioning: stimulus and response. In the early twentieth century, behaviorism introduced a theory of learning that was empirical, observable, and measurable. This earliest theory of learning emphasized overt action that which was most easily apparent and accessible for study: behavior. Behaviorists studied how we act and what impacts and changes how we act. Based on clinical experiments with animals, behaviorist thinkers discovered that a response to certain stimuli would be repeated and could be observed, controlled, and quantified. To be considered a “science,” behaviorism had to adhere to rigid positivist principles based upon rigorous “objectivity” and ignore or dismiss “subjectivity” and anything to do with introspection or mental states (called mentalism at the time). In contrast to Freudian theory, in behaviorist theory what was in the mind was not considered accessible for study. The mind was viewed as a largely irrelevant black box, and, therefore, educational practice based on behaviorist terms could not take the mind into account. The emphasis was on environmental stimulus and observed response. To be considered scientific, research had to employ an experimental method, manipulating one variable to determine if changes in one caused changes in another (Fig. 1).

4

Learning Theories: The Role of Epistemology, Science, and Technology

83

Fig. 1 The behaviorist formula argues stimulus goes directly to response

Behaviorist learning theory emphasized two major types of conditioning: • Classical conditioning: for example, Pavlov’s dog experiments in which behavior becomes a reflex response to a stimulus • Operant conditioning: the example of Skinner’s rat experiments, which refer to the reinforcement of a behavior by a reward or punishment

Pavlov: Classical Conditioning The development of behaviorism is associated most famously with the Russian physiologist, Ivan Pavlov (1849–1936), who is considered the intellectual founder of behaviorist learning theory. Pavlov was famous for his theory of classical conditioning and his experiments with a dog, food, and a bell. Pavlov, a physiologist involved in medical research, studied reflexes and argued that reflexes are automatic behaviors caused by stimulus in the environment. For example, the smell of food cooking causes us to salivate. In 1904, Pavlov won the Nobel Prize in Medicine (Physiology) in recognition of his work on the physiology of digestion. It was this pioneering work on digestion that led him to serendipitously discover what he subsequently called conditioned reflexes. Pavlov was studying the physiology of digestion in dogs, when he discovered that in addition to salivating in the presence of meat powder, dogs had begun to salivate in the presence of the lab technician who fed them, even if there was no meat powder present. The dogs had learned to associate food with the person who fed them; this person became the stimulus for the food, and his presence would cause salivation even in the absence of food. Pavlov began to study the stimulus and response in dog salivation, using a bell (a neutral stimulus), which became associated with feeding time and thus became a conditioned stimulus as a result of consistent pairing with the unconditioned stimulus, meat powder in this example. Pavlov referred to a relationship that can be learned as conditioned reflex, as opposed to natural unconditioned reflexes. This became the theory of classical conditioning. Pavlov manipulated the situation of stimulus–response, by linking a conditional stimulus (the bell) to the unconditional stimulus (the food), and eventually took the unconditional stimulus away. The dog now salivated to the bell. This demonstrated that behavior could be manipulated through conditioning: responses could be manipulated or learned.

84

L. Harasim

Fig. 2 Classical conditioning: Pavlov’s dog experiment

Classical conditioning refers to a theory about how behavior is learned and was first applied to animals and then to humans. Pavlov proved that a conditional stimulus could cause a response on its own, demonstrating that classical conditioning succeeded (Fig. 2). The above experiment may seem simplistic but the results are widely regarded as representing the first major theory of learning, that is, a theory based on scientific evidence that is replicable and observable. Behaviorism emphasized that the repetition of a certain behavioral pattern makes that pattern automatic. If it is replicable and observable, then it is real. This became the underlying behaviorist theory of learning. Behaviorism was based upon empirical evidence and arguably, therefore, part of the emerging stream of scientific processes, reflecting modern science.

Watson John B. Watson (1878–1958) was the first American psychologist to use Pavlov’s ideas and is credited with coining the term “behaviorism.” In 1913, Watson published Psychology as the Behaviorist Views It, in which he writes that “Psychology as the behaviorist views it is a purely objective experimental branch of natural science. Its theoretical goal is the prediction and control of behavior. Introspection forms no essential part of its methods, nor is the scientific value of its data dependent upon the readiness with which they lend themselves to interpretation in terms of consciousness” (Watson, 1913, p. 158).

4

Learning Theories: The Role of Epistemology, Science, and Technology

85

Watson described psychology as the process whereby behavior is predictable and controlled, and he argued that terms such as consciousness, mind, or images do not have a place in psychology.

Thorndike: Connectionism Edward L. Thorndike’s work (1874–1949) is referred to as connectionism within the behaviorist school. Thorndike was interested in the association or connection between sensation and impulse, and he studied learning connected to action. Thorndike’s experiments with a “puzzle box” measured the amount of time it took an animal to operate the latch of the box and escape. In his experiments, animals were repeatedly returned to the puzzle box while Thorndike observed their efforts to escape. The amount of time taken to escape decreased with exposure. These experiments supported the view that learning is the result of associations forming between stimuli (S) and responses (R). According to Thorndike, such associations or “habits” become strengthened or weakened by the nature and frequency of the S–R pairings. Thorndike’s S–R theory was based on the concept of trial-and-error learning in which certain responses come to dominate others due to rewards. Connectionism (like all behavioral theory) posited that learning could be adequately explained without referring to any unobservable internal states. Skinner: Operant Conditioning The American psychologist, Burrhus Frederic Skinner (1904–1990), is also famously associated with behaviorist learning theory. Skinner’s work, however, differed from his Pavlovian predecessors in that Skinner focused on voluntary or operant behavioral conditioning, whereas Pavlov focused on what is known as classical conditioning. Operant conditioning emphasized the use of positive and negative reinforcement to manipulate or teach new behaviors. What distinguished operant conditioning was its focus on voluntary behavior rather than involuntary reflexive responses. Through experimentation, Skinner discovered that behavior can be conditioned using both positive and negative reinforcement. One well-known example is that of a laboratory rat learning to find cheese in a maze. Positive reinforcement conditions the rat to find the end of the maze through successive approximations or steps. Operant refers to the process of operating on the environment. The subject, in this case a rat, is doing whatever it does in the box shaped like a maze. While so doing (operating), the rat encounters a special stimulus, cheese. The cheese is called a reinforcing stimulus. This special stimulus has the effect of changing or modifying the behavior of the subject, tending to reinforce the tendency to repeat the behavior in future. The stimulus will cause the rat to make the correct turn in the maze to find the cheese. If the cheese is moved, the rat must learn to follow the pathway until another reward is discovered (by taking another turn in the maze) and so on. If the cheese disappears, the operant behavior is extinguished (Fig. 3).

86

L. Harasim

Fig. 3 Operant conditioning: Skinner’s rat and cheese maze

Experiments demonstrated that many repetitions were required before laboratory animals (mice, rats) learned that certain responses resulted in a reward of food (stimulus). Skinner found that such changes in behavior took considerable time and required many successive approximations of behavior. Often a big change in behavior would be induced by breaking it down into many smaller acts or components repeated over a long period. Skinner’s research led him to conclude that simply rewarding a series of small acts could condition complex forms of behavior. Skinner’s (1953) book, Science and Human Behavior, was a nonfiction consideration of how operant behavior could function in such social institutions as education, economics, law, religion, and government. Operant conditioning, Skinner argued, could shape behavior through such mechanisms as positive reinforcement (reward), negative reinforcement, non-reinforcement, and punishment. However, there were significant problems with Skinner’s own science, in particular a disturbing disjuncture between his model and the empirical results of his experiments. Some researchers argued that his claims exceeded his evidence and that he could not prove or demonstrate empirically that the responses were the result of a particular stimulus. Skinner responded to these criticisms by creating a set of highly controlled conditions in which a discriminating stimulus could be defined and linked to a specific and particular response. Behaviorist learning theory was most successful or relevant in contexts where the learning objectives were unambiguous and where their attainment could be judged according to commonly agreed upon criteria of successful performance in taskoriented learning. Examples might include learning accountancy procedures, learning to swim, or learning to operate a sophisticated machine. The prevailing pedagogy of behaviorist learning theory aims at achieving the correct (intended) behavior. It focuses on predictability, that is, ensuring that what is intended is achieved and that the link between a stimulus and the response it evokes is reliable, in other words: consistent, automatic, and replicable, time after time. A correct response to a stimulus would receive a positive reward. An incorrect response to a stimulus would yield a negative response (punishment). Behavioral

4

Learning Theories: The Role of Epistemology, Science, and Technology

87

pedagogies were rigidly adopted in some quarters within the field of learning theory. Memorization, repetition, reinforcement of correct answers, examinations, and the organization of the curriculum content into specific behavioral objectives resulted. This approach continues to influence instructional approaches today.

Cognitivist Learning Theory Cognitivist learning theory emerged as an extension of and a reaction to behaviorist theory, while retaining certain aspects of behaviorist theory. The rise of cognitivist learning theory was a response to behaviorism’s rigid emphasis on the direct link between “stimulus and response.” Cognitivist psychologists argued that the link between stimulus and response was not straight forward and that a number of other factors intervened to mitigate or reduce the “predictability” of a response to a stimulus (Winn & Snyder, 1996). As such, cognitivist theory was concerned with what comes between stimulus and response and understanding the processes of the mind – processes that behaviorists had rejected. If behaviorism treated the organism as a black box, cognitive theory recognized the importance of the mind. Cognitive theorists did not reject behaviorist science altogether, but shifted the emphasis to understanding the mental processes that operated on the stimulus and intervened to determine whether or not there was a response and, if so, which response? Cognitivists argued that these processes are what constitute human learning and determine how we think and act and hence must be studied. Hence, the key difference between behaviorist and cognitivist theories of learning was the importance accorded to what goes on between the stimulus or input and the resultant behavior. Cognitivists were interested in modeling the mental structures and processes that operated in the mind in order to explain behavior. The rise of cognitive learning theory in the mid-twentieth century was influenced by developments in such fields as linguistics, neurology, psychology, education, and the nascent field of computer science. It soon replaced behaviorism as the major school of thought and experimental paradigm. Elements from the behaviorist tradition were reshaped and incorporated into the cognitivist model of learning: stimuli became inputs and behaviors were outputs. In particular this was accomplished because of the invention of the computer which had a powerful impact on cognitive theory. Metaphors such as “mind as computer” and “human information processing” came to dominate cognitivist research as it related to educational practice. The first mainframe computer (the ENIAC) was completed in 1946, and computers, with their emphasis on input/output, began to have an increasing impact on teaching and learning theories and practices even though computers were initially viewed as “mathematical machines.” Cognitivists attributed great value to efficiency; whereas the human mind was viewed as slow and frequently faulty, the

88

L. Harasim

computer – based on mathematical formulas and software programs – was fast and potentially flawless. Machines improved on human ability. The didactic principles of Pressey’s teaching machine and the traditional lecture based on CONTENT + QUIZ were revised into software programs for computers. Individualized classroom computing based on discrete learning modules of CONTENT + QUIZ became major forces in educational technology in the 1980s. Computer-based training, also known as courseware, began to be delivered online in the 1990s. Courseware transmitted content efficiently and flawlessly.

Constructivist Learning Theory Constructivist learning theory, like behaviorist theory and cognitive learning theory, is not one unified entity. Rather it is an umbrella term representing a range of perspectives that share some common understandings. Constructivist learning theory argues that people construct their own understanding and knowledge of the world through experience. Encounters with new ideas, new things, and new perspectives require that we reconcile the new with our prior understanding: does the new fit with our previous understanding? And if not, we must choose between discarding it, integrating it with our existing views, or changing our existing beliefs. In the twentieth century, the major theorists associated with constructivist approaches were Jean Piaget and Lev Vygotsky, representing two major camps or perspectives associated with constructivist learning theory: • “Cognitive constructivism” is how the individual learner understands the world, in terms of biological developmental stages (Piaget). • “Social constructivism” emphasizes how meanings and understandings grow out of social encounters (Vygotsky).

Piaget Jean Piaget (1896–1980), a Swiss-born professor of psychology and student of biology, devoted his life to the question of cognitive development and to classifying the stages of human development. Piaget posited that humans learn through the construction of progressively complex logical structures, from infancy through adulthood. Piaget concluded that the logic of children and their modes of thinking are initially entirely different from those of adults and that successive knowledgebuilding activities increase in depth and complexity as humans move from one stage to another in their development: age-based stages. According to Piaget, all humans pass through the same four stages of cognitive development at around the same age: 1. Sensorimotor (birth to approximately 2 years of age): a period in which infants begin to construct an understanding of the world through the senses and through movement.

4

Learning Theories: The Role of Epistemology, Science, and Technology

89

2. Preoperational (2–7 years): at the preoperational stage of development, the child is able to mentally act on objects and to represent objects using words and drawings, but is not yet able to think through actions. 3. Concrete operational (7–11 years): by around the age of 7, a child is able to use logic appropriately and to solve actual problems, although not abstract problems. 4. Formal operational (12+ years): in this stage, individuals move beyond concrete experiences and begin to think abstractly, reason logically, draw conclusions from the information available, as well as apply all these processes to hypothetical situations (Santrock, 2008, pp. 221–223). These four stages of development were posited by Piaget as the psychological states that children pass through. Humans, according to Piaget, internalize knowledge through experience and make sense of these experiences through adaptation involving processes of assimilation, accommodation, and equilibration/ disequilibration. Assimilation, he argued, occurs when we apply preexisting mental structures to interpreting sensory data. Disequilibration occurs when an action cannot be assimilated into preexisting structures or when we cannot achieve the goals we seek (i.e., sucking a thumb not a nipple does not lead to food or when what we learned previously does not accomplish our goal). Accommodation occurs when the person realizes that the activity does not achieve the expected result and that existing schemes or operations must be modified. An instructor, for example, seeks to stimulate conceptual change by challenging a student’s existing concepts in order to create cognitive disequilibration. The student will try to restore equilibrium or resolve the problem. Through a process of disequilibration and restoring equilibration, the student constructs new cognitive structures. As with any major school of thought, there were many critiques of Piaget’s work. He published huge numbers of scholarly articles, was an expert in many fields, and was able to speak the “language” of many disciplines. This in itself caused confusion and obstacles for readers. Papert, who introduced constructivist computing to school children, noted that Piaget was a deep thinker whose real interests and contributions were to epistemology, an area overlooked by educators. Papert wrote about Piaget in Time Magazine’s 1999 special issue on the “Century’s Greatest Minds”: Piaget never thought of himself as a child psychologist. His real interest was epistemology – the theory of knowledge . . . The core of Piaget is his belief that looking carefully at how knowledge develops in children will elucidate the nature of knowledge in general. Whether this has in fact led to deeper understanding remains, like everything about Piaget, controversial. (p. 105)

Vygotsky Lev Semyonovich Vygotsky (1896–1934), a Russian psychologist, is the scholar today most prominently associated with social constructivism. He proposed a theory

90

L. Harasim

of cognitive development that emphasized underlying processes rather than the ultimate stage of human development, and he focused on the social rather than individual context of human cognitive development. Vygotsky’s view of constructivism focused on the relationship between the cognitive process and a subject’s social activities, emphasizing the social context of human development and learning, whereas Piaget focused on biological human development. Where Piaget placed the developmental stage before learning, Vygotsky placed learning before development. Vygotsky’s theories are most famously presented in his book Thought and Language (1962), written shortly before his early death, in which he asserts that thought and language are integrally connected. He argued that humans, even as infants, engage in internal dialogue, and it is the expression of this dialogue that leads to speech and thought. All humans are taught language by adults, and others, who speak to the child, point at and name things and introduce language to make meaning of the child’s experiences. Vygotsky’s approach to human development was fundamentally different from that of other developmental psychologists. Rather than focusing on a particular period of development, Vygotsky posed research questions with a broader perspective: what is the process of intellectual development from birth to death? What was of key importance for Vygotsky was the role of social and cultural factors: biological development does not occur in isolation. He argued that social interactions are an essential part of human cognitive development. Thus while other animals may also use tools, humans went beyond that to develop social speech. Vygotsky viewed socialization as leading to higher (individual) cognitive functions, of which human speech is a key example. He emphasized both egocentric speech and social speech. Based on his experiments, Vygotsky concluded that as children become more aware of themselves as individuals within a social world, their egocentric speech becomes subvocal and inner-directed. Egocentric speech leads to inner-directed thought; thought then leads to social speech. Vygotsky created the concept of ZPD, the “zone of proximal development” (proximal is a term meaning nearest). According to ZPD, learning takes place when learners solve problems beyond their actual developmental level – but within their level of potential development – under adult guidance or in collaboration with more capable peers. What this means is guided or supported learning. It does not suggest that the instructor guides the learner to the instructor’s intended goal through successive approximations (as in Skinner’s behaviorism), but on the contrary, the more advanced peer or teacher (or parent) supports the learner by providing the tools (language, concepts) needed to advance and eventually independently achieve the learner’s intended goal. Although Vygotsky never used the term “scaffolding,” it has become closely associated with ZPD, in which the peer or adult supports the learner in constructing knowledge. Scaffolds in learning can be compared with the use of scaffolds in the construction of buildings. In the classroom, a scaffold is a set of activities designed by the teacher to assist the learner move through increasingly difficult tasks to master a new skill. The teacher designs the classroom activities based on the student’s prior

4

Learning Theories: The Role of Epistemology, Science, and Technology

91

knowledge. Classroom activities are designed to help move students from point A to point B, to progress from what they know to what they need to know to complete the course or the class unit – to bring them through the zone of proximal development to achieve their potential.

What Is the Role of Technology in Learning Theory? The importance of online technology in the twenty-first century learning has been widely acknowledged. How we live and how we work have been broadly impacted by the rapid advances of computer technology, whereby digital media (defined as the use of AI) are embedded throughout our physiological, personal, social, and professional world in the form of smart technologies in our bodies, homes, appliances, phones, fashion, medical devices, transportation, schools, universities, work, and industry. This is now referred to as the Internet of things: everything in our lives is increasingly connected to the Internet. More than 50% of the planet’s population is currently connected online, and digital media moguls seek to include the remainder of the world population into the net in the near future. In education, computer networking provides new and ever-expanding opportunities for extensive communication and collaboration among diverse groups of peers and instructors in addition to important preparation for living and working an increasingly online world. Learning networks, online courses, electronic pen pals, online learning circles, computer-supported collaborative learning, knowledge forums, LISTSERVs, online seminars, computer-supported cooperative work, online communities, and a host of other terms reflect the power of computer communications to enable in-depth educational opportunities. Most millennials and youth today have grown up collaborating using online social networking technologies. This is important because a key problem is “the serious and persistent gap between how the digital youth of today learn in school and how they interact and work outside of school” (IESD, 2009). Many students are already adept at online group work in a social setting before they reach the classroom, yet classroom work from school through university is not significantly predicated on the online work or collaboration that plays a significant role in modern innovation. The lack of theory to guide pedagogical transformations in an online environment has been a significant obstacle. Teachers, trainers, and faculty have been urged to adapt to an increasingly technological environment without guidance about the educational paradigmatic changes occurring, the implications of educational transformation, and the ways in which teachers can develop and implement new pedagogies that are consonant with these realities. Given the rapid acceleration of computing power in recent decades and the stated intention of many AI scientists to create superintelligence, an AI that is sentient, able

92

L. Harasim

to reproduce itself, and within a few decades growing so powerful as to operate on a level far beyond the ability of humans to understand, this is a critical moment. Whereas computer scientists and AI experts generally seek to develop a “superintelligence,” that is, a super form of artificial intelligence superior to making, another approach is that of augmented human intelligence, in which AI is designed and developed to augment human intelligence rather than replace it. The online education debate for the twenty-first century has become artificial intelligence (AI – objectivist epistemology) versus augmented human intelligence (AHI – constructivist epistemology). Proponents of AI support a determining and controlling role for AI in education. They emphasize the efficiency of technological advancements, such as the ability of AI to rapidly autograde a quiz and respond with lightning speed to guide the learner to the next step on the computer-generated program. Proponents of AHI, on the other hand, sound the alarm about the potential dangers of abdicating responsibility for human learning to artificial intelligence that is outside the control of any but a few corporate entities. They argue that humans must maintain control over AI and its applications to ensure that artificial intelligence augments rather than replaces human learning and endeavor.

Twenty-First Century Learning Theories In post-WWII America, the invention of mainframe computing (the ENIAC, invented in 1946) triggered enormous excitement about new ways of work, mathematical problem solving, and a renewed interest in teaching machines. One path of educational computing continued the vision and objectivist epistemology that motivated Pressey’s teaching machine: to automate teaching through better and more efficient systems to deliver content and apply quizzes as the test of learning. The invention of the personal computer intensified efforts to create computer software that replaced teachers to make “teaching” more efficient. The emphasis from the 1920s onward, in the objectivist epistemology, was on individualized learning. A major shift in educational technology arrived with the invention of computer networking in 1969. Email and computer conferencing in the early 1970s introduced unprecedented possibilities for human communication and education. Educational applications of computer networking began to flourish in 1980s and 1990s, enabling users to freely communicate across obstacles of time and place. They could communicate one to one, one to many, or many to many. The ability for humans to engage in group communication (many–many) across time and place was an unprecedented civilizational advance. In the 1980s, individualized educational computing was largely discredited as a form of “electronic page turning,” reproducing textbooks on the screen, emphasizing drills, quizzes, and simplistic games. The attempt to replace the teacher with a computer was boring to the students and educationally a failure. On the other hand, the invention and introduction of computer networking was accepted by

4

Learning Theories: The Role of Epistemology, Science, and Technology

93

students and many teachers as an exciting opportunity to expand learning opportunities, critical thinking, and innovativeness through peer interaction and collaboration. As school-based learning networks emerged, educators began experimenting with online collaboration through projects such as electronic pen pals, online newsletters, online field science activities, and online cross-cultural exchanges with schools in other countries (Harasim, 1990; Harasim, Hiltz, Teles, & Turnoff, 1995). In 1986, the first totally online credit university course was delivered at the graduate school of education, University of Toronto (Harasim & Smith 1986, 1994). The adoption of online courses and learning networks spread during the 1980s. The decade was characterized by educational activities that emphasized online collaboration and distributed team projects at both the university and school levels, heralding the rise of collaborativist learning theory and practice. By the late 1990s, training organizations began to adopt online delivery, using the Internet to post self-paced, computerized courseware programs that replaced trainers and tutors. Participants could access courseware, also known as elearning, directly online. To date, two major twenty-first century learning theories have been posited to address learning in an online-networked environment: connectivism and collaborativism. They represent divergent views on the role that technology can or should play in human learning. Connectivism promotes technology as having the central and determining role in a learning environment, whereas in a collaborativist learning model, technology plays a supporting and augmenting role.

Connectivist Learning Theory The concept of connectivism as a learning theory was first introduced in 2004 by George Siemens and Stephen Downes, and, despite significant academic and media attention, it remains controversial as both a concept and a theory. It is included here due to the high-profile attention it received and its enduring application in for-profit educational ventures, related to MOOCs in particular. Connectivism is based on a concept and pedagogy of self-organized online learning: a concept of learning that would and could occur without the assistance of an instructor or professor. Proponents asserted that computer intelligence already existing in online networks could replace instructors in connecting people, resources, and technologies with the curriculum – thereby enabling learners to create and control their learning interests and processes without the involvement of teachers or structured coursework. Connectivism attempted to substitute the role of instructor with network-managed learning, whereby network intelligence created and organized the course design and curriculum. George Siemens argued that learning could reside in nonhuman appliances, thereby opening the role of non-sentient intelligence (or artificial intelligence) to manage human learning (Siemens, 2004). The concept was later redefined as

94

L. Harasim

network-managed learning, in which learning resources and links were identified and managed by intelligent networks. One of the claims by founder, George Siemens, was that connectivism was the only learning theory for the digital age because it was the first learning theory to consider technology. This fact, he argued, is what made connectivism a unique theory of learning for the digital age. Critics of connectivism have argued that human learning has always shaped and been shaped by the technology of the time. Technology and learning have been central to human development and civilization: how we live, how we survive, and how we thrive have always been related to the technologies of the time. Human invention of the technologies of speech, writing, printing, and the Internet has been integral to human learning and knowledge building. The invention of these communication technologies has represented key paradigmatic moments when human, societal, and technological development coincided to trigger major social and economic shifts, and great leaps forward in civil progress (Bates, 2015; Harasim, 2017). Behaviorist learning theory, itself, was linked to the notion of automation by Pressey’s teaching machine (1924), based on the view of learning as stimulus!response. Pressey’s teaching machine was developed to automate the role of the teacher and replace the teacher with a mechanical device; the machine provided the student with a short piece of content, followed by a multiple choice question to assess whether the student could accurately select the correct answer. The pedagogy of CONTENT + QUIZ that Pressey sought to mechanize with his teaching machine was refined by B.F. Skinner in the 1950s to become programmed instruction. Skinner was aligned with the rise of computers and their application in the field of education. Cognitivist learning theory was central to the development of computer-based teaching machines, especially software developed for computerassisted learning (CAL), computer-assisted instruction (CAI), computer-based training (CBT), and then, with intelligent tutoring systems (ITS), the increasing role of artificial intelligence in educational applications in online courseware beginning in the 1990s. The pedagogy of CONTENT + QUIZ that Pressey sought to mechanize was now embedded in computer software. Critics of connectivism note that there has been little effort by either Siemens or Downes to acknowledge the historical role of technology in learning or to specify or empirically demonstrate and define connectivism in practice. In 2012, writing about the development of the 2008 Mass Open Online Course (MOOC) with George Siemens, Downes commented: . . .we both decided at the outset that it would be designed along explicitly connectivist lines, whatever those were (emphasis added). Which was great in theory, but then we began almost immediately to accommodate the demands of a formal course offered by a traditional institution. (Downes, 2012)

This was written in 2012, 8 years after connectivism was first proclaimed a theory and practice for the digital age. Despite the enduring vagueness of the concept, the

4

Learning Theories: The Role of Epistemology, Science, and Technology

95

lack of empirical data, and controversy over whether it constituted a new learning theory, the media were enthusiastic proponents of MOOC courses, based on connectivist principles, as revolutionizing education.

George Siemens George Siemens, a writer, researcher, and international speaker, was the originator of the theory of connectivist learning. He was also the instructor and designer of what has been labeled the first MOOC, a course on the subject of “Connectivism and Connectivist Knowledge,” (CCK08), delivered at the University of Manitoba, Canada, in 2008. The term MOOC was coined to describe the fact that this course was a massive, open, online course available to non-credit participants who would link into the course via Internet connections. The concept of an open, online course available worldwide captured the imagination of both the media and higher education and was expected to revolutionize education. Siemens’ major conceptual contributions, such as his 2004 post, Connectivism: A Learning Theory for the Digital Age, and his 176-page online book, Knowing Knowledge (2006, Creative Commons), were blog posts at elearnspace.org. In his writings, Siemens argued that connectivism was unique because it acknowledged technology as an active participant in learning networks. He asserted that network technology is not just a player, but possibly the major or decisive participant. Siemens wrote: A central tenet of most learning theories is that learning occurs inside a person. Even social constructivist views, which hold that learning is a socially enacted process, promotes (sic) the principality of the individual (and her/his physical presence – i.e. brain-based) in learning. These theories do not address learning that occurs outside of people (i.e. learning that is stored and manipulated by technology). (Siemens, 2004)

Learning that occurs outside of people, manipulated by technology, is a theme that recurs throughout the writings of Siemens and Downes. Critics have questioned who or what will control or manipulate the technology, create and control the computer algorithms that manipulate learning interaction, and what theories of learning and epistemologies will drive the algorithm? Siemens linked his concept of connectivism to computational theories such as the theory of connectionism (connectionism is a theory in the field of artificial intelligence). Both Siemens and Downes suggest a relationship between connectionism and connectivism, but this has not been fully explored and documented. Connectionism remains a respected and frequently referenced artificial intelligence research theory that examines how intelligence might emerge from the activity of networks of neuronlike entities. Marvin Minsky and Seymour Papert in their classic book, Perceptrons: An Introduction to Computational Mathematics (expanded edition 1988), built on what were at the time new developments in mathematical tools, psychological models of how the brain works, and the evolution

96

L. Harasim

of fast computers that can simulate neural networks in the brain to advance the theory of connectionism, with special relevance to machine learning and artificial intelligence. Connectivism has not been related to computer science research or theory, and, to date, there have been no substantiated theoretical, methodological, or empirical associations made between connectivism and connectionism. Nonetheless, Siemens has suggested in his writing that there are commonalities. “Connectivism shares some traits of the cognitive science view of connectionism – the view that learning is a process of network formation” (ibid.). . . Connectionism, in terms of neuro/ cognitive science, is focused on neural networks – the manner in which we learn – contrasted with previous views of learning as information processing. Connectivism shares some traits of the cognitive science view of connectionism – the view that learning is a process of network formation. Connectionism is only focused with learning that happens in our heads. . . . Connectivism is strongly focused on the linking to knowledge sources ...not simply trying to explain how knowledge is formed in our own heads. The more rapidly knowledge develops the less likely it will be that we will possess all knowledge internally. The interplay of network, context, and other entities (many which are external) results in a new approach or conception of learning. The active creation of our own learning networks is the actual learning, as it allows us to continue to learn and benefit from our network – compared to a course which has a set start and end date. (Siemens, 2006) Connectivism addresses the principles of learning at numerous levels – biological/neural, conceptual, and social/external. . . .the same structure of learning that creates neural connections can be found in how we link ideas and in how we connect to people and information sources. One scepter to rule them all. (Siemens, undated)

The phrase “one sceptre to rule them all” implies a commonality among biological/neural networks, conceptual networks, and social/external networks: that each facilitate learning and reflect similar network structures and network principles. Siemens writes that, “Connectivism is the application of network principles to define both knowledge and the process of learning.” Implicit is a formula that neural networks (brain learning) + social networks + online networks = connectivist learning (learning networks). This formula, however, has yet to be developed, clarified, or supported by empirical evidence related to connectivism. Abruptly, in late 2015, Siemens recanted his earlier devotion to EdTech. On his elearnspace.ca blog, Siemens wrote Adios Ed Tech. Hola something else: I no longer want to be affiliated with the tool-fetish of edtech. It’s time to say adios to technosolutionism that recreates people as agents within a programmed infrastructure. (Siemens, 2015)

Technosolutionism, Siemens now argues, approaches human learning in the same manner as it would program a machine, with the predesignated code simply imprinted on the human brain. Overall, Siemens concluded that “educational

4

Learning Theories: The Role of Epistemology, Science, and Technology

97

technology is not becoming more human; it is making the human a technology” (ibid.). References to connectivism as a theory of learning have largely faded from view, especially given Siemens’ (2015) departure. With few exceptions, there was little empirical contribution to the theory beyond its initial pronouncement and some experiments with MOOCs around 2008–2012 that were never sufficiently analyzed. Initial efforts to provide results by edX, Coursera, and Udacity illuminated alarmingly high dropout rates, student failure, and very high costs. Moreover, there was little to no effort invested in generating empirical evidence of learning to support the claims of connectivists. The marketing of for-profit MOOCs is flourishing however.

Collaborativist Learning Theory (a.k.a. Online Collaborative Learning) In the early 1980s, long before the concept of connectivism was first proposed, online collaborativist learning activities were taking advantage of the invention of online computer communications including email (1970) and computer conferencing (1972). The field of online educational activities continued to grow and has been adopted to some extent at all levels of education. The rapid growth of the practice, combined with active field research created a large base of empirical data from which patterns could be discerned. It was from these patterns that the theory of online collaborative learning (OCL), later renamed collaborativism, emerged. Collaborativist theory is a model of learning in which students are encouraged and supported to work together to create knowledge. Unlike constructivist or active learning theory, collaborativist theory focuses on the role of collaboration and discourse in learning. While both collaborativist and constructivist learning theories promote the concept of active learning, the role of the learner and of the teacher is not identified nor explained in constructivist or active learning theory. Is the teacher to be a co-participant in activities? What defines or distinguishes “learning” activities and how might these be identified and facilitated? Constructivist learning theory emphasizes “activity” by the learner but offers no definition of learning to guide the “learning activities.” In the collaborativist model, the role of the teacher and of the learner is identified and defined as part of the theoretical framework. Learning activities need to be informed and guided by the norms of the discipline and by a discourse process that emphasizes conceptual learning and builds knowledge. There is a need for students to have a relationship to the knowledge community, mediated by the teacher who represents that community. In the collaborativist approach to learning, the teacher plays a key role as the link between the learners and the knowledge community that represents the state of the art in a given discipline; in other words, the teacher inducts the learners into the knowledge community.

98

L. Harasim

The Role of Discourse and Collaboration in Learning In collaborativism, discourse is viewed as playing the central role in knowledge creation, sharing, dissemination, application, and critique. Discourse is defined as written or spoken discussion, communication, and conversation. It is also posited as the catalyst for the development of civilization and the basis of thought and knowledge. Lev Vygotsky’s (1962) book, Thought and Language, is recognized as a major contribution to understanding the role of language and society in human thought. Vygotsky makes the argument that “A word devoid of thought is a dead thing, and a thought unembodied in words remains a shadow” (p. 153). Vygotsky was an early and major force in positing the importance of collaboration for knowledge construction; he revised learning theory by moving the unit of analysis from the individual per se to the individual in relation to the environment and to interaction with others. He defined learning as a social process, based on language, conversation, and the “zone of proximal development” (ZPD), whereby we learn through contact and discourse with an adult or peer more competent in the field. Kenneth Bruffee (1999) also emphasized the importance of discourse in learning: “We think because we can talk with one another” (p. 134). Knowledge is generated by speech and conversation with one another, a construct of the community’s form of discourse, negotiated and maintained by local consensus, and subject to endless conversation (Bruffee; Kuhn, 1970). “Education initiates us into conversation, and by virtue of that conversation initiates us into thought” (Bruffee, p. 133). Collaborativist learning theory and pedagogy focus on initiating learners into the processes of conversation (discourse) as used by knowledge communities to create knowledge and improve ideas. Michael Tomasello, a world leader in cognitive anthropology, spent over 20 years studying how humans think and learn. His 2014 book “A Natural History of Human Thinking” covers the evolution of human cognitive development over the past million or so years, to present the argument that collaboration is key to human cognitive uniqueness. Early humans had to learn to see the world from multiple perspectives, in order to coordinate and understand their partners’ perspectives. And in order to find partners who wished to collaborate with them. Tomasello’s arguments about human thinking and collaboration emphasize the need to understand how humans think, cooperate, and communicate if we want to facilitate learning. By 400,000 years ago, early humans had become essentially collaborative beings (Tomasello, 2014, p. 48). It is accepted that communication, collaboration, and social cognition are the hallmarks of evolution that distinguish us from other animals (including other great apes) and are key to how humans have survived and thrived over the past half million years. Collaborative learning theory builds on discourse as the foundation of human learning. The theory defines ways of teaching and learning that are based on discourse whereby students co-labor to produce a result whether to solve a

4

Learning Theories: The Role of Epistemology, Science, and Technology

99

problem, discuss or improve an idea, explore a hypothesis, or undertake a project (Harasim, 2004). Collaborativism asserts that it is through the role of the teacher or moderator, and access to new sources of information, that a group arrives at a position of intellectual convergence, a group position (albeit not homogenous) that reflects a deeper understanding of the content and possibly even contributes to advancing the practice and the state of the art.

The Three Processes of Collaborativist Learning Theory Collaborativist learning theory is based on three key learning processes or stages that lead from divergent thinking to intellectual convergence. Divergent thinking refers to a process that generates many questions, ideas, responses, or solutions. It is associated with brainstorming and creative thought, generating questions, and drawing on ideas from different perspectives and many sources (including personal observations and experiences). While divergent thinking involves generating many ideas, the process associated with identifying the best ideas and discarding the weak ones is called convergent thinking. Linus Pauling, the great scientist who won two Nobel prizes in his lifetime, was credited with the following response when asked at a public lecture how one creates scientific theories: Pauling replied that one must endeavor to come up with many ideas – then discard the useless ones. Collaborativist learning models facilitate intellectual convergence in a group environment; participants begin by brainstorming their ideas and experiences on a topic – idea generating or divergence. Then by engaging in group discussion and debate, informed by scientific literature and experience, participants move to a second phase, to identify and discard the weak, irrelevant or useless ideas and to cluster and organize the connections. This is called the idea organizing phase. The third phase, intellectual convergence, is when learners reach a collective position on the topic. Participants cluster on shared positions or decide to “agree to disagree.” In the case where a group product is required, participants will come to a consensus on the process and content (Harasim, 1990, 2002) (Fig. 4). The three steps or stages from divergent to convergent thinking are: 1. Idea generating: brainstorming, verbalization, and generating information 2. Idea organizing: clarifying and organizing new ideas according to their relationships and identifying their strengths and weaknesses, agreeing, disagreeing, clarifying, supporting, and questioning 3. Intellectual convergence: arriving at a shared understanding (including agreeing to disagree) or a mutual contribution to and construction of shared knowledge and understanding These phases may lead to social applications or they may lead to further debate, discussion, and the refinement of the concepts. The process is not circular, but one of

100

L. Harasim

Fig. 4 Collaborativist learning theory

continual growth or advancement in learning based on a feedback spiral. The ultimate goal of the process is to facilitate collaborative learning and to augment human intelligence.

Artificial Intelligence (AI) Versus Augmented Human Intelligence (AHI) The invention of the world’s first fully function computer in 1946 (the ENIAC) at the University of Pennsylvania triggered immense excitement over the potential of its mathematical problem-solving capabilities. The power and possibilities of the computer were enormous and generated excitement and new visions of “how to change the world.” Scientists and researchers worldwide began to consider ways to design, develop, and apply these powers. Visions of what may be possible were bright, and computer pioneers in the 1950s and 1960s were charged with optimism. Two distinct and potentially polar opposite paths emerged regarding the design and role of computing in society and in humanity as a whole: artificial intelligence and intelligence augmentation. Artificial intelligence is based on the idea that advanced computers can reproduce human cognition and function autonomously, ultimately outpacing and replacing the human brain. An autonomous AI (aka superintelligence) would be able to initiate actions on its own, make autonomous decisions, and pursue its own goals. Founders of artificial intelligence include John McCarthy, who coined the term “artificial intelligence” and Marvin Minsky one of the foremost leaders in AI who, together with McCarthy, founded the MIT AI Lab in 1959. Prof. Minsky’s ideas and influence have been wide ranging, including linguistics, mathematics, and robotics. His overarching theme or perspective was that the brain was a “meat machine” whose functions could be replicated and replaced by a computer.

4

Learning Theories: The Role of Epistemology, Science, and Technology

101

Intelligence augmentation (IA) researchers were also computer pioneers; unlike AI scientists, IA scientists viewed computing power as a way to augment and enhance human intelligence. Beginning in the 1950s and 1960s, these pioneers took a path that represented a constructivist epistemology. Rather than seek to develop the computer as a form of superintelligence that could gain autonomy and intentionality over humans, developers of concepts such as intelligence augmentation, intellectual amplification, and the augmentation framework sought to develop computing powers that would supplement and facilitate human thinking, analysis, and development. Founders of the IA path include early computer pioneers such as J.C.R. Licklider, Ted Nelson, and Douglas Engelbart. There is urgency in the AI versus AHI developments, epistemologies, theories, and implications. They represent not just competitive but clashing views that have as their conclusion the future of humanity. The role of the computer in the AI scenario is to act alone; the role of the computer in the IA scenario is to interact with humans. The key issue is the astounding speed at which computer intelligence has developed within seven decades, as a result of the immense level of investment in AI. Whereas humans took millions of years to evolve into their current intellectual capacity, computers, with the aid of humans, are evolving at a far faster rate. In 70 years, computers have reached many measures of human intelligence. AI and robots, moreover, threaten to destroy not only millions of jobs but to replace manual and also cognitive labor professions. There are very strong voices warning against AI. Famous physicist, Stephen Hawking, one of Britain’s preeminent scientists, has stated that efforts to create thinking machines pose a threat to our very existence. “The development of full artificial intelligence could spell the end of the human race,” the world-renowned physicist told the BBC’s Rory Cellan-Jones during a 2014 interview. “It would take off on its own and re-design itself at an ever increasing rate. Humans, who are limited by slow biological evolution, couldn’t compete, and would be superseded” (Cellan-Jones, 2014). Hawking has been voicing this apocalyptic vision for a while. In response to Transcendence, the sci-fi movie about the singularity starring Johnny Depp, Hawking criticized researchers for not doing more to protect humans from the risks of AI. “If a superior alien civilization sent us a message saying, ‘We’ll arrive in a few decades,’ would we just reply, ‘OK, call us when you get here – we’ll leave the lights on’? Probably not – but this is more or less what is happening with AI” (Hawking, Russel, Tegmark, & Wilczek, 2014). Elon Musk, founder and CEO of the rocket-making company SpaceX and the Tesla car and proponent of the HyperLoop, similarly warns that AI is “our biggest existential threat.” At a conference at MIT in October 2014, Musk likened the investments to improve artificial intelligence to “summoning the demon.” Musk also warned that AI could be more dangerous than nuclear weapons (Luckerson, 2014). AI would reduce or replace the role of humans on earth, because of its exponential rate of growth, the way it is being designed, and the enormous investment by government, military, and for-profit commercial interests. What began in the 1950s

102

L. Harasim

as university-based research and development, funded by the US military, has become by the twenty-first century a massive commercial and military venture. A venture that is also causing great concern even among AI researchers, scientists, and technology experts and entrepreneurs. In July 2015, over 1,000 high-profile artificial intelligence experts and leading researchers signed an open letter warning of a “military artificial intelligence arms race” and calling for a ban on “offensive autonomous weapons” (Gibbs, 2015). The letter, presented at the International Joint Conference on Artificial Intelligence in Buenos Aires, Argentina, was signed by 1,000 AI and robotics researchers and states: AI technology has reached a point where the deployment of [autonomous weapons] is – practically if not legally – feasible within years, not decades, and the stakes are high: autonomous weapons have been described as the third revolution in warfare, after gunpowder and nuclear arms. (ibid.)

A computer program that can take control of its own destiny and actually think for itself may be available within a few decades. This significantly affects educators and learners and humanity – worldwide.

Artificial Intelligence AI runs on algorithms that predict or direct human behavior. Enormous investment by the private sector in educational technologies such as MOOCs, adaptive learning systems (ALS), and personalized learning systems (PLEs) aims to replace live teachers with computer algorithms designed and owned by corporate interests. As governments and universities purchase and adopt these systems, education becomes increasingly controlled by the corporations who own and control those algorithms. Bates (2016) points out: These algorithms though are not transparent to the end users. To give an example, learning analytics are being used to identify students at high risk of failure, based on correlations of previous behaviour online by previous students. However, for an individual, should a software program be making the decision as to whether that person is suitable for higher education or a particular course? If so, should that person know the grounds on which they are considered unsuitable and be able to challenge the algorithm or at least the principles on which that algorithm is based? Who makes the decision about these algorithms – a computer scientist using correlated data, or an educator concerned with equitable access? The more we try to automate learning, the greater the danger of unintended consequences, and the more need for educators rather than computer scientists to control the decision-making. (ibid.)

In the media hoopla that accompanied the creation of the first massive open, online course (MOOC), little consideration was given to the unintended consequences of such an approach to education. Excitement over the possibility of making courses available online to millions of people worldwide overshadowed a somber consideration of potential unintended consequences.

4

Learning Theories: The Role of Epistemology, Science, and Technology

103

Is human intelligence advanced by requiring learners to repeat a “correct” answer, a form of obedience not just to technology and the content, but to the owners of the technology which delivers their message? Is simply learning to “click” on the correct answer the path to genuine learning? If there is no instructor to question, to address nuances in information, or to introduce concepts outside of the technologically established parameters, where does that leave the learner? What if the student disagrees with the material? What if they have ideas that would advance our understanding of the subject matter? It is a closed box. A major misunderstanding today is that educational access and quality is primarily an issue of technology rather than pedagogy. In reality, it is both. Behaviorist and cognitivist theories promote technology over pedagogy as the solution to educational progress, arguing that efficiency and massive delivery of content are more important than effectiveness or understanding. Technology is presented as superior not only to pedagogy, but as superior to human educators. Technology can deliver more content, more often, and to more people: it is more efficient at delivery. But does that mean that delivery systems are actually better at facilitating learning, understanding, and knowledge construction? Learning theories that promote technology as content transmission/delivery systems do not address this more complex and important issue. Proponents of connectivism, for example, argued that intelligent networks were superior to instructors. Stephen Downes wrote in 2007 that computer networks would organize resources for learning “without prejudice (or commercial motivation).” A self-organizing network would “never need to be searched – it would flex and bend and reshape itself minute by minute according to where you are, who you’re with, what you’re doing, and would always have certain resources ‘top of mind’ would could be displayed in any environment or work area” (Downes, 2007, Italics added-LH). Downes presents the ability of the network to track/monitor each user all the time as a positive feature. Technology is superior to human teachers; “it works ‘well and without prejudice (or commercial motivation)’” (Downes). The technologies and pedagogies of behaviorist, cognitivist, and connectivist approaches are all based on individualized learning rather than collaboration, on telling rather than discussing or constructing knowledge. Theories of learning within this objectivist epistemology emphasize quantity over quality and on the explicit superiority of technology over human teachers. Technology is deemed to be more pure than the algorithms or the values of the computer scientists who first programmed them. The efficiency of technology marks its superiority over humanity and over such issues as ethics, principles, or values. As major investments in computer networking and artificial intelligence transform AI from computer-based programs designed by humans to systems which program themselves, human irrelevance and mass unemployment become very real dangers. In this scenario, humans are superseded by technology. Ray Kurzweil, a chief engineer at Google, is the major proponent of the singularity, the point at which he argues machine intelligence will match and exceed human intelligence. His 2006 book The Singularity is Near: When Humans

104

L. Harasim

Transcend Biology promotes the superiority of technology and advocates the union of humans with technology. Others write about the dangers of creating sentient beings more powerful than us, artificial super intelligence that humans will not be able to control and which will easily be able to outsmart, outwork, and outcompete humans in all ways. Books such as James Barratt’s Our Final Invention: Artificial Intelligence and the End of the Human Era (2013) and Nick Bostrom’s book on Superintelligence (2014) are among a myriad that consider the negative implications of artificial super intelligence and the future of the human race. Adherents of AI and the singularity believe that they are creating a “solution,” a “species” superior to humanity which will embody intentions and decisions that are pure and superior to any produced by mankind. Other challenges of AI to education are the enormous investments in commercializing AI. Massive “donations” by the private sector who seek to control how their “money” should be used are creating a tsunami of purchase power that could ultimately destroy public education. Donations with strings, such as the Zukerberg’s 2015 donation of $45 billion explicitly designated for the funding and purchase of PLEs (personalized learning environments), seem more like directives for public education than traditional donations which enable the benefactors to decide how best to use the “gift.” Shoshona Zuboff has studied how corporate interest in education is about changing human behavior; to teach/train participants to do the will of the company or the market. She writes: Among the many interviews I’ve conducted over the past three years, the Chief Data Scientist of a much-admired Silicon Valley company that develops applications to improve students’ learning told me, “The goal of everything we do is to change people’s actual behavior at scale. When people use our app, we can capture their behaviors, identify good and bad behaviors, and develop ways to reward the good and punish the bad. We can test how actionable our cues are for them and how profitable for us.” (Zuboff, 2016)

The intent to change people’s actual behavior at scale is, as the “chief data scientist” in the article noted, based on punishment and reward to yield human behavior that is profitable for the company. The goal is not to create a better workforce, a knowledge society, nor to augment human ability to think; it is to control behavior in order to create profit. Marvin Minsky, in one of the last interviews before his death, said in 2015 that he felt there had been “very little growth in artificial intelligence” in the past decade, adding current work had been “mostly attempting to improve systems that aren’t very good and haven’t improved much in two decades.” By contrast, he said “the 1950s and 1960s were wonderful – something new every week” (Lee, 2016). And he hinted he was against large technology companies such as Google and Facebook getting involved in the field of AI. “We have to get rid of the big companies and go back to giving support to individuals who have new ideas because attempting to commercialise existing things hasn’t worked very well,” Minsky said (ibid.).

4

Learning Theories: The Role of Epistemology, Science, and Technology

105

Augmented Human Intelligence Augmented human intelligence (AHI) is a concept coined to describe the augmentation of human intelligence through education, communication, human experience, and technology (Harasim, 2017). It posits the human being as central, rather than peripheral or inferior to technology. Thus, AHI requires that advanced technologies be designed to enhance and augment rather than replace or reduce human intelligence. AHI represents a constructivist epistemology. The concept of AHI reflects many perspectives: that human intelligence has been amplified or augmented by a variety of technologies invented for this purpose over the past hundreds of thousands of years of human development, that augmentation is not a new concept but is organic with human civilization and progress, and that advances in computing should not be designed to outpace and replace humanity, but to encourage human innovation, collaboration, communication, civility, and creativity. The invention of computer networking technologies has roots in a vision of collaboration, community, learning, and active knowledge construction. One of the earliest technological precursors is hypertext, a concept and technology viewed as the precursor and inspiration for the World Wide Web. The history of hypertext began in 1945 with Vannevar Bush’s article in the Atlantic Monthly entitled “As We May Think,” about a futuristic technology that he called memex, “a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory” (p. 108). Bush’s groundbreaking vision of a technology to enhance thought pre-dated the computer. Nonetheless, Bush’s article and his concept of the memex directly influenced and inspired the two Americans generally credited with the invention of hypertext – Ted Nelson and Douglas Engelbart. Nelson coined the words “hypertext” and “hypermedia” in 1965 and worked to develop a computer system that enabled writing and reading that was nonsequential and presented the potential for cross-referencing and annotating (Nelson, 1974). In Project Xanadu, Nelson sought to create a computer networking system that enabled users to create linkages among ideas and information resources, to explore the interconnections and generate multiple perspectives on a topic (Nelson, 1987) (Fig. 5). Douglas Engelbart, like Vannevar Bush two decades earlier, was concerned with enhancing the intellectual capacity of people. In 1962, Engelbart published his seminal work, Augmenting Human Intellect: A Conceptual Framework, proposing to use computers to augment training. With his colleagues at the Stanford Research Institute, Engelbart developed a computer system to augment human abilities, including learning. The system was simply called the oNLine System (NLS) debuted in 1968 and later marketed as “Augment.” One of the most notable design features of Augment was the emphasis on providing tools to support collaborative knowledge work.

106

L. Harasim

Fig. 5 A timeline of epistemologies and technologies of learning

The Augment project “placed the greatest emphasis on collaboration among people doing their work in an asynchronous, geographically distributed manner” (Engelbart & Lehtman, 1988, p. 245). Augment enabled idea structuring, as well as idea sharing. While linkages among ideas and authors are supported by Augment, the system employs a hierarchical structure. Xanadu and Augment “were the first systems to articulate the potential of computers to create cognitive and social connectivity: webs of connected information and communication among knowledge workers” (Harasim, 1990, p. 41). The initial concept of a global information network came from J.C.R. Licklider in the late 1950s. At a time when computers were viewed as giant calculators, Licklider envisioned the use of networked computers to facilitate an online community, online personal communication, and active informed participation in government (Hafner & Lyon, 1996, p. 34). Licklider’s 1950s visions were prescient and one of the earliest precursors to the rise of personal computers and computer networking. In 1960, Licklider published his seminal paper “Man–Computer Symbiosis” in which he proposes the potential of computers to transform society. He put forward a vision that anticipated collaborative learning, emphasizing the potential of the computer to support group discussion, networking, multiple perspectives, active

4

Learning Theories: The Role of Epistemology, Science, and Technology

107

participation, and community practice. Although Licklider left the Arpanet project before it was completed, his vision of Arpanet as a knowledge network remained. Another very important technological development related to human communication and collaboration was computer conferencing. Computer conferencing was invented to support group communication and decision-making, and the first system, EMISARI, was developed by Murray Turoff in 1971. In 1974 Turoff founded the Computerized Conferencing and Communications Center at the New Jersey Institute of Technology (NJIT) and developed the EIES computer conferencing system. Other conferencing systems developed in the early to mid-1970s were PLANET, Confer, and *Forum. Computer conferencing is important to the history of online education because many of the earliest ventures in online course delivery involved computer conferencing. The history of online education, and collaborativist learning theory, reflect the continuation, development, and refinement of these early computer pioneers. The concept of AHI needs serious consideration by educators and society as a counter to the promotion of artificial intelligence in education and the growing media support of automated education. Augmenting human intelligence should be the focus of developments in computing technology, socioeconomic development, and education. The emphasis in education must shift from technology as automating education to technology as empowering and augmenting the educational community of teachers and learners. In 2016, Tony Bates warned that AI systems in education are increasingly automating rather than empowering the learner: The danger then with automation is that we drive humans to learn in ways that best suit how machines operate, and thus deny humans the potential of developing the higher levels of thinking that make humans different from machines. For instance, humans are better than machines at dealing with volatile, uncertain, complex and ambiguous situations, which is where we find ourselves in today’s society. (Bates, 2016)

AHI emphasizes the design and use of technologies and pedagogies that support human learning while extending and advancing human thinking. Technologies that augment human intelligence are designed to serve and advance human intelligence, rather than reduce or replace it. AHI builds on the work and directions of the computer pioneers of intellectual augmentation and amplification in the 1950s and 1960s, but also builds on the story of human development which has been augmented and amplified for hundreds of thousands of years, with the technologies of speech, writing, printing, and other communication and social–intellectual media. In December 2015, Musk and other AI critics announced the launch of OpenAI (Kelly, 2015). OpenAI’s goal is to develop AI safely and share its research widely. The purpose of OpenAI is specifically meant to be used in ways that will benefit humanity. Huge megacorporations, such as Google, Apple, Microsoft, and Facebook, have been investing heavily in AI for their own private profit and knowledge.

108

L. Harasim

To counter this, OpenAI’s backers – a group that includes Musk, Peter Thiel, Reid Hoffman, and Y Combinator’s Sam Altman and Jessica Livingston – are committing $1 billion to the project. “I believe it’s better to empower human kind with distributed artificial intelligence than a central artificial intelligence controlled by a single company,” said Altman in an interview. Altman has concerns about the technology but instead of being worried and doing nothing, he feels it’s better to be active in the field. “I sleep better knowing I can have some influence now,” he said. (Kelly, 2015)

Musk warned that “Humanity’s position on this planet depends on its intelligence, so if our intelligence is exceeded, it’s unlikely we will remain in charge of the planet” (ibid.). Likewise, education’s challenge, then, is to design and implement educational and socioeconomic strategies to contribute to an enlightened society, and a knowledge age on a global scale, that augments human learning and progress, and avoids the negative implications of a reliance on artificial intelligence and programs based on motives of increasing profit rather than ensuring thinking skills. This scenario may well be possible if educators immediately take up the challenge and begin to engage in conversations with one another, locally and globally, about the nature of these challenges and our potential to recreate and transform teaching and learning in order to augment human thinking, creativity, equity, ethics, and civil understanding. Theories such as collaborativism encourage active learning and knowledge building capabilities in which technology enhances and amplifies but does not replace human creativity, autonomy, and control. Collaborativist learning theory provides a theory, technology, and pedagogy for advancing AHI, rather than AI. We need to pay attention to epistemology and truly consider and understand the forces behind a particular theory or educational technology and what it is promoting (Fig. 6).

Conclusion For hundreds of thousands of years, humans have developed and used technology to benefit and advance civilization. Our entire development path has been augmented and amplified by technologies – such as speech, writing, and printing which aligned with profound socioeconomic shifts and paradigmatic transformation. Education and learning have been central and essential to human development, augmented by the use of technologies. During the last 100 years, psychologists, educators, anthropologists, and scientists from a variety of disciplines have identified many components of human learning. From the earliest Freudian theories and behaviorism to the twenty-first century theories of online learning, we have acquired a great deal of knowledge about the methods and environments through which human learning occurs.

4

Learning Theories: The Role of Epistemology, Science, and Technology

109

Epistemology: How we view knowledge

Scientific Thought

Metaphysical Thought

Belief

Scientific Objectivism

Scientific Constructivism

Behaviorist LT

Constructivist LT

Cognitivisit LT

Collaborativist LT

Connectivism LT

Fig. 6 Learning theories, technologies, and epistemologies

Throughout the invention of computer networking in 1969, email in 1970, computer conferencing in 1972, the public Internet in 1989, and the World Wide Web in 1993, online technologies have continued to be viewed as largely beneficent forces for humanity that have provided new avenues for learning. These are the forces that created the notion and practice of online communities of interest, of practice, of art, of learning, and many other social, cultural, and professional knowledge-sharing and knowledge-building communities. These are the forces that brought us new ways of working, computer-supported collaborative learning, computer-supported cooperative work, and many other network-related initiatives. The challenge for educators in the twenty-first century is to take the best of what is known about human learning and address the new issues posed by technology and the rise of artificial intelligence. Renowned physicist, Stephen Hawking, issued a powerful observation and a powerful call to action for humanity, one especially poignant and relevant to educators when he wrote: We are not going to stop making progress, or reverse it, so we have to recognize the dangers and control them. I’m an optimist, and I believe we can. . .. It’s important to ensure that these

110

L. Harasim

changes are heading in the right directions. In a democratic society, this means that everyone needs to have a basic understanding of science to make informed decisions about the future. (Griffin 2016)

Educators are one of the key channels for ensuring that “these changes are heading in the right directions” and that everyone has “a basic understanding of science to make informed decisions about the future” (ibid.).

References Allen, I. E., & Seaman, J. (2004). Entering the mainstream: The quality and extent of online education in the United States, 2003 and 2004. Needham, MA: The Sloan Consortium. Retrieved November 26, 2009, from www.sloan-c.org/publications/survey/pdf/entering_main stream.pdf Allen, I. E., & Seaman, J. (2007). Online nation: Five years of growth in online learning. Needham, MA: The Sloan Consortium. Retrieved November 26, 2009, from www.sloan-c. org/publications/survey/pdf/online_nation.pdf Allen, I. E., & Seaman, J. (2008). NASULGC–Sloan national commission on online learning benchmarking study: Preliminary findings. Needham, MA: The Sloan Consortium. Retrieved November 26, 2009, from www.sloan-c.org/publications/survey/nasulgc_prelim Andre, T., & Phye, G. D. (1986). Cognition, learning and education. In G. D. Phye & T. Andre (Eds.), Cognitive classroom learning. Orlando, FL: Academic Press. Bates, T. (2015). Is there a future in online learning? Online learning and distance education resources. Retrieved April 5, 2016, from Http://www.tonybates.ca/2015/09/07/is-there-afuture-in-online-learning/ Bates, T. (2016). Automation or empowerment: Online learning at the crossroads, Online learning and distance education resources. Retrieved April 5, 2016, from http://www.tonybates.ca/2016/ 01/11/automation-or-empowerment-online-learning-at-the-crossroads/#sthash.75h5FZnl.dpuf Bates, A. W., & Pool, G. (2003). Effective teaching with technology in higher education. Foundations for success. San Francisco, CA: Jossey-Bass. Bodner, G. M. (1986). Constructivism: A theory of knowledge. Journal of Chemical Education, 63, 873–878. Bredo, E. (2006). Conceptual confusion and educational psychology. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates. Bruffee, K. A. (1999). Collaborative learning: Higher education, interdependence, and the authority of knowledge (2nd ed.). Baltimore, MD: Johns Hopkins University Press. Bruner, J. S. (1962). Introduction. In L. S. Vygotsky (Ed.), Thought and language. Cambridge, MA: MIT Press. Cellan-Jones, R. (2014, December 2). Stephen Hawking warns artificial intelligence could end mankind. BBC Interview. Retrieved May 16, 2016, from http://www.bbc.com/news/technology30290540 Chomsky, N. (1959). A review of B. F. Skinner’s verbal behaviour. Language, 35(1), 26–58. Clarck, A. (2001). Mindware: An introduction to the philosophy of cognitive science. New York, NY: Oxford University Press. Dewey, J. (1896). The reflex arc concept in psychology. Psychological Review, 3, 357–370. Downes, S. (2007, March 29). What I’m working on, from Downes’ Blog. Retrieved January 19, 2016, from http://www.downes.ca/post/40879

4

Learning Theories: The Role of Epistemology, Science, and Technology

111

Downes, S. (2012). Creating the connectivist course. Retrieved January 6, 2012, from http://www. downes.ca/post/57750 Duffy, T. M., & Cunningham, D. J. (1996). Constructivism: Implications for the design and delivery of instruction. In D. H. Jonassen (Ed.), Handbook for research for educational communications and technology. New York, NY: Schuster Macmillan. Engelbart, D., & Lehtman, H. (1988). Working together. Byte, 13(13), 245–252. Farrell, M. P. (2001). Collaborative circles: Friendship dynamics & creative work. Chicago: University of Chicago Press. Gagne, R. M. (1985). The conditions of learning and theory of instruction. New York, NY: CBS College Publishing. Gagne, R. M., & Driscoll, M. P. (1988). Essentials of learning for instruction (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall. Gagne, R. M., & Medsker, K. (1996). The conditions of learning: Training applications. New York, NY: Harcourt Brace and Company. Gardner, H. (2008). Wrestling with Jean Piaget, my paragon. In What have you changed your mind about? Edge Foundation, Inc. Retrieved November 26, 2009, from www.edge.org/q2008/q08_ 1.html#gardner Gibbs, S. (2015, July 27). Musk, Wozniak and Hawking urge ban on warfare AI and autonomous weapons. The Guardian, UK. Greenfield, P. M. (1984). A theory of the teacher in the learning activities of everyday life. In B. Rogoff & J. Lave (Eds.), Everyday cognition (pp. 117–138). Cambridge, MA: Harvard University Press. Griffin, Andrew (2016). Stephen Hawking: Humanity is going to use science and technology to wipe itself out, professor warns, The Independent, January 19, 2016. Retrieved May 31, 2016 from http://www.independent.co.uk/news/science/stephen-hawking-humanity-a6821341.html. Hafner, K., & Lyon, M. (1996). Where wizards stay up late: The origins of the internet. New York, NY: Simon & Schuster Macmillan. Harasim, L. M. (1990). Online education: Perspectives on a new environment. New York, NY: Praeger. Harasim, L. M. (2002). What makes online learning communities successful? The role of collaborative learning in social and intellectual development. In C. Vrasidas & G. V. Glass (Eds.), Distance education and distributed learning (pp. 181–200). Charlotte, NC: Information Age Publishers. Harasim, L. (2004). Collaboration. In A. DeStafano, K. Rudestam, & R. Silverman (Eds.), Encyclopedia of distributed learning (pp. 65–68). Thousand Oaks, CA: SAGE Publications. Harasim, L. M. (2012). Learning theory and online technologies. New York, NY: Rutledge. Harasim, L. (2017). Learning theory and online technologies (2nd ed.). New York, NY: Rutledge. Harasim, L. & Smith, D.E. (1986). Final report on the ontario women educators’ computer research network. Toronto, ON: Federation of Women Teachers’ Association of Ontario (100 pp). Harasim, L., & Smith, D.E. (1994). Making connections, thinking change together: Women teachers and computer networks. In: Bourne, P. (ed.) Feminism and Education: A Canadian Perspective: Toronto ON: CWSE, OISE Harasim, L. M., Hiltz, S. R., Teles, L., & Turnoff, M. (1995). Learning networks: A field guide to teaching and learning online. Cambridge, MA: MIT Press. Hawking, S., Russel, S., Tegmark, M., & Wilczek, F. (2014, May 1). Stephen Hawking: ‘Transcendence looks at the implications of artificial intelligence – But are we taking AI seriously enough?’. The Independent. Retrieved May 17, 2016, from http://www.independent.co.uk/ news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligenc e-but-are-we-taking-9313474.html Hrdy, S. B. (2009). Mothers and others: The evolutionary origins of mutual understanding. Cambridge, MA: Harvard University Press.

112

L. Harasim

Interactive Educational Systems Design (IESD). (2009). National online survey of district technology directors exploring district use of Web 2.0 technologies. Retrieved January 31, 2010, from www.lightspeedsystems.com/researchsurvey Jonassen, D. H., Beissner, K., & Yacci, M. (1993). Structural knowledge: Techniques for representing, conveying, and acquiring structural knowledge. Hillsdale, NJ: Lawrence Erlbaum Associates. Kelly, H. (2015, December 12). Elon Musk and tech heavies invest $1 billion in artificial intelligence. CNN Money. Retrieved March 18, 2016, from http://money.cnn.com/2015/12/12/technol ogy/openai-elon-musk/index.html Kuhn, T. S. (1970). The structure of scientific revolutions. Chicago, IL: University of Chicago Press. Kurzweil, R. (2006). The singularity is near: When humans transcend biology. New York, NY: Viking Press. Lee, D. (2016, January 26). AI pioneer Marvin Minsky dies aged 88. BBC News Technology. http:// www.bbc.com/news/technology-35409119 Luckerson, V. (2014, December 2). 5 very smart people who think artificial intelligence could bring the apocalypse. Time Magazine. Retrieved March 18, 2016, from http://time.com/3614349/ artificial-intelligence-singularity-stephen-hawking-elon-musk/ Mazur, E. (2009). Education: Farewell, lecture? Science, 323(5910), 50–51. Minsky, M., & Papert, S. (1988). Perceptrons (2nd ed.). Cambridge, MA: MIT Press. Munari, A. (1994). Jean Piaget (1896–1980). Prospects: The quarterly review of comparative education, XXIV(1/2), 311–327. Retrieved November 26, 2009, from ww.ibe.unesco.org/ fileadmin/user_upload/archieve/publications/ThinkersPdf/piagate.PDF Nelson, T. H. (1974). Dream machines. South Bend, IN: The Distributors. Nelson, T. H. (1987). Literary machines. South Bend, IN: The Distributors. Papert, S. (1999, March). Papert on Piaget. Time Magazine – The Century’s Greatest Minds. Piaget, J. (1969). Science of education and the psychology of the child. New York, NY: Viking. Pressey, S. L. (1926). A simple apparatus which gives tests and scores – And teaches. School and Society, 23(586), 373–376. Santrock, J. W. (2008). A topical approach to life span development. New York, NY: McGraw-Hill. Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In K. Swyer (Ed.), Cambridge handbook of learning sciences (pp. 97–118). New York, NY: Cambridge University Press. Siemens, G. (2004). Connectivism: A learning theory for the digital age. Retrieved September 30, 2015, from http://www.elearnspace.org/Articles/connectivism.htm Siemens, G. (2006). Knowing knowledge, creative commons. Retrieved January 19, 2016 from http://www.elearnspace.org/KnowingKnowledge_LowRes.pdf. Siemens, G. (2011). From knowledge to bathroom renovations, July 3, 2011, from www. connectivism.ca (no longer available). Retrieved January 19, 2016, from http://www.scoop.it/ t/connectivism?q=Bathroom+renovations Siemens, G. (2015). Adios Ed Tech. Hola something else. Retrieved September 9, 2015, from http:// www.elearnspace.org/blog/2015/09/09/adios-ed-tech-hola-something-else/#comments) Skinner, B. F. (1948). Walden two. Indianapolis, IN: Hackett Publishing Company. Skinner, B. F. (1953). Science and human behavior. New York, NY: Macmillan. Skinner, B. F. (1964). New methods and new aims in teaching. New Scientist, 122, 483. Tomasello, M. (2014). A natural history of human thinking. Cambridge, MA: Harvard University Press. Vygotsky, L. S. (1962). Thought and language (E. Hanfmann, & G. Vakar, Ed. & Trans.) Cambridge, MA: MIT Press. Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158–177. Wilson, B. G. (1997). Thoughts on theory in educational technology. Educational Technology, 37 (1), 22–27.

4

Learning Theories: The Role of Epistemology, Science, and Technology

113

Winn, W., & Snyder, D. (1996). Cognitive perspectives in psychology. In D. H. Jonassen (Ed.), Handbook for research for educational communications and technology. New York, NY: Macmillan. Zuboff, S. (2016, May 3). The secrets of surveillance capitalism. Frankfurter Allgemeine Feuilleton. Retrieved May 30, 2016, from http://www.faz.net/aktuell/feuilleton/debatten/the-digitaldebate/shoshana-zuboff-secrets-of-surveillance-capitalism-14103616.html

Professor Linda Harasim is a renowned pioneer and practitioner in the field of online education who, in 1986, designed and taught the first-ever completely online university course. Over a 35 year career as a researcher and educator, she developed the Collaborativist Theory and Pedagogy of Learning (2017). In recent years, she has expanded her focus to promote Augmented Human Intelligence through a collaborativist approach to online instruction, warning against the dangers of a growing reliance upon Artificial Intelligence within the field of online education. Harasim has served as a senior consultant to many large-scale and international online education programs, and has published six books and numerous articles worldwide.

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating Opposing Paradigms in the Design of Personal Learning Systems Johannes C. Cronje

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Twenty-first-Century Learning Needs: Connection and Diversity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Rhizome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Principles of the Rhizome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heterogeneity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multiplicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-Signifying Rupture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cartography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Decalcomania . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rhizomatic Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning in a Hyper-connected World . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Techniques and Technologies for Blended Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Injection Quadrant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Construction Quadrant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Immersion Quadrant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Integration Quadrant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Putting It All Together: The Personal Learning Environment (PLE) . . . . . . . . . . . . . . . . . . . . . . . . . . What Are Personal Learning Environments and Why Should We Have Them? . . . . . . . . . . A Framework for Describing Personal Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

116 116 118 118 119 120 120 121 122 122 122 123 126 126 127 128 128 129 131 131 132 134 135

Abstract

In a world where Google gives us the answer before we have finished typing the question, what is left to learn? The emphasis on learning has shifted from the individual to the collective. It is not just the individual who learns it is the whole J. C. Cronje (*) Cape Peninsula University of Technology, Cape Town, South Africa e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_49

115

116

J. C. Cronje

system that learns. In considering how to facilitate such learning, direct instruction and constructivism are often seen as mutually exclusive poles in education. This chapter considers the integration of the two within the context of rhizomatic learning and then presents a framework for the selection of technologies with which learners can construct their own personal learning environments in an information-rich context. Keywords

Rhizome · Direct instruction · Constructivism · Integration · Personal learning environments

Introduction This chapter will first consider the learning needs of the twenty-first century and how they relate to the principles of Deleuze and Guattari’s (1987) rhizome theory in order to explore the emerging concept of rhizomatic learning. Thereafter it will consider a four-quadrant model for the design of integrated learning strategies and look the importance of personal learning environments in the context of rhizomatic, blended learning. Finally the principles of rhizomatic learning and the integrated design are combined for the selection of techniques and technologies for blended learning, from which personal learning environments can be designed. For the purpose of this chapter, learning will be defined as “becoming able to do something that we were not able to do before,” and teaching will be defined as “assisting people (or things) in being able to do things that they were not able to do before.” The aim of this chapter, then, is to arrive at a design framework for personalized learning.

Twenty-first-Century Learning Needs: Connection and Diversity The dawn of the twenty-first century has seen an unprecedented growth in connection and diversity. Thanks to almost ubiquitous Internet connectivity worldwide, anybody can learn from anybody else anywhere. Moreover learning is not limited to people learning from people. As we learn with our (mobile) devices, so our devices – that are attached to the cloud – learn from us, thus creating a learning network known as the semantic web, or Web 3.0 where data on the Web is managed in such a way that machines, through artificial intelligence, can actually understand and act upon it. We are already seeing evidence of this phenomenon when Google returns a different result based on who searches, what geographical location the search is launched from, what previous searches have been conducted by a particular user, what searches are being conducted by other users, etc. In a world where Google even completes the search phrase while the user is typing the question becomes

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating. . .

117

If Google knows the answer before we have even finished typing the question, then what is left to learn?

The fact that the Internet has connected everybody to everybody else, it has also created an unprecedented diversity of learners, and the “demands created by this diversity are exacerbated by the immense changes taking place in the twenty-first century, such as unprecedented richness of information and communication systems, unprecedented mobility, and the technological empowerment of ordinary people to create or find their own personal solutions in a just in time, just enough and just for me fashion. This leads to the conclusion that learning needs will be vastly different, individual and largely unpredicted and unpredictable” (Lian & Pineda, 2014, p. 1). Together with the richness of access and communication comes the rise of the social network where the connected community, rather than the press or the government, is the producer of opinion, and as a consequence the individual in the network becomes not just able to, but responsible for, developing an own opinion. Lian and Pineda argue further that this richness of information “runs the risk of being controlled by large corporations (or even governments) which will provide and, necessarily, filter, monitor, ration and otherwise potentially manipulate what the public is allowed to know, and how it is presented” (Lian & Pineda, 2014, pp. 7–8). They claim that as a result, “we are now encountering what might be called the generalized growth of a research mentality or even of community intelligence (Lian & Pineda, 2014, p. 7). By extension Dave Cormier (Cormier, 2008) argues that, in this connected world of learning, the community does not simply create the curriculum, the community is the curriculum. As the community learns we may see a rise in community intelligence, which indicates an “important shift in our intellectual arsenal from independent thinking/learning to interdependent thinking/learning: we are no longer alone in our efforts to learn, something that educationists have recognized increasingly: we learn best in groups” (Lian & Pineda, 2014, p. 8 their emphasis). This would signal the “opening of the academic world to the ‘ordinary’ person and the existence of a latent interest in society for education. In a nutshell, education is valued more than we might think, but it is not education in its traditional form” (Lian & Pineda, 2014, p. 12). Fundamental to our understanding of learning in the twenty-first century is the fact that in a networked learning world, it is not just the learners who learn, it is the network too. It is in this context that the term Learning 3.0 emerges. “Learning 3.0 will be a facet of an ongoing, limitless symbiotic relationship between human and machine” (Wheeler, 2012). Learning 3.0 may be defined as a type of learning where learners and machines become co-dependent, where machines are able to predict learner needs and act upon it at time of need and learners, by the very act of learning, create more data for machines to act upon. Where we are already used to the term “networked learning” to refer to collaborative learning usually mediated by the Internet, Learning 3.0 adds the dimension that the network itself is also learning. Thus the emphasis shifts from the individual learner to the ever-growing and ever-changing network of learning. The goal of learning is no longer just for the learner to be able to do new things. The goal is for

118

J. C. Cronje

the whole system to be more effective. For this to happen, there needs to be a whole focus of learning. Much of the current content that learners are taught may well be redundant, since they can obtain the information from the Internet, or the information may well be pushed to their devices even before they require it. It does, however, become necessary to teach learners how to make sense of all this new information so that they might develop their own personal learning environment (PLE). The paradigm shifts from a world in which information is scarce to one in which information is overabundant. In this way learning comes to resemble the growth of a rhizome plant: “Learning 3.0 will be user and machine generated, and will in all respects be represented in what I will call ‘rhizonomies’. The rhizonomic organisation of content will emerge from chaotic, multi-dimensional and multi-nodal organisation of content, giving rise to an infinite number of possibilities and choices for learners” (Wheeler, 2012). Rhizonomies and the concept of rhizomatic learning are based on Deleuze and Guattari’s (1987) rhizome theory.

The Rhizome The origin of the rhizome metaphor lies in botany, where “a rhizome is a subterranean stem of a plant, a creeping root stalk, which spreads laterally in multiple directions and surfaces to produce a clone of the original plant in an unexpected location” (Mackness, Bell, & Funes, 2015, p. 81). The rhizome is “a metaphor used to represent a dynamic, open-ended, self-adjusting personal learning network constructed by the learners themselves to meet perceived and actual needs” (Lian & Pineda, 2014, p. 22). It is presented in opposition to the “more traditional, arborescent modes of conceiving and understanding our world. The arborescent or tree-like view of reality tends to rely on hierarchical understandings of our world” (Tillmanns, Holland, Lorenzi, & McDonagh, 2014, p. 6). The absence of hierarchy in the rhizome is attractive in a connected world where “not only are mainstream dominant voices heard but, at least potentially, so are those of the intellectual fringe and of “little” people everywhere” (Lian & Pineda, 2014, p. 5). Cormier (2008) argues that the “The rhizome metaphor, which represents a critical leap in coping with the loss of a canon against which to compare, judge, and value knowledge, may be particularly apt as a model for disciplines on the bleeding edge where the canon is fluid and knowledge is a moving target” (Cormier, 2008). For the purpose of this chapter, the emphasis will be on the reduction of hierarchy. The existence of various emancipatory pedagogies is acknowledged but falls beyond the scope of this chapter.

Principles of the Rhizome Deleuze and Guattari (1987) identified six principles of the rhizome: connection, heterogeneity, multiplicity, a-signifying rupture, cartography, and decalcomania.

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating. . .

119

Table 1 Principles of the rhizome and their relevance to teaching and learning Principle Connection

Heterogeneity

Multiplicity

A-signifying rupture Cartography

Decalcomania

Explanation (Deleuze & Guattari, 1987) “A rhizome ceaslessly establishes connections between semiotic chains. . .” (p. 6) “There is no ideal speaker-listener, there is [no] homogeneous linguistic community” (p. 6) There is no unity to serve as a pivot in the object or to divide in the subject” (p. 7) “A rhizome may be broken [] but it will start up again on one of its old lines, or on new lines” (p. 8) “. . .[a] map that is always [] modifiable and has mutliple entryways and exits and its own lines of flight” (p. 22) “The tracing has [] translated the map into an image; it has already transformed the rhizome into roots and radicles” (p. 13)

Relevance to teaching and learning (Mackness et al., 2015) Encourage ceaseless connection and diversity in people, ideas, and resources. The system has no beginning or end and can be entered at any point Design is a-centered and antihierarchical. It allows for breakaway groups or individual learners to reorganize in locations of their choice

Learners create and follow selfselected, individual pathways and embrace uncertainty without attempts to predict learning outcomes

Synthesized from Mackness et al., 2015, pp. 82–83

These are explained briefly in Table 1, which is a synthesis of two infographics by Mackness et al. (2015). These six principles of the rhizome have very important connections to the general nature of teaching and learning in a Web 3.0 environment, as will be seen from the discussion below.

Connection The emergence of social media, as well as the cloud, which connects almost all devices – and thus by implication their users – to one another, means that we have never been so connected physically, emotionally, and educationally. In a sense social media are mirroring what is happening in the physical world where “learners typically rely on lunchtime discussions, student organizations, brown bag sessions and study groups for peer support and informal learning networks” (Dabbagh & Kitsantas, 2012, p. 4). Out of this connectivity in learning arises the learning theory of connectivism. George Siemens (2005, p. 5) defines connectivism as “the integration of principles explored by chaos, network, and complexity and self-organization theories.” He identifies the following eight principles:

120

J. C. Cronje

1. 2. 3. 4. 5. 6. 7.

Learning and knowledge rests in diversity of opinions. Learning is a process of connecting specialized nodes or information sources. Learning may reside in non-human appliances. Capacity to know more is more critical than what is currently known. Nurturing and maintaining connections is needed to facilitate continual learning. Ability to see connections between fields, ideas, and concepts is a core skill. Currency (accurate, up-to-date knowledge) is the intent of all connectivist learning activities. 8. Decision-making is itself a learning process. Choosing what to learn and the meaning of incoming information is seen through the lens of a shifting reality. While there is a right answer now, it may be wrong tomorrow due to alterations in the information climate affecting the decision. (Siemens, 2005, p. 6) Connectivism in a Learning 3.0 context thus implies that since it is not just humans who learn but also their appliances, we may well have a responsibility toward one another to make sure that our appliances do learn. An example of such a collaborative learning event can be seen in contemporary GPS navigation systems where the device not only tells individual users where they are, it also makes their location available to the system, and based on that information, the system can calculate the best route for each individual thus avoiding traffic jams or at least minimizing their effect.

Heterogeneity One of the key criteria of learning is that what is learnt should be transferable to a different context. Heterogeneity is about the celebration of difference and the ability to effect such transfer. Thus, a facilitator of learning “must be aware of the nature of skill acquisition, the heterogeneity of constituent skills involved and their underlying learning processes, [and] the need for transfer of acquired skills to new situations” (Van Merriënboer, 1997, p. 1). Furthermore heterogeneity of learning tasks leads to the reduction of rote responses. As early as 1966, Traub advocated the use of heterogeneous learning tasks, since “results suggest that the heterogeneous subtask problems were better because they reduced the probability of making stereotyped or omitted-response errors” (Traub, 1966, p. 54). Thus heterogeneity leads to creativity, since it allows for so many different perspectives and requires “instruction of the same teaching content in different ways” (Xiao-yan & Yu-xiang, 2014, p. 1).

Multiplicity In terms of the rhizome, Deleuze and Guattari (1987) point out that the multiple becomes the unit. A rhizome is by definition a collection of multiple linkages, and, because of the principle of decalcomania which will be discussed later, the rhizome

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating. . .

121

is connected also to other rhizomes, thus, possibly forming the one big rhizome that consists of the multiples of other rhizomes. In teaching and learning, James Levin and colleagues identify six types of multiplicity, “instructional media, instructional formats, student learning activities, assessment techniques, contexts for learning, and evaluation approaches” (Levin, Levin, & Waddoups, 1999, p. 256). These multiplicities can be broken down even further. Instructional media can be classified as technologies for communication, expression, inquiry, and construction. Instructional formats include large reading discussion groups, online office hours, simulations, whole class student presentations, electronic field trips, online reading and text books, as well as lectures. Student learning activities vary between learning to use technologies for participating in the online classroom and learning to implement technology into classrooms. Contexts of learning include learning in the online class context, learning by doing in the students’ everyday settings, learning by doing in simulated everyday settings, and learning in informal learning groups. Finally, multiple assessment techniques involve assessment by classmates, assessment by the professor, assessment of self, and assessment by a wider audience (Levin et al., 1999). In investigating the effect of these multiplicities, they demonstrated that “Multiplicity decreases efficiency in the short run, but encourages the development of powerful new learning and teaching environments in the longer term” (Levin et al., 1999, p. 269). In the face of such overwhelming multiplicity, the individual learner, of course, tends to disappear and needs to be taught how to survive as an individual in a world of multiples.

A-Signifying Rupture A-signifying rupture holds that when one piece of the rhizome is broken off, it can be made to grow elsewhere – much as a cutting of a ginger plant will produce another. By the same token, something that was learnt in one context should be able to be “broken off” and allowed to grow in another context. Transfer of learning “occurs when learning in one context enhances (positive transfer) or undermines (negative transfer) a related performance in another context” (Perkins & Salomon, 1992, p. 6452). It could be regarded as “the ultimate aim of teaching” (Bray, 1928, p. 443). Nevertheless transfer of learning seems to be hard to achieve and “students often fail to transfer what they have learned about one problem to a structurally similar problem” (Marini & Genereux, 1995, p. 1). A review of the literature has shown that teaching principles and concepts will be more likely to lead to transfer than will the rote learning of facts. Transfer does not usually occur naturally and often should be taught along with “self-monitoring practices and potential applications in varied contexts” (Billing, 2007, p. 483). Cooperative learning encourages explanation, and the generation of principles enhances transfer, specifically showing learners how learning that occurs in one domain influences another, and finally “Learning to use meta-cognitive strategies is especially important for transfer” (Billing, 2007, p. 483). Such metacognitive strategies as thinking aloud, using checklists, rubrics, and organizers, be overtly designed

122

J. C. Cronje

into a personal learning system so that the learners end up learning how to learn in ways that they had not learnt before.

Cartography The metaphor of learning as a journey is common. Along with that has been the concept of the path of learning as a map. The principle of cartography associated with rhizome theory is that learning is a map, and not a tracing (Deleuze & Guattari, 1987). Each learner should develop his or her own map of concepts and relationships. They should not simply trace that which is given by the teacher. This resonates with Paulo Freire’s emphasis on the development of students’ critical thinking about their educational situation, which allows them to “recognize connections between their individual problems and experiences and the social contexts in which they are embedded” (Freire, 1970). The value of concept maps “to help learners learn, researchers create new knowledge, administrators to better structure and manage organizations, writers to write, and evaluators assess learning” (Novak & Cañas, 2008, p. 31) has been well demonstrated. Of importance here, though, is that such concept maps should be generative and based on the learner’s critical understanding of the source and value of the individual connections in the maps. The maps should resonate with learners’ own individual experiences and contexts.

Decalcomania Bloom’s revised taxonomy (Anderson, Krathwohl, & Bloom, 2001) identifies evaluation and creation as the highest levels. Decalcomania refers to the creation of endless patterns as certain decals are repeated endlessly. Evaluation means being able to recognize patterns and predicting the consequences of such patterns. Web 3.0, the semantic web, is based upon recognizing patterns rather than looking for individual words. Similarly learning in a Web 3.0 environment would require of learners to look for patterns and trends rather than to rely on rote memorization of facts that can easily be found through a search engine.

Rhizomatic Learning Although the principles of the rhizome have been discussed sequentially and in isolation, it must be stressed that the rhizome does not work like that. It is specifically the whole interconnectedness that makes it such a good metaphor for learning. In a way it mimics the neural networks of the brain, and it mimics the way in which in any person’s own mind connections are made anyway. Often the “wrong” connections are also made, and, then again, the term “wrong” is dependent on who is doing the evaluating and from what perspective. Thus, what may be mathematically incorrect may make perfect sense in an artwork – or even in an ironic poem.

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating. . .

123

It is in the nature of the rhizome that all its principles are also inexorably connected to one another. A rhizomatic view of teaching and learning views education “as distributed, interconnected, co-constructed and emancipatory through educational processes involving critical consideration of the complex interplay of human and non-human entities” (Tillmanns et al., 2014, p. 6). Thus, without connection and multiplicity, there can be no decalcomania. Without heterogeneity there cannot be multiplicity, and without multiplicity no connection. A-signifying rupture is a cause of multiplicity and a result of a break in connectivity – while at the same time, it becomes the source of new connections, because “in the rhizomatic model of learning, curriculum is not driven by predefined inputs from experts; it is constructed and negotiated in real time by the contributions of those engaged in the learning process” (Cormier, 2008). As such the rhizome becomes the ultimate metaphor and a useful tool for describing and analyzing learning in a hyperconnected world.

Learning in a Hyper-connected World Rhizomatic learning may require a rethink of the traditional divide between the so-called objectivist and constructivist approaches to designing for learning that were so prevalent in the 1990s (Clark, Kirschner, & Sweller, 2012; Cooper, 1993; Dörr & Seel, 2014; Jonassen, 1991; Vrasidas, 2000). In the traditional dichotomous perspective at the objectivist end of the scale, Richard E. Clark and colleagues argue for direct instruction of novice learners (Clark et al., 2012), while at the other end, Kalyuga, Ayres, Chandler, and Sweller argue against it for experienced learners (Kalyuga, Ayres, Chandler, & Sweller, 2003). Thus it would seem that there might still be a tendency to place objectivism (or direct instruction) and constructivism (or problem-based learning and a host of other forms) as linear opposites. Nevertheless, as Richard E. Mayer points out, constructivism is a way of understanding how we learn, rather than a teaching strategy (Mayer, 2009); thus it may not be possible to plot them along a straight line. Furthermore there is increasing evidence of teaching and learning that occurs in both paradigms simultaneously. Jeroen van Merriënboer, for example, distinguishes between “learning processes that re/construct schemas (schema construction) and learning processes that automate these schemas (schema automation)” (Van Merriënboer, 2016, p. 15) and has identified “induction and elaboration as basic learning processes that re/construct cognitive shemas” (Van Merriënboer, 2016, p. 16). He also identifies “knowledge compilation and strengthening as basic learning processes that automate these cognitive schemas” (Van Merriënboer, 2016, p. 16). This move by van Merriënboer indicates a very clear move away from the dichotomous relationship between objectivism and constructivism toward a clearly complementary model. “4C/ID shows how an educational program can be designed in such a way that all four basic learning processes occur simultaneously in a process of complex learning and how, eventually, transfer of learning can be realized” (Van Merriënboer, 2016, p. 24).

124

J. C. Cronje

Renkl (2014) suggests four overlapping phases of instruction by combining learning from worked examples, observational learning, and analogical reasoning. The four overlapping phases are relying on analogs, forming declarative rules and fine-tuning automation and flexibilization. Since these phases involve both direct instruction and knowledge construction and given that they overlap, it becomes evident that they cannot be plotted on a straight line between objectivism and constructivism. These integrative models by van Merriënboer and Renkl are better suited to assist with our understanding of learning in the twenty-first century where “students are integrating social media in their academic experience both formally and informally. Furthermore, college faculty is increasingly using social media to support teaching and learning activities” (Dabbagh & Kitsantas, 2012, p. 4). In a similar vein, Tom Brown identifies “An emerging paradigm shift within management and information sciences suggests that the focus is shifting from knowledge management to sensemaking” (Brown, 2015, p. 230). There is a model that treats the two approaches to learning not as linear opposites but as two different dimensions of a four-quadrant orthogonal plane as shown in Fig. 1 (Cronje, 2000, 2006). Over time this model has gained some support since some more recent theories of teaching and learning have drawn on using both direct instruction and constructivist techniques simultaneously (Renkl, 2014; Van Merriënboer, 2012, 2016). The model begins with a point 0, which is low in both constructivist and objectivist characteristics. There is no cognitive scaffolding, no prompting and fading, and no predefined learning problem. There is also no clearly specified objective and to linear progression or control of learning. Although it would seem that no learning can occur under such circumstances, of course, much serendipitous learning occurs there. It is where babies learn to talk. Sometimes by direct instruction and sometimes by trial and error, but never programmed and curriculated. At the vertical extreme is where constructivism would exist in its purist form, and the horizontal extreme would be the domain of the classical behaviorists. Figure 1 shows the four-quadrant model that arises when one plots constructivism and objectivism at right angles. Four quadrants emerge that have been called construction, immersion, injection, and integration. The construction quadrant is the domain of “constructionism” and other forms of problem-based learning. The instruction quadrant is where programmed instruction, drill and practice, and other classical, linear models would resort. The immersion quadrant is the domain of experiential, incidental, and even accidental learning, while the integration quadrant is where learning would take place such as described by van Merriënboer and Renkl where, quite deliberately, the instructional designer has selected from both paradigms. As a part of his doctoral studies, Elander (2012) devised an instrument that tested the extent to which objectivist and constructivist elements were present in a given course and tested it on a sample of 214 instructional designers. The results, shown in Fig. 2, indicate that in fact the majority of courses showed some form of combination of the two paradigms, with a distinct bias toward the “integration” and “injection”

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating. . .

125

Constructivism 10 9 8

Construction

Integration

Immersion

Injection

7 6 5 4 3 2 1 0 0

2

4

6

8

10 Objectivism

Fig. 1 The four-quadrant model integrating constructivism and objectivism (Cronje, 2000, 2006)

Course Learning Approach Orientation 40

Integration

Construction

35

Constructivist

30 25 Instructors Instructional Designers

20

Course Developer

15

Others Legend for Multiples* Symbols

10

Shape

5 Injection

Immersion 0

0

5

10

15

20 25 Objectivist

30

35

40

Total# with the same combination

Red Circle Black Circle Black Square

2 3 4

Hexagon

5

*Multiples-refers to multiple responses on the same vector point

Fig. 2 Various courses plotted along the objectivist/constructivist matrix (Elander & Cronje, 2016; Elander, 2012)

126

J. C. Cronje

quadrant. If a course adhered to a single paradigm only, that course would have been plotted against either the horizontal axis (if it were purely objectivist) or against the vertical axis (if it were purely constructivist). The scattering of courses toward the integrated and objectivist quadrant provides some support for the rhizomatic nature of learning, as well as for the prevalent results of research on teaching and learning by van Merriënboer and Renkl. The low number of courses in the “immersion” quadrant can be explained by the fact that it is the quadrant of incidental and serendipitous learning and it is unlikely that course designers would deliberately design a course based primarily of the luck of the draw. If one were to analyze people’s day-to-day experiences though, it would be very possible that much of our everyday learning occurs in that quadrant. The low numbers in the “construction” quadrant can be explained by the fact that the courses under investigation were formal taught courses rather than workshops or studio sessions where the construction quadrant might have figured more prominently. In the following section, the four-quadrant model will be applied to make suggestions regarding the development of blended learning solutions in a rhizomatic context.

Techniques and Technologies for Blended Learning There are many definitions for blended learning, but most consider two dimensions – face to face as opposed to distance and minimum as opposed to maximum involvement of technology. For the purpose of this chapter, however, the blend will be defined as a blend between an objectivist and a constructivist approach to learning. The rationale for this decision is the so-called No Significant Difference Phenomenon (Russell, 1999), which argues that when all variables are controlled for, the medium of transmission in teaching subject matter does not translate to a difference in learning outcome. It is therefore more important to blend the approach to teaching and learning depending upon the outcome that is required than it is to blend the technology or the proximity of the instructor. In developing a personal learning environment, it may be useful to borrow from the domain of knowledge management, since essentially a personal learning environment aims at managing the process of acquiring knowledge, skills, and attitudes. The four-quadrant model described above resonates with Kurtz and Snowden’s (2003) Cynefin framework that identifies four domains of knowledge: complex, chaos, known, and knowable, as shown in Fig. 3. It could be argued that should learning outcomes be in a particular quadrant of the Cynefin framework, then the blend of learning techniques and technologies should be in the corresponding quadrant of the integrative model.

Injection Quadrant This quadrant corresponds with the “known” quadrant of the Cynefin framework, where cause and effect relationships have already been established. This is

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating. . .

COMPLEX - Cause and effect are only coherent in retrospect and do not repeat - Pattern management - Perspective filters - Complex adaptive systems - Probe-Sense-Respond

CHAOS - No cause and effect relationships perceivable - Stability-focused intervention - Enactment tools - Crisis management - Act-Sense-Respond

127

KNOWABLE - Cause and effect separated over time and space - Analytical/Reductionist - Scenarios planning - Systems thinking - Sense-Analyse-Respond

KNOWN - Cause and effect relationships repeatable, perceivable and predictable - Legitimate best practice - Standard operating procedures - Process reengineering - Sense-Categorise-Respond

Fig. 3 The Cynefin framework (Kurtz & Snowden, 2003, p. 468)

knowledge that is in the canon and where very little new is required of the learner other than to master the content. Much of our traditional “training” happens in this quadrant, and it has been shown that traditional instruction works best: “Evidence from controlled, experimental (a.k.a. “gold standard”) studies almost uniformly supports full and explicit instructional guidance rather than partial or minimal guidance for novice to intermediate learners” (Clark et al., 2012, p. 11). This is the quadrant where learners would add to their personal learning environments technologies such as drill and practice software, content sites such as YouTube and Khan Academy, as well as online game-based testing programs such as Kahoot and make a contribution.

Construction Quadrant This quadrant corresponds with the complex quadrant of Kurtz and Snowden. Cause and effect only becomes evident in retrospect and where we are likely to work with experienced learners. We need to work in this quadrant to avoid the “expertise reversal effect” which states that “instructional techniques that are highly effective with inexperienced learners can lose their effectiveness and even have negative consequences when used with more experienced learners” (Kalyuga et al., 2003, p. 23). This

128

J. C. Cronje

is because “The involvement of different (schema-based and instruction-based) cognitive constructs for dealing with the same units of information may consume sufficient resources to cause cognitive overload compared with instruction that relies more heavily on preexisting schemas for guidance” (Kalyuga et al., 2003, p. 24). To the personal learning environment, this quadrant adds collaborative programs such as Google Docs and AnswerGarden, as well as other shared whiteboards and notebooks are used. Of course it is also the domain of the currently popular Maker Movement.

Immersion Quadrant The immersion quadrant explains what happens when people are “thrown into the deep end.” It corresponds with the chaos quadrant of the Cynefin framework. Clark et al. warn against this type of learning in a formal classroom: “In real classrooms, several problems occur when different kinds of minimally guided instruction are used. First, often only the brightest and most well-prepared students make the discovery. Second, many students, as noted above, simply become frustrated. Some may disengage, others may copy whatever the brightest students are doing—either way, they are not actually discovering anything. Third, some students believe they have discovered the correct information or solution, but they are mistaken and so they learn a misconception that can interfere with later learning and problem solving. Even after being shown the right answer, a student is likely to recall his or her discovery—not the correction. Fourth, even in the unlikely event that a problem or project is devised that all students succeed in completing minimally guided instruction is much less efficient than explicit guidance. What can be taught directly in a 25-minute demonstration and discussion, followed by 15 minutes of independent practice with corrective feedback by a teacher, may take several class periods to learn via minimally guided projects and/or problem solving” (Clark et al., 2012, p. 8). “Formal learning is described as learning that is institutionally sponsored or highly structured, i.e., learning that happens in courses, classrooms, and schools, resulting in learners receiving grades, degrees, diplomas, and certificates, whereas informal learning is learning that rests primarily in the hands of the learner and happens through observation, trial and error, asking for help, conversing with others, listening to stories, reflecting on a day's events, or stimulated by general interests” (Dabbagh & Kitsantas, 2012, p. 4). For a personal learning environment, this is the domain of the Google search and the self-directed learner. It is also where people record their learning in blogs and on shared bookmarking sites such as Delicious.

Integration Quadrant The integration quadrant is particularly important, since “most learning experiences are a blend of both formal and informal learning” (Dabbagh & Kitsantas, 2012, p. 5).

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating. . .

129

This is because “cognitive activity can happen with or without behavioral activity, and behavioral activity does not in any way guarantee cognitive activity. In fact, the type of active cognitive processing that students need to engage in to “construct” knowledge can happen through reading a book, listening to a lecture, watching a teacher conduct an experiment while simultaneously describing what he or she is doing, etc” (Clark et al., 2012, p. 8). While working in the integration quadrant, designers should be mindful of how and why they select from direct instruction methods or self-discovery methods. In this respect it is useful to know that “more-skilled learners tend to learn more with less-guided instruction, but less-skilled learners tend to learn more with more-guided instruction” (Clark et al., 2012, p. 8). In designing a personal learning environment for the integration quadrant, while the rhizome explains the ubiquity of learning, it is still important that learners learn how to learn, since “in order for students to use Web 2.0 technologies as formal learning tools they need training” (Dabbagh & Kitsantas, 2012, p. 5). Bjork, Dunlosky, and Kornell present a few very good guidelines in the form of questions and answers that have been paraphrased and tabulated in Table 2. Even something as traditionally objectivist as drill and practice can fit well into the integration quadrant, as Benjamin Bloom calls for the development of “automaticity” in order to create economy of effort, rapidity, and accuracy and to allow that “other conscious brain functions may occur simultaneously with the automatic functions” (Bloom, 1986, p. 74). It would seem that automaticity is related to items stored in long-term memory, as Clark et al. point out “When dealing with previously learned, organized information stored in long-term memory, these limitations disappear. Since information can be brought back from long-term memory to working memory as needed, the 30-second limit of working memory becomes irrelevant. Similarly, there are no known limits to the amount of such information that can be brought into working memory from long-term memory” (Clark et al., 2012, p. 9). Or even more directly put “Automatic processing of schemas requires minimal working memory resources and allows problem solving to proceed with minimal effort” (Kalyuga et al., 2003, p. 24). A way of combining the two approaches lies in the worked example. “A worked example is just what it sounds like: a problem that has already been solved (or “worked out”) for which every step is fully explained and clearly shown; it constitutes the epitome of direct, explicit instruction” (Clark et al., 2012, p. 9). This is the quadrant of the learning management system and also of formal collaborative sites such as Academia.edu, Researchgate.net, and LinkedIn.

Consolidation Regardless of the quadrant in which a learning experience is cast, it is impossible to deliver a one-size-fits-all solution for all learners. Since there is such a high level of diversity – in learners – in what has to be learnt, in ways to learn it, and in resources and technologies, it is simply no longer possible to “batch process” information for

130

J. C. Cronje

Table 2 Useful questions for learning how to learn Question What is the format of the upcoming test?

I study by copying my notes. Is that a good idea?

Does cramming work?

I did so much worse than I expected. What happened?

How much time should I spend studying?

How should I study to get good grades and succeed in school?

Short answer You will do best if you assume the exam will require that you truly understand and can produce from memory, the sought-after information, whether doing so involves recalling facts, answering comprehension questions, or solving problems Verbatim copying is a passive process and not very effective Rewriting one’s notes, however, or reorganizing them, exercises active organizational and elaborative processing. Studying one’s notes and then trying to reproduce them without the notes being visible are another active process and take advantage of the learning benefits of retrieval practice If the student’s goal is merely to obtain enough information to pass (or even do well on) an upcoming test, then cramming may work fine. If, however, a student’s goal is to retain what they learn for a longer period of time (e.g., until they take a more advanced course on the same topic), cramming is very ineffective compared to other techniques. If good performance on an upcoming test and good long-term retention is the goal, then students should study ahead of time and space their learning sessions across days and then study the night before the exam Take a meaningful self-test without checking the answers until you are done. Only then can you be confident that you know the information (and even then, forgetting can still occur) Students cannot excel without both (a) studying effectively and (b) spending enough time doing so. Compounding the problem, it is difficult to monitor one’s own study time – because study sessions, even attending class, can include email, online shopping, social networks, YouTube, and so on Some strategies, such as self-testing and spacing of practice, do seem generally effective across a broad set of materials and contexts, but many strategies are not so broadly effective and will not always be useful. It makes sense to summarize what one is reading, for example, yet writing summaries does not always benefit learning and comprehension and is less effective for students who have difficulty writing summaries. Moreover, summarizing a physics problem set may not be appropriate. Studying with other students may be effective if done well (e.g., if students take turns testing one another and providing feedback), but certainly will not work well if such a session turns into a social event or one group member takes the lead and everyone else becomes a passive observer

Paraphrased from Bjork, Dunlosky, and Kornell, 2013, pp. 16.20–16.21

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating. . .

131

all learners. It thus becomes necessary for every individual learner to learn to create their own personal learning environment (PLE) and to realize that such an environment will be with them for life. The correspondence between Cronje’s four-quadrant model and Kurtz and Snowden’s (Kurtz & Snowden, 2003) knowledge management model is particularly useful to show the relationship between knowledge management and learning in the creation of personal learning environments because “in order to successfully leverage social media towards the creation of PLEs, students must acquire and apply a set of personal knowledge management (PKM) skills” (Dabbagh & Kitsantas, 2012, p. 5).

Putting It All Together: The Personal Learning Environment (PLE) To answer the original question: “If Google knows the answer before we have even finished typing the question, then what is left to learn?” it would seem then that learners have to be taught how to create personal learning environments (PLEs).

What Are Personal Learning Environments and Why Should We Have Them? PLEs have been defined as “tools, communities, and services that constitute the individual educational platforms that learners use to direct their own learning and pursue educational goals” (EDUCAUSE Learning Initiative (ELI), 2009, p. 1). A personal learning environment assists learners in creating histories of their own learning, which “enable us to function by helping to organize or categorize the world and, at the same time, limit us to what is contained in the categories we create. They literally tell us what is relevant and what is not. Most of all, from a learning perspective, they account for the countless variables which distinguish us from one another and make each of us truly unique and experience specific learning needs” (Lian & Pineda, 2014, p. 14 their emphasis). Although there are a number of commercial derivatives of learning management systems that purport to be personal learning environments, it is more likely that a personal learning environment will be different for each learner and will consist of their own amalgamation of software and hardware, like “experience- and resourcesharing tools such as Delicious, WordPress, and Twitter that enable online/social bookmarking, blogging, and microblogging; wiki software such as PBworks that enables the creation of collaborative workspaces; media sharing tools such as Flickr and YouTube that enable social tagging; social networking sites (SNS) such as Facebook and LinkedIn that enable social networking; and web-based (cloud-computing) office tools such as Google Apps that enable document and calendar sharing and editing among other things” (Dabbagh & Kitsantas, 2012, p. 3). These tools can be classified into three pedagogical levels: “(1) personal information management, (2) social interaction and collaboration, and (3) information aggregation and management” (Dabbagh & Kitsantas, 2012, p. 6).

132

J. C. Cronje

The necessity for personal learning environments arises from the complex nature of the twenty-first-century learning environment and the increasingly rhizomatic response to it. It becomes a new challenge for the instructional designer: “organizing learning resources available at a PLE into meaningful learning activities towards achieving set goals can as well be considered as an act of instructional design” (Türker & Zingel, 2008, p. 4) and as such “the PLE marks a fundamental change in the role resources (people and media) play in teaching and learning. In an environment where information is ubiquitous and needs only to be located, there is a greater premium on skills that support fast and accurate access to information and on the ability to assess that information. In this regard, teaching is less a matter of data transmission and more a collaborative exercise in collection, orchestration, remixing, and integration of data into knowledge building. The goal for the student shifts from a need to collect information to a need to draw connections from it—to acquire it, disseminate it, and collaborate in its use” (EDUCAUSE Learning Initiative (ELI), 2009, p. 2).

A Framework for Describing Personal Learning Environments A personal learning environment serves two purposes, to assist learners in seeking, assimilating, creating, and disseminating knowledge and to act as a record of such learning. Much as learning is an individual achievement, it is an increasingly social endeavor. One should bear in mind that in the context of Learning 3.0, it is not just the learner who learns, it is the network of people and machines to which the learner belongs that learns. The personal learning environment is important because it is a tool to tap into the rhizome while at the same time it helps the rhizome to grow. The more links a learners share on Delicious, for instance, the more likely they are to get followers who share things with them. In selecting technologies from which to assemble a personal learning environment, one might consider the most appropriate match between the relative teaching and learning quadrant (immersion, injection, construction, or integration) and the various principles of the rhizome. So, for instance, if a personal learning environment needs to accommodate “injection” type of learning from a perspective of multiplicity, then a learner might consider a host of sources of direct instruction such as Khan academy, Code academy, or Lynda.com. If on the other hand the learners need to collaborate in the “construction” quadrant, then the cameras on their mobile devices for sharing images on LinkedIn would be more appropriate tools. In the “immersion” quadrant, it would be more important for learners to bookmark the various experiences that they have had using a social bookmarking site such as Delicious, while in the “integration” quadrant, learners and their instructors together might want to select and discuss materials together in a format similar to a cMOOC. Table 3 provides a worked example of what such a design tool might look like. The table is not prescriptive. It simply shows how one might consider elements of the rhizome and map them onto one of the four quadrants of learning (immersion, injection, construction, and integration). One would then be able to address each

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating. . .

133

Table 3 Web 3.0 applications mapped onto a rhizomatic exploration of the four quadrants of learning Rhizome Connection

Immersion Random connections between new experiences and existing knowledge Social bookmarking (Delicious, DIIGO)

Heterogeneity

Every new experience is different and often unexpected Blogging and microblogging (Twitter, Facebook, Blogspot)

Multiplicity

Learning everywhere and all the time implies multiple contexts Delicious

a-signifying rupture

Each new experience is a rupture of existing knowledge structures and has the potential to lead to other new experiences Tumblr

Cartography

Map of learning drawn as the learning takes place Evernote

Injection Explicit connections to existing knowledge, based on Gagne’s events of instruction Learning Management System (Moodle, Blackboard) Deliberate application of knowledge in different contexts to enhance transfer Productivity software (Google docs) Multiple iterations of increasing complexity as learning progresses Khan academy Code Academy Linda.com Deliberate and controlled rupture to encourage transfer of learning Delicious

Tends to be a tracing rather than a map Visio

Construction Deliberately created “gaps” where the connections should be made in the problemsolving process Shared editing tools (Google Docs)

Integration Negotiated connections to obtain the “best fit” between learner, content and context Massive (or mini) open online courses (cMoocs)

Different perspectives, usually encouraged by cooperative learning and debate Blogs and virtual worlds

Different perspectives encouraged with similarities and differences pointed out and explained Closed online discussion groups Multiplicity first generated in the preparatory phases and then reduced to select the best solutions cMooc

Multiple resources provided and multiple solutions created LinkedIN Device camera

Construction based upon pieces that have already ruptured from somewhere else. Learners encouraged to break existing bonds Pinterest Vine Instagram Each map is unique – usually having the learner in the center C-map

“Inoculation” of new rhizome structures that are incubated during the learning process to encourage new growth Makerspaces Evernote

Various individual maps but with the same start and end points C-map (continued)

134

J. C. Cronje

Table 3 (continued) Rhizome

Decalcomania

Immersion C-Map Delicious Very high level of randomness in the generation of patterns. Little control over the accuracy of the pattern Pinterest

Injection

Construction

Integration

Clearly described patterns that are easy to recognize. Patterns usually explicitly taught Flickr

Reinforcement of existing patterns while at the same time creating opportunity for new experience Instagram

Comparison of new patterns with existing ones and the development of new “best practice” patterns Google photos

element of the rhizome from a particular quadrant and derive a rationale for learning. Once that has been done, one might consider a specific technological solution. The table is descriptive rather than prescriptive. A designer of a personal learning environment would first do an environmental scan and determine which principles of the rhizome are at play and what learning is happening in which quadrant, before matching appropriate solutions to each cell. Moreover, given the learning task and the learning outcomes, not all cells need to be filled.

Conclusion This chapter set out to show how an increase of diversity that has been brought about by increased connectivity as a result of Web 3.0 has led to the emergence of rhizome theory as a way to make sense of teaching and learning in the early twenty-first century. It has also indicated that the traditional divide between objectivism and constructivism no longer holds, since emerging new theories of teaching and learning, such as van Merriënboer’s 4C/ID and Renkl’s four-stage models, work across those paradigms simultaneously. A possible solution would be to develop a four-quadrant model of teaching and learning where, depending on the context, learning could take place by “immersion,” “injection,” “construction,” or “integration.” However, regardless of what mode of learning takes place, the principles of the rhizome and the increased diversity of learners have put increased pressure on individual learners to design and develop their own personal learning environments. Such environments, by virtue of their being personal, will have to be designed and constructed individually for each leaner, and it would be the learner’s personal responsibility to do that. In a world where Google knows the answer before we have even finished typing the question, it is still necessary for us to learn how to learn and to learn how to curate the tools with which we learn. Essentially learning becomes not so much about acquiring knowledge, skills, and attitudes but about designing, growing, and nurturing the network from which such learning will follow.

5

Twenty-First-Century Learning, Rhizome Theory, and Integrating. . .

135

References Anderson, L. W., Krathwohl, D. R., & Bloom, B. S. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Boston: Allyn & Bacon. Billing, D. (2007). Teaching for transfer of core/key skills in higher education: Cognitive skills. Higher Education, 53(4), 483–516. Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64(1), 417–444. https://doi.org/10.1146/annurev-psych113011-143823. Bloom, B. S. (1986). Automaticity: “The hands and feet of genius.” Educational Leadership, 43(5), 70–77. Bray, C. W. (1928). Transfer of learning. Journal of Experimental Psychology, 11(6), 443. Brown, T. H. (2015). Exploring new learning paradigms: A reflection on Barber, Donnelly, and Rizvi (2013): “An avalanche is coming: Higher education and the revolution ahead”. International Review of Research in Open and Distance Learning, 16(4), 227–234. Clark, R. E., Kirschner, P. A., & Sweller, J. (2012). Putting students on the path to learning the case for fully guided instruction. American Educator, 36(1), 6–11. Cooper, P. A. (1993). Paradigm shifts in designed instruction: From behaviorism to cognitivism to constructivism. Educational Technology, 33(5), 12–19. Cormier, D. (2008). Rhizomatic education: Community as curriculum. Innovate: Journal of Online Education, 4(5), 6. Retrieved from http://nsuworks.nova.edu/innovate/vol4/iss5/2 Cronje, J. C. (2000). Paradigms lost: Towards integrating objectivism and constructivism. ITForum. ITForum. Retrieved December 16, 2015, from http://itforum.coe.uga.edu/paper48/paper48.htm Cronje, J. C. (2006). Paradigms regained : Toward integrating objectivism and constructivism in instructional design and the learning sciences. Educational Technology Research and Development, 54(4), 387–416. Dabbagh, N., & Kitsantas, A. (2012). Personal Learning Environments, social media, and selfregulated learning: A natural formula for connecting formal and informal learning. The Internet and Higher Education, 15(1), 3–8. https://doi.org/10.1016/j.iheduc.2011.06.002. Deleuze, G., & Guattari, F. (1987). A thousand plateaus: Capitalism and schizophrenia. London: Athlone Press. Dörr, G., & Seel, N. M. (2014). Instructional delivery systems and multimedia environments. Instructional Design. International Perspectives, 2, 145–181. EDUCAUSE Learning Initiative (ELI). (2009). Personal Learning Environments. The seven things you should know about. . .. https://doi.org/10.1101/gr.10.4.516 Elander, K., & Cronje, J. C. (2016). Paradigms revisited: A quantitative investigation into a model to integrate objectivism and constructivism in instructional design. Educational Technology Research and Development, 64(3), 389–405. https://doi.org/10.1007/s11423-016-9424-y. Elander, K. R. (2012). Merging paradigms: The integration of objectivist and constructivist approaches in university settings. Minneapolis, MN: Capella University. Freire, P. (1970). Pedagogy of the oppressed (MB Ramos, Trans.). New York: Continuum, 2007. Jonassen, D. H. (1991). Objectivism versus constructivism: Do we need a new philosophical paradigm? Educational Technology Research and Development, 39(3), 5–14. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38(1), 23–31. https://doi.org/10.1207/S15326985EP3801_4. Kurtz, C. F., & Snowden, D. J. (2003). The new dynamics of strategy: Sense-making in a complex and complicated world. IBM Systems Journal, 42(3), 462–483. Levin, J., Levin, S. R., & Waddoups, G. (1999). Multiplicity in learning and teaching: A framework for developing innovative online education. Journal of Research on Computing in Education, 32(2), 256–269. Lian, A., & Pineda, M. V. (2014). Rhizomatic Learning: “As. . . When. . . and If. . .” A Strategy for the ASEAN Community in the 21. Beyond Words, 2(1), 1–28.

136

J. C. Cronje

Mackness, J., Bell, F., & Funes, M. (2015). The Rhizome: A problematic metaphor for teaching and learning in a MOOC. Australasian Journal of Educational Technology, forthcoming, 32(1), 78–91. https://doi.org/10.14742/ajet.v0i0.2486. Marini, A., & Genereux, R. (1995). The challenge of teaching for transfer. In A. McKeough, J. Lupart, & A. Marini (Eds.), Teaching for transfer: Fostering generalization in learning (pp. 1–19). New York: Routledge. Mayer, R. E. (2009). Constructivism as a theory of learning versus constructivism as a prescription for instruction. In S. Tobias & T. M. Duffy (Eds.), Constructivist instruction: Success or failure? (pp. 184–200). New York: Routledge/Taylor & Francis Group. Novak, J. D., & Cañas, A. J. (2008). The theory underlying concept maps and how to construct and use them. Technical Report IHMC CmapTools 2006-01 Rev 01-2008. Institute for Human and Machine Cognition. http://cmap.ihmc.us/docs/pdf/TheoryUnderlyingConceptMaps.pdf Perkins, D. N., & Salomon, G. (1992). Transfer of learning. International Encyclopedia of Education, 2, 6452–6457. Renkl, A. (2014). Toward an instructionally oriented theory of example-based learning. Cognitive Science, 38(1), 1–37. https://doi.org/10.1111/cogs.12086. Russell, T. L. (1999). The no significant difference phenomenon: A comparative research annotated bibliography on technology for distance education: As reported in 355 research reports, summaries and papers. Raleigh: North Carolina State University. Siemens, G. (2005). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2(1), 3–10. Tillmanns, T., Holland, C., Lorenzi, F., & McDonagh, P. (2014). Interplay of rhizome and education for sustainable development. Journal of Teacher Education for Sustainability, 16(2), 5–17. https://doi.org/10.2478/jtes-2014-0008. Traub, R. E. (1966). Importance of problem heterogeneity to programed instruction. Journal of Educational Psychology, 57(1), 54. Türker, M. A., & Zingel, S. (2008). Formative interfaces for scaffolding self-regulated learning in PLEs. Elearning Papers, 14(9), 1–15. Van Merriënboer, J. J. G. (1997). Training complex cognitive skills: A four-component instructional design model for technical training. Englewood Cliffs, New Jersey: Educational Technology. Van Merriënboer, J. J. G. (2012). Four-component instructional design. In Encyclopedia of the sciences of learning (pp. 1320–1322). Springer. Van Merriënboer, J. J. G. (2016). How people learn. The Wiley Handbook of Learning Technology, 15–34. Vrasidas, C. (2000). Constructivism versus objectivism: Implications for interaction, course design, and evaluation in distance education. International Journal of Educational Telecommunications, 6(4), 339–362. Wheeler, S. (2012). Next generation learning | Learning with “e”s. Learning with “e”s. Retrieved April 9, 2015, from http://steve-wheeler.blogspot.com/2012/11/next-generation-learning.html Xiao-yan, H., & Yu-xiang, Z. (2014). ON heterogeneity in instructional design. Journal of Tongling University, 3, 32.

Johannes C. Cronje is the Dean of the Faculty of Informatics and Design at the Cape Peninsula University of Technology. He started his career as a schoolmaster at Pretoria Boys High School, then became a Lecturer in Communication at Pretoria Technikon, and later a Professor of Computers in Education at the University of Pretoria. He holds two master’s degrees and a doctorate from the University of Pretoria and was visiting professor at universities in Norway, Finland, Sudan, Ethiopia, and Belgium. He has supervised more than 72 masters and 55 doctoral students and has published more than 45 academic articles and chapters in books.

6

Cognitive Load Theory: What We Learn and How We Learn John Sweller

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Categories of Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Biologically Primary Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Biologically Secondary Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Human Cognitive Architecture When Dealing with Biologically Secondary Information . . . . The Information Store Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Borrowing and Reorganizing Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Randomness as Genesis Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Narrow Limits of Change Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Environmental Organizing and Linking Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cognitive Load Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Intrinsic Cognitive Load . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Extraneous Cognitive Load . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Altering Element Interactivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cognitive Load Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Element Interactivity Effects Related to Worked Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Transient Information Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

138 139 139 140 141 141 142 142 142 142 143 143 144 145 146 146 148 150 151

Abstract

The information that humans acquire can be divided into two categories. One category, biologically primary knowledge, is largely generic in nature leading to generic cognitive skills. It is critically important, and so we have evolved to acquire such skills without explicit tuition or conscious thought. The other

J. Sweller (*) School of Education, University of New South Wales, Sydney, NSW, Australia e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_50

137

138

J. Sweller

category, biologically secondary knowledge, is largely domain specific, leading to domain-specific concepts and skills. This category consists of cultural knowledge that we are able to acquire but without the specific acquisition mechanisms of primary knowledge. Biologically secondary knowledge is the subject of almost all teaching and learning in educational contexts. Because we have not evolved to specifically acquire this knowledge, it is best acquired with explicit instruction and conscious effort. Cognitive load theory uses evolutionary educational psychology to determine the cognitive processes needed to acquire biologically secondary knowledge and the instructional procedures that, in accord with those cognitive processes, best facilitate learning. This chapter describes the theory and some of the more recent instructional procedures developed using the theory. Keywords

Cognitive load theory · Evolutionary educational psychology · Cognitive processes and instructional design

Introduction Consider a person attempting to understand a difficult prose passage or understand and solve a difficult mathematics problem. What does the person need to learn to handle these and similar tasks, and how should we organize the learning environment to facilitate learning? Commonly assumed answers to the two questions, What do we learn? and How do we learn?, are that in educational contexts, what we learn is similar in structure and function to what we learn in the external, natural environment, and so we should teach in a similar, natural manner rather than the somewhat artificial environment found in many classrooms. Intuitively, the argument makes sense and seems to be implicit in many instructional design recommendations. Furthermore, with the advancing sophistication of educational technology, it has become easier to provide instructional contexts that mimic natural, non-educational environments. We can emphasize video and animation instead of static graphics, speech instead of written material, and interaction instead of a unidirectional presentation of information. Nevertheless, in this chapter, I will suggest and provide data indicating that many of the commonly accepted assumptions that underlie the naturalistic argument are profoundly misdirected. I will suggest that what we learn outside of educational contexts is very different to what we learn within educational contexts and that accordingly, how that information should be taught is similarly different. My argument is based on evolutionary educational psychology that provides assumptions concerning human cognitive architecture. Those assumptions lead to cognitive load theory, and in turn, that theory provides potential answers to questions associated with what and how we learn. I will begin by considering categories of knowledge.

6

Cognitive Load Theory: What We Learn and How We Learn

139

Categories of Knowledge Knowledge can be categorized in a vast variety of ways, but categories that matter in an educational context usually are ones that require different instructional procedures for each category. If two or more categories of knowledge require the same instructional procedures, then from an instructional perspective, they can be treated as a single category. One categorization scheme that has deep instructional consequences was provided by Geary (2005, 2007, 2008, 2012). Within an evolutionary educational psychology context, he distinguished between biologically primary and biologically secondary knowledge and skills.

Biologically Primary Knowledge Biologically primary knowledge is knowledge we have evolved to acquire. Examples are learning to listen and speak, learning to recognize faces, and learning to use general problem-solving strategies. This category of knowledge has several important characteristics. Primary knowledge is modular. The skills required may be unrelated to each other, and we may have evolved to acquire them during different evolutionary epochs. For example, our ability to listen and speak is likely to be unrelated to our ability to recognize faces, with both evolving independently. While we have evolved to acquire and use primary knowledge, that knowledge can be generalized to a variety of contexts. We have evolved to learn to listen and speak a native language, but that skill applies to any native language. We will learn to listen to and speak the language of our culture irrespective of the characteristics of that language. Primary knowledge is adapted to local conditions. Because we have evolved to acquire biologically primary knowledge, it is acquired automatically, without tuition and without conscious effort. For example, we do not need to teach most children how to organize their tongues, lips, breath, and voice in order to speak their native language. Most children will acquire this immensely complex skill unconsciously and without tuition merely by hearing others speak. We have evolved to acquire a native language. Primary knowledge overlaps very heavily with generic cognitive knowledge and skills (Tricot & Sweller, 2014). Our most important skills are generic cognitive skills, and because of their importance, we have specifically evolved to acquire them. Many generic cognitive skills are far too important to be left to the vagaries of a sole reliance on environmental conditions for their acquisition. We need to be primed to acquire them by an evolutionary impetus. The general problem-solving strategy, means-ends analysis (Newell & Simon, 1972), provides an example. This problemsolving strategy is general in the sense that it can be used to solve any problem that involves the transformation of problem states into other problem states. An algebra problem such as a/b = c, solve for a, provides an example, as does any maze-type problem. The strategy requires problem solvers to find differences between their current problem state (e.g., a/b = c) and the goal state (a = ?) and then find

140

J. Sweller

problem-solving operators (the rules of algebra) that will reduce the difference between the two states. In a recursive process, once a new state has been obtained, the process is repeated until the goal is reached. I know of no evidence that this means-ends strategy is teachable. We certainly do need to learn it, but we all do learn it automatically and without tuition. It is unteachable because even as young children, we have already learned it. We have evolved to learn generic cognitive strategies such as the use of a means-ends strategy because of their critical importance. This importance has led many educational theorists to gravitate toward studying generic cognitive strategies with a heavy emphasis on processes such as cognitive and metacognitive skills. We certainly can teach learners that a generic cognitive skill is relevant to a new domain (YoussefShalala, Ayres, Schubert, & Sweller, 2014), but that is not the same as teaching the skill itself. It is difficult to find evidence that teaching learners how to use a generic cognitive skill results in improved performance, as opposed to teaching them that an already learned skill is relevant in a particular domain. Once we know they are needed in a given context, we are very good at using generic cognitive skills. In that sense, they cannot be taught. Their importance results in our having evolved to acquire them unconsciously as biologically primary knowledge.

Biologically Secondary Knowledge Secondary knowledge is knowledge we need to acquire for cultural reasons. Examples are learning to read and write and most topics taught in educational institutions. We developed educational institutions precisely because they deal with information that is not readily acquired in the outside world. It is not learned in the same manner as biologically primary knowledge because we have not evolved to acquire specific varieties of secondary knowledge. We are able to acquire biologically secondary knowledge, but the conditions under which that knowledge is acquired are very different from the conditions under which biologically primary knowledge is acquired. All subject domain areas have similar biologically secondary characteristics unlike biologically primary knowledge whose characteristics differ between domains. One of the characteristics of secondary knowledge is that it tends to be domain specific rather than consisting of generic cognitive skills. As indicated above, it is possible that all generic cognitive skills are biologically primary because of their importance. In contrast, most of the knowledge acquired in educational contexts is domain specific. For example, we have not evolved to acquire the domain-specific knowledge that the translation of the English word “dog” is the French word “chien,” that the Treaty of Versailles constituted one of the causes of the Second World War, or that to solve the problem, (a + b)/c = d, solve for a, the first step is to multiply out the denominator on the left side. Such valuable knowledge is largely useless outside of its own domain. It is very different from generic cognitive skills such as knowing how to generalize from the solution of one problem to another, similar problem. We can survive as humans without the domain-specific knowledge taught in educational establishments, but we cannot survive as humans

6

Cognitive Load Theory: What We Learn and How We Learn

141

without generic cognitive skills such as knowing how to generalize. Accordingly, we have not specifically evolved to acquire each of the biologically secondary, domainspecific skills taught in educational establishments, but we have evolved to acquire the far more important, biologically primary, generic cognitive skills. Unlike biologically primary knowledge, biologically secondary knowledge consists of information that needs to be explicitly taught (Kirschner, Sweller, & Clark, 2006; Klahr & Nigam, 2004; Mayer, 2004) and consciously learned. As a consequence, while attempts to teach general problem-solving strategies such as meansends analysis may be impossible because we all know how to use the strategy without tuition, explicitly teaching domain-specific strategies such as how to multiply out a denominator when solving an algebraic equation may be essential. Unlike primary knowledge, if secondary knowledge is not explicitly taught, the vast majority of any population will fail to acquire it. As an example, humans devised writing and learned how to read that writing several thousand years ago, but until the advent of mass education a little over a hundred years ago, the vast majority of people never learned to read or write. People do not learn to read and write in the same manner that they learn to listen and speak, simply by immersion in a reading and writing society. They need to be explicitly taught. Organizing instruction of biologically secondary knowledge with the expectation that it will be automatically assimilated by immersion in order to make it more “natural” is likely to result in failure. Listening and speaking are natural activities, while reading and writing are not and need to be explicitly taught and learned in a vastly different manner to learning to listen and speak. The cognitive architecture associated with biologically secondary information will be discussed next.

Human Cognitive Architecture When Dealing with Biologically Secondary Information Biologically secondary information is processed according to a set of cognitive processes that constitute a cognitive architecture. The information processing rules of that architecture are analogous to the information processing rules of biological evolution. Both are examples of natural information processing systems (Sweller & Sweller, 2006). The suggestion that human cognition and evolution by natural selection are analogous can be traced back to Darwin (1871/2003) and more recently Campbell (1960) and Popper (1979), among others. There are many ways of expressing that analogy (Sweller, 2003). One way is in terms of five basic principles.

The Information Store Principle Long-term memory is able to store very large amounts of information (Simon & Gilmartin, 1973). Analogously, genomes also store large amounts of information. Both act as a massive information store.

142

J. Sweller

The Borrowing and Reorganizing Principle We obtain most of the information stored in long-term memory from other people by imitating what they do (Bandura, 1986), listening to what they say, and reading what they write. The obtained information is usually reorganized by combination with previously stored information. We are motivated to do so. The evolutionary analogue is sexual reproduction under which genetic information is obtained from ancestors after reorganization.

The Randomness as Genesis Principle While we can obtain information from others using the borrowing and reorganizing principle, that information must be created in the first instance. Novel information is created during problem-solving by generating moves that are random with respect to the goals of the problem followed by tests of effectiveness. If we are faced with a problem state for which we have no knowledge that will assist us in choosing a move, we have no choice but to randomly choose a move and test it for effectiveness. Effective moves are retained, while ineffective moves are jettisoned. Random mutation provides a genetic analogue.

The Narrow Limits of Change Principle The randomness as genesis principle has structural consequences. There are only six permutations of three elements but over 3.5 million permutations of 10 elements. Dealing with more than a very small number of novel elements at a time is difficult and may be impossible. Accordingly, working memory when dealing with novel elements is extremely limited in both duration (Peterson & Peterson, 1959) and capacity (Miller, 1956) resulting in the addition of new information to long-term memory being slow and incremental. In that way, the fidelity of long-term memory can be protected as it is changed. The epigenetic system (Jablonka & Lamb, 2005; West-Eberhard, 2003) acts as a link between the genetic system and the external world in the same manner as working memory acts as a link between long-term memory and the external world. The epigenetic system can increase or decrease mutations, but the rate of effective genetic change is slow and incremental thus protecting a successful genome in the same way as the rate of change of long-term memory is slow and incremental to protect its contents.

The Environmental Organizing and Linking Principle This principle provides purpose to the preceding principles. Environmental signals allow the transfer of unlimited amounts of appropriate, organized, stored information from long-term to working memory. Once in working memory, that information can

6

Cognitive Load Theory: What We Learn and How We Learn

143

generate action that is appropriate to the environment. The limits to the amount of information that can be transferred to working memory from the environment disappear when organized information is transferred from long-term to working memory. There are no known limits to either the amount of information that can be transferred to working memory or how long it can be held in working memory. Similarly, the epigenetic system switches genes on or off depending on environmental signals. Again, there are no known limits to how much information can be switched on to determine action resulting in phenotypical structures.

Cognitive Load Theory Cognitive load theory uses this cognitive architecture to generate instructional procedures. The concept of element interactivity is central to the theory. Element interactivity provides an estimate of the cognitive complexity of information that learners must deal with when acquiring information. Through the narrow limits of change principle, working memory is particularly sensitive to the complexity of information to be stored in long-term memory. Whether using the borrowing and reorganizing principle or the randomness as genesis principle, the limitations of working memory prevent large amounts of novel, biologically secondary information from being transferred to long-term memory at any given time. While that mechanism protects the fidelity of long-term memory, it also means that instructional procedures must be calibrated to ensure that they do not require impossibly large amounts of information to be assimilated at any given moment. That requirement can be difficult to meet because due to element interactivity, some learning tasks under some circumstances include more elements of novel information that can be simultaneously assimilated (Pollock, Chandler, & Sweller, 2002). There are two sources of element interactivity associated with information that learners must process. One source is intrinsic to the information and is referred to as intrinsic cognitive load, while the other source concerns the manner in which the information is presented and is referred to as extraneous cognitive load. A third source that is sometimes identified, germane cognitive load (Sweller, van Merrienboer, & Paas, 1998), refers to the cognitive load associated with acquiring information. That form of cognitive load can be associated with intrinsic cognitive load by assuming that it refers to the mental resources required to deal with intrinsic cognitive load rather than as an independent source of cognitive load (Sweller, 2010). That formulation will be used in the current treatment.

Intrinsic Cognitive Load Consider a task such as learning the vocabulary of a second language or learning the symbols of the chemical periodic table. While such tasks are difficult, they do not impose a heavy working memory load. Each element can be assimilated easily without reference to any other element because the elements do not interact. For

144

J. Sweller

example, we can learn the symbol for copper independently of learning the symbol for iron or the translation of the word “cat” independently of the translation of the word “dog” into another language. The elements do not interact and so are referred to as low element interactivity tasks. Some low element interactivity tasks such as these are difficult and may take years to learn to perform because there are many elements that need to be assimilated. The difficulty of such tasks does not reside in the number of interacting elements that need to be assimilated simultaneously. While these tasks have many elements, very few need to be assimilated simultaneously. Instead, they can be assimilated serially over long periods of time without a heavy working memory load. For such tasks, element interactivity is low, and so the intrinsic working memory (or cognitive) load imposed is low. Element interactivity and intrinsic working memory load are high when in order to understand and learn to perform a task, multiple elements must be processed simultaneously. Learning to solve algebraic problems such as (a + b)/c = d, solve for a, provides an example. No change can be made to any part of the equation without simultaneously considering the entire equation including all or most of its elements as well as considering whether the new equation that is generated by the change is useful in reaching the goal of the problem. The number of elements that must be processed when learning to solve this problem is a small fraction of the number of elements that must be processed when learning the translation of words in a foreign language, but the number that must be processed simultaneously is vastly greater. As a consequence, learning to solve algebraic equation problems is difficult for an entirely different reason to learning the translation of words in a foreign language. In the case of learning the translation of some of the words of a foreign language, there are many elements, but many of the elements can be learned independently of each other, while in the case of learning to solve algebra equation problems, there are far fewer elements in total but far more elements that must be processed simultaneously because they interact. Learning to solve algebra equation problems is high in element interactivity and so imposes a high intrinsic working memory load.

Extraneous Cognitive Load Element interactivity also determines extraneous as well as intrinsic working memory load. Extraneous working memory load is increased when instructional procedures unnecessarily increase element interactivity. For example, levels of instructional guidance can determine extraneous working memory load. If learners are provided with a solution to the above algebra problem, element interactivity is decreased compared to learners who must generate their own solution. To find a solution using means-ends analysis, learners must generate a variety of possible moves at each choice point such as multiplying out the denominator on the left side of the equation or attempting to subtract the addend instead. If knowledge held in long-term memory is unavailable, the randomness as genesis principle can be used to generate moves. The consequences of possible moves must be compared with the

6

Cognitive Load Theory: What We Learn and How We Learn

145

move that results in either attainment of the goal or in a problem state that is closer to the goal chosen before the process is repeated from the new problem state. For novices, a large number of elements must be simultaneously processed in working memory imposing a heavy cognitive load. If instruction provides the solution as a worked example for learners to study, there is a large reduction in element interactivity. Rather than using the randomness as genesis principle to generate moves, learners can use the borrowing and reorganizing principle, obtaining knowledge from someone else. Instead of generating and comparing alternative moves, what all learners must do is consider and make sure they understand the move that is presented at each step in the worked example. The reduction in element interactivity reduces extraneous cognitive load and permits more working memory resources to be devoted to transferring information from working to long-term memory, thus facilitating learning.

Altering Element Interactivity As can be seen from the above analysis, the difficulty of a learning task is determined by two independent factors: the total number of elements that must be processed and the total number of elements that must be processed simultaneously because they interact. These two factors should not be confused because they have vastly different instructional implications. Cognitive load theory only is concerned with the consequences of altering element interactivity or the number of elements that must be considered simultaneously. It is not concerned with alterations in the total number of elements. The extent to which element interactivity can be altered depends on whether its source is intrinsic to the task or extraneous. Intrinsic cognitive load, as the name implies, is intrinsic to the task, and so for a given task presented to an individual with a given level of expertise, it cannot be changed. Changing the task may change intrinsic cognitive load as well as changing the learner to one with a different level of expertise. With respect to changes in levels of expertise, solving the above algebra problem may impose a heavy cognitive load on a novice because of the large number of interacting elements. Solving the same problem for an expert may constitute a trivial task. For an expert, both the problem and its solution will have been stored in longterm memory, possibly as a single element. Using the environmental organizing and linking principle, all of this biologically secondary, domain-specific information can be transferred from long-term to working memory providing an immediate solution. In effect, an expert may use the environmental organizing and linking principle to reduce element interactivity and so reduce the resources required by working memory to process the information. A novice cannot similarly reduce element interactivity and working memory load using the environmental organizing and linking principle because a novice, by definition, has not stored appropriate knowledge in long-term memory. In this manner, expertise has a dramatic effect on element interactivity and intrinsic cognitive load.

146

J. Sweller

While element interactivity sourced from intrinsic cognitive load cannot be changed other than by changing the task or the person interacting with the task, element interactivity associated with extraneous cognitive load can be readily changed by changing instructional procedures. Indeed, the major purpose of cognitive load theory has been to devise instructional procedures that reduce extraneous cognitive load. Instruction interacts to some extent with intrinsic cognitive load in that the tasks presented to learners should optimize intrinsic load, but within a cognitive load theory framework, the major purpose of instructional manipulations is to reduce the element interactivity associated with extraneous cognitive load.

Cognitive Load Effects Cognitive load theory has been used to generate a variety of cognitive load effects. Each effect is based on randomized, controlled trials comparing an instructional design derived from the theory to more conventional, currently used instruction. As is essential when running such trials, each experiment ensures that only one variable is altered at a time for each factor (Hsu, Kalyuga, & Sweller,. 2015). When an instructional procedure generated from cognitive load theory is repeatedly superior to conventional alternatives, a cognitive load effect is demonstrated. The worked example effect, demonstrating that studying worked examples is superior to solving the equivalent problems and discussed above, provides the most commonly replicated cognitive load effect. Summaries of many other effects can be found in Sweller (2010, 2012); Sweller, Ayres, and Kalyuga (2011). Very recent work on the worked example effect and element interactivityy will be discussed here followed by a summary of the transient information effect that is particularly relevant to computer-based instruction.

Element Interactivity Effects Related to Worked Examples While the worked example effect has been studied and replicated over many years, another effect, the generation effect, had been studied outside of a cognitive load theory context for an even longer period (Slamecka & Graf, 1978). The two effects seemed to have diametrically opposed findings. While the worked example effect indicated that superior performance could be obtained by providing learners with problem solutions compared to having them generate the solutions themselves, the generation effect indicated that the generation of responses resulted in more learning than providing learners with appropriate responses. Both the worked example and the generation effect each were supported by a very substantial literature consisting of dozens of well-run experiments from a variety of researchers from around the globe. Neither group seemed to be aware of the other with limited or no crossreferencing of the opposing findings. Most researchers were working in independent silos for several decades.

6

Cognitive Load Theory: What We Learn and How We Learn

147

The concept of element interactivity has the potential to provide a resolution to the paradox (Chen, Kalyuga, & Sweller, 2015). The worked example effect was based on relatively complex, high element interactivity tasks found in common, educational contexts. Mathematics, science, and technology problems predominated. For such tasks, with a high intrinsic cognitive load, it was important to reduce the extraneous cognitive load because the combination of a high intrinsic and high extraneous cognitive load ran the risk of overwhelming working memory. The use of instructional guidance via worked examples accomplished this aim. In contrast to the worked example effect literature that derived from cognitive load theory and educationally relevant tasks, the genesis of the generation effect lays in the experimental investigation of the human memory system. Much of that literature relied on studying memory phenomena using word lists. For example, one group of participants might be presented a list of paired associates consisting of words and their opposites (e.g., hot-cold, tall-short, etc.), while the other group might be presented the first word of each pair and asked to generate the opposite themselves. Both groups would then be tested on their memory of the second word of each pair by being asked to list as many of those words as they could. The typical result was that the generation group was able to list more words than the presentation group. Chen et al. (2015) tested the hypothesis that the generation effect was more likely to be obtained using low element interactivity information, while the worked example effect was more likely using high element interactivity information. In two phases of instruction, learners were first asked to memorize several geometric formulae such as the area of a parallelogram is equal to the length of its base multiplied by its height (A = BxH). Half of the participants were presented the name of the formula and the actual formula twice (the presentation group), while the other half were presented the same information once followed by the name of the formula alone with a request to generate the formula themselves (the generation group). During a test, learners were asked to reproduce as many of the formulae as they could. This first phase was a test of the generation effect. In the second phase, learners were required to solve problems using the previously memorized formulae. Half were presented worked examples to study, while the other half were presented problems to solve. In the test of this phase, learners were given problems to solve. This second phase was a test of the worked example effect. The results indicated a conventional generation effect for the first phase with generation group superiority and a conventional worked example effect for the second phase with worked example group superiority. In a second experiment, Chen et al. (2015) provided additional confirmation that the contrary results of the first and second phases were due to differences in element interactivity. As indicated above, element interactivity not only depends on the characteristics of the information being processed but also on the levels of expertise of the learners. If more expert learners are used, element interactivity should be reduced due to the environmental organizing and linking principle, and the worked example effect found in the second phase of the first experiment should be reversed, resulting in a generation effect in both phases. Chen et al. obtained that result.

148

J. Sweller

The reversal of the worked example effect with increases in expertise provides an instance of the expertise reversal effect, another cognitive load effect (Kalyuga, Ayres, Chandler, & Sweller, 2003). It is an effect directly due to the environmental organizing and linking principle and the consequences of that principle for element interactivity. Cognitive load effects generally are only obtainable using novices because they rely on high levels of element interactivity and novices are more likely to experience high levels of element interactivity than more expert learners. Accordingly, as expertise increases, most cognitive load effects first decrease, and with further increases in expertise, they are likely to disappear and then reverse. Within cognitive load theory, the explanation for this reversal is usually attributed to another cognitive load effect, the redundancy effect (Chandler & Sweller, 1991). This effect occurs when learners are provided information that they do not need. Processing unnecessary information may increase extraneous cognitive load. In the case of the worked example effect, novices need to study worked examples in order to learn how to solve classes of problems. More expert learners do not need to study worked examples although they may need to continue practicing solving the problems. Studying redundant worked examples may increase cognitive load above solving the problems for more expert learners (Kalyuga, Chandler, Tuovinen, & Sweller, 2001). When comparing novices and more expert learners on the same material, the result is the expertise reversal effect. For more expert learners, generating a problem solution results in superior performance to studying a worked example, in line with the generation effect. The generation effect is not usually explained in cognitive load theory terms. It may be explained by the redundancy effect as above by assuming that learners presented information that they do not need such as indicating that the opposite of “hot” is “cold” find it redundant and learn less than if they generate the opposite word themselves. Nevertheless, there are many alternative explanations of the generation effect, and at this point it is not clear which should be preferred.

The Transient Information Effect The transient information effect (Leahy & Sweller, 2011; Wong, Leahy, Marcus, & Sweller, 2012) is another recently discovered effect that is particularly important to anyone using information technology for educational purposes. It occurs when information that can be presented in permanent form is transformed into a transient form. Transient information is particularly susceptible to working memory limitations. The transient information effect was discovered via the modality effect. Working memory can be divided into multiple processors (Baddeley, 1999) with partially separate processors for handling auditory and visual information. Potentially, the use of both processors for different information may increase the total capacity of working memory. Consider someone studying a diagram and its associated written text. The visual processor must be used to process both the textual and diagrammatic information, although the textual information may subsequently be transformed into and processed as natural, linguistic information in the same way as spoken information. Having to use the visual processor for both the diagram and written text may

6

Cognitive Load Theory: What We Learn and How We Learn

149

place a heavy load on that processor. Alternatively, by presenting the text in spoken rather than visual form allows the load to be spread over both the visual and auditory processor reducing the load on the visual processor. This theory provided the basis for the modality effect. Consider a learner studying a geometry worked example. Rather than reading “Angle ABC equals Angle XBY (vertically opposite angles),” the learner can hear the same, spoken statement. Information is shifted from the visual to the auditory processor reducing the cognitive load on the visual processor. Typically, comparing dual modality to single modality presentations indicates increased learning for the dual-modality condition (Mousavi, Low, & Sweller, 1995), providing an example of the modality effect. The modality effect, like all instructional effects, has limiting conditions. It will only be obtained if the visual and auditory components are unintelligible in isolation and must be mentally integrated before they can be understood. If, for example, the verbal material is simply a reiteration of the diagrammatic information (i.e., redundant), the effect will not be obtained. More importantly, for current purposes, the modality effect will not be obtained if the auditory information is complex and lengthy. Lengthy, high element interactivity and auditory information should never be presented in auditory form. The transient information effect provides the rationale. Writing, a biologically secondary activity, was invented to transform the biologically primary activity of speaking from transient to permanent form. In the normal course of events, when listening to spoken information, we find it is constantly disappearing to be replaced by the next elements of information. Furthermore, barring recording devices or someone with a memory of what was spoken, it cannot be retrieved. In that sense, spoken information is transient. If it is lengthy and high in element interactivity, it will impose a very high working memory load. For lengthy, high element interactivity spoken text, we will need to remember preceding text in order to understand current text. If previous text cannot be retained in working memory while listening to current text, what is heard will be unintelligible. In contrast, if the same information is presented in written form, it is permanent, and so we can go over the same text as often as we wish until it is understood. These factors underlie the transient information effect. Consider the modality effect again. If the verbal component of the instruction is relatively short, it can be held in working memory allowing the modality effect to occur. As the verbal component increases in length and complexity, the advantage of a dual-mode presentation will decrease. Eventually, there may be a disadvantage to using transient, spoken information with written information proving to be superior. Rather than obtaining a modality effect, a reverse modality effect may be obtained due to the transient nature of spoken information. Leahy and Sweller (2011) and Wong et al. (2012) obtained this effect. A reverse modality effect was obtained using lengthy, verbal information. Visual-only information was superior to dual-modality information. By shortening the information, a conventional modality effect with dualmodality superiority was reinstated. Wong et al. (2012) obtained similar results comparing static graphical presentations with animations. Animations are transient with each depiction replacing previous depictions. If element interactivity is high and if previous information is needed to understand current information, any advantage of animations may be lost due to their transient nature.

150

J. Sweller

These results have implications for the use of information technology in education. Information technology permits us to present information in a more “natural” manner. Instead of using written text, we can use spoken text. Instead of using static graphics, we can use animations. There can be advantages to using more natural forms of information presentation but under some conditions, especially when we are dealing with high element interactivity, biologically secondary information that is not “natural” may need to be presented in a manner that takes into account our cognitive architecture. For such information, written text or static graphics may be superior to spoken text and animations, respectively.

Summary and Conclusions The current version of cognitive load theory is intended to link evolutionary biology, categories of knowledge, human cognitive architecture, and human information processing with instructional design. Evolutionary educational psychology can be used to distinguish between two categories of knowledge, biologically primary and secondary knowledge, that have instructional implications. Biologically primary information frequently consists of generic cognitive skills and is more important than biologically secondary information that is usually domain specific. We have evolved to acquire the more important primary skills, but for that reason, primary knowledge cannot be taught because it is acquired automatically. In contrast, education is essential to the acquisition of biologically secondary knowledge because without education, the necessary skills are rarely acquired. The acquisition of biologically secondary knowledge is governed by a cognitive architecture that has well-defined and relatively well-known characteristics. An analysis of those characteristics suggests that human cognitive architecture processes information using similar structures and functions to those required by evolutionary theory. Both evolutionary theory and human cognition when dealing with biologically secondary knowledge require a large store of information (long-term memory in the case of human cognition) that is acquired largely from other information stores (other people) and to a lesser extent by a process of random generation followed by tests of effectiveness (problem-solving). To ensure that the acquisition of new information does not interfere with the utility of current information, there are structures in place to ensure that only limited amounts of novel information are transferred to the long-term information store (via a limited working memory). Once information is stored in the information store, large amounts of that information can be used to govern action that is appropriate to a particular environment (long-term working memory). This cognitive architecture has been used via cognitive load theory to determine instructional procedures. It is particularly relevant to information that because of its structure imposes a heavy working memory load. Elements of information that interact are best assimilated simultaneously, and simultaneous assimilation of many elements imposes a heavy working memory load. Accordingly, cognitive load theory has been used primarily to generate instructional procedures that reduce

6

Cognitive Load Theory: What We Learn and How We Learn

151

unnecessary working memory load. The relative effectiveness of all procedures generated by cognitive load theory is tested using randomized, controlled trials. Because the characteristics of working memory differ dramatically depending on whether it is processing novel information or organized, stored information from long-term memory, the effectiveness of any given instructional procedure changes equally dramatically depending on learners’ levels of expertise. Accordingly, instructional procedures need to change with changing knowledge levels. Additionally, the use of instructional technology can easily and accidentally overwhelm working memory. No instructional techniques should be introduced without considering their information processing characteristics and the manner in which those characteristics interact with the human cognitive system. Cognitive load theory was devised to assist in this process.

References Baddeley, A. (1999). Human memory. Boston, MA: Allyn & Bacon. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewoods Cliffs, NJ: Prentice Hall. Campbell, D. (1960). Blind variation and selective retention in creative thought as in other knowledge processes. Psychol Rev, 67, 380–400. Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cogn Instr, 8, 293–332. Chen, O., Kalyuga, S., & Sweller, J. (2015). The worked example effect, the generation effect, and element interactivity. J Educ Psychol, 107, 689–704. Darwin C (1871/2003) The descent of man. London: Gibson Square Geary, D. (2005). The origin of mind: Evolution of brain, cognition, and general intelligence. Washington, DC: American Psychological Association. Geary, D. (2007). Educating the evolved mind: Conceptual foundations for an evolutionary educational psychology. In J. S. Carlson & J. R. Levin (Eds.), Psychological perspectives on contemporary educational issues (pp. 1–99). Greenwich: Information Age Publishing. Geary, D. (2008). An evolutionarily informed education science. Educ Psychol, 43, 179–195. Geary, D. (2012). Evolutionary educational psychology. In K. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook (Vol. 1, pp. 597–621). Washington, DC: American Psychological Association. Hsu, C.-Y., Kalyuga, S., & Sweller, J. (2015). When should guidance be presented in physics instruction? Arch Sci Psychol, 3, 37–53. Jablonka, E., & Lamb, M. J. (2005). Evolution in four dimensions: Genetic, epigenetic, behavioral, and symbolic variation in the history of life. Cambridge, MA: MIT Press. Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. J Educ Psychol, 93, 579–588. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educ Psychol, 38, 23–31. Kirschner, P., Sweller, J., & Clark, R. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential and inquiry-based teaching. Educ Psychol, 41, 75–86. Klahr, D., & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. Psychol Sci, 15, 661–667. Leahy, W., & Sweller, J. (2011). Cognitive load theory, modality of presentation and the transient information effect. Appl Cogn Psychol, 25, 943–951.

152

J. Sweller

Mayer, R. (2004). Should there be a three-strikes rule against pure discovery learning? The case for guided methods of instruction. Am Psychol, 59, 14–19. Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychol Rev, 63, 81–97. Mousavi, S. Y., Low, R., & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. J Educ Psychol, 87, 319–334. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall. Peterson, L., & Peterson, M. J. (1959). Short-term retention of individual verbal items. J Exp Psychol, 58, 193–198. Pollock, E., Chandler, P., & Sweller, J. (2002). Assimilating complex information. Learn Instr, 12, 61–86. Popper, K. (1979). Objective knowledge: An evolutionary approach. Oxford, UK: Clarendon. Simon, H., & Gilmartin, K. (1973). A simulation of memory for chess positions. Cogn Psychol, 5, 29–46. Slamecka, N., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. J Exp Psychol Hum Learn Mem, 4, 592–604. Sweller, J. (2003). Evolution of human cognitive architecture. In B. Ross (Ed.), The psychology of learning and motivation (Vol. 43, pp. 215–266). San Diego, CA: Academic. Sweller, J. (2010). Element interactivity and intrinsic, extraneous and germane cognitive load. Educ Psychol Rev, 22, 123–138. Sweller, J. (2012). Human cognitive architecture: Why some instructional procedures work and others do not. In K. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook (Vol. 1, pp. 295–325). Washington, DC: American Psychological Association. Sweller, J., & Sweller, S. (2006). Natural information processing systems. Evol Psychol, 4, 434–458. Sweller, J., van Merrienboer, J. J., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educ Psychol Rev, 10, 251–296. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York, NY: Springer. Tricot, A., & Sweller, J. (2014). Domain-specific knowledge and why teaching generic skills does not work. Educ Psychol Rev, 26, 265–283. doi:10.1007/s10648-013-9243-1. West-Eberhard, M. (2003). Developmental plasticity and evolution. New York, NY: Oxford University Press. Wong, A., Leahy, W., Marcus, N., & Sweller, J. (2012). Cognitive load theory, the transient information effect and e-learning. Learn Instr, 22, 449–457. doi:10.1016/j. learninstruc.2012.05.004. Youssef-Shalala, A., Ayres, P., Schubert, C., & Sweller, J. (2014). Using a general problem-solving strategy to promote transfer. J Exp Psychol Appl, 20, 215–231.

John Sweller My research reputation is associated with cognitive load theory, an instructional theory based on our knowledge of human cognitive architecture. I initiated work on the theory in the early 1980s. Subsequently, “ownership” of the theory shifted to my research group at UNSW and then to a large group of international researchers. The theory is now a contributor to both research and debate on issues associated with human cognitive architecture, its links to evolution by natural selection, and the instructional design consequences that follow. It is one of the few theories to have generated a large range of novel instructional designs from our knowledge of human cognitive architecture. The following instructional design effects have flowed from cognitive load theory: goal-free, worked example, split attention, redundancy, modality, element interactivity, isolatedinteracting elements, imagination, expertise reversal, completion, variable examples, guidance fading, transient information, and collective working memory effects. These effects have been studied by many groups of researchers from around the globe. Based on any commonly used citation index, the work has been cited on between 10,000 and 20,000 occasions.

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA) P. A. Kirschner, B. Park, S. Malone, and H. Jarodzka

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Birth and Growth of Computer-Based Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Traditional Assessment and Item Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Limitations of Cognitive Theories on Multimedia Learning when Applied to Assessment . . . Cognitive Theory of Multimedia Learning (CTMML) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cognitive Load Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Is This the Good Approach to Assessment? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design Principles in Multimedia Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Applying Cognitive Principles to the Presentation of Test Items . . . . . . . . . . . . . . . . . . . . . . . . . . Using More Sophisticated Response Modes in Multimedia Assessment . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

154 156 158 159 160 162 163 164 164 167 168 170

Abstract

Much is known about assessment in all its forms and the corpus of theory, and knowledge is growing daily. In a similar vein, the use of multimedia for learning also has a sound basis in research and theory, such as the cognitive load theory (CLT; Sweller, Van Merriënboer, & Paas, (1998). Educational Psychological P. A. Kirschner (*) Open University of the Netherlands, Heerlen, The Netherlands Oulu University, Oulu, Finland e-mail: [email protected] B. Park · S. Malone Saarland University, Saarbrücken, Germany e-mail: [email protected]; [email protected] H. Jarodzka Open University of the Netherlands, Heerlen, The Netherlands Lund University, Lund, Sweden e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_53

153

154

P. A. Kirschner et al.

Review, 10, 251–296), human information processing (e.g., Atkinson & Shiffrin (1968). Human memory: A proposed system and its control processes. In K. W. Spence & J. T. Spence (Eds.), The psychology of learning and motivation: Advances in research and theory (Vol. 2, pp. 89–192). New York: Academic Press; Miller (1956). Psychological Review, 63, 81–97; Paivio (1986) Mental representations: A dual coding approach. New York: Oxford University Press), and praxis in the form of evidence-informed design principles often based on the cognitive theory of multimedia learning (CTMML; Mayer (2005b). Cognitive theory of multimedia learning. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 31–48). New York: Cambridge University Press). However, the combination of the two lacks both theoretical underpinnings and practical design principles. Multimedia assessment (MMA) is, at best, either a translation of paper-based assessment and assessment principles to the computer screen or an attempt to make use of the theory and principles underlying multimedia learning (i.e., CTMML). And this is the problem. In the first place, MMA needs, just as multimedia learning (MML), its own theory and principles. Just as MML was not simply the translation of paper-based learning to the computer screen, MMA requires its own place. In the second place, the application of CTMML and its principles to assessment leads to problems. The CTMML is based upon the idea that learning should be facilitated by the proper use of CTMML principles and its underlying theories (CLT, human information processing). In cognitive load terms, germane load is increased, while extraneous load is avoided so as to facilitate effective and efficient learning. But the goal of assessment is not learner facilitation, but rather separating the wheat from the chaff. Those who do not possess the knowledge and skills need to not be able to answer the question, while those who do have the knowledge and skills need to answer correctly. This may mean that certain forms of extraneous load need to be increased, while germane load needs to be minimized. This chapter will kick off the road to a cognitive theory of multimedia assessment (CTMMA). Keywords

Assessment · Multimedia · Instructional design · Cognitive load

Introduction In education and training using both paper-based and computer-based learning materials, we see both a convergence of opinions on and an adoption of instructional design principles and practices for their use. Instructivists and constructivists have found a certain degree of common ground in most if not all of the guidelines and principles found in the cognitive theory of multimedia learning (CTMML; Mayer, 2001) and cognitive load theory (CLT; Sweller, Van Merriënboer, & Paas, 1998). These guidelines and principles, arising from paper-based instructional materials, have been expanded and specified for the increasing use of computers and computer-

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

155

based instructional materials and learning environments and specifically multimedia materials and learning environments. Multimedia is defined here as a combination of text, audio, still images, animation, and/or video content. Now that multimedia learning materials have become commonplace and educators, trainers, instructional designers, and educational policy makers (including politicians) have embraced the ability of such materials to personalize teaching, training, and learning leading to more effective, efficient, and possibly more enjoyable teaching and learning experiences, a concomitant increase in attempts is observable with respect to designing, developing, and implementing multimedia assessment (e.g., in the Netherlands: De Boer, 2009; in Germany: Dennick, Wilkinson, & Purcell, 2009; Hamm & Robertson, 2010; Hartig & Klieme, 2007). We have deliberately chosen for the word “assessment” and not for “testing” as assessment is, in the context of this chapter, a much broader concept which includes testing. Testing (sometimes called examination) is almost always used in a summative way to determine what someone knows or has learned. Testing is actually subsumed by assessment in that it is a form of assessment which is intended almost exclusively to measure a test taker’s knowledge, skill, aptitude, and physical fitness; in other words it classifies a person by assigning her/him a level or score. Assessment expands this to include the process of measurably documenting the progress of the learner (i.e., her/his knowledge, skills, attitudes, and beliefs) in measureable terms to make improvements in and help guide that process. And here is where one can encounter problems that are counterintuitive, counterproductive, and possibly detrimental to assessment. While the design and use of multimedia for instruction are based upon sound and often tested theories (i.e., CTMML, CLT) with concomitant guidelines, using multimedia for computer-based assessment (CBA) is not. On the one hand, CBA is often based upon traditional design principles that have been developed and tested – for better or for worse – for paper-based applications, which are quite limited as compared to CBA with regard to presentation and response formats. The question is whether tried and tested instructional guidelines can simply be transferred to multimedia assessment and which aspects of CBA require their own proper principles. On the other hand, some designers use the CTMML and/or the CLT for the design and development of assessment. Different indicators are used for measuring cognitive processing during learning and its measurable immediate or delayed consequences (for an overview see also Brünken, Seufert, & Paas, 2010; Van Mierlo, Jarodzka, Kirschner, & Kirschner 2012). These include subjectively self-rated cognitive load or mental effort (Leppink, Paas, Van der Vleuten, Van Gog, & Van Merrienboer, 2013; Paas, 1992), objectively measured cognitive load via the dual-task paradigm (Brünken, Plass, & Leutner, 2004; Brünken, Steinbacher, Plass, & Leutner, 2002; DeLeeuw & Mayer, 2008; Park & Brünken, 2015), cognitive load measured with different eye movement phenomena (Jarodzka, Janssen, Kirschner, & Erkens, 2015; Knörzer, Brünken, & Park, 2016; Marshall, 2002; Mayer, 2010; Park, Knörzer, Plass, & Brünken, 2015; Park, Korbach, & Brünken, 2015), and different levels of learning performance distinguishing, for example, between retention and transfer performance (e.g., Marcus, Cooper, & Sweller, 1996) or knowledge about processes and structures (Park,

156

P. A. Kirschner et al.

Münzer, Seufert, & Brünken, 2016) or combined measures. The question here is whether principles meant to make learning from multimedia materials effective and efficient can be directly transferred to and used in an assessment situation since the goals of learning and assessment are different from and may even conflict with those of learning. For example, while the goal of using certain learning materials or types of learning materials might be to reduce extraneous cognitive load so as to facilitate learning (Sweller et al., 1998), the goal of introducing assessment or types of assessment materials might be to increase extraneous cognitive load so as to better distinguish between novices and experts. In this regard, the International Test Commission refers in their guidelines on computer and Internet testing to the use of advanced multimedia features in assessment, stating that these “should be used only where justified by validity” (p. 147). To overcome this research gap, the present article introduces and defines a cognitive theory of multimedia assessment (CTMMA) and presents the first derived principles for the design of multimedia assessment materials.

Birth and Growth of Computer-Based Assessment With increasing technical development, the use of computers for assessment is inevitable to take over assessment in general. Though original computer-based assessment was actually only computer-based testing (i.e., was only designed and used for making summative decisions), we will still use the term assessment. Educational researchers and designers should be prepared to deal with this and be able to provide guidelines, while CBA is being introduced instead of only reacting to students being confronted with bad design in CBA. Thus, the question is not about media whether assessment should be on paper or on computers, but instead about the methods: How to design CBA so that it does not hamper students’ performance nor its assessment, but instead uses its full potential to capture students’ level of knowledge, skills, and potential performance as adequately as possible so as to facilitate their progress in learning, skills attainment, and attitude adoption. Avoiding incorrect diagnostic decisions is an important goal for assessment for many reasons. Inappropriate assessment can even affect safety concerns. Theoretical driving tests have, for example, been recently adapted to computer-based assessment systems in many countries in the world. As this test is to assess aspects of driving competence, its function is to identify those applicants who are not yet competent enough to drive safely and therefore need further training. Besides, high norms for assessment are also required in less standardized contexts of assessment, for example, in multimedia learning studies. Appropriate assessment of learning outcomes is required for conclusions about the effects of different instructional methods, adaptive instruction and adaptive assessment, and real-time feedback. To address this methodological question, the advantages and the dangers of CBA due to the media change from paper-based to computer-based and due to different possibilities that CBA affords are considered next. One crucial change coming from CBA is that it allows for automated assessment and analysis of outcomes. This

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

157

automated assessment requires less time to prepare the tests for their administration (i.e., there is no need to prepare hundreds of printouts), makes it easier to prepare parallel forms so that students sitting next to each other cannot just copy what their smarter neighbors have answered, and requires less time to evaluate their test performance. As “time is money,” CBA can have – even at this very basic level – a distinct and measurable economic advantage over its paper-based peer (cf. Jurecka & Hartig, 2007). Furthermore, CBA ensures quality criterions by providing enhanced standardization of administration, scoring, and interpretation of the results. The automated analysis of the test performance leaves less room for careless mistakes made by teachers/markers and is thus more reliable and valid. In addition, CBA makes the analysis of the test as a whole (e.g., test-retest reliability, item analysis, item-test reliability, etc.) quicker and easier. Moreover, CBA allows for adaptive assessment that is, for example, recommended for adaptive learning systems when considering an individual difference perspective concerning prior knowledge (i.e., Kalyuga, Ayres, Chandler, & Sweller, 2003) or learner characteristics like spatial ability (i.e., Korbach, Brünken, & Park, 2016; Münzer, 2012, 2015; Münzer, Seufert, & Brünken, 2009; Park, Korbach, & Brünken, 2015; Park et al., 2016). With a large enough database of items that are well designed accompanied by an adaptivity algorithm, it is possible to easily provide different versions of a test to different groups of learners in different situations. Hence, the assessment can be adapted to each student’s knowledge level and, thus, not only be conducted more quickly (by avoiding too difficult and too simple questions for specific learners) but also be more accurate by carving out the abilities of a student in detail. Added to this, CBA allows the use of very different forms and combinations of media to students (e.g., sound, video, animation) that may represent a certain task better than only text and static pictures – often also only in black and white – such as in paper-based assessment (PBA). Besides the presentation of various stimuli, CBA offers the opportunity to record aspects of the participant’s behavior (e.g., response times) that cannot be logged by the means of PBA. Finally, CBA can be used in different places at different times and thus reach students across the world without the need to be physically present at a certain place at a certain time. However, the possibilities of CBA can be negated and/or even have serious disadvantages. For instance, when introducing CBA one can easily be tempted to simply put a PBA on a computer. This change of medium without adaptation to it can, for example, cause disadvantages to processing the information (e.g., paper pages that can easily be turned vs. computer pages that cannot be revisited) or to responding to task demands (e.g., using a pen vs. using the keyboard). Especially, for speed tests these difficulties that come along with CBA can lead to a test bias (i.e., participants, who rarely use a computer can be at a disadvantage). A second problem is that, since there are no explicit guidelines for CBA, the technical possibilities that it makes possible can easily lure test designers to implement advanced media (e.g., 3D visualizations, hypertext links), just because it is possible, without considering the consequences of this for its demands on processing and on test reliability and validity. This can lead to what designers call “Christmas tree” designs with lots of colorful trinkets and candy hanging on it taking away from the functionality needed.

158

P. A. Kirschner et al.

Finally, the ultimate goal of assessment is to reliably and validly distinguish someone who knows something or can do something from someone who cannot/does not and/or determine who is a novice, who is an expert, and where someone is on the continuum between the two. Improper use of the possibilities of multimedia in the assessment situation can easily lead to false positives (i.e., Type I errors where a learner who lacks the knowledge and skills is classified as having them because the multimedia made the test items answerable with the required prerequisite knowledge and/or skills) or false negatives (i.e., Type II errors where a learner who has the knowledge and skills is classified as not having them because the multimedia and their use made the test items unanswerable/unreliable).

Traditional Assessment and Item Design According to Lienert (1969) (or more recently Moosbrugger & Kelava, 2012), a test is a scientific routine to examine one or several personality features to make a quantitative statement about the relative degree of this feature’s characteristic. Such assessments can measure very different aspects. As the present paper focuses on performance (cf. Bortz & Döring, 2013), each assessment has to meet three quality standards, that is, objectivity (i.e., different coders must come to the same results), reliability (i.e., when repeating the assessment under similar circumstances, similar outcomes must be reached), and validity (i.e., individuals with a similar degree of a feature characteristic must come to a similar outcome). The latter is considered to be very important, as it demands evidence for a strong relation between the construct that is proposed to be assessed and the features that are actually assessed. Among others, two aspects of validity can be distinguished: criterion and ecological validity. Criterion validity refers to the relation between score and a certain criterion beyond the assessment situation. Ecological validity concerns the extent to which the assessment demands are similar to the demands of typical tasks in the respective domain. The process of constructing an assessment can be divided into six phases, namely, planning the assessment, design of the assessment + construction of the assessment items, analysis of assessment items, exploitation of item analysis, empirical testing of assessment quality criteria, and standardization of the assessment (Lienert & Raatz, 1994). The present paper focuses in particular on the second phase, that is, the design of the assessment and the construction of the assessment items. An item consists of two parts: a stimulus part and a response part. From principles on item design, many guidelines can be drawn on how to formulate questions or statements given to the participant (e.g., not to use ambiguous terms), which response modes to choose when (e.g., multiple-choice questions vs. open-answer formats), and how to compose several items into one test (e.g., according to discrimination power). However, though certain principles on the actual layout design of items exist for paper-based items, there are actually no principles for multimedia-based items. This is surprising as multimedia has the powerful potential to increase ecological validity of a test, because it can reflect and/or simulate many

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

159

aspects of real-life tasks in more detail than traditional paper-based assessment (PBA). On the other hand, research, which is derived from or refers to cognitive theories on learning (see the next section), shows that the use of multimedia instruction must be very carefully considered and the design of such multimedia material must take human cognitive architecture into account to not hamper performance. Within traditional item development, there is a focus on advantages and disadvantages of different response formats. Tasks with closed-response formats (e.g., multiple-choice questions, matching tasks) are on the one hand easy to evaluate, ensure high objectivity, and are economic for different reasons. On the other hand, such formats are often difficult to create (e.g., presenting the necessary number of plausible alternatives in a multiple-choice test), and aspects of their validity are questionable (e.g., in a car driving exam, the answers of the multiple-choice questions can often be simply learned by heart). Advantages of open-response formats (e.g., open questions, essays) are high content validity and easy development. Disadvantages are uneconomic (e.g., evaluation takes a lot of time, the need for a second assessor) and unstandardized evaluation (e.g., that two or more assessors reach the same evaluation of the answer). Beyond these classical response modes, items may be more authentic, in terms of context authenticity or the authenticity of the response formats (Meyer, 1992). Those formats that are most ecologically valid are often also most unstandardized. As all these kinds of response formats have pros and cons that are mutually exclusive in PBA, the decision for a certain response format always involves a conflict. This dilemma can be addressed by multimedia assessment, which allows for standardized and ecologically valid assessment at the same time. Ecological validity can be achieved by providing authentic sensations (e.g., animations instead of still pictures to visualize motion or sound instead of pronunciation notation) and realistic tasks (e.g., simulations). At the same time, internal validity can be assured by standardized test implementation, task assignment, and interpretation of results, which would be only possible to a limited extent within authentic assessment. CBA should make use of this advantage to provide valid assessment at all points.

Limitations of Cognitive Theories on Multimedia Learning when Applied to Assessment Even though not much is known on how to design computer-based multimedia assessment (CBMMA) environments, one possible source of recommendations and guidelines can be found in those principles that are available for the design of multimedia in computer-based instruction, though this must be done with caution. Two leading theories in this field of research are the cognitive theory of multimedia learning (Mayer, 2001, 2005b) and the cognitive load theory (Sweller et al., 1998). Both theories are based on similar assumptions that lead to similar recommendations

160

P. A. Kirschner et al.

for instructional design of learning material. What do these theories tell us about the idea of learning and why does this not work for assessment?

Cognitive Theory of Multimedia Learning (CTMML) CTMML is targeted on creating a plausible theoretical construct, which is consistent with known principles of research on learning and instruction. CTMML (Mayer, 2001, 2005b, 2009) is based on three assumptions, as already summarized in numerous publications like, for example, by Park (2010) as follows. The first assumption, which is a main assumption for many cognitive theories, comprises that human working memory, the cognitive subsystem for processing current information, is limited in its capacity for processing (Baddeley, 1992; Chandler & Sweller, 1991; Miller, 1956; Miyake & Shah, 1999). The second assumption is that meaningful learning requires active processing of information by the learner (cf. Fig. 2). For active processing different cognitive processes are necessary such as focusing the attention on the relevant learning content (i.e., selection), mentally organizing information in a coherent way (i.e., organization), and integrating new information with existing knowledge (i.e., integration). These three essential cognitive processes result in the so-called SOI model (selection-organization-integration; Mayer, 1996) summarizing active processing of an engaged learner. The last assumption of the CTMML is the dual channel assumption (cf. Fig. 1), which is derived from the dual-coding theory of Paivio (1986). Two channels of information processing have to be differentiated: verbal information is processed in the verbal/ auditory channel, while pictorial information is processed via the visual/pictorial channel, and limited capacity is assumed for each channel. In detail, active processing of pictures and words begins with the perception of these external representations via sensory memory. After that, the selection of relevant information begins within the working memory and results by means of an organization process in pictorial or verbal mental models. These internal representations are integrated by an active integration process to a coherent mental model ending in storage in long-term memory. In sum, it is possible to empirically test hypotheses, which can be derived from CTMML. This is what Mayer and other researchers successfully showed, documented in three handbooks of multimedia learning (Mayer, 2001, 2005a, 2009). In the most recent version of his handbook of multimedia learning, Mayer distinguishes between principles for reducing extraneous cognitive processing (i.e., coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principle), principles for managing essential cognitive processing (i.e., segmenting, pretraining, and modality principle), and principles for fostering generative processing in multimedia learning (i.e., multimedia, personalization, voice, and image principle). These principles, however, cannot be simply translated to assessment. For instance, one essential principle that is derived from CTMML is about coherence of the learning material. As, according to CTMML, learning consists in developing a

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

MULTIMEDIA PRESENTATION

SENSORY MEMORY

Words

Ears

Pictures

Eyes

161 LONG-TERM MEMORY

WORKING MEMORY

selecting words

selecting images

Sounds

Images

organizing words

organizing words

Verbal Model

intergating

Prior Knowledge

Pictorial Model

Fig. 1 Cognitive theory of multimedia learning (Mayer and Moreno based, in part, on the dualcoding theory of Paivio)

Fig. 2 Triarchic model of cognitive load theory (Adapted from Moreno & Park, 2010, © Cambridge University Press, reprinted with permission)

coherent mental representation of the learning contents, this can be fostered best by avoiding incoherence in presented learning materials. The corresponding reasonable deduction from CTMML for multimedia assessment is that tasks for assessments are to assess, whether the student has a correct coherent mental model. However, the question that arises in this context now is whether the assessment materials should be designed to be coherent, too. With the present paper, it is hypothesized that the opposite assumption might be true. Criterion validity of a task is expected to be higher if incoherencies appear within the assessment materials. Dealing with incoherencies can be an indicator of competence; because of their coherent mental model, experts can compensate or block out incoherencies. Similar questions arise when trying to transfer design principles that have been derived from CLT to assessment principles as described in the following section.

162

P. A. Kirschner et al.

Cognitive Load Theory Like many working memory models (Baddeley, 1992; Mayer, 2001; Paivio, 1986), CLT assumes that the capacity of working memory is limited and therefore learning is hampered when capacity is exceeded. In detail, CLT assumes that (1) different learning issues can be distinguished by complexity of the learning task; (2) human working memory, the cognitive subsystem for processing current information, is limited in its capacity for processing (Baddeley, 1992; Miyake & Shah, 1999); and (3) learned content is stored in capacity-unlimited long-term memory by using meaningful structured complex mental representations, in the form of schemata (Rumelhart & Ortony, 1976; Schank & Abelson, 1977). The description of CLT exists already in numerous publications, and the following one is out of a paper on cognitive and affective processes in multimedia learning by Park, Flowerday, and Brünken (2015). CLT (Kirschner, 2002; Plass, Moreno, & Brünken, 2010; Sweller, Ayres, & Kalyuga, 2011) assumes that knowledge acquisition depends on the efficiency of the use of available (limited) cognitive resources. The extent of cognitive load is thereafter determined by three components. First, intrinsic cognitive load (ICL) is related to the complexity of the learning content in terms of number of elements and the interactivity between those elements. Thus, intrinsic load depends on the number of elements and the relationships between them that must be simultaneously processed in working memory to learn the material being taught. The larger the number of elements of the material that needs to be learned and the higher the interactivity of those elements, the higher the intrinsic load of the material. Second, extraneous cognitive load (ECL) is caused by the cognitive demands imposed by instructional design that is not conducive to learning. The better the learning material is presented, considering the cognitive architecture and empirically proved instructional design principles, the lower the extraneous cognitive load. Instructional material, which does not specifically lead to learning and/or distracts from learning (e.g., search behavior which is not part of the learning goal), should thereafter be avoided. Finally, germane cognitive load (GCL) is the load that results from engaging in learning activities that effectively and efficiently foster schema acquisition. Germane cognitive load is thereafter also elicited by instructional material that facilitates or is beneficial for effective and efficient learning processes and therefore beneficial for the learning outcome. Whereas extraneous sources of load hinder learning, intrinsic sources of load reflect the complexity of the given learning task in relation to the learners’ level of expertise, and germane sources of load promote learning by helping students engage in the process of schema formation and automation. A basic assumption of CLT is that the total cognitive load experienced during learning is additively composed of these three load types, the so-called additivity hypothesis (Moreno & Park, 2010). If total cognitive load is excessive, learning and problem-solving will be inhibited. The triarchic model of CLT is shown in Fig. 2 that is adapted from a summary on the historical development of CLT by Moreno and Park. In sum, CLT results in the practical implication that extraneous cognitive load can be reduced by optimization of instructional design in order to free up capacity for

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

163

germane cognitive load. Especially reducing extraneous load is therefore assumed to facilitate learning. For the present paper, the question arises if the principle to reduce extraneous load is also relevant for and transferable to CBMMA.

Is This the Good Approach to Assessment? Both theories, CTMML and CLT, assume that the human information processing system is limited in terms of capacity and durability. As a consequence, they recommend minimizing the amount of information that needs to be processed at any one time. A second joint assumption is that information of different modalities is initially processed in different parts of the human information processing system. Hence, to make optimal use of this limited system, both parts should be used. The third assumption is that for information to be stored durably, it must be processed actively. Optimal learning material should support active processing of the to-belearned information. All these joint assumptions are necessary and relevant when investigating information processing during learning and instruction. However, when looking from the other side of the coin on information processing that is from the retrieval of stored information, other relevant aspects need to be considered. With the goal to create a plausible theoretical construct, which is consistent with known principles of research on CBA, all principles that have been derived from CTMML have to be proved carefully with respect to its suitability for CBMMA. As already mentioned, criterion validity of a task is, for example, expected to be higher if incoherencies appear within the assessment materials. Moreover, the following question arises when considering CLT from the other – assessment – side of the coin: Is there such a thing as intrinsic, extraneous, and germane assessment load, and if so, what are they? And is reducing extraneous cognitive load the right thing to do for assessment? A constructive dilemma exists between fostering instructional understanding by reducing extraneous load and ensuring ecological validity in assessment by keeping this load relatively high. Fostering instructional understanding can be achieved by reducing extraneous load, which is essential for learning as well as for assessment, as aspects of reliability and validity can be ensured because the measuring error is being reduced. However, for some tasks, especially within the assessment of complex skills, minimizing extraneous cognitive load would mean reducing the task’s complexity and, at the same time, making it less similar to (i.e., more discriminable from) the tasks that usually exist in the specific domain (i.e., low ecological validity). This would make it highly problematic – if not impossible – to determine whether the assessee has acquired the knowledge, skills, and/or competencies required. In sum, multimedia principles derived from CTMML need to be varied or even reversed in most cases of CBMMA. In addition, design principles, which are derived from CLT, seem not to be simply transferable to CBMMA in the same way. For instance, the ways to reduce cognitive load in multimedia learning described by Mayer and Moreno (2003) as principles to foster learning can appropriately be used in assessment when varying cognitive load in these ways for two purposes: ensuring

164

P. A. Kirschner et al.

ecological validity in assessment by keeping cognitive assessment load (CAL) relatively high and opening the possibility to test the limits of learners by varying CAL (from low until high) within the CBMMA. Thus, the following multimedia effects explicated by Mayer and Moreno could be used to vary CAL: modality, segmentation, pretraining, coherence, signaling, spatial contiguity, redundancy, temporal contiguity, voice, and personalization. In detail, the variation of CAL can be assigned to different kinds of assessment load: 1. Intrinsic assessment load (IAL) varies, for example, by using more or less complex assessment tasks or by using differing amounts of pretraining or explanations to already known labels or procedures of the assessment material that could differ from learning material as it is often the case in transfer assessment tasks. 2. Extraneous assessment load (EAL) associated with most of the principles mentioned varies, for example, by using incoherencies or redundant assessment material or additional load by using, for example, dual-task methods. 3. Germane assessment load (GAL) could vary, for instance, by using animating material to foster the learners’ assessment performance such as positive feedback or other methods, which increase the learners’ engagement within the assessment situation.

Design Principles in Multimedia Assessment On the basis of CTMML and CLT, various principles to guide instructional design have been formulated and empirically studied. Can design principles that originate from learning be appropriate for assessment? Tasks usually consist of two parts: a stimulus part and a response part. Both parts of a task have different functions: Whereas the stimulus part relates to information presentation, the response part is about what kind of reaction is demanded from the participant. It should be discussed for the both parts of a task separately, whether the use of instructional design principles for the purpose of designing tasks for assessment is promising.

Applying Cognitive Principles to the Presentation of Test Items Regarding the stimulus part of a task, there is clear evidence against the adoption of instructional guidelines to assessment. The expertise reversal effect (Kalyuga et al., 2003) – where instructional techniques that are highly effective with novices lose their effectiveness and even have negative consequences when used with more experts and vice versa – can be interpreted as an indicator for the inappropriateness of many design principles for assessment. In particular, the criterion validity of a test is expected to be threatened by an uncritical adoption of the common multimedia

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

165

learning design principles to multimedia assessment situations. An empirical indicator for criterion validity is a clear performance difference (with lower total load) in favor of domain experts compared to novices. According to the expertise reversal effect, the application of some design principles in multimedia learning supports novices and hampers experts. This question is also related to the question on cognitive load posed on an assessee: How can an optimal level of extraneous load be reached that allows instructional understanding and ecological validity at the same time? According to general expertise research, experts clearly outperform novices in a specific domain: they solve problems faster and make fewer mistakes (Ericsson, Charness, Hoffman, & Feltovich, 2006; Posner, 1988). Thus, for CBA this means that adding time pressure (or logging time-on-task) to the environment may help to distinguish between individuals of different levels of expertise. In terms of CLT, this expertise-related difference in performance is expected to be caused by different amounts of intrinsic and germane cognitive load in experts as compared to novices, which in turn is due to a difference in knowledge structuring. While intrinsic assessment load is the load that arises from the subjective difficulty of a certain task, germane load in assessment can be defined as load that is produced by the processes of information retrieval and problem-solving. Experts and novices differ with respect to intrinsic load: Experts have more prior knowledge than novices as well as having this prior knowledge organized differently in their schemata (i.e., they have larger and more complex schemata which function as one chunk or information element). For some domains, this knowledge structuring is not only encapsulating in continuously larger chunks, but this structure is entirely different than the one of novices or even intermediates (e.g., in medicine: Boshuizen & Schmidt, 1992). The same task, thus, is expected to be more difficult for novices than for individuals with higher expertise (e.g., intermediates, experts). A good task for skill assessment is expected to reveal this difference in intrinsic load: experts are assumed to be able to effectively and efficiently solve a complex task, while novices are assumed to fail. Especially for less demanding tasks (low intrinsic load), an optimal level of induced extraneous load can support assessment. With an optimal amount of extraneous load, experts still have free resources to accomplish the tasks, while the novices’ complete cognitive capacity will be consumed by intrinsic and extraneous load (cf. Fig. 3). Basically, real experts must be able to perform also under suboptimal circumstances. Malone and Brünken (2013) provide empirical evidence for this assumption. They assessed car driving-related knowledge in an expert-novice comparison and applied either useful animations or static pictures to visualize the same dynamic processes in traffic scenarios. They found an interaction effect between presentation mode (static vs. dynamic) and expertise. Experts outperformed novices only in the static version of the test. The animations were helpful for the novices, as they were relieved from the need to infer motion from a static picture. In contrast, the expert drivers did not benefit from the presentation of animations because, based on their

166

P. A. Kirschner et al.

Fig. 3 Optimal division of intrinsic (IAL), extraneous (EAL), and germane assessment load (GAL) in tasks for multimedia assessment

experience, they were able to mentally animate the static pictures, easily. The authors showed in their experiment how introducing helpful features in assessment (e.g., by providing animations) could interfere with criterion validity. Another study by Brünken, Steinbacher, Schnotz, and Leutner (2001) also provides evidence that CBA efforts, such as using codality, have to be considered in the frame of CBMMA to guarantee the required validity of assessments. They showed that effects of learning can be detected more easily when posttest materials are presented in the same codality as learning materials. It depends on the learning goal if this really is valid assessment. Two other studies (Brünken et al., 2002, 2004) show the same effects for valid and reliable measures of cognitive load when considering the modality principle in the frame of using dual-task methods. The used modality should be the same in both the CBMMA and the previously presented computer-based multimedia material that often also includes narration instead of text (i.e., audio files). In other words, dual-task methods appear to be modality specific, at least when using visual or auditory prompts within the dual-task method for measuring cognitive load. And this specificity can be used in an advantageous way to filter out the corresponding interesting cognitive processes. As summarized by Park and Brünken (2015), within the dual-task paradigm, cognitive load is measured by the performance of a secondary task executed parallel to/simultaneously with the primary learning task. In detail, the dual-task method measures cognitive load at different times of measurement during learning (primary task) with the help of the secondary task performance (e.g., reaction time to a signal), which reflects the amount of cognitive load in the primary task. In other words, differences in a learner’s resource consumption caused by different presentations of the learning material can be measured by differences in performance on the secondary task. The established secondary tasks usually include either an auditory or visual cue in the instruction. For example, Brünken et al. (2004) asked participants to monitor a letter in the upper part of the computer screen and react by pressing the space bar when a color change was observed. In a recent study by Park and Brünken (2015) using a continuous, intraindividual, and behavioral measure, the new task is achieved by utilizing internalized cues. More specifically, a previously practiced rhythm is executed continuously by foot tapping (i.e., the secondary task) while learning (i.e., primary task). Execution precision was used as indicator for cognitive load; the greater the precision, the lower the load. This is a variation of dual task that may provide a general indicator for cognitive load in that it is not modality specific

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

167

for executive control processes (Baddeley, 1992), but this needs further empirical testing. It is likely, thus, that experts outperform novices only on those tasks that match their area of expertise (Chi, 2006). To best detect the specific level of expertise, one has to find a “standardized set of tasks” that are most “representative” for a domain (Ericsson & Lehmann, 1996; Ericsson & Smith, 1991). For few domains these tasks are static and grayscale and thus easily presentable on paper (even though most of expertise research has been conducted in such tasks: Reingold & Sheridan, 2011). Only recently, expertise is investigated in more authentic and thus ecologically valid tasks and also capturing the relevant underlying processes, for instance, with eye tracking (e.g., Balslev et al. 2012; Jaarsma, Jarodzka, Nap, Van Merriënboer, & Boshuizen, 2015; Jarodzka, Scheiter, Gerjets, & Van Gog, 2010; Van Meeuwen et al., 2014; Wolff, Jarodzka, Van den Bogert, & Boshuizen, 2016). Only such research can unravel the exact processes underlying different levels of expertise and thus ultimately allow for their assessment and prediction.

Using More Sophisticated Response Modes in Multimedia Assessment Theoretical assumptions and empirical evidence do not support the uncritical transfer of design guidelines for multimedia learning to multimedia assessment. Processes of expertise development can on the one side explain why the transfer won’t work, and on the other hand, they point to possible resolutions for the problem. Van Gog, Ericsson, Rikers, and Paas (2005) have already addressed a part of the problem in their theoretical paper on the need for special guidelines to design instructional materials for advanced learners. The authors discuss why many design principles that work for novice learners might be inappropriate for advanced learners (expertise reversal effect) and that there is a need for special instructional design guidelines for learners which already have gained prior knowledge and made experiences in a domain. In order to do that, Van Gog et al. (2005) advise to take research findings on expert-novice differences, expertise acquisition, and factors that have proven to foster expertise into account. The authors emphasize the need for appropriate knowledge and skill assessment to be able to design adapted instruction. In their literature review, Ericsson, Krampe, and Tesch-Römer (1993) found that while practice is essential for skill and expertise development, whether performance is maximized by practice depends on how something is practiced. Particularly, the amount of deliberate practice is crucial for expertise development. Deliberate practice involves the explicit aim to improve one’s skill, permanent effort, phases of direct instruction, and immediate feedback. Expertise assessment should reveal if a person has engaged in deliberate practice and has acquired correct schemata. Expert performance is defined by Ericsson and Lehman (1996) as “an extreme adaptation to task constraints” (p. 291). The assessment of expertise therefore requires the selection of the essential aspects of expert performance, the identification of relevant real task constraints, and the creation of representative tasks for the

168

P. A. Kirschner et al.

specific domain. This approach ensures ecological as well as criterion validity of the assessment. For complex dynamic domains, such as many sports, these two validity criteria, ecological and criterion validity, are considered to be related. As Hodges, Huys, and Starkes (2007) report, the increase of ecological validity of stimulus and response modes of the tasks (facilitated by the means of new media) makes them more sensitive to expert-novice differences, which indicate criterion validity. This finding leads to the conclusion that valid tests need to include tasks that are representative for a certain domain. Representative tasks (task demands match the requirements imposed usually in the specific domain) can be created by the means of multimedia assessment by the application of sophisticated response modes (e.g., reaction time measure, car/truck/flight simulation). However, it is not only relevant what type of response mode we use for assessment but also how the assessees process them. Research focusing on the processes underlying multimedia assessment indicates that two issues are crucial (Jarodzka et al., 2015; Ögren, Nyström, & Jarodzka, 2016): First, the students must carefully process the main question posed to them. Second, they must integrate this question with the multimedia material (e.g., in forms of integrative saccades). Only such a processing behavior could be related to higher assessment scores.

Conclusion We conclude this chapter with a number of general considerations with respect to the application of CTMMA and an elucidation (see Table 1) of the similarities and differences between CTMML and CTMMA. From the perspective of cognitive load research, at first, intrinsic cognitive load can be considered in the given assessment tasks by varying the complexity. This is already considered in several studies by using, for example, retention versus comprehension and transfer tasks (e.g., Marcus et al., 1996) or using tasks asking for learning outcomes of processes in contrast to knowledge about structures (e.g., Park et al., 2016). Second, extraneous cognitive load can be considered by varying this type of load to ensure ecological validity (i.e., increase extraneous cognitive load) and assess the limits of the learner (i.e., present different tasks with increasing extraneous cognitive load levels). Integrating at this point also the perspective of research on multimedia learning, optimal extraneous cognitive load should be imposed by “ignoring” many of the instructional design principles given to reduce extraneous cognitive load. However, this should be operationalized carefully, not overdoing it. In addition, response modes have to be considered, as these allow for representative tasks by means of employing sophisticated response modes (i.e., authenticity measures are needed). Third, when considering germane cognitive load, the learner’s engagement in the assessment situation should be varied in order to test the limits of the learner. In Table 1, some concluding CTMMA principles are summarized hinting at the concrete possible operationalizations for considering CTMMA in future assessment.

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

169

Table 1 CTMML and CTMMA principles, including possible operationalizations of CTMMA in assessment Principle Modality

Segmentation

Pretraining

Coherence

Signaling

CTMML People learn better from graphics and narrations than from animation and on-screen text, especially when the graphic is complex, the words are familiar, and the lesson is fast paced People learn better from a multimedia lesson that is presented in user-paced segments rather than as a continuous unit People learn better from a multimedia lesson when they know the names and characteristics of the main concepts People learn better when extraneous words, pictures, and sounds are excluded rather than included. Adding interesting but irrelevant materials to e-learning courses may distract the learner People learn better when cues that highlight the organization of the essential material are added

Spatial contiguity

People learn better when corresponding words and pictures are presented near rather than far from each other on the page or screen

Temporal contiguity

People learn better when corresponding words and pictures are presented simultaneously rather than successively

CTMMA Pictures can actually “trick” assessees into confirming a statement (cf. Ögren et al., 2016). Hence, they should be used scarcely and cautiously

It is easier to distinguish between individuals of higher and lower expertise, if the task or problem is presented as a continuous unit (cf. whole task) It is easier to distinguish between individuals of higher and lower expertise, if no pretraining on the test material was given It is easier to distinguish between individuals of higher and lower expertise, if the amount of coherence of the testing material corresponds to the coherence found in the real world It is easier to distinguish between individuals of higher and lower expertise, if no additional cues or highlights are given It is easier to distinguish between individuals of higher and lower expertise, if the spatial contiguity of the given material corresponds to the realworld situation: the assessment itself is that the assessees select and integrate the relevant information autonomously It is easier to distinguish between individuals of higher and lower expertise, if all information is presented in such a way as it would occur in the real-world task: for some situations this may mean that people need to integrate a lot of information at the same time or that they need to remember information for later usage (continued)

170

P. A. Kirschner et al.

Table 1 (continued) Principle Redundancy

Emotional design and emotion induction including personalization and voice

Self-pacing

CTMML People learn better from graphics and narration than from graphics, narration, and on-screen text. The visual text information presented simultaneously to the verbal information becomes redundant People learn better from multimedia lessons when words are in conversational style rather than formal style. People learn better when the narration in multimedia lessons is spoken in a friendly human voice rather than a machine voice

Learners learn better from selfpaced than from system-paced multimedia lessons

CTMMA It is easier to distinguish between individuals of higher and lower expertise, if the amount of redundant information is as high as it would be in the according realworld task or problem Emotional design, emotion induction or personalization, and the use of human voice within the assessment situation could help to distinguish between individuals of higher and lower expertise because experts are known to be capable to compensate effects of emotionalized material, induced emotions, or formal instead of conversational style As experts are known to execute tasks faster than novices do, putting temporal restrictions to assessment (presentation and answer time) may help easily distinguish between individuals of higher and lower expertise

Moreover, important general issues that have to be kept in mind when designing multimedia assessment are the aims/goals of assessment and level of expertise of the person/group to be assessed, the content of the assessment tasks and the type of knowledge and skills that the assessment is intended to capture, as well as the fact that the design of the assessment tasks on the computer screen is quite different from doing this on paper. Acknowledgments This work was – in part – supported by the German Federal Ministry of Education and Research (01PL12057).

References Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control processes. In K. W. Spence & J. T. Spence (Eds.), The psychology of learning and motivation: Advances in research and theory (Vol. 2, pp. 89–192). New York, NY: Academic. Baddeley, A. D. (1992). Working memory. Science, 255, 556–559. https://doi.org/10.1126/ science.1736359.

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

171

Balslev, T., Jarodzka, H., Holmqvist, K., De Grave, W. S., Muijtjens, A., Eika, B., et al. (2012). Visual expertise in paediatric neurology. European Journal of Paediatric Neurology, 16, 161–166. https://doi.org/10.1016/j.ejpn.2011.07.004. Bortz, J., & Döring, N. (2013). Forschungsmethoden und evaluation [Research methods and evaluation]. Heidelberg, Germany: Springer-Verlag. Boshuizen, H. P. A., & Schmidt, H. G. (1992). Biomedical knowledge and clinical expertise. Cognitive Science, 16, 153–184. Brünken, R., Steinbacher, S., Schnotz, W., & Leutner, D. (2001). Mentale Modelle und Effekte der Präsentations- und Abrufkodalität beim Lernen mit Multimedia [Mental models and effects of presentation and retrieval coding when learning with multimedia]. Zeitschrift für Pädagogische Psychologie, 15, 16–27. https://doi.org/10.1024//1010-0652.15.1.16. Brünken, R., Steinbacher, S., Plass, J., & Leutner, D. (2002). Assessment of cognitive load in multimedia learning using dual-task methodology. Experimental Psychology, 49, 109–119. https://doi.org/10.1027//1618-3169.49.2.109. Brünken, R., Plass, J. L., & Leutner, D. (2004). Assessment of cognitive load in multimedia learning with dual-task methodology: Auditory load and modality effects. Instructional Science, 32, 115–132. https://doi.org/10.1023/B:TRUC.0000021812.96911.c5. Brünken, R., Seufert, T., & Paas, F. (2010). Measuring cognitive load. In J. L. Plass, R. Moreno, & R. Brünken (Eds.), Cognitive load theory (pp. 181–202). Cambridge, UK: University Press. Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8, 293–332. https://doi.org/10.1207/s1532690xci0804_2. Chi, M. T. H. (2006). Two approaches to the study of experts’ characteristics. In K. A. Ericsson, N. Charness, R. R. Hoffman, & P. Feltovich (Eds.), The Cambridge handbook of expertise and expert performance (pp. 21–30). Cambridge, UK: Cambridge University Press. De Boer, N. (2009). De computer bij de centrale examens. Duidelijk digitaal 2 [The computer at the national exams. Clearly digital 2]. http://www.cito.nl/VO/ce/compex/introductie/cve_comp_ bij_ce_duidelijk_digitaal_2.pdf DeLeeuw, K. E., & Mayer, R. E. (2008). A comparison of three measures of cognitive load: Evidence for separable measures of intrinsic, extraneous, and germane load. Journal of Educational Psychology, 100, 223–234. https://doi.org/10.1037/0022-0663.100.1.223. Dennick, R., Wilkinson, S., & Purcell, N. (2009). Online eAssessment: AMEE guide no. 39. Medical Teacher, 31, 192–206. https://doi.org/10.1080/01421590902792406. Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance: Evidence of maximal adaptation to task constraints. Annual Review of Psychology, 47(1), 273–305. https:// doi.org/10.1146/annurev.psych.47.1.273. Ericsson, K. A., & Smith, J. (1991). Prospects and limits in the empirical study of expertise. In K. A. Ericsson & J. Smith (Eds.), Towards a general theory of expertise: Prospects and limits (pp. 1–38). Cambridge, MA: University Press. Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363–406. https://doi.org/ 10.1037/0033-295X.100.3.363. Ericsson, K. A., Charness, N., Hoffman, R. R., & Feltovich, P. (Eds.). (2006). The Cambridge handbook of expertise and expert performance. Cambridge, UK: University Press. Fischer, F., Waibel, M., & Wecker, C. (2005). Nutzenorientierte Grundlagenforschung im Bildungsbereich [Benefit-oriented basic research in the field of education]. Zeitschrift für Erziehungswissenschaft, 8, 427–442. https://doi.org/10.1007/s11618-005-0149-7. Hamm, S., & Robertson, I. (2010). Preferences for deep-surface learning: A vocational education case study using a multimedia assessment activity. Australasian Journal of Educational Technology, 26, 951–965. https://doi.org/10.14742/ajet.1027. Hartig, J., & Klieme, E. (Eds.). (2007). Möglichkeiten und Vorraussetzungen technologiebasierter Kompetenzdiagnostik [Possibilities and prerequisites of technology-driven competence diagnostics]. Bonn, Berlin: Bundesministerium für Bildung und Forschung (BMBF).

172

P. A. Kirschner et al.

Hodges, N. J., Huys, R., & Starkes, J. L. (2007). Methodological review and evaluation of research in expert performance in sport. In G. Tenenbaum & R. C. Eklund (Eds.), Handbook of sport psychology (Vol. 3, pp. 161–183). Hoboken, NJ: Wiley. Jaarsma, T., Jarodzka, H., Nap, M., Van Merriënboer, J. J. G., & Boshuizen, H. P. A. (2015). Expertise in clinical pathology: Bridging the gap. Advances in Health Sciences Education, 20, 1089–1106. https://doi.org/10.1007/s10459-015-9589-x. Jarodzka, H., Scheiter, K., Gerjets, P., & Van Gog, T. (2010). In the eyes of the beholder: How experts and novices interpret dynamic stimuli. Journal of Learning and Instruction, 20, 146–154. https://doi.org/10.1016/j.learninstruc.2009.02.019. Jarodzka, H., Janssen, N., Kirschner, P. A., & Erkens, G. (2015). Avoiding split attention in computer-based testing: Is neglecting additional information facilitative? British Journal of Educational Technology, 46, 803–817. https://doi.org/10.1111/bjet.12174. Jurecka, A., & Hartig, J. (2007). Computer- und netzwerkbasiertes Assessment [Computer- and network-based assessment]. In J. Hartig & E. Klieme (Eds.), Möglichkeiten und Voraussetzungen technologiebasierter Kompetenzdiagnostik (pp. 37–48). Bonn, Berlin: Bundesministerium für Bildung und Forschung (BMBF). Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38, 23–32. https://doi.org/10.1207/S15326985EP3801_4. Kirschner, P. A. (2002). Cognitive load theory: Implications of cognitive load theory on the design of learning. Learning and Instruction, 12, 1–10. https://doi.org/10.1007/s11251-009-9110-0. Knörzer, L., Brünken, R., & Park, B. (2016). Facilitators or suppressors: Effects of experimentally induced emotions on multimedia learning. Learning and Instruction, 44, 97–107. https://doi. org/10.1016/j.learninstruc.2016.04.002. Korbach, A., Brünken, R., & Park, B. (2016). Learner characteristics and information processing in multimedia learning: A moderated mediation of the seductive details effect. Learning and Individual Differences, 51, 59–68. https://doi.org/10.1016/j.lindif.2016.08.030. Leppink, J., Paas, F., Van der Vleuten, C. P. M., Van Gog, T., & Van Merrienboer, J. J. G. (2013). Development of an instrument for measuring different types of cognitive load. Behavioral Research, 45, 1058–1072. https://doi.org/10.3758/s13428-013-0334-1. Lienert, G. A. (1969). Testaufbau und Testanalyse [Test construction and test analysis] (3., durch einen Anh. über Faktorenanalyse erg. Aufl.). Weinheim, Germany: Beltz. Lienert, G. A., & Raatz, U. (1994). Testaufbau und Testanalyse [Test construction and test analysis] (5. völlig neu bearbeitete und erweiterte Auflage). Weinheim, Germany: Beltz. Malone, S., & Brünken, R. (2013). Assessment of driving expertise using multiple choice questions including static vs. animated presentation of driving scenarios. Accident Analysis & Prevention, 51, 112–119. https://doi.org/10.1016/j.aap.2012.11.003. Marcus, N., Cooper, M., & Sweller, J. (1996). Understanding instructions. Journal of Educational Psychology, 88, 49–63. https://doi.org/10.1037/0022-0663.88.1.49. Marshall, S. P. (2002). The index of cognitive activity: Measuring cognitive workload. Proceeding of the 2002 I.E. 7th Conference, Human Factors and Power Plants, 2002. https://doi.org/ 10.1109/HFPP.2002.1042860. Mayer, R. E. (1996). Learning strategies for making sense out of expository text: The SOI model for guiding three cognitive processes in knowledge construction. Educational Psychology Review, 8, 357–371. https://doi.org/10.1007/BF01463939. Mayer, R. E. (2001). Multimedia learning. New York, NY: Cambridge University Press. Mayer, R. E. (2005a). The Cambridge handbook of multimedia learning. New York, NY: Cambridge University Press. Mayer, R. E. (2005b). Cognitive theory of multimedia learning. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 31–48). New York, NY: Cambridge University Press. Mayer, R. E. (2009). Multimedia learning. Cambridge, UK: University Press. Mayer, R. E. (2010). Unique contributions of eye-tracking research to the study of learning with graphics. Learning and Instruction, 20, 167–171. https://doi.org/10.1016/j. learninstruc.2009.02.012.

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

173

Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38, 43–52. https://doi.org/10.1207/S15326985EP3801_6. Meyer, C. A. (1992). What’s the difference between authentic and performance assessment? Educational Leadership, 49, 39–40. Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81–97. https://doi.org/10.1037/h0043158. Miyake, A., & Shah, P. (Eds.). (1999). Models of working memory: Mechanisms of active maintenance and executive control. New York, NY: Cambridge University Press. Moosbrugger, H., & Kelava, A. (2012). Testtheorie und Fragebogenkonstruktion [Test theory and questionnaire design]. Berlin, Germany: Springer. Moreno, R., & Park, B. (2010). Cognitive load theory: Historical development and relation to other theories. In R. M. R. B. J. L. Plass (Ed.), Cognitive load theory (pp. 9–28). New York, NY: Cambridge University Press. Münzer, S. (2012). Facilitating spatial perspective taking through animation: Evidence from an aptitude-treatment-interaction. Learning and Individual Differences, 22, 505–510 http://dx.doi. org/10.1016/j.lindif.2012.03.002. Münzer, S. (2015). Facilitating recognition of spatial structures through animation and the role of mental rotation ability. Learning and Individual Differences, 38, 76–88 http://dx.doi.org/10. 1016/j.lindif.2014.12.007. Münzer, S., Seufert, T., & Brünken, R. (2009). Learning from multimedia presentations: Facilitation function of animations and spatial abilities. Learning and Individual Differences, 19, 481–485. https://doi.org/10.1016/j.lindif.2009.05.001. Ögren, M., Nyström, M., & Jarodzka, H. (2016, online). There’s more to the multimedia effect than meets the eye: Is seeing pictures believing? Instructional Science. https://doi.org/10.1007/ s11251-016-9397-6 Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84, 429–434. https://doi.org/ 10.1037/0022-0663.84.4.429. Paivio, A. (1986). Mental representations: A dual coding approach. New York, NY: Oxford University Press. Park, B. (2010). Testing the additivity hypothesis of cognitive load theory (Doctoral dissertation, Saarland University, Saarbrücken, Germany). Retrieved from http://scidok.sulb.uni-saarland.de/ volltexte/2010/3478/. Park, B., & Brünken, R. (2015). The rhythm method: A new method for measuring cognitive load: An experimental dual-task study. Applied Cognitive Psychology, 29, 232–243. https://doi.org/ 10.1002/acp.3100. Park, B., Flowerday, T., & Brünken, R. (2015). Cognitive and affective effects of seductive details in multimedia learning. Computers in Human Behavior, 44, 267–278. https://doi.org/10.1016/j. chb.2014.10.061. Park, B., Knörzer, L., Plass, J. L., & Brünken, R. (2015). Emotional design and positive emotions in multimedia learning: An eyetracking study on the use of antropomorphisms. Computers & Education, 86, 30–42. https://doi.org/10.1016/j.compedu.2015.02.016. Park, B., Korbach, A., & Brünken, R. (2015). Do learner characteristics moderate the seductive-detailseffect? A cognitive-load-study using eye-tracking. Journal of Educational Technology & Society, 18, 24–36 http://www.ifets.info/journals/18_4/3.pdf, Creative Commons CC-BY-ND-NC 3.0. Park, B., Münzer, S., Seufert, T., & Brünken, R. (2016). The role of spatial ability when fostering mental animation in multimedia learning: An ATI-study. Computers in Human Behavior, 64, 497–506. https://doi.org/10.1016/j.chb.2016.07.022. Plass, J. L., Moreno, R., & Brünken, R. (2010). Cognitive load theory. New York, NY: Cambridge University Press. Posner, M. I. (1988). Introduction: What is it to be an expert? In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise. Hillsdale, NJ: Erlbaum.

174

P. A. Kirschner et al.

Reingold, E. M., & Sheridan, H. (2011). Eye movements and visual expertise in chess and medicine. In S. P. Liversedge, I. D. Gilchrist, & S. Everling (Eds.), Oxford handbook of eye movements (pp. 523–550). Oxford, UK: Oxford University Press. Rumelhart, D. E., & Ortony, A. (1976). The representation of knowledge in memory. San Diego, CA: Center for Human Information Processing, Department of Psychology, University of California. Schank, R., & Abelson, R. (1977). Scripts, goals, and understanding. Hillsdale, NJ: LEA. Sharkey, N. E., & Mitchell, D. C. (1985). Word recognition in a functional context: The use of scripts in reading. Journal of Memory and Language, 24, 253–270. https://doi.org/10.1016/ 0749-596X(85)90027-0. Sweller, J., Van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychological Review, 10, 251–296. https://doi.org/10.1023/b: truc.0000021808.72598.4d. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York, NY: Springer. The International Test Commission. (2006). International guidelines on computer-based and internet-delivered testing. International Journal of Testing, 6, 143–171. https://doi.org/ 10.1207/s15327574ijt0602_4. van Gog, T., Ericsson, K. A., Rikers, R. M. J. P., & Paas, F. (2005). Instructional design for advanced learners: Establishing connections between the theoretical frameworks of cognitive load and deliberate practice. Educational Technology Research and Development, 53, 73–81. https://doi.org/10.1007/BF02504799. Van Meeuwen, L. W., Jarodzka, H., Brand-Gruwel, S., Kirschner, P. A., De Bock, J. J. P. R., & Van Merriënboer, J. J. G. (2014). Identification of effective visual problem solving strategies in a complex visual domain. Learning and Instruction, 32, 10–21. https://doi.org/10.1016/j. learninstruc.2014.01.004. Van Mierlo, C. M., Jarodzka, H., Kirschner, F., & Kirschner, P. A. (2012). Cognitive load theory and e-learning. In Z. Yan (Ed.), Encyclopedia of cyber behavior. Hershey, PA: IGI Global. Wolff, C. E., Jarodzka, H., Van den Bogert, N., & Boshuizen, H. P. A. (2016). Teacher vision: Comparing expert and novice teachers’ perception of problematic classroom management scenes. Instructional Science, 44(3), 243. https://doi.org/10.1007/s11251-016-9367-z.

Paul A. Kirschner is a Distinguished University Professor at the Open University in the Netherlands as well as a Visiting Professor of Education with a special emphasis on Learning and Interaction in Teacher Education at the University of Oulu, Finland. He was previously a Scientific Director of the Learning and Cognition program at CELSTEC, Open University in the Netherlands. He is an internationally recognized expert in the fields of educational psychology and instructional design. He is a past president (2010–2011) of the International Society for the Learning Sciences and a former member of the Dutch Educational Council and, as such, was an advisor to the Minister of Education (2000–2004). He is also a member of the Scientific Technical Council of the Foundation for University Computing Facilities (SURF WTR), chief editor of Journal of Computer Assisted Learning, and associate editor of Computers in Human Behavior. As for books, he is a coauthor of the recently released book Urban Myths about Learning and Education as well as of the highly successful book Ten Steps to Complex Learning and editor of two other books (Visualizing Argumentation and What We Know About CSCL). Babette Park is an Assistant Professor (Juniorprofessorin) at the Department of Education at Saarland University in Germany. She has a Diploma in Psychology and finished her German Ph.D. in Educational Psychology in 2010. Since 2012 she holds the chair “Empirical Research and Didactics in Higher Education” funded by the German Federal Ministry of Education and Research (01PL12057) within the project “Studying with Profile: Competence in Research and Practice.” Babette Park teaches students of psychology (diploma/bachelor/master), of the international and interdisciplinary study course educational technology (master), and of teacher education.

7

Toward a Cognitive Theory of Multimedia Assessment (CTMMA)

175

Additionally, she is a teacher and counselor for academic staff at the Centre for Key Skills and Didactics in Higher Education. Her research group runs three different research lines. Within basicoriented research, suitable instruments are developed and validated for measuring cognitive load. With instructional-psychological research, cognitive and affective processes are investigated in multimedia learning. Finally, application-oriented research focuses on analyzing conditions and determinants of successful teaching in higher education from an educational-psychological perspective. Babette Park is an active member of an international network in research on cognitive load and multimedia learning. Since 2015 she is a member of the Cognitive Load Theory Advisory Committee for the annual meeting International Cognitive Load Theory Conference. Sarah Malone works as a postdoctoral research fellow at the Department of Education, Saarland University, Germany. Her research focuses on learning and assessment with multiple representations in mathematics and traffic psychology (hazard perception assessment, computer-based driver training). Halszka Jarodzka works as an Assistant Professor at the Open University in the Netherlands, where she is the Chair of the topic group “Processes of learning and expertise development in information-rich environments.” Moreover, she works part time as a visiting scholar at a large eye tracking laboratory at Lund University in Sweden. Her research deals with the use of eye tracking to understand and improve learning and its instruction. In that, she investigates three topics: first, the instructional design of computer-based learning and testing environments; second, the characteristic and development of visual expertise in diverse professions; and third, training of perceptual skills with eye movement modeling examples (EMME). She is a cofounder and chair of a special interest group on “online measures of learning” within the European Association of Research on Learning and Instruction.

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs, and Learning Strategies on Help Seeking Silke Schworm and Hans Gruber

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Help Seeking and Learner-Related Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Help-Seeking Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Help Seeking and Learners’ Epistemic Beliefs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Help Seeking and Learning Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Help Seeking and Context-Related Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Impact of Goal Orientations, Epistemic Beliefs, and Learning Strategies on Help Seeking: Two Empirical Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Study 1: The Impact of Goal Orientations, Epistemic Beliefs, and Learning Strategies on Help Seeking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Study 2: The Impact of Prompting Adaptive Help Seeking on Activity in a Virtual Workspace, Acceptance of the Learning Environment, and Learning Outcome . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

178 179 180 181 182 183 184 185 191 194 195

Abstract

In this chapter, an important part of theories on learning and instruction is presented: the state of the art of research about help seeking of learners in academic settings is outlined. It is argued that help seeking is a demanding part of learning activities, should it be used adequately. Both learner-related factors and context-related factors impact help seeking. Although context-related factors are discussed as well, most parts of the chapter are devoted to present research S. Schworm (*) University of Regensburg, Regensburg, Germany e-mail: [email protected]; [email protected] H. Gruber University of Regensburg, Regensburg, Germany University of Turku, Turku, Finland e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_54

177

178

S. Schworm and H. Gruber

about the three most important learner-related factors: goal orientations, epistemic beliefs, and learning strategies. While many studies on help seeking were performed in school contexts, higher education contexts might pose even more challenging questions. Learners are increasingly working in virtual environments, and there they are faced with the requirement to contribute to complex collaborative learning processes. Therefore, two studies are presented, which investigated in such learning contexts the impact of goal orientations, epistemic beliefs, and learning strategies on help seeking. In study 1 cluster analyses helped to categorize students into adaptive help seekers and help avoiders, based on their profiles of goal orientations, epistemic beliefs, and learning strategies. Study 2 tested instructional consequences drawn from these results. Keywords

Epistemic beliefs · Goal orientations · Help seeking · Learning strategies

Introduction “Help seeking can be defined as the process of seeking assistance from other individuals or other sources that facilitate accomplishing desired goals” (Karabenick & Berger, 2013, p. 238). It is essential for the successful mastery of complex learning settings. In contrast to other strategies of self-regulated learning, help seeking requires interaction with teachers, peers, or computer-based learning environments (Karabenick & Newman, 2010). Several models outlined the path through a situation where help is needed (Mercier & Frederiksen, 2007; Nelson-LeGall, 1981; Newman, 1994). Karabenick and Dembo (2011) summarized that those models distinguish eight parts of the helpseeking process: (1) identifying the problem, (2) recognizing need for help, (3) deciding to seek help, (4) deciding which kind of help to seek, (5) searching for an appropriate source of help, (6) soliciting help (7), obtaining help, and (8) processing the help received. Learners make decisions and adapt their help-seeking process according to available cognitive, affective, and social competencies and resources (Karabenick & Berger, 2013; Karabenick & Dembo, 2011). Some help-seeking activities run automatically, and some others are controlled by cognitive processes, as Table 1 indicates. Therefore, many reasons can be identified why the process may be interrupted before successfully being completed. Help seeking is regarded to be successful when it enables or facilitates the completion of an academic task. There are two groups of factors influencing the success of a help-seeking process, learner-related factors, and context-related factors. Among learner-related factors, goal orientations, epistemic beliefs, and learning strategies most extensively have been studied. Context-related factors comprise general factors of the learning environment or specific factors of the technical

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs. . .

179

Table 1 Competencies and resources supporting the help-seeking process (Karabenick & Dembo, 2011, p. 34) Competencies/resources Stage Determine whether there is a problem Determine whether help is needed Decide whether to seek help Decide on the type of help Decide on whom to ask Solicit help Obtain help Process the help received

Cognitive X

Social

X

Contextual emotional

X

X X X X

Affective emotional

X X

X X X X X X

X

X X

environment (Mäkitalo-Siegl & Fischer, 2013; Ryan & Shim, 2011; Schworm & Nistor, 2013; Shim, Kiefer, & Wang, 2013). Both groups of factors are introduced in this chapter, but the main focus is on the impact of learner-related factors.

Help Seeking and Learner-Related Factors Karabenick (2003) analyzed the relationship between students’ attitudes toward help seeking and learner-related factors. A hierarchical cluster analysis revealed four groups of help seekers. The first two clusters comprised students with an instrumental (adaptive) help seeking; they just differed in the preference of formal help sources. Students of both groups do not feel threatened by a need of help, do not avoid help seeking, and are not just aiming for fast solutions. The third and fourth cluster comprised students who show little adaptive help seeking, feel threatened by the need of help, and tend toward avoiding help seeking. Students of the first two clusters were identified as self-regulated, intrinsically motivated, and mastery oriented, and they employed a high level of learning strategies. They showed superior course performance compared to the other students. Karabenick’s (2003) study gives insight into factors related with help seeking. To understand maladaptive help seeking or even the absence of help seeking, it is crucial to analyze learners’ helpseeking attitudes and their relation to other learner-related factors and to find effective learning contexts and instructional arrangements to foster help seeking. Of particular relevance in academic settings are one’s perceptions which topics are worth investing effort in (goal orientations), one’s interpretation of the nature of knowledge (epistemic beliefs), and one’s usage of learning strategies.

180

S. Schworm and H. Gruber

Help-Seeking Goals According to Mercier and Frederiksen (2007), an impasse indicates a learner’s need for help. Metacognitive competencies are required to effectively monitor one’s understanding and to avoid illusions of understanding (Tobias & Everson, 2009). Learners have to decide to actively seek help before setting concrete help goals. Contrary to expectations, a greater need for help does not necessarily lead to a higher level of help seeking. Rather, the relationship is that of an inverted U shape: those who need help the most refuse to ask for it (Karabenick & Knapp, 1988; Wood & Wood, 1999). Successfully self-regulated students “are not more likely to seek help but rather are more likely to seek help if needed” (Karabenick & Berger, 2013, p. 242). Not all ways of help seeking are equally beneficial for learning. A learner’s goal in seeking help may be to merely complete a task without striving for deeper understanding. Accordingly, executive help seeking (looking for a task solution) is just a short-term perspective. A learner’s goal to enhance one’s understanding, however, leads to instrumental help seeking (requesting help considering understanding and future performance) (Nelson-LeGall, 1981; Nelson-LeGall, Kratzer, Jones, & DeCooke, 1990). This kind of help seeking is considered to be adaptive and appropriate (Karabenick & Berger, 2013). Thus, help may not be appropriate if it only supports a learner to complete a task. Schworm and Renkl (2006) showed that in a computer-based learning environment, learners’ self-explaining activity decreased when they were provided with instructional explanations which give answers to the questions asked. Karabenick (2004) showed that executive help seeking results from learners’ intentions to minimize their effort. Goal orientations are expected to impact learners’ efforts, in particular the quality of learning (Zimmerman & Moylan, 2009). Much research on help seeking is framed within the goal orientation theory (Butler, 2006; Ryan & Pintrich, 1997). Goal orientations characterize ways to approach a task (Dweck & Leggett, 1988; Nicholls, 1990), with a distinction between mastery orientation and performance orientation. Masteryoriented students aim at understanding tasks and developing their own competencies. In contrast, performance-oriented students focus on the result of a task, on external evaluation, and on social comparison. Mastery orientation is negatively related with feeling threatened by need for help and help avoidance. It is positively related with instrumental help seeking. Performance orientation is positively related with help-seeking avoidance and with executive help seeking (Arbreton, 1998; Karabenick, 2003, 2004; Ryan & Pintrich, 1997). Research on goal orientations further distinguishes approach goal orientation and avoidance goal orientation. Approach-oriented students strive for looking smart, competing with others, and showing their abilities. Avoidance-oriented students are threatened by potential negative judgment, and they avoid situations in which a lack of ability might become public (Elliot, 1999; Middleton & Midgley, 1997). Evidence exists that mastery-oriented students are more likely to seek adaptive help, feel less threatened by help seeking, and are less likely to avoid seeking help. In contrast, performance-oriented students feel more threatened by help seeking, tend

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs. . .

181

to avoid seeking help, and, if seeking for help, prefer executive help (Karabenick, 2003, 2004; Ryan & Pintrich, 1997; Schworm & Gruber, 2017). Social goals play an important role for understanding learning engagement and achievement. They are conceptualized as distinct orientations toward social competency, and they are linked to adjustment in learning settings (Horst, Finney, & Barron, 2007; Ryan & Shim, 2008). Adolescents’ self-reports of social goals are closely related to their help seeking (Ryan, Hicks, & Midgley, 1997). Ryan and Shim (2011) found that adolescents’ social demonstration approach goals were negatively related to adaptive help seeking as observed by their teachers.

Help Seeking and Learners’ Epistemic Beliefs Learners’ beliefs about knowledge and learning are significantly related to quantity and quality of learning processes and learning outcomes (Hofer, 2001; Kardash & Howell, 2000; Schommer, 1993). In contrast to metacognition – which refers to knowledge about one’s own understanding, knowledge, and strategies – epistemic beliefs are more fundamental assumptions about the limits, certainty, and criteria of knowing and learning. They also include aspects from which sources knowledge can be acquired. Schommer (1990) was the first to propose a multidimensional conception of epistemological beliefs. A continuum of patterns of such beliefs was identified, ranging from a naïve realistic perspective to an elaborated, sophisticated perspective. (In accordance with Moschner and Gruber, 2017, we prefer to use the concept of “epistemic beliefs” rather than “epistemological beliefs.”) Naïve epistemic beliefs indicate learners’ beliefs that the knowledge to be learned consists of a stock of certain facts, which are additively related to each other and whose veracity is guaranteed by an authority. Such facts, once found, represent the world unambiguously. During the educational development, students become aware that knowledge is more complex and less “guaranteed” (King & Kitchener, 2002). More sophisticated epistemic beliefs usually are related with higher-quality learning processes and with better learning outcomes. Hofer and Pintrich (1997) developed a model which consists of four dimensions, (1) certainty of knowledge, (2) simplicity of knowledge, (3) justification of knowledge, and (4) source of knowledge. Hofer (2004) regards epistemic beliefs as parts of metacognition. She suggested to assign beliefs about certainty and simplicity to declarative metacognitive knowledge and beliefs about justification and source to metacognitive monitoring. Epistemic processes in complex learning arrangements may include questions like “How do I know this?” which are considered to be metacognitive reflections (Hofer, 2004). Such metacognitive reflection may encompass thoughts about the source of knowledge. Those may range from naïve beliefs that knowledge resides in external authorities to the sophisticated belief that knowledge is actively constructed in interaction with the environment and with others. Beliefs about transmission of knowledge versus active construction of knowledge lead to different learning strategies (Bromme, Pieschl, & Stahl, 2009).

182

S. Schworm and H. Gruber

Dweck (1999) and Dweck and Leggett (1988) showed that learners’ beliefs about whether their ability is fixed was associated with their goal orientations and with their willingness to invest effort into the learning process. She showed that the belief that intelligence is shapeable and can be enhanced by effort was associated with mastery goal orientation. On the other hand, performance goal orientation was associated with the belief that intelligence is fixed, which implies that it might not be worth investigating effort or even that effort is an indicator of low ability. Thus, learners’ epistemic beliefs impact their preference for particular learning strategies. For example, beliefs about changeability of abilities are related with the willingness to invest effort (Nicholls, 1984, 1990). Dupeyrat and Mariné (2005) confirmed the model of Dweck (1999) in the field of adult education. Their path analysis showed that striving for mastery goals positively was related with learning activities and outcomes. Mastery goals positively were related with learning outcome through the mediation of learners’ willingness to invest effort. Evidence exists that epistemic beliefs are of particular relevance for learning in computer-based settings and that they are related to learners’ help seeking (Aleven, Stahl, Schworm, Fischer, & Wallace, 2003). Bartholomé, Stahl, Pieschl, and Bromme (2006) showed that learners with the belief that knowledge in the content domain of the learning environment is uncertain and unstructured accessed contextsensitive help more often than those who believed knowledge to be more certain and structured.

Help Seeking and Learning Strategies Help seeking in academic settings is an important strategy of self-regulated learning and can foster the acquisition of cognitive skills and abilities (Karabenick & Berger, 2013). Help seeking thus is a strategy of social self-regulation (Zimmerman, 2008) which may be considered as a part of resource-based learning activities. A distinction is made between cognitive, metacognitive, and resource-based learning strategies. Cognitive learning strategies include rehearsal, organization, and elaboration of the learning material. Elaboration activities relate the current learning content to prior knowledge, often through the generation of examples and analogies (Weinstein & Mayer, 1986). One higher-order cognitive strategy, critical reflection, comprises the discussion and evaluation of an issue from different perspectives, which is considered to be a prerequisite of sophisticated epistemic beliefs (Kuhn, 1991). It fosters the understanding of the material to be learned (Entwistle, Entwistle, & Tait, 1993; Pintrich & de Groot, 1990). Metacognitive strategies aim at monitoring and regulation of learning processes. Acknowledging that parts of the learning materials are not yet fully understood is a crucial antecedent of the perception of a need for help. Active help seekers usually show, and prefer, higher-order learning strategies (Karabenick, 2003). Resource-based learning strategies include learners’ endeavors to organize the learning context by adequate time management, cooperation with other learners, or research of relevant literature. Cooperative learning settings positively impact learners’ help seeking (Webb & Farivar, 1999).

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs. . .

183

Help Seeking and Context-Related Factors The motivational characteristics of the learning context (e.g., classroom or university course) play a role in shaping students’ help seeking, too. Ryan, Gheen, and Midgley (1998) showed in a survey study involving 516 fifth graders from 63 mathematics classrooms that students’ self-reported avoidance of help seeking was related to their perception of the classroom goal orientations. Students who perceived the classroom goals as mastery oriented reported lower levels of avoidance, whereas students who perceived the classroom goals as performance oriented reported higher levels of avoidance. Newman (1998) suggested an independent impact of contextual goals on students’ help seeking. He examined the relations between personal goal orientations, contextual goals, and help seeking. Contextual goals were experimentally induced goals that were used to simulate classroom goals. Seventy-eight fourth and fifth grade students were asked to solve mathematics puzzles. Some students were told that doing these puzzles would greatly help them to improve their skill in mathematics (mastery goal). The other students were told that the experimenter wanted to assess how smart they were in mathematics and how they performed compared to other children (performance goal). The results showed that when both contextual and personal goals emphasized performance, students were most reluctant to seek help. For students with personal performance goals, contextual learning goals helped to overcome individual tendencies, resulting in more help seeking. Newman and Schwinger (1993), who examined the relation between help seeking and contextual goals, also found that contextual performance goals more often lead to maladaptive help seeking. University students are increasingly confronted with the current developments of learning. An increasing number of university courses are supplemented with or even replaced by virtual learning environments. Here, the student-to-teacher ratio is often worse than in face-to-face settings, and students are to a greater extent expected to self-regulate planning and organization of the course tasks and of the according learning processes. Typically in a traditional university course, students leave the room after completion of a session, and many do not think again about the course content until the next session. They often do not restudy course contents between sessions although sessions usually build upon each other. However, complex cooperative learning tasks usually cannot be completed within the time frame of a course session. Virtual workspaces offer good opportunities to integrate complex cooperative learning tasks into a university course, but pose new challenges concerning learning activities. Synchronous and asynchronous communication tools like chats and forums enable (and force) students to communicate and work together across space and time. If students have difficulties in understanding the learning content or cannot successfully complete their part of the cooperative learning task, they have to fill this gap by asking for help. Using a virtual workspace is a promising possibility to do so. However, many students’ help-seeking activities are far from optimal (Webb, Ing, Kersting, & Nemer, 2006) which holds true for computer-based settings (Aleven, McLaren, Roll, & Koedinger, 2006). Blended learning environments may

184

S. Schworm and H. Gruber

be used to implement instructional support, but little is known in how far they differ in their contextual goals from traditional classroom settings and how such differences might affect students’ help-seeking activities. In the following two exemplary studies about university students’ help seeking are presented, parts of which already have been published elsewhere (Schworm & Gruber, 2012, 2017). The studies are reanalyzed and extended in order to address and illustrate the theoretical statements of this chapter. Statistical analyses are supplemented according to the common focus on learner-related factors of help seeking. In study 1, students were categorized into adaptive help seekers and help avoiders, based on their profiles of goal orientations, epistemic beliefs, and learning strategies. Study 2 tested whether instructional prompts of help seeking impacted learning processes and fostered learning outcomes.

The Impact of Goal Orientations, Epistemic Beliefs, and Learning Strategies on Help Seeking: Two Empirical Studies Figure 1 summarizes the theoretical background of the studies presented. It shows which groups of variables are conceptualized as learner-related factors or as contextrelated factors. The overlap indicates that both learner-related factors and contextrelated factors impact help seeking. Study 1 aimed to investigate the relationship of help seeking with learner-related factors. Goal orientations, epistemic beliefs, and learning strategies were analyzed in relation to learners’ attitude toward help seeking. As the learning context constrains or affords learner activities and help seeking, it is a matter of instructional methods to take into account learners’ attitudes and beliefs and to implement an instructional context which actively supports adaptive help seeking. Therefore, in study 2 an elicitation function was investigated through prompts which aimed at supporting learners’ help seeking in a distributed learning environment, a course which contained face-to-face as well as virtual components. In the set of both studies, thus three research questions were addressed: 1. Are there clearly distinguishable profiles of attitudes toward help seeking regarding students? 2. How are students’ goal orientations, epistemic beliefs, and learning strategies related to their attitude toward help seeking? 3. Can students’ help seeking be fostered effectively? In order to address the first and second research questions, students’ attitudes toward help seeking and their relations to goal orientations, epistemic beliefs, and learning strategies were investigated (study 1). The third research question addresses students’ actual help seeking in university courses. A virtual workspace was implemented in a blended learning course offering various opportunities for cooperation and help seeking. A pretest/posttest control group study was designed,

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs. . .

185

Fig. 1 Goal orientations, epistemic beliefs, and learning strategies and their relation to help seeking

integrating prompts on effective help seeking in the workspace of the experimental group to foster students’ adaptive help seeking (study 2).

Study 1: The Impact of Goal Orientations, Epistemic Beliefs, and Learning Strategies on Help Seeking Two-hundred and ten students voluntarily took part in the study. All of them were students of undergraduate courses of educational science and participated in a course on qualitative research methodologies. Their mean age was 22.73 (SD = 3.15). Based on the help-seeking questionnaire of Karabenick (2003), 20 items measured students’ attitudes toward instrumental help seeking, executive help seeking, formal help sources, threat of help seeking, and help avoidance. The items were integrated in the questionnaire on epistemological beliefs by Moschner and Gruber (2017). This questionnaire includes 53 items aggregated in seven scales (absolute knowledge, reflexivity of knowledge, cultural bound ways of knowledge, social component of knowledge, gender-related ways of knowledge, learning to learn, value of knowledge). It uses a six-point (1–6) response scale and is anchored with the statements “not true at all” and “completely true.” Students’ self-reported use of learning strategies was assessed by the LIST (Wild & Schiefele, 1994), a German questionnaire based on the Motivated Strategies for Learning Questionnaire (Pintrich, Smith, Garcia, & McKeachie, 1991). Seventy-

186

S. Schworm and H. Gruber

seven items assessed students’ cognitive learning strategies (rehearsal, elaboration, organization, critical reflection), their metacognitive strategies (planning, monitoring, controlling of learning processes), and their resource-based learning strategies (management of time, effort, attention, learning contexts, cooperation with other students, literature search). A five-point response scale (1–5), anchored with the statements “very rarely” to “very often,” is used. Achievement goal orientation was measured by the SELLMO-ST (Spinath & Schöne, 2003) which includes 31 items in four achievement goal orientation scales (mastery orientation, approach performance orientation, avoidance performance orientation, work avoidance). SELLMO-ST uses a five-point response scale (1–5), anchored with the statements “not true at all” and “completely true.” Students’ attitudes towards help-seeking were measured by 20 items based on the help-seeking questionnaire of Karabenick (2003). See Table 2 for scale descriptive statistics and internal consistencies. Looking at the descriptive data, it seems that students are rather adaptive help seekers and do neither feel threatened by the necessity of seeking help nor avoid it. Correlations showed a positive relation between help-seeking threat and help avoidance (r = 0.67). Both indicators are negatively correlated with instrumental help seeking (help threat, r = 0.40; help avoidance, r = 0.53). Instrumental help seekers preferred formal help sources (r = 0.37), while help threat (r = 0.33) and help avoidance (r = 0.34) showed an inverse relationship to the teacher as preferred help source. However, the differences between the two kinds of help seeking which are reported in the literature (Karabenick, 2003; Nelson-LeGall, 1981) could not be replicated. A significant relationship was observed between the attitudes toward instrumental and executive help seeking (r = 0.62). A hierarchical cluster analysis was done to group students by help-seeking indicators. K-means clustering was applied, trying to identify groups of students with similar help seeking. Wards method and squared Euclidean distances were used. Two groups of students were found that showed similar help-seeking patterns. Table 2 shows the z-standardized means of the help-seeking indicators of the two clusters. The pattern of means of cluster 1 indicates that those students were seeking instrumental as well as executive help preferably from formal sources. Thus 70% of the participants can be called adaptive and formal help seekers. These participants show instrumental as well as executive help seeking. Thirty percent of the students, being part of cluster 2, clearly feel threatened by their need of help and avoid help seeking (avoidant help seekers). Table 3 presents the means of goal orientations, epistemic beliefs, and learning strategies of both clusters.

Epistemic Beliefs and Help Seeking To analyze the relationship between epistemic beliefs and attitudes toward help seeking, a stepwise multiple regression analysis was computed. Instrumental help seeking was used as criterion and epistemic beliefs as predictors. A solution with two predictors resulted, with a significant squared multiple correlation of R2 = 0.09 (adjusted R2 = 0.08) resulted (F(2209) = 10.15, p < 0.01). Students’ belief that

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs. . .

187

Table 2 Scale descriptive statistics, internal consistencies (Cronbach’s alpha) Scale Help seeking (max.: 6) Instrumental help seeking Executive help seeking Help-seeking source Help-seeking threat Help-seeking avoidance Epistemological beliefs (max.: 5) Absolute knowledge Cultural bound ways of knowledge Social component of knowledge Gender-related ways of knowledge Learning to learn Value of knowledge Learning strategies (max.: 5) Rehearsal Elaboration Organization Critical reflection Metacognition Cooperation Effort Time management Search for literature Attention Learning environment Achievement goal orientation (max.: 5) Mastery approach Performance approach Performance avoidance Work avoidance

Items

Mean

SD

α

4 4 4 4 4

3.99 3.94 3.13 1.99 1.92

0.96 0.89 0.94 0.88 0.87

0.74 0.70 0.69 0.78 0.79

12 7 5 10 6 5

3.20 4.56 2.25 3.88 4.82 3.45

0.63 0.94 0.84 0.89 0.69 0.89

0.74 0.47 0.60 0.85 0.72 0.75

7 8 8 8 11 7 8 4 4 6 6

3.41 3.40 3.58 2.93 3.59 3.16 3.48 2.85 3.56 3.08 3.68

0.90 0.70 0.71 0.76 0.48 0.76 0.59 1.01 0.78 0.88 0.72

0.53 0.85 0.80 0.82 0.67 0.83 0.74 0.86 0.79 0.75 0.78

8 7 8 8

4.30 2.86 2.06 1.97

0.46 0.72 0.80 0.68

0.76 0.81 0.89 0.86

SD standard deviation

knowledge is absolute (ß = 0.24, p < 0.01) as well as their belief that learning can be learned (ß = 0.17, p < 0.01) predicted the degree of instrumental help seeking. The stepwise multiple regression analysis about the impact of epistemic beliefs on the avoidance of help seeking led to a solution with two predictors. A significant squared multiple correlation of R2 = 0.07 (adjusted R2 = 0.07) resulted (F(2209) = 8.31, p < 0.01). Students’ belief that learning cannot be learned predicted their degree of help avoidance (ß = 0.19, p < 0.01). Students’ belief that there are gender-related ways of knowing (ß = 0.18, p < 0.01) predicted help avoidance. Correlation analyses revealed that this holds especially true for the female students. Within this subgroup, the belief that there are gender-related ways

188

S. Schworm and H. Gruber

Table 3 Z-standardized means (standard deviations in brackets) of help-seeking indicators, separated for help-seeking clusters Scale Instrumental help seeking Executive help seeking Help-seeking threat Help-seeking avoidance Help-seeking source

Active help seekers 0.79 (0.79) 0.81 (0.78) 0.39 (0.53) 0.51 (0.67) 0.75 (0.69)

Help avoiders 0.61 (0.66) 0.62 (0.64) 0.30 (1.17) 0.39 (1.05) 0.58 (0.79)

Table 4 Z-standardized levels of goal orientations, epistemic beliefs, and learning strategies, separated for help-seeking clusters Scale Goal orientations Learning goal orientation Performance approach Performance avoidance Work avoidance

Active help seekers

Help avoiders

0.17 (0.82) 0.02 (0.86) 0.04 (1.24) 0.15 (1.10)

0.13 (1.12) 0.02 (1.11) 0.03 (0.80) 0.12 (0.92)

Epistemic beliefs Absolute knowledge Social component of knowledge Gender-related ways of knowledge Cultural bound ways of knowledge Learning to learn Value of knowledge

0.13 (0.93) 0.07 (0.87) 0.04 (0.94) 0.03 (0.98) 0.32 (0.71) 0.05 (0.92)

0.10 (1.06) 0.05 (1.11) 0.03 (1.07) 0.02 (1.04) 0.25 (1.13) 0.04 (1.08)

Learning strategies Organization Elaboration Critical reflection Rehearsal Metacognition Effort Attention Time management Learning environment Cooperation Search for literature

0.05 (0.94) 0.14 (0.94) 0.22 (1.18) 0.03 (1.05) 0.17 (1.18) 0.31 (1.02) 0.17 (0.87) 0.24 (1.12) 0.33 (0.95) 0.14 (0.93) 0.12 (1.09)

0.04 (1.06) 0.10 (1.01) 0.17 (0.82) 0.03 (0.99) 0.13 (0.85) 0.24 (0.94) 0.13 (1.09) 0.18 (0.87) 0.26 (0.98) 0.11 (1.06) 0.09 (0.94)

of knowing positively correlates with help avoidance (r = 0.23). There is no such relationship within the subgroup of male students. Sexes do not differ in their belief that there are gender-related ways of knowing (t(208) = 1.86, p > .10). MANOVA was used to analyze differences between the two groups of helpseeking students. A significant difference was found between the clusters concerning

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs. . .

189

students’ belief about the ability to learn (F(1209) = 5.43, p < 0.05, eta2 = 0.03). In contrast to help avoiders, adaptive help seekers believe that learning can be learned.

Learning Strategies and Help Seeking Help seeking is regarded as a resource-based learning strategy. Thus its relationship to other learning strategies is of particular interest. As there was a significant correlation between the two forms of help seeking (instrumental help seeking, executive help seeking), more analyses concerning both kinds of help seeking were made. Correlation analyses showed that there was a strong relationship between help seeking and cognitive learning strategies. Both kinds of help seeking also significantly correlated with the use of metacognitive learning strategies. Resource-based learning strategies were directly related with help seeking, with the arrangement of one’s learning environment, and with students’ strategies on literature search. A strong relationship was found between help seeking and the preference of cooperative learning (r = 0.58). Differences between instrumental help seeking and executive help seeking were found in their relationship to students’ appreciation of effort to be relevant for learning and in difficulties in keeping the attention focused on the learning subject. While there was no significant relationship with executive help seeking, instrumental help seekers regarded effort as relevant for learning (r = 0.26), and they did not report problems in focusing their attention (r = 0.14). There was only one significant relationship between the perceived threat of help seeking and the use of learning strategies: students feeling threatened by seeking help avoided cooperative learning settings (r = 0.17). Help avoidance was strongly negatively correlated with several learning strategies (cognitive, metacognitive, and resource-based ones). Help avoiders avoided cooperative learning setting (r = 0.39), they did not regard effort as relevant for learning outcomes (r = 0.15), and they did neither use metacognitive (r = 0.23) nor organizing strategies (r = 0.28). The pattern of the student groups as revealed by the cluster analysis corresponds to these results. Groups of help seekers could clearly be distinguished by their cognitive strategies (organization, F(1210) = 7.52, p < 0.01, eta2 = 0.04; critical reflection, F(1210) = 6.84, p < 0.01, eta2 = 0.03). They could also be distinguished by their metacognitive strategies (F (1210) = 10.33, p < 0.01, eta2 = 0.05) and their resource-based learning strategies (cooperative learning strategies F(1210) = 23.34, p < 0.01, eta2 = 0.10; literature research F(1210) = 16.23, p < 0.01, eta2 = 0.07; effort F(1210) = 5.08, p < 0.05, eta2 = 0.02). The pattern of learning strategies clearly distinguished strategic and formal help seekers from the avoidant seekers, with the adaptive help seekers using all available resources for learning and showing to a larger extent deeper cognitive strategies and metacognitive strategies. Goal Orientations and Help Seeking The stepwise multiple regression analysis concerning the relationship between achievement goal orientations and instrumental help seeking led to a solution with one predictor. A significant squared multiple correlation of R2 = 0.09 (adjusted

190

S. Schworm and H. Gruber

R2 = 0.09) resulted (F(1209) = 21.29, p < 0.01). Students’ avoidance performance orientation predicted their degree of not seeking instrumental help (ß = 0.31, p < 0.01). The stepwise multiple regression analysis about the impact of goal orientations on the avoidance of help seeking led to a solution with two predictors. A significant squared multiple correlation of R2 = 0.19 (adjusted R2 = 0.18) resulted (F(2209) = 24.54, p < 0.01). Students’ avoidance performance orientation predicted their degree of avoiding help seeking (ß = 0.39, p < 0.01), whereas learning orientation loaded significantly negative (ß = 0.17, p < .01). Students with learning orientation do not avoid help seeking. A MANOVA was computed to reveal differences between the clusters of help seekers concerning the achievement goal orientations. Results show that the two clusters can clearly be distinguished by their goal orientations, with avoidant help seekers showing a higher approach performance orientation (F(1210) = 5.06, p < 0.05, eta2 = 0.02) as well as an avoidance performance orientation (F(1210) = 37.71, p < 0.01, eta2 = 0.15) and work avoidance (F(1210) = 8.70, p < 0.01, eta2 = 0.04). In contrast, adaptive help seekers show to a larger degree a learning-oriented goal orientation (F(1210) = 4.04, p < 0.05, eta2 = 0.02).

Discussion of Study 1 Study 1 revealed that adaptive help seekers prefer formal help sources. They do neither feel threatened by a need of help nor do they avoid help seeking in general. On the other hand, learners who feel threatened by their need of help tend to avoid seeking help at all. This distinction is supported by a cluster analysis which led to a two-cluster solution: (1) adaptive, formal help seekers and (2) avoidant help seekers. These groups differ in their interpretations of the components involved in learning and knowledge acquisition. Of particular interest are learners’ goal orientations, epistemic beliefs, and learning strategies. Considering epistemic beliefs, the belief of learners that learning can be learned is a significant predictor of adaptive help seeking and (with negative load) of avoidance of help seeking. Thus one possible intervention to foster adaptive help seeking and to reduce avoidance of help seeking is to develop students’ beliefs in their ability to learn how to learn. Females who think that knowledge is gender related tend to avoid seeking help. There is only speculation about the reason of this relationship. Attributional processes might lead women to think that they lack knowledge (e.g., in mathematics) due to genderspecific disabilities and in a consequence perhaps try not to be judged as “silly woman.” Performance goal orientation, in particular avoidance goal orientation, proved to be a relevant predictor of help avoidance or of not seeking instrumental help. Learning orientation on the other hand reduces avoidance of help seeking. Thus fostering learning orientation and reducing avoidance performance orientation should be important instructional goals to be addressed in educational efforts.

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs. . .

191

Study 2: The Impact of Prompting Adaptive Help Seeking on Activity in a Virtual Workspace, Acceptance of the Learning Environment, and Learning Outcome Study 1 showed that using cooperative learning strategies and metacognitive learning strategies is a paramount way to distinguish adaptive help seekers from nonadaptive and avoidant ones. The analyses of the correlative patterns of help-seeking indicators and learning strategies reveal clear relations between cooperative learning as a resource-based learning strategy and metacognitive learning strategies on the one hand and help seeking as a resource-based learning strategy on the other hand. As cooperative learning strategies have proven to be strongly related to a positive attitude to help seeking, in study 2 help seeking and its relation to cooperative learning in a virtual learning environment were focused. In this study it was asked to what extent the relationships revealed in study 1, a questionnaire study, could be transferred to actual learning in a university course. In order to investigate help seeking, a prompting procedure was implemented. Prompts are questions or elicitations which aim to induce meaningful learning activities by eliciting learning strategies and learning activities that the students are capable of, but do not show spontaneously (King, 1996; Pressley et al., 1992). Prompts were assumed to reduce perceived help threat and to foster instrumental help seeking. Thirty-nine students took part in study 2. All were students of educational science and attended an undergraduate course on qualitative research methods. Their mean age was 23.90 years (SD = 4.20 years). Eleven were male and 28 female. The subjects were randomly assigned to one of two conditions of a control group design (experimental group, with prompts on effective help seeking; control group, without prompts on effective help seeking). The course comprised 11 sessions. The first session was an introductory session. In the last session, the students presented the results of their cooperative work in a poster session. In addition, the end-of-course test was applied, and a questionnaire was handed out assessing students’ self-reported learning activities in the virtual workspace and their feedback to the course environment. The course was supplemented by group activities which were organized in a virtual workspace. During the course all students had to complete a cooperative task (doing an interview study) and to work on two individual tasks. During the course, the experimental group received two written prompts indicating the importance of effective help seeking for successful learning. The control group did not receive any prompts. A self-report questionnaire assessed students’ help seeking and their activity in the virtual workspace. Content analysis was made to analyze students’ actual activities in the forums while working on the cooperative and individual tasks. Perceived threat of help seeking, degree of cooperative work, and acceptance of the prompts were assessed by a questionnaire. Prior knowledge and learning outcome were assessed by tests.

192

S. Schworm and H. Gruber

Help-Seeking Indicators, Goal Orientations, Epistemic Beliefs, and Learning Strategies Students were rather adaptive help seekers and did neither feel threatened by the necessity of seeking help nor avoided it. Correlations revealed a similar pattern as in study 1. A strong significant correlation was found between executive and instrumental help seeking (r = 0.71). Thus, in further analyses the scores were aggregated to a new score “adaptive help seeking.” The expected significant positive relation was found between help-seeking threat and help avoidance (r = 0.35). Both indicators were negatively correlated with adaptive help seeking (help threat, r = 0.45; help avoidance, r = 0.57). Results confirmed that adaptive help seekers prefer formal help sources (r = 0.57). Learners with high scores on help threat (r = 0.28) and help avoidance (r = 0.29) on the other hand tended to avoid the teacher as help source; the correlations, however, are not statistically significant. To analyze the impact of goal orientations and epistemic beliefs on help seeking, a regression analysis was calculated including the paramount variables of study 1. The regression analysis including the epistemic belief scales “learning to learn” and “absolute knowledge” as well as the achievement goal orientation scale “avoidance performance goals” led to a significant squared multiple correlation of R2 = 0.24 (adjusted R2 = 0.18) (F(3,38) = 3.70, p < 0.05). Students’ beliefs that learning can be learned (ß = 0.45, p < 0.05) and that knowledge is something absolute (ß = 0.49, p < 0.01) both predicted help-seeking activities. However, students’ avoidance performance orientation did not predict their degree of not seeking help (ß = 0.05, p > 0.10), as was suggested by study 1. Help-seeking avoidance was not significantly predicted by epistemic beliefs and achievement goals (F < 1). By and large the results of study 1 concerning learning strategies could be replicated. Adaptive help seeking was positively related with cooperative work (r = 0.41) and metacognitive learning strategies (r = 0.35), as well as with organizational strategies (r = 0.34) and the belief that effort is relevant for learning (r = 0.48). Amazingly, there were no significant correlations between learning strategies and students’ help avoidance, except that organizational strategies were negatively correlated with help avoidance (r = 0.33). Learning Outcomes Prompting help-seeking activities led to significant differences in learning outcomes. At the end of the course, both groups differed in their knowledge about qualitative research methods (t(37) = 0.1.84, p < 0.05, one-sided). Some other factors also impacted learners’ posttest results. Correlative analyses revealed positive relationships between learning outcome and learners’ activity in the virtual workspace (r = 0.43), perceived difficulty of the course (r = 0.38), cooperative learning activities (r = 0.42), instrumental help seeking (r = 0.60), and executive help seeking (r = 0.48). A regression analysis included those variables and learning outcome led to a solution with four significant predictors. A significant squared multiple correlation of R2 = 0.53 (adjusted R2 = 0.45) resulted (F(5,36) = 86.92,

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs. . .

193

p < 0.001). Instrumental help seeking (ß = 0.33, p < 0.05) and executive help seeking (ß = 0.29, p < 0.05) predicted students’ learning outcome, as well as perceived difficulty of the course (ß = 0.38, p < 0.01) and activity in the virtual workspace (ß = 0.32, p < 0.05). Qualitative analyses of the learners’ activities as provided by their forum posts display interesting details.

Activity in the Virtual Workspace The course was divided into two virtual parts. Each virtual group had the opportunity to take part in eight forums, dealing, for example, how to develop a questionnaire or how to analyze qualitative data. Students were offered to adaptively participate in those forums. An analysis of the forums of revealed considerable participation rates in both groups (234 posts in the control group, 201 posts in the experimental group; posts from the teachers were excluded). Overall participation in the control group was even higher than in the experimental group. However, qualitative differences were found. In the experimental group, there were 92 starts of new discussions, compared to 40 in the control group. The participants of the prompted group seemed to have been more initiative. Categorization of posts was used to distinguish the categories “organization” (statements about, e.g., coordination of group work) and “content” (statements about the learning content). In the experimental group, 77% of the post dealt with learning content, compared to 57% in the control group. The content of the posts might serve as an indicator of help seeking. The posts were distinguished as addressing either “exchange of information” or “requests.” In the experimental group, 24% of the posts contained questions, compared to 17% in the control group. Discussion of Study 2 Study 2 revealed that prompting students on adaptive help seeking as a strategy of self-regulated learning fosters learning in a blended learning course. Students’ evaluation of the prompts were positively related to their self-reported use of the virtual workspace (r = 0.61). Actively working in the virtual workspace thus was identified as a relevant predictor of learning outcomes. Although the two experimental groups did not differ concerning the predictors of learning outcome (i.e., instrumental help seeking, executive help seeking, perceived difficulty of the course, activity in the virtual workspace), they differed in their participation at learning activities in the virtual workspace. Students who were prompted on help seeking started more often new discussions in the forum. Contributions were more often focused on course-relevant content, and questions were more frequently asked. No substantial relationships were found between students’ attitudes toward help seeking and their actual help seeking. However, correlates of students’ help seeking proved to be relevant for learning outcome. Students asking for instrumental help judged the help-seeking prompts as correct and helpful (r = 0.54). They reported a frequent use of the virtual workspace (r = 0.36) and of cooperative learning arrangements (r = 0.44).

194

S. Schworm and H. Gruber

Concerning attitudes toward adaptive help seeking, the main parts of the pattern revealed in study 1 could be replicated. Adaptive help seekers believe that learning can be learned and that knowledge is something absolute. They prefer cooperative learning settings and make much use both of metacognitive and of resource-based learning strategies. The results concerning help-seeking avoidance from study 1 could not be replicated. This may have been due to the fact that many students in study 2 were highly adaptive help seekers, but not help-seeking avoiders.

Conclusion Studies 1 and 2 show that there are clearly distinguishable patterns of attitudes toward help seeking in groups of students of educational science. These patterns are related to students’ goal orientations, epistemic beliefs, and learning strategies. The results of study 1 indicate that adaptive help seekers prefer formal help sources and do not feel threatened by a need of help nor do they avoid help seeking in general. On the other hand, learners who feel threatened by their need of help tend to avoid seeking help at all. The analysis of goal orientations, epistemic beliefs, and learning strategies and their relationship to learners’ attitude toward help seeking revealed that the belief of learners that learning can be learned is a significant predictor of adaptive help seeking and as well, with negative load, of avoidance of help seeking. Performance goal orientation, in particular avoidance goal orientation, proved to be a relevant predictor of help avoidance, while learning orientation on the other hand is negatively related with the avoidance of help seeking. Cooperative learning strategies and metacognitive learning strategies are strongly positively related to adaptive help seeking. However, the data assessed mainly is based on student’s self-reports. There were no direct relationships measured between students’ questionnaire data, their actual help seeking, and learning outcome which of course weakens the external validity of the results found on the relevance of beliefs and strategy use. Behavioral data, e.g., assessed by social badges, would be helpful and future research should definitely take this into focus. However, prompting students on adaptive help seeking impacted actual learning activities, namely, help seeking and learning outcome. Students’ evaluations of the prompts were positively related with their self-reported use of the virtual workspace. Actively working in the virtual workspace was identified as relevant predictor of learning outcomes. Students’ prompted on help seeking started more discussions in the forums, their contributions more often focused on course-relevant content, and they posed questions more frequently. It is a rewarding aim of future research to further investigate the impact of instructional settings on learners’ actual help seeking, in particular in university settings. Implementing cooperative settings in university courses and supporting learning goal orientations seem to be promising steps in order to help students to become self-regulated learners who actively seek for help – if necessary.

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs. . .

195

References Aleven, V., McLaren, B. M., Roll, I., & Koedinger, K. R. (2006). Toward metacognitive tutoring: A model of help seeking with a cognitive tutor. International Journal of Artificial Intelligence in Education, 16, 101–130. Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003). Help seeking and help design in interactive learning environments. Review of Educational Research, 73, 277–320. Arbreton, A. (1998). Student goal orientation and help seeking strategy use. In S. A. Karabenick (Ed.), Strategic help seeking. Implications for learning and teaching (pp. 95–116). Mahwah, NJ: Erlbaum. Bartholomé, T., Stahl, E., Pieschl, S., & Bromme, R. (2006). What matters in help seeking? A study of help effectiveness and learner related factors. Computers in Human Behavior, 22, 113–129. Bromme, R., Pieschl, S., & Stahl, E. (2009). Epistemological beliefs are standards for adaptive learning: A functional theory about epistemological beliefs and metacognition. Metacognition and Learning, 5, 7–26. Butler, R. (2006). An achievement goal perspective on student help seeking and teacher help giving in the classroom: Theory, research, and educational implications. In S. A. Karabenick & R. S. Newman (Eds.), Help seeking in academic settings: Goals, groups, and contexts (pp. 15–44). Mahwah, NJ: Erlbaum. Dupeyrat, C., & Mariné, C. (2005). Implicit theories of intelligence, goal orientation, cognitive engagement, and achievement: A test of Dweck’s model with returning to school adults. Contemporary Educational Psychology, 30, 43–59. Dweck, C. S. (1999). Self-theories: Their role in motivation, personality, and development. Philadelphia: Taylor & Francis. Dweck, C. S., & Leggett, E. (1988). A social cognitive approach to motivation and personality. Psychological Review, 95, 256–273. Elliot, A. J. (1999). Approach and avoidance motivation and achievement goals. Educational Psychologist, 34, 169–189. Entwistle, N. J., Entwistle, A., & Tait, H. (1993). Academic understanding and contexts to enhance it: A perspective from research on student learning. In T. Duffy, J. Lowyck, & D. H. Jonassen (Eds.), Designing environments for constructive learning (pp. 331–357). Berlin, Germany: Springer. Hofer, B. K. (2001). Personal epistemology research: Implications for learning and teaching. Journal of Educational Psychology Review, 13, 353–383. Hofer, B. K. (2004). Epistemological understanding as a metacognitive process: Thinking aloud during online-searching. Educational Psychologist, 39, 43–55. Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67, 88–140. Horst, S. J., Finney, S. J., & Barron, K. E. (2007). Moving beyond academic achievement measures: A study of social achievement goals. Contemporary Educational Psychology, 32, 667–698. Karabenick, S. A. (2003). Seeking help in large college classes: A person-centered approach. Contemporary Educational Psychology, 28, 37–58. Karabenick, S. A. (2004). Perceived achievement goal structure and college student help seeking. Journal of Educational Psychology, 96, 569–581. Karabenick, S. A., & Berger, J. (2013). Help seeking as a self-regulated learning strategy. In H. Bembenutty, T. J. Cleary, & A. Kitsantas (Eds.), Applications of self-regulated learning across diverse disciplines (pp. 237–261). Charlotte, NC: Information Age. Karabenick, S. A., & Dembo, M. H. (2011). Understanding and facilitating self-regulated help seeking. New Directions for Teaching and Learning, 126, 33–43. Karabenick, S. A., & Knapp, J. R. (1988). Effects of computer privacy on help seeking. Journal of Applied Social Psychology, 18, 461–472.

196

S. Schworm and H. Gruber

Karabenick, S. A., & Newman, R. S. (2010). Seeking help as an adaptive response to learning difficulties: Person, situation and developmental influences. In E. Baker, P. L. Peterson, & B. McGraw (Eds.), Instructional encyclopaedia of education (3rd ed., pp. 653–659). Amsterdam: Elsevier. Kardash, C. M., & Howell, K. L. (2000). Effects of epistemological beliefs and topic-specific beliefs on undergraduates’ cognitive and strategic processing of dual-positional text. Journal of Educational Psychology, 92, 524–535. King, A. (1996). Teaching students to generate questions: A review of the intervention studies. Review of Educational Research, 66, 181–221. King, P. M., & Kitchener, K. S. (2002). The reflective judgment model: Twenty years of research on epistemic cognition. In B. K. Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 37–61). Mahwah, NJ: Erlbaum. Kuhn, D. (1991). The skills of argument. Cambridge, UK: Cambridge University Press. Mäkitalo-Siegl, K., & Fischer, F. (2013). Help seeking in computer-supported collaborative science learning environments. In S. A. Karabenick & M. Puustinen (Eds.), Advances in help seeking research and applications: The role of emerging technologies (pp. 99–120). Charlotte, NC: Information Age. Mercier, J., & Frederiksen, C. H. (2007). Individual differences in graduate students’ help seeking process in using a computer coach in problem based learning. Learning and Instruction, 17, 184–203. Middleton, M., & Midgley, C. (1997). Avoiding the demonstration of the lack of ability: An underexplored aspect of goal theory. Journal of Educational Psychology, 89, 710–718. Moschner, B., & Gruber, H. (2017). Erfassung epistemischer Überzeugungen mit dem FEE [Measuring epistemological beliefs with the FEE]. In A. Bernholt, H. Gruber, & B. Moschner (Eds.), Wissen und Lernen – in der Sicht von Lehrenden und Lernenden. Wie epistemische Überzeugungen Schule, Universität und Arbeitswelt beeinflussen. Münster, Germany: Waxmann. Nelson-Le Gall, S. (1981). Help seeking: An understudied problem-solving skill in children. Developmental Review, 1, 224–246. Nelson-Le Gall, S., Kratzer, L., Jones, E., & DeCooke, P. (1990). Children’s self-assessment of performance and task-related help seeking. Journal of Experimental Child Psychology, 49, 245–263. Newman, R. S. (1994). Adaptive help seeking: A strategy of self-regulated learning. In D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulation of learning and performance: Issues and educational applications (pp. 283–301). Hillsdale, MI: Erlbaum. Newman, R. S. (1998). Students’ help seeking during problem solving: Influences of personal and contextual achievement goals. Journal of Educational Psychology, 90, 644–658. Newman, R. S., & Schwager, M. T. (1993). Students’ help seeking during problem solving: Effects of grade, goal, and prior achievement. American Educational Research Journal, 32, 352–376. Nicholls, J. G. (1984). Achievement motivation: Conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 91, 328–346. Nicholls, J. G. (1990). What is ability and why are we mindful of it: A developmental perspective. In R. Sternberg & J. Kolligian (Eds.), Competence considered (pp. 11–40). New Haven, CT: Yale University Press. Pintrich, P. R., & de Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33–40. Pintrich, P. R., Smith, D., Garcia, T., & McKeachie, W. (1991). The motivated strategies for learning questionnaire (MSQL). Ann Arbor, MI: University of Michigan. Pressley, M., Wood, E., Woloshyn, V. E., Martin, V., King, A., & Menke, D. (1992). Encouraging mindful use of prior knowledge: Attempting to construct explanatory answers facilitates learning. Educational Psychologist, 27, 91–109.

8

Learning Theories: The Impact of Goal Orientations, Epistemic Beliefs. . .

197

Ryan, A. M., Gheen, M. H., & Midgley, C. (1998). Why do some students avoid asking for help? An examination of the interplay among ‘students’ academic efficacy, ‘teachers’ social-emotional role, and the classroom goal structure. Journal of Educational Psychology, 90, 528–535. Ryan, A. M., Hicks, L., & Midgley, C. (1997). Social goals, academic goals, and avoiding seeking help in the classroom. Journal of Early Adolescence, 17, 152–171. Ryan, A. M., & Pintrich, P. R. (1997). “Should I ask for help?” The role of motivation and attitudes in ‘adolescents’ help seeking in math class. Journal of Educational Psychology, 89, 329–341. Ryan, A. M., & Shim, S. S. (2008). An exploration of young ‘adolescents’ social achievement goals and social adjustment in middle school. Journal of Educational Psychology, 100, 672–687. Ryan, A. M., & Shim, S. S. (2011). Help seeking tendencies during early adolescence: An examination of motivational correlates and consequences for achievement. Learning and Instruction, 21, 247–256. Schommer, M. (1990). Effects of beliefs about the nature of knowledge on comprehension. Journal of Educational Psychology, 82, 498–504. Schommer, M. (1993). Epistemological development and academic performance among secondary students. Journal of Educational Psychology, 85, 406–411. Schworm, S., & Gruber, H. (2012). E-learning in universities: Supporting help seeking processes by instructional prompts. British Journal of Educational Technology, 43, 272–281. Schworm, S., & Gruber, H. (2017). Academic help seeking: The influence of epistemological beliefs, learning strategies and goal orientation. In A. Bernholt, H. Gruber, & B. Moschner (Eds.), Wissen und Lernen – in der Sicht von Lehrenden und Lernenden. Wie epistemische Überzeugungen Schule, Universität und Arbeitswelt beeinflussen. Münster, Germany: Waxmann. Schworm, S., & Nistor, N. (2013). Elements of social computing in online help design. Fostering help seeking activities in communities of practice. In S. A. Karabenick & M. Puustinen (Eds.), Advances in help seeking research and applications: The role of emerging technologies (pp. 179–203). Charlotte, NC: Information Age. Schworm, S., & Renkl, A. (2006). Computer-supported example based learning: When instructional explanations reduce self-explanations. Computers and Education, 46, 426–445. Shim, S. S., Kiefer, S. M., & Wang, C. (2013). Help seeking amongst peers: The role of goal structure and peer climate. The Journal of Educational Research, 106, 290–300. Spinath, B., & Schöne, C. (2003). Ziele als Bedingungen von Motivation am Beispiel der Skalen zur Erfassung der Lern- und Leistungsmotivation (SELLMO) [Goals as a precondition of motivation. The SELLMO scales of learning orientation and achievement orientation]. In J. Stiensmeier-Pelster & F. Rheinberg (Eds.), Diagnostik von Motivation und Selbstkonzept (pp. 29–40). Göttingen, Germany: Hogrefe. Tobias, S., & Everson, H. T. (2009). The importance of knowing what you know: A knowledge monitoring framework for studying metacognition in education. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook of metacognition in education (pp. 107–128). New York, NY: Routledge. Webb, N. M., & Farivar, S. (1999). Developing productive group interaction in middle school. In A. M. O’Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 117–149). Mahwah, NJ: Erlbaum. Webb, N. M., Ing, M., Kersting, N., & Nemer, K. M. (2006). Help seeking in cooperative learning groups. In S. Karabenick (Ed.), Strategic help seeking: Implications for learning and teaching (pp. 45–115). Mahwah, NJ: Erlbaum. Weinstein, C. E., & Mayer, R. E. (1986). The teaching of learning strategies. In C. M. Wittrock (Ed.), Handbook of research in teaching (pp. 315–327). New York: Macmillan. Wild, K.-P., & Schiefele, U. (1994). Lernstrategien im Studium. Ergebnisse zur Faktorenstruktur und Reliabilität eines neuen Fragebogens [Learning strategies at university. Factorial structure and reliability of a new questionnaire]. Zeitschrift für Differentielle und Diagnostische Psychologie, 15, 185–200.

198

S. Schworm and H. Gruber

Wood, H., & Wood, D. (1999). Help seeking, learning and contingent tutoring. Computers and Education, 33, 153–169. Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45, 166–183. Zimmerman, B. J., & Moylan, A. R. (2009). Where metacognition and motivation intersect. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook of metacognition and education (pp. 299–315). New York: Routledge.

Silke Schworm (born 1973) is Professor of Educational Science at the University of Regensburg (Germany) since 2012. Her research interests lie in the field of cognitive learning processes with a special focus on multimedia learning, academic help seeking, instructional design, and higher education. Hans Gruber (born 1960) is Full Professor of Educational Science at the University of Regensburg (Germany) since 1998 and Visiting Professor at the Faculty of Education, University of Turku (Finland) since 2015 which conferred an Honorary Doctorate to him. His research interests lie in the field of professional learning, expertise, workplace learning, social network analysis, and higher education. He served as Vice-Rector for Study Affairs at the University of Regensburg (Germany) and repeatedly as a member of the Review Board “Education Sciences” of the German Research Foundation (Deutsche Forschungsgemeinschaft). Currently he is President of the “European Association for Research on Learning and Instruction” (EARLI) and Dean of the Faculty of Psychology, Educational Science and Sports Science at the University of Regensburg (Germany). He serves as reviewer for many international journals, book series, and research organizations.

9

Networked Societies for Learning: Emergent Learning Activity in Connected and Participatory Meshworks Lucila Carvalho

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Networked Learning as Connected and Participatory Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Understanding Meshworks: Analysis and Design for Networked Learning . . . . . . . . . . . . . . . . . . . Socio-material Approaches in Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Theory of Entanglement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Foregrounding Key Components and Entanglement in MONA and CmyView . . . . . . . . . . . . . . . Case Study 1: Museum of Old and New Art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Case Study 2: CmyView . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Connection and Participation: The Emergent Activity of Networked Learners . . . . . . . . . . . . Conclusion: Designing for Learning in the Digital Age . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

200 201 203 203 205 207 207 211 215 216 217

Abstract

Within richly networked societies, a new culture of learning is operating – a culture where knowledge is perceived as fluid and constantly evolving, where technology is used to promote connections, and where individual experiences are enhanced and refined through collective encounters. Learning in the Digital Age is often mediated through connections in networks of people and things – it involves understanding what are the best tools and social strategies that may support collaboration and sustain people’s connections in networked structures. In order to help the twenty-first century learners to develop skills and knowledge necessary to successfully navigate the digital realm, educational designers need analytical tools to help them address the complexity of contemporary learning situations. This chapter discusses a networked learning approach to the analysis of complex learning situations, which combines ideas from the Activity-Centred L. Carvalho (*) Institute of Education, College of Humanities and Social Sciences, Massey University, Auckland, New Zealand e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_55

199

200

L. Carvalho

Analysis and Design (ACAD) framework, and the notion of entanglement of things and humans. The application of the networked approach is illustrated through two case studies – one involves learning within a museum setting and the second one is an outdoor activity with university students. The approach helps reveal deeper and innovative understandings of the learning connections and modes of participation that are at play in contemporary networked learning situations. Keywords

Networked learning · Design for learning · Digital literacy · Entanglement

Introduction We live in a rapidly changing world, a place where most of our everyday activities are carried out through our connections to networks of people and things. People work, shop, and interact with others, while tapping on personal mobile devices. These innovative technologies enable a quick turnaround of (and access to) knowledge and information and have transformed socioeconomic practices in the fastpaced Digital Age (Laurillard, 2002; Thomas & Brown, 2011). By and large, the “stable infrastructure” of the twentieth century was overturned by a more “fluid infrastructure,” where technology is both constantly creating change and disruption and an instrument to respond to these alterations (Gee & Hayes, 2011; Goodyear, Carvalho, & Dohn, 2016; Thomas & Brown, 2011). This is partly due to the nature of innovative technologies and their increased accessibility, capability, and ubiquity, coupled, of course, with an efficient infrastructure to support and enable their integration into all aspects of our lives. As a result, new possibilities for learning, working, and leisure are also opening up. Learning implies changes in both individuals and environments – it involves people’s ability to match their actions to alterations in their surroundings (Damsa & Jornet, 2016; Ingold, 2013). Learning in the twenty-first century, however, is also associated with an understanding of how to successfully navigate the digital realm and what are the best tools and social strategies that may support collaboration and social interactions and sustain our connections in networked structures (de Laat & Dawson, 2017; Jandric & Boras, 2015). Lack of access, or lack of understanding on how to productively use digital tools, may negatively impact both: people’s individual lives and community prosperity (OECD, 2015; Warschauer & Ware, 2008). Therefore, a crucial question for those in education revolves around how to best engage in design for learning in ways that acknowledge the complexity of the learning situations we find ourselves in – networked learning involves multifaceted assemblages of artifacts, tools, places, ideas, and people. Goodyear and Carvalho’s (2014) networked learning perspective conceptualizes learner’s activity as an emergent phenomenon, as something that cannot be entirely predicted or controlled (only influenced) by an educators’ design choices of tools,

9

Networked Societies for Learning: Emergent Learning Activity in. . .

201

tasks, and social arrangements that will be at play at “learntime” – the moment when learners engage and interact with the assemblage of elements in a learning situation. This perspective is in accord with recent developments in the learning sciences that transcend binary notions of formal and informal and physical and digital to highlight relationality (Gourlay & Oliver, 2016). Goodyear and Carvalho (2014) connect designable elements and activity – with a focus on what emerges as a meshwork of people and things at learntime. They stress that design for networked learning must acknowledge learning activity as epistemically, socially, and physically situated (Goodyear & Carvalho, 2014; Carvalho, Goodyear, & de Laat, 2017). This chapter describes the Activity-Centred Analysis and Design framework (ACAD) (Goodyear & Carvalho, 2014; Carvalho et al., 2017) – and its useful analytical tools to understand complex learning situations in the Digital Age. The ACAD framework is used in combination with concepts from the theory of entanglement (Hodder, 2012), which notes people’s evolving dependence and dependency on material artifacts, thoughts, and things. The application of this networked learning approach is illustrated through the empirical analysis of two learning networks – one connected to learning in a museum setting and another with university students. Within this scenario, this chapter has two main aims. Firstly, it introduces analytical tools which may help educators identify key elements of learning networks, framing the structural components of learning situations and illustrating how these elements connect to one another. Secondly, the chapter shows how the approach may deepen the empirical analyses of contemporary learning situations, conceptualizing the multiple interconnected dimensions of learning networks, which, in turn, bring important and innovative research questions to the fore. The chapter shows how the approach helps reveal connections between the qualities of design elements and the emergent learner’s activity that ensues. The chapter concludes with implications for those involved in design for learning in richly networked societies, emphasizing the need to acknowledge networked learning as an emergent phenomenon, which transcends old binary notions that often separate formal and informal education, or strictly distinguish between physical and digital realms.

Networked Learning as Connected and Participatory Practices Networked learning is defined as the use of information and communication technologies “to promote connections: between one learner and other learners, between learners and tutors; between a learning community and its learning resources” (Goodyear, Banks, Hodgson, & McConnell, 2004, p. 3). Ultimately, networked learning is not so much about the technology itself but implies “a different way of thinking about relationships between digital technologies, and the processes of learning and education” (Jones, 2015, p. 5). Grounded on openness and fluidity, networked learning has its roots in the 1990s, within the context of higher education in the United Kingdom. It combined, what was then, a recent rise of technologies

202

L. Carvalho

with ideas from critical theory, emphasizing an active social role and individual agency for learners and teachers (Jandric & Boras, 2015; Jones, 2015). Networked learning came as a response to a perceived shift in learning, acknowledging the need to go beyond the written text, to encompass other forms of academic literacy, such as images, graphic representations, and audio (Jones, 2008). Since then, networked learning has expanded in scope to embrace learning situations beyond university settings, including learning in schools, museums, libraries, professional, and other non-formal settings (Bonderup Dohn, Crammer, Sime, de Laat, & Ryberg, 2018; Hodgson, de Laat, McConnell, & Ryberg, 2014). As technology became more portable and ubiquitous, different learning opportunities flourished, with digital enhancements that augment all sorts of learning activity within physical spaces (Carvalho et al., 2017). It is undeniable that technology has enabled a new “culture of learning” to emerge, challenging educators to rethink learning and pedagogical practices in the Digital Age (Beetham & Sharpe, 2013; Thomas & Brown, 2011). The traditions of teacher-centered approaches and notions of transmission and acquisition of knowledge, as well as beliefs associated with the usefulness of standardized performance, are all being questioned, as educators and learners embrace learning as a more open, connected, and participatory endeavor. In networked societies for learning, learners are no longer seen as passive consumers of information but as both consumers and producers (Ito et al., 2013; Jenkins, Purushotma, Weigel, Clinton, & Robison, 2009; Jones, 2008). As such, the ability to find, critically analyze, evaluate, and use digital information and the ability to engage in the creation of digital content are some of the key skills and knowledge associated with “digital literacy.” Digital literacy is about “capabilities which fit an individual for living, learning and working in a digital society” (JISC, 2014, online) and involves people’s digital behaviors, practices, and identities, which influence one’s academic and professional situated practices, but which also may change over time and across contexts. Digital literacy is, therefore, intrinsically important to networked societies – people need to be proficient in the digital behaviors and practices that will successfully support their activity in networked structures. Not surprisingly, schools and universities have devised and incorporated digital literacy competencies as part of their curricula (OECD, 2015; Warschauer & Ware, 2008), recognizing that further in-depth knowledge of the digital world enables people to rapidly adapt to new contexts and engage in co-creation activities in the new era (Adams Becker et al., 2017; Miller & Barlett, 2012). Although competencies associated with digital literacies have been progressively added to schools’ and universities’ curricula, these skills, practices, behaviors, and identities are not necessarily confined to formal education (Ito et al., 2013; Jenkins et al., 2009; Thomas & Brown, 2011). As Thomas and Brown (2011) remind us, a new culture of learning has emerged and is powerfully influenced by the interplay between two key factors. On the one hand, we have now access to massive amounts of information, networks, and resources, where people can learn about all sorts of topics. On the other hand, these are also bounded and structured spaces, with

9

Networked Societies for Learning: Emergent Learning Activity in. . .

203

possibilities for unlimited agency, as people build and experiment with different opportunities within those boundaries. Play, innovation, and the cultivation of imagination are seen as foundational elements for learning. Knowledge and information are now perceived as fluid and constantly evolving, and individual experiences are often enhanced and refined through collective encounters (Thomas & Brown, 2011). Networked and connected learning share core values in critical pedagogy (Cronin, 2016; Gogia, 2016; Jones, 2015). Connected learning usually implies a focus on media production and social practices of young people in out-of-school settings (Ito et al., 2013), while networked learning tended to be seen as related to adults’ education (Jones, 2015). To understand how activity unfolds in complex learning situations where technology is used to promote connections and to figure out how to best design for networked learning within these situations, we need analytical tools to help us identify key elements of a learning network. We need analytical tools that acknowledge that the ways and settings where people learn are being transformed and extended. We need analytical tools that can cope with learning activity that emerges not only as a result of experiences in formal educational (e.g., university, schools, etc.) but in other less formal settings. In the next section, the Activity-Centred Analysis and Design framework (Goodyear & Carvalho, 2014) is described, along with some core concepts from Hodder’s (2012) theory of entanglement. The application of a networked learning approach, which combines ACAD and entanglement, is then illustrated through the analysis of two case study scenarios: in a museum environment and in an outdoor activity conducted with university students.

Understanding Meshworks: Analysis and Design for Networked Learning Socio-material Approaches in Education In the past decade or so, socio-material theories have been gaining attention, challenging approaches that see social, cultural, and personal as the only defining factors for what it means to learn (Ellis & Goodyear, 2018; Fenwick & Edwards, 2011; Fenwick, Edwards, & Sawchuk, 2011; Gourlay & Oliver, 2016; Sørensen, 2009; Orlikowski, 2007). Socio-materiality argues for the foregrounding of the material in learning activity, for noticing the relevance of networks of humans and things, as well as the relational interactions they enable (Ingold, 2011, 2012). As pointed out by Fenwick and Edwards (2011): Any changes we might describe as policy — new ideas, innovations, changes in behavior, transformations — emerge through the effects of relational interactions and assemblages, in various kinds of more-than-human networks entangled with one another, that may be messy and incoherent, spread across time and space. (Fenwick & Edwards, 2011, p. 712)

204

L. Carvalho

On this view, learning is to be seen as a situated and embodied experience – not only we need to acknowledge the physicality of the space itself, but we also should embrace its relationship with individual, social, cultural, and political contexts (Boddington & Boys, 2011). Indeed, although we are surrounded by material things and have our feet firmly grounded on physical spaces, we do not often pay enough attention to these elements and how they might inspire, suggest, induce, and shape what we do. It is time that educators notice the qualities of the materials in a given scenario, as well as the activity of those interacting in these spaces, searching for how these relate, organize, and connect to the processes of learning, teaching, and researching. Such a socio-spatial perspective argues for space to be understood in connection to how it is occupied (Boddington & Boys, 2011). These connections are particularly relevant for scholars in embodied cognition, where an ecological view of learning emphasizes that people’s concepts and beliefs are linked to their perceptualaction experiences with things – bodies, minds, and technologies are seen as intrinsically connected (Clark, 2008; Kirsh, 2013).

Activity-Centred Analysis and Design (ACAD) Framework In studying the architecture of formal and informal learning networks, Goodyear and Carvalho (2014; Carvalho et al., 2017) foreground networked learning as involving complex assemblages of interconnected elements, such as artifacts, tools, places, ideas, and people. In this relational approach, the focus is on the influence of design on people’s activity, or on what people actually do, their thoughts and feelings, at learntime. Goodyear and Carvalho (2014) stress that when design comes along, it comes as part of a flux, and as such, design is best understood in the context of a world that is in constant movement, continually under construction – where design at the same time precipitates and is a response to the activity of its inhabitants (Gatt & Ingold, 2013). In other words, when design comes along, it needs to mesh with the ongoing existing actions of people (Goodyear & Dimitriadis, 2013). The ACAD framework acknowledges learning as socially, physically, and epistemically grounded, suggesting that what learners do, think, and feel is central, but best understood as an emergent phenomenon, as something that cannot be designed or entirely predicted in advance. What learners do is, nevertheless, partially shaped or influenced by the qualities of a situation, which are influenced by the tools, tasks, and social arrangements chosen by educators – teachers, educational designers, and those involved in design for learning. Four structural dimensions are proposed (Fig. 1), but only three of these are seen as open to adjustments and referred to as the designable components. Structures of place, such as artifacts, tools, and digital or material resources that may come to hand at learntime, are components in set design. Structures of tasks relate to knowledge and ways of knowing, foreground what learners are asked to do, and involve the selection, sequencing, and pacing of information. These are components of epistemic design. Elements related to learners’ organization, such as whether they will work in groups and pairs or follow scripted roles, are components of social design. Part of the work of educational designers (teachers, instructional designers, subject matter specialists, and others) is about making choices for elements in set,

9

Networked Societies for Learning: Emergent Learning Activity in. . .

205

Fig. 1 Activity-Centred Analysis and Design (ACAD) framework

epistemic, and social design (Fig. 1), which will result in a certain assemblage. At learntime, when learners engage with the structural composition of tasks, tools, and people, they are likely to reshape what has been proposed. This fourth dimension – co-creation and co-configuration activity – is emergent and acknowledges people’s agency to co-configure whatever is proposed. The ACAD framework has been applied into the analysis and design of different complex learning situations at varied contexts, including schools (Thibaut, Curwood, Carvalho, & Simpson, 2015; Yeoman, 2018), universities and MOOCs (Garreta-Domingo, Sloep, Hérnandez-Leo, & Mor, 2017), and informal spaces such as museums (Carvalho, 2017) and to frame the processes of educational designers (Martinez-Maldonado, Goodyear, Kay, Carvalho, & Thompson, 2016).

Theory of Entanglement The work of anthropologist Hodder (2012, 2013) aligns well with relational approaches in the social sciences and humanities, as it advocates that humans and things need to be seen as intrinsically connected. Hodder offers an interesting perspective to complement the ACAD framework, as a way to further theorize and reveal connections between humans, things, and structural elements of learning networks.

206

L. Carvalho

Hodder’s theory of entanglement (2012) focuses on understanding dependencies (rather than relationality alone), or on how humans may get caught in dynamic relations with things. Entanglement is about the reliance of humans on tools, and their role in supporting individuals and society, as humans evolve or adapt to their environment. Hodder (2012, 2013) poses that people often rely on things to develop their personal goals, but this reliance requires work in maintaining and caring for those things, so that they can be relied upon. As a result, people often become “entrapped” in the lives and temporalities of things. As such, the notion of entanglement involves considerations about the dialectic dependence and dependency between humans and things (Hodder, 2012, 2013). Dependence expresses a person’s reliance on material artifacts, thoughts, or things, in a relationship that may be both productive and enabling. In this scenario, a person’s use of things is related to “enabling an action,” for example, to use a cup to drink a beverage or the use of a phone to interact with others (Hodder, 2012). Dependency, on the other hand, is about constraints between humans and things and is used to describe situations in which humans may become too reliant on things, in ways that somewhat limit their ability to evolve as individuals or as a society, for example, when someone refuses to go on holidays unless there is reliable access to mobile data. Hodder (2016) has applied these analytical concepts in archaeological research, for example, when exploring the notion of “entrapment” or “path dependency” in relation to the origins of farming and settled life in the Middle East in the Çatalhöyük site. As such, these concepts have helped render visible new explanations or ideas that initially may have seemed counterintuitive, for example, revealing a social and ritual entanglement connected to hunting and adoption of farming practices – where the take-up of farming is associated with hunting being more effectively pursued and to continue to occupy its place as the core of social and ritual life in that society. Shove (2017) explores ideas from entanglement theory in discussions that challenge the conceptual foundations of “energy efficiency,” arguing that energy is intimately interwoven into infrastructures and devices that not only define but also are defined by people’s interactions with them. Empirical research by Yeoman and Carvalho (2014, Yeoman, 2015) also explored the notion of entanglement within a school context through detailed analysis of moments of interaction between teachers and students, extracted as vignettes. The relevance of these concepts to our particular context is that they allow us to investigate dependence and dependency aspects in the connections between humans and things in our networked learning environs. In placing things as fluid and in conceptualizing things as being in themselves flows of matter and energy, rather than static matters, a different view of networks emerges – not as a collection of stable nodes, links, and edges but instead as composed by relational, mixed, and heterogeneous elements to be analyzed in context. In sum, these analytical lenses enable us to further elaborate existing relationships that tie together tools, tasks, and people in the emergent collective activity. As highlighted by Hodder (2016):

9

Networked Societies for Learning: Emergent Learning Activity in. . .

207

human and things do not just relate to each other. Rather they are dependent on each other in ways that are entrapping and asymmetrical. Entanglement argues that things are so caught up in other things and in other human-thing dependences, that daily practices are directed down specific pathways, that humans are drawn in specific directions that create further entanglements. (Hodder, 2016, p. 9)

In applying these concepts and mapping the dependences and dependencies of a learning network into graphical representations, it becomes easier to abstract the complexity of learning networks into more digestible bites. Visualizations and snapshots highlight aspects of its functionality, not only showing how elements may be relationally linked but suggesting how some of these may be tied up on relationships that express dependences and dependencies between humans and things. The next section of this chapter applies concepts from the ACAD framework and Hodder’s notations of T (Things) and H (Humans) to illustrate and discuss core relationships in the networks. Graphical images – or “tanglegrams” – are produced to depict abstractions of entanglement in the two case studies.

Foregrounding Key Components and Entanglement in MONA and CmyView The two case studies in this section illustrate how concepts from ACAD in combination with Hodder’s theory of entanglement (2012) may be used to explore structural relations between people and things, thus revealing a complexity of connections between networked elements, and the learning activity they enable. The analyses reveal not only the array of heterogeneous elements that are part of these learning networks (or meshworks) but also the types of connections and participatory practices that emerge in contemporary networked learning. The first case is set in a museum scenario, exploring the networked activity of visitors within the physical space of the Museum of Old and New Art (MONA) in Hobart, Australia (Carvalho, 2017). Data collection involved observations conducted at the museum premises and secondary analysis of articles about MONA. The second case is based on the use of a “tool” and “method” called CmyView, in an activity conducted with four university students, enrolled in an undergraduate architecture course at an Australian university (Carvalho & Garduño Freeman, 2018; Garduño Freeman, 2017a). This case study is based on a non-formal activity, part of an evaluative study about the tool and method. Data collection and analysis involved artifacts produced by students (images and audio files) and a survey questionnaire.

Case Study 1: Museum of Old and New Art Recent research investigated museums as informal place-based spaces for networked learning, discussing the influence of technology on people’s experiences of cultural heritage (Carvalho, 2017). This research illustrated how one

208

L. Carvalho

element of a network calls for a pull of other elements, as all of them function together in relation to an end – creating opportunities for visitors’ engagement in (networked) learning activity. The dimensions in the ACAD framework supported the analysis and identification of elements that contributed to the networked activity of museum visitors at the Museum of Old and New Art (MONA). The O is a novel digital device which has been specially designed for use in MONA. One of the characteristics of MONA is the lack of writing information in its surroundings, as there is no signage directing or suggesting curatorial information to visitors. Visitors to the museum can borrow an O and a pair of headphones at the museum entrance, and they use the device to learn more about the objects in the museum. In framing certain elements of MONA’s network through set, epistemic, and social design, it is possible to establish relationships between design choices and visitors’ (learning) activity (Fig. 2). For example, digital and material elements available in the museum space, such as specific characteristics of its physical spaces, the art objects and their positioning, special lighting features, as well as the O and its specific digital features and interface would all be part of set design. In looking closely at some of these elements, one may notice, for example, the grandiosity of some of MONA’s physical spaces in connection to the absence of written signs in the museum walls. PHYSICALLY SITUATED

MOBILE TECHNOLOGY & THE MUSEUM SURROUNDINGS (ART OBJECTS, LIGHTS)

SET DESIGN EPISTEMICALLY SITUATED

Artefacts; tools, resources and texts INFORMATION DISPLAYED ON THE O

EPISTEMIC DESIGN looking/finding recording listening reflecting

OUTCOMES

EMERGENT ACTIVITY

MODIFYING THE APP CONTRIBUTING AUDIO RE-INTERPRETING THE ART OBJECTS

Dyads, groups and teams; roles and division of labour

SOCIAL DESIGN INDIVIDUAL BUT CONNECTING TO OTHERS EXPERIENCES

SOCIALLY SITUATED

Fig. 2 MONA in the ACAD framework

9

Networked Societies for Learning: Emergent Learning Activity in. . .

209

The museum invites its audience to start their visit, going down three levels, from the bottom up, and away from the light. The lack of tags in walls and the absence of light (set design) could both be considered unusual for a museum setting, but in this case, light and darkness are crucial elements of the MONA space, not only guiding visitors in subtle ways (e.g., lighting features suggesting directions) but also because low light levels are actually needed to enhance the readability of the screen on the O, where curatorial information is made available to visitors. Interesting features of MONA’s epistemic design are connected to the display of curatorial information, carefully selected, sequenced, and paced to be entirely mediated by the O. By stripping written words and signs within the museum space (set design), visitors are put “in control” of what objects they want to engage with and learn more information about (epistemic design). Electronic sensors locate visitors and display objects in surrounding area (set design), but it is up to the visitor to open up and choose what information to engage with (epistemic design). For example, there are choices for stories about art objects (composed by museum curators), about its relevance and history, as well as about interviews with artists and art personalities. Other features in the app invite visitors to vote whether they “love” or “hate” a certain object, subtly provoking a deeper type of engagement with the art work, as the visitor ponders their feelings – e.g., What is my reaction here? Do I like this object enough to “love” it, or do I dislike enough to “hate” it? And how have others felt in relation to this art object? As visitors’ activity is mostly mediated through an O, part of epistemic design also results in data collection about the audience’s interactions – creating opportunities for museum staff to learn about what objects visitors engage with, what objects they express feelings about (e.g., when voting on objects they “hate” or “love”), what exhibits they want to know more about, what is the chosen trajectory of visitors, etc. These analytics are gathered and would allow museum staff to get deeper insights about its audience and their activity, which can be used to inform the (re)design of new exhibitions. A museum visit is often considered as mostly an individual experience, even if people usually go to museums accompanied by friends or family. The social design of MONA, however, fosters a number of subtle interactions between visitors. There are features of the O that encourage visitors to communicate, for example, checking with companions for the information in their individual devices (e.g., not all the information in the different Os are exactly the same). After voting for an object – “hate” or “love” – the device allows users to visualize statistics of how other visitors voted that same object. In so doing, it situates the visitor’s opinion of an object in relation to others, offering an indirect way of connecting to previous visitors. Another way that social design enables a current user of an O to connect to invisible others involves the audio-recording of messages, sharing information about their favorite objects, which can then be accessed by the future users of that specific device. How then this assemblage of elements in the set, epistemic, and social design at MONA shape people’s emergent activity? The absence of written information or

210

L. Carvalho

signs hanging on the walls with curatorial notes affects the activity of both visitors and museum staff. Curators and designers are able to experiment with new ways of designing exhibitions and gallery spaces, as there is no need to produce wall signs and tags. What is more, visitors do not need to get close to tags on the walls, in order to read them, and their absence in turn modifies the way visitors walk around and move within such a museum space. With no entry panels to suggest where an exhibition starts, or what direction should be followed, museum visitors have more agency to decide their directions. The only indication is at the start of their entire museum visit – as visitors are invited to descend the floors and start their visit three levels down. From then on, visitors are free to follow their own trajectories, and through their interactions with the O, they select objects they want to engage with and learn more about. Hodder’s (2012) concepts allow us to expand this networked scenario a little further. As illustrated below, the relationship between a piece of technology, a specific exhibit, and learning activity (Fig. 3) can be further elaborated through its relationship to several other elements (Fig. 4). Each of these plays a part in contributing to the functioning of the whole and, in so doing, further reveals the complexity of the network. Figure 3 shows the novel handheld digital device (the O). In our example in Fig. 3, the device is being used to learn more about a gold Roman coin. The visitor may look at the gold coin, in the physical space, but the O may also display supplementary text about the coin. Sensors in the O allow the device to “know” where it is in the museum, so it can display relevant textual, audio, and visual information about the objects in that area. The “HT” annotation in Fig. 3 denotes that, in this case, the Human user is dependent on a Thing (the O) to learn more about the coin (Hodder, 2012). Figure 4 helps to visualize a broader network of relationships in which the O and the artwork sit.

Fig. 3 The O and artwork

9

Networked Societies for Learning: Emergent Learning Activity in. . .

211

Fig. 4 MONA and the network of humans and things

For the museum visitor to use the O to learn more about the gold coin, a number of additional relationships have to exist. In some of these, the Thing is dependent on a Human (TH – as with a curator providing information to be embedded in the O). In others, Things are dependent on Things (TT – as with the O’s dependence on sensors and a battery), and Humans are dependent on Humans (HH – as with the curator interviewing an artist to gather more information for the O). Museum visitors can also use the O to digitally tag museum artifacts that they liked. These implicit recommendations can be accessed by later museum visitors, and in this case, they augment the service being provided by the museum.

Case Study 2: CmyView This second example highlights structural elements in an outdoor activity conducted with architecture university students (Garduño Freeman, 2017a). CmyView shares many similarities with MONA’s case study, where activity is also precipitated by a piece of mobile technology. However, this second scenario is more bounded – students are asked to participate in specific tasks involving the creation, sharing, and reflection of images and audio-recordings of sites of interest. CmyView is described as a “mobile tool” but also as “a way” to explore people’s attachments to places (Carvalho & Garduño Freeman, 2018). CmyView invites people to collect and share “views,” through the gathering of images and production of audio-recordings of personally meaningful sites that people see while walking outdoors in the natural or built environment. Each walking trajectory is captured and

212

L. Carvalho

becomes a traceable artifact, which can be shared with other individuals in future walks. Beyond the design of the system and the activity of its users, CmyView envisages people’s participation in community assessment of social value, through their engagement in curatorial practices of place, and the aggregation and analyses of the “views” produced by people in the community (Carvalho & Garduño Freeman, 2018). In this example, elements in set design would include the mobile technology (e.g., iPhone) and the app that allows functionality for users to capture images and voice record their impressions. In addition, set design would also include elements in the built environment, as these are the objects of interest, which students are asked to engage with in the proposed task (epistemic design). Users of CmyView engage in two types of tasks as part of epistemic design. In the “collect mode,” they interact with the tool and the surroundings to capture images and audio-recordings of sites of interest. In the “sharing mode,” users interact with the system to gain access to and view someone else’s images and audio-recordings. In this case study, the first task – “collect mode” – asked students to take a walk and make “views” of sites of significance, within the surrounding built environment of their university campus, as a way of collectively documenting social value (epistemic design). In devising the task, care was taken to not direct students to specific locations nor to ask them to focus on places of positive or negative personal value. Four students were asked to “walk” for about 30 min, and each contributed with 6–12 “views.” A week later, these students went into the same area using CmyView to locate the photographs taken by their peers and to listen to their audiorecordings (epistemic design). They then completed a short survey, while reflecting on their experience. Elements in social design are similar to those in MONA’s case study, where a user is envisaged to participate in an individual activity, but with an overarching aim of connecting people to others, through shared views (Fig. 5). As the CmyView research reveals, students’ perceptions of places become more salient as they walked around imbued with the task of capturing and sharing representations of sites that were of interest to them (Carvalho & Garduño Freeman, 2018). Their connections to others arose, as a student chooses to experience and see sites of interest that were captured and shared by another, or as one person experiences a walking trajectory previously created by peers. They reported a feeling of intimacy as they listened to the voice of their peers narrating their perspective of a specific “view.” What is more, through engagement in a network of peer learners, students contributed to a common repository of potential walks, enabling each individual to discover new perspectives (the perspective of others) and how others have significantly different perceptions and forms of attachment to places. As such, learning experiences and connections do not end as one finishes a walk but continue through an evolving meshwork of elements, which includes their contribution to assessment of social value, as people gather data (through their “views”), and these are added to the repository, modifying the system and generating new data, which can then be used to inform government and corporate decisions. In

9

Networked Societies for Learning: Emergent Learning Activity in. . .

213

Fig. 5 CmyView in the ACAD framework

so doing, CmyView contributes to form and inform communal heritage practices, with ideas about social value and people’s varied attachment to places. In further abstracting this learning situation, we may look closely on dependence and dependency aspects between things and humans, through Hodder’s (2012) concepts. As illustrated in Fig. 6, the complexity of the relationship between the piece of technology, the built environment, and the learning activity that the technology enables can be further conceptualized by bringing several other elements into the mix, depicted in Fig. 7. Similar to MONA’s example, each of these relationships is important and contributes to the functioning of the whole. As students use CmyView to collect and share representations of their attachment with the built environment, a richer network of elements is revealed. Figure 7 is a representation of the broader network of relationships between things and humans, surrounding the activity of students engaged in the collect and sharing activities. For the student to use CmyView and participate in the “collect” and “sharing” tasks, other additional relationships also have to be in place. Similar to MONA, in some of these, the Thing is dependent on a Human (TH – as with CmyView being dependent on students to create “views” that will be embedded in the device). There are also relationships where Things are dependent on Things (TT – as with CmyView dependence on created artifacts to become a repository of walks, or on batteries to power the device) and Humans are dependent on Humans

214

L. Carvalho Student depends on CmvView (to collect ‘views’) (HT) (to share ‘views’ (HT))

Built environment

Fig. 6 CmyView

Depends on humans (to create these constructions) (TH) Depends on materials (to build these construction) (TT)

Student depends on CmvView (to collect ‘views’) (HT) (to share ‘views’) (HT)

Students depend on Lecturer (to explain ‘collect’ and ’ sharing’ tasks) (HH)

Depends on sensors and satellite (to detect geo location) TT

Built environment Information displayed depends on students (to create and share views) (TH)

camera voice memos

Depends on apps (to enable functionality of capturing and recording) (TT) (to let students create a view) (HT) (to let students see a view created by another) (HT)

Depends on batteries (to power device) TT (and to enable students use) TH Depends on created artefacts (to enable a repository of walks)(TT) (to inform policy makers) (TH)

Fig. 7 CmyView and the network of humans and things

(HH – as with the students hearing about the task from the teacher). As students use CmyView to digitally tag sites they liked, they create recommendations for other users of the app (HH – new users depend on previous users), and in this case, they also augment CmyView.

9

Networked Societies for Learning: Emergent Learning Activity in. . .

215

Connection and Participation: The Emergent Activity of Networked Learners The empirical analyses of MONA and CmyView illustrate how a networked learning approach helps in framing the key elements likely to influence the emergent learning activity of these learning networks. The ACAD framing (Goodyear & Carvalho, 2014) highlights relationships between the designable components (in terms of set, epistemic, and social design) and what networked participants actually do – or their emergent activity. Once the overarching framing is established, it is possible to further conceptualize the richer network of elements that needs to be in place as the learning activity unfolds, through a “path of dependence” and the entanglement between humans and things (Hodder, 2012). It is then possible to examine how what emerges, reverberates, and contributes to processes of learning, in a continuously evolving network. Both case studies – MONA and CmyView – illustrated the use of technology to enable learning activity and connections between people (Goodyear et al., 2004). Each involved technology use in a different setting (museum and outdoors), where learning took place under diverse narratives (art appreciation and attachment to places) and in relation to different learning objects (art pieces and built environment). The two cases, nevertheless, also share correspondences, as both are situated within curatorial and heritage practices, and have some similar ways of promoting connections. The educational design in both situations seems to account for processes of learning that place people not only as central actors but also as co-producers (Ito et al., 2013; Jenkins et al., 2009; Jones, 2008) and as actively engaged in knowledge building practices. In MONA, interesting connections and participatory practices emerged (Giaccardi, 2012), for example, through visitors’ engagement with a simply posed question (whether they “hate” or “love” an object) displayed via the O. Such design not only has the potential effect of provoking a deeper engagement with the object viewed but asks for the input of the user, which adds on to the analytics related to a particular object. Moreover, in releasing data about other visitors’ views on a particular object, people are given opportunities to situate and connect their own impressions to the views of previous visitors. Visitors’ responses and interactions with the O also then become analytical data to be interpreted and used by curators/museum staff in their own learning journeys – e.g., for evaluation and planning of future exhibitions. As MONA visitors add their personal audio-recordings to the particular O that was offered to them, visitors are co-creating – again modifying the information available in that device. They are not only consuming information but they are also producing (Ito et al., 2013; Jenkins et al., 2009; Jones, 2008) and adding their audiorecordings, which then become available to future users of that device, implicitly creating new possibilities for connections with others. These modes of participation are markedly different from the linear threaded discussions of the online university forums of the 1990s – they open up new perspectives in the ways people learn, connect, and participate in knowledge building processes in the Digital Age. They allow for exploration of learning at multiple levels: at the individual level, in relation to other visitors, and from the perspective of the museum staff.

216

L. Carvalho

In graphically representing dependences and dependencies related to the use of the O for learning in MONA, the complex network of elements that need to be in place becomes explicit. It is possible to then establish part-whole relationships and consider the multiple effects of walls that are stripped out of writing materials and the subtle design of lighting. These interconnected elements change people’s way of interacting with the collection, the paths they navigate within the museum, and the ways they experience and learn about the objects in display, and yet they are elements that educational designers (or museum staff in this case) would need to consider when planning their designs. The analysis of CmyView also illustrates new forms of connections that are enabled through the use of technology, as students capture and share views about the built environment. In the act of interacting with the app, adding images of significant places and audio-recordings, people – like in the MONA case study – are also connecting with future users of the device. They are sharing their personal experiences of the physical surroundings, and their social values about the sites of interest, with these future users. In the case of CmyView, it is argued that these types of activity could reflect networked learning practices that challenge traditional practices in the field of heritage (Garduño Freeman, 2017b Giaccardi, 2012). Beyond the university context, the CmyView type of activity has the potential to shift the role of a curator – as the expert with valuable knowledge about what constitutes a place of cultural significance – to a more collective effort (by a community), as other users of the technology curate and share places of significance to them. As such CmyView could allow for a form of community curatorship of place. In sum, these two analyses illustrate types of emergent activities that cross boundaries of time and space and formal and informal learning, with people sharing experiences of the physical environment while being apart. As the above analyses reveal, the networked learning approach can be applied across contexts, while focusing the analysis on the structural composition of a network. In so doing, it opens up a discussion about the types of participatory practices that emerge in complex learning situations where technology is used to promote connections and where learners participate in processes of knowledge creation and/or knowledge building practices, in or out of formal educational settings.

Conclusion: Designing for Learning in the Digital Age As mobile and ubiquitous technology evolve, the ways and settings where people learn are also being transformed and extended; and as the complexity of networked learning situations increases, so does our need for analytical tools that may help educators understand learning activity as involving a complex mix of heterogeneous elements. Recent research in networked learning acknowledges such a shift, highlighting networked learning activity as crossing boundaries and going beyond higher education contexts and binary notions that separate formal and informal education, or that strictly distinguish between physical and digital realms (Bonderup Dohn, Crammer, Sime, de Laat, & Ryberg, 2018; Hodgson et al., 2014).

9

Networked Societies for Learning: Emergent Learning Activity in. . .

217

The original definition of networked learning, crafted by Goodyear et al. (2004), is still fitting to the present context. Essentially, networked learning still involves people’s connections to resources and to other people with the support of innovative technologies. However, contemporary networked learning practices are taking new forms, which are vividly different from the types of interactions that were possible in the late 1990s and the early 2000s. As such, it is crucial that educators and researchers understand how designable components relate to the types of participatory learning practices emerging in contemporary networked societies. In adopting a relational view to analysis and design, educational designers can identify part-whole relationships existing between the complex mix of heterogeneous elements involved in networked learning. It becomes then possible to conceptualize the influence of these elements in the emergent process, to account for the different forms of collective participation and connections, and to examine how these co-exist and co-evolve and, importantly, their impact on the kinds of learning activity that are now possible. Schools, universities, museums, libraries, and other spaces for learning have been changing its physical infrastructure to accommodate the use of digital devices as part of the toolset of the twenty-first-century learners. What is more, social media sites such as YouTube, Pinterest, Wikipedia, Facebook, Flickr, and many others are becoming common places for people to express and share their connections and ideas, in informal learning networks that revolve around activities of collection, preservation, and interpretation of digital artifacts (Garduño Freeman, 2017b; Giaccardi, 2012). Within the richly networked societies, this new culture of learning already exists, with blurred boundaries between the physical and digital realms and between formal and informal spaces for learning. The networked learning approach to analysis and design discussed in this chapter helps reveal structural elements in participatory practices that connect people and resources. In so doing, it conceptualizes processes of learning in ways that place people as central, as co-producers, and as actively engaged in knowledge building practices, while acknowledging the pool of networked elements involved in such process. The approach helps educators deal with the complexity of designing for learning in the Digital Age – not only because it offers analytical tools to abstract and identify the key elements at play, and a way to map out how these elements connect to one another, and to emergent activity. The approach also supports educators in (re) creating designs for learning that are more likely to successfully engage learners in knowledge creation and knowledge building practices, accounting for learner’s co-production and co-creation, in networked societies for learning.

References Adams Becker, S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C., & Ananthanarayanan, V. (2017). NMC horizon report: 2017 higher education edition. Austin, TX: The New Media Consortium. Beetham, H., & Sharpe, R. (2013). Rethinking pedagogy for a digital age: Designing for 21st century learning. New York, NY: Routledge.

218

L. Carvalho

Boddington, A., & Boys, J. (Eds.). (2011). Re-shaping learning: A critical reader—The future of learning spaces in post-compulsory education. Rotterdam, Netherlands: Sense Publishers. Bonderup Dohn, N., Crammer, S., Sime, J., de Laat, M., & Ryberg, T. (Eds.). (2018). Networked learning: Situating networked learning: Looking back – Moving forward. Cham, Switzerland: Springer International Publishing. Carvalho, L. (2017). The O in MONA: Reshaping museum. In L. Carvalho, P. Goodyear, & M. de Laat (Eds.), Place-based spaces for networked learning. New York, NY: Routledge. Carvalho, L., & Garduño Freeman, C. (2018). CmyView: Learning by walking and sharing social values. In N. Bonderup Dohn, S. Cranmer, J. Sime, M. de Laat, & T. Ryberg (Eds.), Networked learning: Situating networked learning: Looking back – Moving forward (pp. 167–186). Cham, Switzerland: Springer International Publishing. Carvalho, L., Goodyear, P., & de Laat, M. (Eds.). (2017). Place-based spaces for networked learning. New York, NY: Routledge. Clark, A. (2008). Supersizing the Mind. Oxford University Press. Cronin, C. (2016). Open, networked and connected learning: Bridging the formal/informal learning divide in higher education. In S. Cranmer, N. B. Dohn, M. de Laat, T. Ryberg, & J. A. Sime (Eds.), Proceedings of the 10th international conference on networked learning (pp. 76–84). Lancaster University. Damsa, C., & Jornet, A. (2016). Revisiting learning in higher education – Framing notions redefined through an ecological perspective. Frontline Learning Research, 4(4), 39–47. de Laat, M., & Dawson, S. (2017). Is there anybody out there? Place-based networks for learning. In L. Carvalho, P. Goodyear, & M. de Laat (Eds.), Place-based spaces for networked learning (pp. 100–110). New York, NY: Routledge. Ellis, R., & Goodyear, P. (Eds.). (2018). Spaces of teaching and learning: Integrating perspectives on research and practice. Singapore: Springer. Fenwick, T., & Edwards, R. (2011). Considering materiality in educational policy: Messy objects and multiple reals. Educational Theory, 6(6), 709–726. Fenwick, T., Edwards, R., & Sawchuk, P. (2011). Emerging approaches to educational research. Oxford, UK: Routledge. Gatt, C., & Ingold, T. (2013). From description to correspondence: Anthropology in real time. In: W. Gunn, T. Otto, and R. C. Smith, (Eds.), Design anthropology: Theory and practice (pp. 139–158). London: Bloomsbury Academic. Garduño Freeman, C. (2017a). Informal networked learning as teamwork in design studio CmyView: Using mobile digital technologies to connect with student’s everyday experiences. In R. Tucker (Ed.), Collaboration and student engagement in design education (pp. 188–208). Hershey, PA: IGI Global. Garduño Freeman, C. (2017b). Participatory culture and the social value of an architectural icon: Sydney Opera House. New York, NY: Routledge. Garreta-Domingo, M., Sloep, P. B., Hérnandez-Leo, D., & Mor, Y. (2017). Design for collective intelligence: Pop-up communities in MOOCs. AI & Society: Journal of Knowledge, Culture and Communication, 1–10. https://doi.org/10.1007/s00146-017-0745-0 Gee, J. P., & Hayes, E. (2011). Language and learning in the digital age. Abingdon, UK: Routledge. Giaccardi, E. (Ed.). (2012). Heritage and social media: Understanding heritage in a participatory culture. London, England: Routledge. Gogia, L. (2016). Collaborative curiosity: Demonstrating relationships between open education, networked learning and connected learning. In S. Cranmer, N. B. Dohn, M. de Laat, T. Ryberg, & J. A. Sime (Eds.), Proceedings of the 10th international conference on networked learning (pp. 85–92). Lancaster University. Goodyear, P., Banks, S., Hodgson, V., & McConnell, D. (Eds.). (2004). Advances in research on networked learning. Dordrecht, The Netherlands: Kluwer Academic Publishers. Goodyear, P., & Carvalho, L. (2014). Framing the analysis of learning network architectures. In L. Carvalho & P. Goodyear (Eds.), The architecture of productive learning networks (pp. 259–276). New York, NY: Routledge.

9

Networked Societies for Learning: Emergent Learning Activity in. . .

219

Goodyear, P., Carvalho, L., & Dohn, N. (2016). Artefacts and activities in the analysis of learning networks. In T. Ryberg, C. Sinclair, S. Bayne, & M. de Laat (Eds.), Research, boundaries and policy in networked learning (pp. 145–164). Cham, Switzerland: Springer. Goodyear, P., & Dimitriadis, Y. (2013). In medias res: Reframing design for learning. Research in Learning Technology, 21(19909), 1–13. Gourlay, L., & Oliver, M. (2016). It’s not all about the learner: Reframing students’ digital literacy as sociomaterial practice. In T. Ryberg, C. Sinclair, S. Bayne, & M. de Laat (Eds.), Research, boundaries, and policy in networked learning (pp. 77–92). Cham, Switzerland: Springer. Hodder, I. (2012). Entangled: An archaeology of the relationships between humans and things. Chichester, UK: Wiley-Blackwell. Hodder, I. (2013). Human-thing evolution: The selection and persistence of traits at Çatalhöyük, Turkey. In S. Bergerbrant & S. Sabatini (Eds.), Counterpoint: Essays in archaeology and heritage studies in honour of Professor Kristian Kristiansen (pp. 583–591). Oxford, UK: Archaeopress. Hodder, I. (2016). Studies in human-thing entanglement. Creative Commons Attribution (CC BY 4.0). Retrieved from: http://www.ian-hodder.com/books/studies-human-thing-entanglement. Accessed 2 June 2018. Hodgson, V., de Laat, M., McConnell, D., & Ryberg, T. (Eds.). (2014). The design, experience and practice of networked learning. New York, NY: Springer. Ingold, T. (2011). Being alive: Essays on movement, knowledge and description. Oxford, UK: Routledge. Ingold, T. (2012). Toward an ecology of materials. Annual Review of Anthropology, 41, 427–442. Ingold, T. (2013). Making: Anthropology, archaeology, art and architecture. Oxford, UK: Routledge. Ito, M., Gutiérrez, K., Livingstone, S., Penuel, B., Rhodes, J., Salen, K., . . . Watkins, S. C. (2013). Connected learning: An agenda for research and design. Irvine, CA: Digital Media and Learning Research Hub. Jandric, P., & Boras, D. (2015). Critical learning in digital networks. Cham, Switzerland: Springer. Jenkins, H., Purushotma, R., Weigel, M., Clinton, K., & Robison, A. J. (2009). Confronting the challenges of participatory culture: Media education for the 21st century. Chicago, IL: MIT Press. JISC. (2014). Developing digital literacies. https://www.jisc.ac.uk/guides/developing-digital-liter acies. Accessed 28 Sept 2017. Jones, C. (2008). Networked learning – A social practice perspective. In V. Hodgson, C. Jones, T. Kargidis, D. McConell, S. Retalis, D. Stamatis, & M. Zenios (Eds.), Proceedings of the 6th international conference on networked learning (pp. 616–623). Halkidiki, Greece: Lancaster University. Jones, C. (2015). Networked learning: An educational paradigm for the age of digital networks. Cham, Switzerland: Springer International Publishing. Kirsh, D. (2013). Embodied cognition and the magical future of interaction design. ACM Transactions on Computer-Human Interaction, 20(1), 3:1–3:20. Laurillard, D. (2002). Rethinking teaching for the knowledge society. Educause Review, 37(1), 16–25. Martinez-Maldonado, R., Goodyear, P., Kay, J., Carvalho, L. & Thompson, K. (2016). An actionable approach to understand group experience in complex, multi-surface spaces. In CHI’16, May 07–12, 2016, San Jose, CA. Miller, C., & Bartlett, J. (2012). Digital fluency: Towards young people’s critical use of the internet. Journal of Information Literacy, 6(2), 35–55. OECD. (2015). Students, computers and learning: Making the connection. PISA. Paris, France: OECD Publishing. Orlikowski, W. J. (2007). Sociomaterial practices: Exploring technology at work. Organization Studies, 28(9), 1435–1448.

220

L. Carvalho

Shove, E. (2017). What is wrong with energy efficiency? Building Research & Information. https:// doi.org/10.1080/09613218.2017.1361746. Retrieved from: https://www.tandfonline.com/doi/ full/10.1080/09613218.2017.1361746. Accessed 2 June 2018. Sørensen, E. (2009). The materiality of learning: Technology and knowledge in educational practice. Cambridge, UK: Cambridge University Press. Thibaut, P., Curwood, J. S., Carvalho, L., & Simpson, A. (2015). Moving across physical and online spaces: A case study in a blended primary classroom. Learning, Media and Technology, 40(4), 458–479. Thomas, D., & Brown, J. S. (2011). A new culture of learning: Cultivating the imagination for a world of constant change. Warschauer, M., & Ware, M. (2008). Learning, change, and power: Competing discourses of technology and literacy. In J. Coiro, M. Knobel, C. Lankshear, & D. J. Leu (Eds.), Handbook of research on new literacies (pp. 215–240). New York, NY: Lawrence Erlbaum Associates. Yeoman, P. (2015). Habits & habitats: An ethnography of learning entanglement. The University of Sydney. Retrieved from: http://hdl.handle.net/2123/13982. Accessed 15 Mar 2018. Yeoman, P. (2018). The material correspondence of learning. In R. Ellis & P. Goodyear (Eds.), Spaces of teaching and learning: Integrating perspectives on research and practice. Singapore: Springer. Yeoman, P., & Carvalho, L. (2014). Material entanglement in a primary school learning network. In S. Bayne, C. Jones, M. de Laat, T. Ryberg, & C. Sinclair (Eds.), Proceedings of the 9th international conference on networked learning (pp. 331–338). Edinburgh, UK: Networked Learning Conference Orgniser. Lucila Carvalho is a Senior Lecturer in e-Learning & Digital Technologies at Massey University (Auckland, New Zealand). Her research explores how knowledge and social structures shape the design and use of technology and how technology influences social and educational experiences. Her interest is on understanding the relationships between (i) knowledge, (ii) physical and digital resources, (iii) human interaction, and (iv) how these elements combine to form productive learning networks. Dr. Carvalho earned her Ph.D. in architecture from the University of Sydney, Australia. She has published articles in education, sociology, design studies, and software engineering. She co-edited the books The Architecture of Productive Learning Networks (with Peter Goodyear, Routledge 2014) and Place-Based Spaces for Networked Learning (with Peter Goodyear and Maarten de Laat, Routledge 2017). She is currently working on a new book, Learning to Teach in Innovative Spaces: A Toolkit for Action (co-authored with Pippa Yeoman, Routledge).

Technology-Enhanced Learning: A Learning Sciences Perspective

10

Eleni A. Kyza

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Foundations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Learning Sciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Guiding Theories of Technology-Enhanced Learning in the Learning Sciences . . . . . . . . . . The Design of Technology-Enhanced Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Authentic Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scaffolding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Computer-Supported Collaborative Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Technology-Enhanced Learning and Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design-Based Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Challenges and Promises in a Learning Sciences Perspective of Technology-Enhanced Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning in Informal Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using Big Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Equity, Democratic Participation and Technology-Enhanced Learning . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

222 223 223 224 226 226 227 230 231 232 234 234 236 237 238 239

Abstract

Technology-enhanced learning (TEL) can be broadly defined as contexts that incorporate ICT technologies in support of learning. There are multiple definitions of TEL in the literature, each influenced by the theoretical perspective in which it is grounded and by the emphasis sought. There is no doubt, though, that TEL is an interdisciplinary and dynamic field, constantly in a process of redefinition as new technologies emerge and their niche in education is explored. To overcome the trade-offs brought upon by the fluidity of technology and context, it E. A. Kyza (*) Department of Communication and Internet Studies, Cyprus University of Technology, Limassol, Cyprus e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_56

221

222

E. A. Kyza

is important to situate and explore TEL within the predominant learning paradigms. This contribution discusses TEL from the perspective of the learning sciences (LS). Anchoring the design, implementation, research, and evaluation of technology-enhanced learning in the LS can offer a theory-grounded perspective that can focus on, and explain, the added value of technology and connect theory with practice. This chapter begins with a discussion of the foundational aspects of the learning sciences, which are relevant to TEL. It then discusses key aspects of TEL, which relate to the design, implementation, assessment, and evaluation of technology-enhanced learning environments. The chapter then concludes with a discussion of areas that are still under-researched in technology-enhanced learning contexts, pertaining to the above issues. Keywords

Technology-enhanced learning · Learning sciences · Learning theories · Designbased research

Introduction Technology has always been linked with human and societal change; however, the rapid pace of technological evolution and the transformative impact of innovations, such as the internet, on everyday life have forced us to reexamine what it means to learn in the twenty-first century, and how technologies can support this learning (Säljö, 2010). In contrast to how education was conceived in the industrial age, dominant ideas about learning in the twenty-first century emphasize the development of lifelong learning skills. Digital technologies, a collective term that includes a variety of ICT technologies and resources, is an important driver for the sweeping changes we see all around us in how we learn, work, and socialize and are at the heart of technology-enhanced learning (TEL). To harness the affordances of technology for the purposes of learning, one needs to situate its use in how people learn and in theories that have been validated in the real word. As argued in the literature, the use of digital technologies for learning is highly dependent on how one conceptualizes and operationalizes learning (Mayer, 2003) and is filtered through the research perspective one adopts (Merriënboer & Bruin, 2014). This chapter presents a learning sciences perspective to how digital technologies can contribute to the transformation of learning sought in the twenty-first century. The chapter begins with a discussion of the foundational issues and progresses to present the theories that support a learning sciences (LS) perspective to TEL. Following this, I explain why a design-based approach is important in creating learning environments that can be useable in real-world settings. I then discuss the contexts of application of digital technologies and conclude with a discussion of some of the main challenges facing technology-enhanced learning in the coming years. By design, the chapter discusses the learning aspects and implications of technology and does not delve in issues of teaching and teacher preparation, even

10

Technology-Enhanced Learning: A Learning Sciences Perspective

223

though these are extremely important in formal learning contexts. The chapter discusses what digital technologies can offer to twenty-first century learning and teaching, if bootstrapped by what we currently know about how people learn and led by the goal to investigate the practices and processes that contribute to meaningful interactions in technologically mediated environments. The path from the adoption of a potentially innovative digital technology to transformative learning is not linear but is dependent on the learning environment, the learner, and their interactions. To be able to use digital technologies to achieve transformative learning, one needs to attend to moments of success, as well as to moments of failure; this is one of the merits of adopting a learning sciences perspective to understand learning-in-context.

Foundations The Learning Sciences The learning sciences (LS) is an interdisciplinary field of research that seeks to investigate and integrate perspectives of learning, drawing from formal and informal contexts, and coupling the development of theory about learning and teaching with useable applications in real-world contexts. The LS emerged in the US in the 1990s because of a growing dissatisfaction with how learning was being studied, as led especially by lab-based research at the time, and the limitations in the implications of this type of research about how learning occurs in complex, real-world situations (Kolodner, 2004). On the contrary, learning sciences research seeks to understand learning in authentic real-world contexts, using interdisciplinary methodologies, and approaching learning with an agentic perspective (Bandura, 2001), which places emphasis on bringing about change. The latter is evident in the predominance of design, as a key characteristic of research in the learning sciences. One could argue that LS’s mission is of social, theoretical, and practical nature. For LS, the purpose of educational research is to understand and improve learning; understanding only, which is often the goal of traditional educational research, is not sufficient. LS’ social agenda indicates that, often, learning scientists take an interventionist approach (Penuel, Cole, & O’Neill, 2016), engaging in research to respond to educational problems and challenges, in the real world, based on social priorities, such as equity, participation, and critical inquiry. According to Hoadley and Van Haneghan (2011), the following five characteristics distinguish the learning sciences perspective from instructional design and other educational research paradigms: (a) embracing multiple perspectives on learning and its research to accommodate the complexity of educational practice and research in the real world; (b) an explicit effort to investigate and seek solutions to problems in learning and teaching; (c) the emphasis on design-based research to investigate the complexities of learning in authentic contexts; (d) a belief that informal settings need to be attended to, as people spend much more time learning in such settings than in formal educational settings during their lifetime; and (e) the emphasis on the use of

224

E. A. Kyza

technologies to support learning, and especially new and emerging technologies that can transform the learning and teaching experience. In a review of articles in four top-ranking learning sciences journals from 2003 to 2012 (Journal of the Learning Sciences, International Journal of ComputerSupported Collaborative Learning, Cognition & Instruction, and Instructional Science), Koh, Cho, Caleon, and Wei (2014) concluded that LS journal trends emphasized: the topics of technology and learning environments; conceptual understanding, development of skills; the role of affect and beliefs; students and teachers, at the secondary and higher education level, and using a variety of pedagogical strategies (i.e., collaboration, scaffolding, inquiry, reflective practices). Children were the most targeted group for research and implementation.

Guiding Theories of Technology-Enhanced Learning in the Learning Sciences It has been repeatedly argued that education is in dire need for reform (Elmore, 1990; Hiebert et al., 1996). In a second-order meta-analysis of the effectiveness of the introduction of computers in instruction, Tamim, Bernard, Borokhovski, Abrami, and Schmid (2011) indicate that computer-based learning is reported to be more effective than instruction without computers. Digital technologies have often been discussed as catalysts for change, if used in ways which can support appropriate learning objectives, and can promote the learning of higher order cognitive skills (Roschelle, Pea, Hoadley, Gordin, & Means, 2000). On the other hand, there is also the danger that technology will be used to fit old practices and that technologyenhanced learning will not succeed to fulfill the transformative vision that it has been often associated with (Cuban, 1993). However, not using technology to facilitate learning is no longer an option; therefore, it is detrimental that any discussion about technology-enhanced learning is framed in terms of the type of learning which technology seeks to enhance and grounded in sound pedagogy that can help achieve the sought-after goals. A learning sciences epistemology emphasizes the transformative potential of technology. Theoretical and epistemological commitments frame and filter the way technology is employed in the design and research of technology-enhanced learning environments. An important goal of the learning sciences perspective is to gain a deeper understanding of the processes of learning and design environments that can promote such deep learning. Learning sciences researchers base their investigations of learning processes on constructivist learning theories (Bransford, Brown, & Cocking, 1999), using methodological approaches or theoretical frameworks from multiple or interdisciplinary contexts, such as cognitive science, computer science, cognitive psychology, education, sociology, or anthropology, to name a few. Many researchers converge in that there is no singular learning theory that can capture all the idiosyncrasies or needs of learning. Therefore, it seems appropriate to state that effective support for learning may also include elements from different theories of learning (Lowyck, 2014), despite the dominance of the constructivist paradigm.

10

Technology-Enhanced Learning: A Learning Sciences Perspective

225

Constructivism is not a unitary learning theory but rather an amalgamation of learning theories which are based on common guiding principles first introduced primarily by Dewey, Piaget, and Vygotsky. One of the goals of the learning sciences is to investigate what contributes to the effectiveness of novel technologies in learning in real-world contexts and fine-tune our current understanding of how people learn. One such example is the Knowledge Integration Framework, which can be viewed as a refinement of constructivist ideas of how to help students understand complex scientific phenomena (Williams & Linn, 2002). This work has been conducted over a period of years and was facilitated by coupled development and research of learning environments using the Web-based Inquiry Science Environment (WISE) online platform. Other such research-based elaborations of how people learn in social and constructivist based learning environments have led to the development of theories on computer-supported collaborative learning, such as how to support online collaboration (Kobbe et al., 2007), knowledge building, and learning communities (Chan & Aalst, 2008), design principles for fostering learning in the disciplines (Engle & Conant, 2002), and how to support collaborative knowledge-building at the level of community using web 2.0 tools (Slotta & Najafi, 2013). The unique advantage of a learning sciences approach can be seen in the joint development of theory and products, i.e., new digital technologies provide opportunities for extending our understanding of new technological impacts while also imparting this knowledge to create technology-enhanced learning environments which can be used in instructional settings. For instance, learning analytics, which is a recent technological advance, now offers the opportunity to explore personalizing learning and supporting individual and group processes and just-in-time learning. Such an understanding will, in turn, lead to improved learning environments that include analytics. This opportunity would have been impossible if research and design were not coupled in this way (Rose, 2018). Even though the learning sciences grew out of research in cognitive psychology, it differentiated itself from the beginning by embracing a broader epistemology that emphasizes socio-cultural perspectives in learning (Bransford et al., 1999; Sawyer, 2006). In recent years, there has also been an increased attention to the role of social context on learning, which is more aptly characterized and explored through the lens of a Vygotskian socio-cultural framework (Daniels, 2011); the latter represents a social approach to learning and is one of the foundational theories of learning for a learning sciences perspective to technology-enhanced learning (Sawyer, 2006). The socio-cognitive perspective accepts that individuals construct their knowledge but also places emphasis on the role of the socio-cultural context; research on the role of mechanisms such as scaffolding and collaboration is expanding to explain how, for instance, tool mediation supports articulation and contributes to the internalization of knowledge. Therefore, a learning sciences perspective to TEL acknowledges that the use of technology is situated and deeply embedded in socio-cultural practices that need to be accounted for when analyzing and designing for learning. Such a social constructivist approach seeks to attend to the ecology of learning, in an effort to understand how different contexts (e.g., learning in formal and informal spaces) and

226

E. A. Kyza

types of human interaction (e.g., learning alone or with peers) contribute to learning and the role that technology can have in such processes.

The Design of Technology-Enhanced Learning Environments Amidst discussions among numerous stakeholders in education worldwide (i.e., policy makers, researchers, teachers, and parents), regarding the need to reform education (Collins & Halverson, 2009; Cuban, 1990; Dewey, 1903; Elmore, 1990; Gago et al., 2004; Hannafin & Land, 1997), the form learning environments take in consideration of the dominant theoretical commitments and epistemological assumptions about how people learn is extremely important. One of the contributions of the learning sciences has been the emphasis on gaining insights into learning processes in real-world contexts, often through the design of technology-based learning environments, and understanding how to support learning in such contexts. As such, learning environments are designed around authenticity, scaffolding, collaboration, and learning across time and contexts. To achieve a transformative role, digital technologies can be best seen as psychological tools, rather than an end to themselves. According to Vygotsky, psychological tools, such as signs, texts, and other external representations, can support the processes of internalization (Kozulin, Gindis, Ageyev, & Miller, 2003). Mediation can take two forms, human and symbolic (Kozulin et al., 2003). Of the two, symbolic mediation is the one explicitly related to the use of digital technologies for learning; mediational concepts, such as scaffolding (Quintana et al., 2004; Wood, Bruner, & Ross, 1976) and cognitive apprenticeship (Collins, Brown, & Holum, 1991), are of utmost importance to designing technology-enhanced learning environments. This emphasis has led to increased discussion about the elaboration of theoretical frameworks on these topics, the articulation of empirically-based design principles to support the design of future learning environments, and the refinement of pedagogical strategies that can support their implementation in real contexts. The next sections provide an overview of these contributions.

Authentic Learning Authenticity of the learning experience is a crucial construct in engaging learners in meaningful learning experiences, and, thus, it is of great import in designing powerful technology-enhanced learning environments. Authentic learning, a pedagogical approach that has its roots in situated cognition, emphasizes the need to recontextualize learning so that the connection between abstract, theoretical knowledge, typically found in formal schooling settings, and real-life applications of concepts is made clear (Herrington & Oliver, 2000). There are many characteristics of authentic learning environments detailed in the literature (e.g., Bransford et al., 1999; Herrington & Oliver, 2000), all of them revolving around experiences that help motivate interest, create opportunities for sustained engagement with the

10

Technology-Enhanced Learning: A Learning Sciences Perspective

227

learning content, and help learners comprehend the application of the knowledge they are learning in real-world contexts. Researchers have cautioned about the difference between authentic environments and authentic tasks, arguing that the authenticity of the task is more crucial in helping students learn than the authenticity of the environment (Gulikers, Bastiaens, & Martens, 2005). Authentic learning activities have real-world relevance, are ill-structured, engage students in activities that resemble real-world practices, and promote students’ initiative and ownership of the activity (Herrington & Oliver, 2000). Ultimately, even though we can design for authenticity, the latter is subjective and emerges from the designed activities and the interactions between task, participants, and technologies rather than being something predetermined (Barab, Squire, & Dueber, 2000). At the same time, technology can provide the tools to engage the students in the activity and contribute to situated learning practices, facilitated through inquiry, collaboration, problem-solving, and reflection that can make a task be experienced as authentic. Inquiry-based learning, an idea that dates to John Dewey, is a central instructional and pedagogical framework within which authentic learning can be promoted. Inquiry has been discussed more extensively in the literature of science education. The US National Research Council – NRC- (1996) has defined inquiry in science learning as a multifaceted activity that involves making observations; posing questions; examining books and other sources of information to see what is already known in light of experimental evidence; using tools to gather, analyze, and interpret data; proposing answers, explanations, and predictions; and communicating the results. Inquiry requires identification of assumptions, use of critical and logical thinking, and consideration of alternative explanations. (p. 23)

This definition can be extended to other disciplines, such as mathematics, history, and literacy. Digital technologies have a significant role to play in such contexts, as they can provide the tools to engage in many of the activities referred to by the NRC definition, and which are akin to competencies identified in twenty-first century learning documents. Nonetheless, inquiry-based learning has not been uncontested, as several debates have been documented in the literature regarding the merits and effectiveness of inquiry approaches (Hmelo-Silver, Duncan, & Chinn, 2007; Kirschner, Sweller, & Clark, 2006; Kuhn, 2007; Schmidt, Loyens, Van Gog, & Paas, 2007). With appropriate scaffolding, inquiry has been proven to be more beneficial than non-inquiry learning contexts as documented by large-scale studies (Granger et al., 2012) and syntheses of the literature (Furtak, Seidel, Iverson, & Briggs, 2012).

Scaffolding Scaffolding, a term first coined by Wood et al. (1976), is a widely-accepted term which is based on Vygotskyan ideas of the importance of social interactions for the development of learning. Scaffolding operationalizes how, through temporary

228

E. A. Kyza

support, a learner can move through their zone of proximal development so that when the support is withdrawn (or faded) they can perform the task with their own cognitive resources. This scaffolding can be provided by a more knowledgeable person, such as the teacher or a peer, or can be provided through features of learning technologies. Drawing from studies of multiple software designed to support complex reasoning in science education, Quintana et al. (2004) presented a scaffolding design framework for supporting students’ use of learning technologies for scientific inquiry practices; this framework discusses how different software features can support three main important learning aspects: sense-making, process management, and articulation and reflection. The framework is based on a task analysis, an analysis of the obstacles usually faced by the learner, and offers scaffolding guidelines to support learners to overcome challenges and obstacles. This approach offers a principled, theory-based, but practice-grounded perspective to how one can approach the scaffolding of a cognitive task. Studies in technology-mediated inquiry learning have investigated strategies in scaffolding the learning experience, reporting on principles to design educational software tools and on how to provide scaffolds outside of software so that the learners’ specific learning needs are addressed, and learners can gradually progress within their own zone of proximal development (ZPD). Technology has, again, a special role to play in this process, as it can work synergistically with the teacher and provide explicit and implicit support for acquiring skills and disciplinary knowledge (Tabak & Baumgartner, 2004; Tabak & Reiser, 2008). In own work with colleagues, using an online platform scaffolding data-rich inquiry investigations, we have found that structuring the task and using targeted reflective inquiry prompts to remind and guide students in articulation and reflection can support lower achievement dyads in constructing evidence-based explanations (Kyza, Constantinou, & Spanoudis, 2011), support middle-school students in attending to alternative explanations of their data (Kyza, 2009), or help high school students attend to the credibility of data as evidence (Nicolaidou, Kyza, Terzian, Hadjichambis, & Kafouris, 2011). Many other references in the literature support similar findings about the importance of scaffolding inquiry-based learning in technology-mediated learning (Baker & Lund, 1997; Davis, 2003; Donnelly, Linn, & Ludvigsen, 2014; Järvelä, Häkkinen, Arvaja, & Leinonen, 2004; Linn, Clark, & Slotta, 2003; Rienties et al., 2012). The amassing evidence seems to argue for the importance of design in developing meaningful learning environments. Tools can scaffold learners by providing multiple representations of a phenomenon, to support them in connecting the representation to their own cognitive schemata. Such tools may often be redesigned to address the needs of the learner, as identified in task analyses and in the analyses of the learning obstacles. For instance, Model-It (Jackson, Stratford, Krajcik, & Soloway, 1994) employed scaffolding strategies to help novice learners engage in expert tasks, such as offering tools for qualitative modeling in lieu of the quantitative modeling that experts would perform; without this scaffolding, most learners, at the secondary school level, would not have been able to engage in the modeling of dynamic phenomena at all. In other approaches to scaffolding, learning scientists have been examining data mining

10

Technology-Enhanced Learning: A Learning Sciences Perspective

229

and learning analytics approaches to automate the provision of scaffolding in the context of the BioLogica software tool, using Bayesian Knowledge Tracing models with positive results (Buckley et al., 2004). According to Reiser (2004), scaffolding should/can enable the learner to perform a task that they would not be otherwise able to do on their own, by structuring the task; at the same time, it should problematize the task for the learner, by keeping it challenging but within their ZPD, to create opportunities for cognitive processes that support deep learning. One example that Reiser gives is related to learners who tend to engage in nonreflective work; adding prompts to the workspace may remind the learner to reflect and guide them to creating explanations of the phenomena they are investigating. The topic of scaffolding in technology-enhanced learning has been steadily discussed in the publications of the four learning sciences journals identified by Koh et al. (2014) in their own bibliometric study, as well as in other publications, with a noticeable increase on citations and papers written as the technological affordances to scaffold learning increase over time. Research on scaffolding, its affordances but also its limitations in supporting learning is oftentimes amassed incrementally based on longitudinal programs of research. For example, work on Knowledge Forum (Scardamalia & Bereiter, 2006), a web-based environment for fostering knowledge-building communities, is continuing with researchers exploring scaffolds to support students’ engagement in settings such as multiplayer epistemic games (Bielaczyc & Ow, 2014). Furthermore, the emergence of more powerful learning technologies enable researchers to investigate their contribution to adapting learning to one’s needs, using intelligent tutoring systems and pedagogical agents. This effort often entails the development of scaffolded technologies and the concurrent investigation of scaffolding strategies and their interplay with learner characteristics and contextual factors (e.g., Feyzi-Behnagh et al., 2014). As learning scientists explore new contexts of technology-supported learning, such as online forums, social media, virtual, and augmented reality technologies, the emphasis is not only placed on the positive impact of technology but also on understanding where scaffolding and learning with technology fails. Documenting, communicating, and explaining failures expands shared knowledge and can provide useful starting points for future research and development efforts. The examination of the effect of scaffolding is not only limited to software-realized scaffolding but also expands to cover synergistic and distributed scaffolding which may be provided alongside software-based scaffolding (Reiser & Tabak, 2014). Such scaffolding can often be found in the interactions of peers or students with their teacher. Nowadays, technology can virtually support several important cognitive and social activities that are important to learning; for instance, scaffolded scientific tools support the progression from novice to expert (de Jong, 2006); social tools such as web 2.0 media and discussion forums (such as the knowledge forum) can help learners capitalize on learning from each other. The examples are too numerous to fairly address in any one chapter; what is important to keep in mind is that the integration of technology through the careful design of the technology-enhanced learning environment, including a consideration of other factors that influence the

230

E. A. Kyza

uptake of the technology by the learner (such as prior knowledge and conceptions, existing infrastructure, pragmatics like time, and classroom management) can be decisive for the affordances that the designed environment will carry.

Computer-Supported Collaborative Learning Socio-cognitive and socio-cultural approaches to learning strongly suggest the need to attend to cognition occurring at different levels, namely to attend to learning happening at the individual and at the collective level. Group learning is an emergent phenomenon which has been studied for many years (Stahl, 2014), as a fundamentally human activity that influences decision-making both in the professions and in our personal lives. Humans are social beings, by nature, and are, thus, continuously exposed to multiple stimuli, whether these are deliberately sought or osmotically obtained. Computer-supported collaborative learning (CSCL) is a term first used in the 1990s (Stahl, Koschmann, & Suthers, 2014), to capture and study those unique, enabling qualities that technology can offer to collaborative and group learning. CSCL differs from other approaches to technology-mediated learning, such as distance learning or e-learning, both epistemologically and pedagogically, as it emphasizes the need for scaffolding the learning process around meaningful, technology-mediated interactions, rather than expecting that technological tools or digitized materials will provide all necessary support to the learning -as is the case, most often, in e-learning contexts (Stahl et al., 2014). Such an approach is warranted by studies indicating that scaffolding is required to support students in engaging with online materials. Kirschner, Beers, Boshuizen, and Gijselaers (2008) discussed three aspects that need to be attended to in CSCL environments: tasks, social conditions, and the learning environment. According to Kirschner and colleagues, tasks need to be authentic to support meaningful collaboration; the need for conducive social conditions (such as feelings of trust and acceptance of each other’s positions) should not be ignored in favor of engaging with the cognitive aspects of the task, whereas the learning environment should enable and facilitate engagement with the task. Attending to these issues can be complex and the success of such environments cannot be equated with the mere presence of technological affordances. A variety of technological tools can be used to support different cognitive and social aspects of CSCL which have been discussed in the literature as vital to successful collaborative learning. Jeong and Hmelo-Silver (2016) argued that the role of CSCL tools should be on: establishing a joint task, facilitating communication, enabling the sharing of resources, engaging in productive learning processes, supporting coconstruction, facilitating metacognitive monitoring and regulation, and supporting the creation of collaborative groups and learning communities. Some of the CSCL tools which present these affordances are commercial products and can be used off the shelf, while other tools are especially designed by researchers. For instance, technologies such as email, asynchronous forums, chat tools, and wikis have been reported to facilitate communication between group members, whereas

10

Technology-Enhanced Learning: A Learning Sciences Perspective

231

other tools, such as the Knowledge Forum (Scardamalia & Bereiter, 2006), have been especially designed and has been validated through years of empirical research. The CSCL tools that can support these critical group activities are diverse and many, and as such, a complete discussion of examples of tools in each of these categories is beyond the scope of this chapter. However, the literature includes several research reports that can allow the interested reader a deeper insight into the pedagogical practices, contexts, and results of related research, such as that on: self-, co-, and shared-regulation of learning (e.g., Järvelä et al., 2015; Miller & Hadwin, 2015); tools for reflective inquiry and collaborative evidence-based explanation building (Kyza et al., 2011; Linn et al., 2003); argumentation tools (for a review see Scheuer, Loll, Pinkwart, & McLaren, 2010); learning to learn together to overcome individual challenges (Schwarz, de Groot, Mavrikis, & Dragon, 2015); collaboration scripts (Kollar, Fischer, & Slotta, 2007); etc. Readers may also wish to refer to the article by Jeong and Hmelo-Silver (2016) for more examples of types of tools that exemplify these affordances.

Technology-Enhanced Learning and Assessment Assessment is important for informing the iterative cycles of design-based research and for evaluating the effectiveness of technology integration. Assessment can be distinguished as formative and summative; Wiliam and Black (1996) presented the difference of the two types of assessment as lying in their different functionalities: that is, formative assessment is meant to be a tool for monitoring and improving learning processes, whereas summative assessment bears a more evaluative nature. The continuous development of technological assessment tools can facilitate both functions. However, it is the formative assessment that is of most interest to the learning scientists. The call for authentic learning environments that exemplify the complexities of learning in the real world creates the need for additional or nontraditional assessments of learning, capable of capturing the essence of new pedagogical practices and the rich learning they may afford (Anderson & Shattuck, 2012). One of the challenges of new innovative learning environments is the mismatch between pedagogical approaches, new modes of learning, and the assessments that are employed to capture this learning, for formative or summative use. As such, a main question of interest refers to the types of technologies which can contribute to more valid assessments. In a review of research published between the years of 1993 and 2004 on the topic of assessing technology-mediated collaborative inquiry, Yang and van Aalst (2015) reported that little published research exists on this topic for K-14 education. In their review, Yang and van Aalst discussed various technological methods which supported assessment: creating shared spaces for documenting the learners’ work, tools encouraging self-regulation and reflection, and tools facilitating the provision of feedback. In their examination of 57 studies, Yang and van Aalst concluded that: (a) only embedded assessments of collaborative inquiry, that is assessments that supported students’ metacognitive awareness, monitoring and

232

E. A. Kyza

regulation of their learning process, contributed to student learning to a greater extent and (b) assessing for learning was not dependent on a sole characteristic of the technology employed or of the learning environment but rather appeared to be an emergent phenomenon that was constituted by synergies between technology, pedagogical approach, and scaffolding. On the other hand, recent technological developments have brought about more dynamic technology-based assessments of learning, in the form of recommender systems and learning analytics. Recommender systems perform dynamic analyses of the learning environment in conjunction to learner decisions and actions in that environment and provide recommendations on next steps; for instance, a recommender system can monitor the learner’s contributions to a blog and suggest additional references they should read, based on the initial monitoring (Manouselis, Drachsler, Vuorikari, Hummel, & Koper, 2011). Digital technologies leave rich traces of data, which can be used to support formative assessment. The purpose of collecting such rich and multimodal information is to contribute to a richer understanding of the learning process by the learner and by the teacher mentor. Learning analytics use data mining and analytical techniques to track, document, and report on students’ actions in digitized environments (Drachsler & Greller, 2012). Learning analytics differ from educational data mining in the customization of the data the system generates each time. Through the use of computational approaches, such as user modeling and visualization of data and processes, learning analytics can, in principle, provide real-time feedback to the learner or the teacher via teacher or student dashboards; for example, visual data analytics presented to the students themselves can support multiple learning purposes, such as inform the students of the state of their collaboration and invite reflections on own and peer assessment (Walkington, 2013). While learning analytics can be of use in multiple contexts, it has the capacity to support learning with big data and learning at scale. The latter is crucial in learning contexts such as massive open online courses (MOOCs), which include a great number of participants.

Design-Based Research Knowledge in the learning sciences amasses through an in-depth examination of learning processes using mixed methods approaches and interdisciplinary methodological approaches; these characteristics been reported as a distinguishing feature between the LS and fields such as Educational Technology and Educational Psychology (Koh et al., 2014). A learning sciences perspective places heavy emphasis on design-based research (DBR) as a principled approach in understanding the effectiveness of the designed learning environments in complex settings and in developing theories of how people learn (Barab & Squire, 2004; Brown, 1992; Collins, 1992; Penuel et al., 2016). One of the distinguishing characteristics of DBR is that it attends to both theory-building and real-world practice. Several researchers have argued the merits of coupling the development of educational theory with the design of authentic learning environments, especially as these also

10

Technology-Enhanced Learning: A Learning Sciences Perspective

233

related to technology-enhanced learning (Barab, 2006; Brown & Campione, 1998; Collins, 1992; Design-Based Research Collective, 2003; Edelson, 2002; Joseph, 2004; Wang & Hannafin, 2005). The iterative refinement of theory, through cycles of design, implementation, data collection, and evaluation is an important aspect of DBR. DBR represents an epistemology, rather than a method, as this approach is inclusive and accepting of a repertoire of research methods, to answer the questions of interest. These questions share the goal of seeking to understand the processes through which learning occurs, as well as understanding how these processes can be redesigned so that they can address the goals of learning, and thus have an impact on real-world practice. In this way, DBR differs from other research paradigms of learning technologies. Reviews of work using the DBR approach suggest that this approach may yield informative conclusions about how theory and practice can be coupled and improved (Anderson & Shattuck, 2012). It has also been proposed that DBR can contribute towards achieving the potential of learning technologies in education, a claim often made but also brought down by the failures of technology alone to bring about change (Amiel & Reeves, 2008). Researchers have been working on design principles to support technologyenhanced learning in general (e.g., Wang & Hannafin, 2005), or grounding them in disciplines such as science education (e.g., Kali & Linn, 2008). Wang and Hannafin (2005) presented a set of important principles guiding DBR, which refer to the coupling of research and design, conducting systematic and iterative research in authentic settings, involving key stakeholders in the design process, and extracting design principles to foster reflection and extract context-sensitive information that can inform other participants. Contextualization may refer to several different aspects, such as instructional, cultural, or disciplinary settings. For instance, Kali and Linn (2008) have elaborated design principles guiding technology-enhanced inquiry science education contexts, based on their prior DBR work using the knowledge integration framework in real classroom settings (Scardamalia & Bereiter, 2006). Kali and Linn used a design principles database, with contributions from over 50 researchers, to identify meta-principles relating to technology-enhanced learning in science. The accumulation of research using design-based approaches has led to the refinement of theory around DBR, to the extent that researchers have proposed different forms of design research which reflect variations in priorities and goals for the design. Three such examples are social design experimentation (Gutiérrez, 2016), formative interventions (Engeström, 2011), and design-based implementation research (DBIR) (Fishman, Penuel, Allen, Cheng, & Sabelli, 2013). These research methods share the principles and goals of DBR but move the approach forward by focusing on different priorities and proposing methods to address them. For instance, social design experimentation seeks to democratize education by prioritizing social transformation and the inclusion of marginalized populations by involving stakeholders in the design activity; formative interventions examine the learners’ observed conflicts of activity, from the perspective of the learner themselves, with the researcher taking a secondary role in the process; finally, design-based

234

E. A. Kyza

implementation research emphasizes the scalability of interventions and seeks to examine the ecosystem of interacting elements, and how they contribute to the change process by explicitly focusing on understanding how change can be brought about and sustained over time. Even though these approaches are not technologycentric, they exemplify how the understanding of the broader context of technologyenhanced learning can lead to a deeper conceptualization of how, when, and to what goals learning technologies can contribute. Nonetheless, this research approach is still maturing and more work is needed to establish it as a widely acceptable research approach, in collaboration with other means of inquiry into educational research. For instance, participatory design, including codesigning with teachers, has only recently been discussed as a valid form of research, despite participatory design’s appeal in fields such as humancomputer interaction and design for many decades now. Participatory design approaches have the potential to make designs more authentic and meaningful to participants and can, thus, more validly inform research and theory-building. In terms of technology-enhanced learning, this also means that challenges and concerns can also be considered more readily and the next iteration of design can offer improved experiences for participants.

Challenges and Promises in a Learning Sciences Perspective of Technology-Enhanced Learning This section of the chapter discusses three big challenges in technology-enhanced learning which are underexplored but important in the learning sciences: understanding and scaffolding learning in informal settings, using big data to support learning, and issues of equity in technology-enhanced learning. These, of course, are not the only challenges that face the research community in the learning sciences, as we try to gain a deeper understanding of learning and how to support it. For a list of additional grand challenges in technology-enhanced learning, the reader can refer to a recent effort, conducted at European level, identifying major stakeholders’ viewpoints on grand challenges relating to TEL via a series of Delphi studies, culminating to the publication of a collection of twelve such grand challenges (Eberle, Lund, Tchounikine, & Fischer, 2016).

Learning in Informal Settings Several of the challenges identified by Eberle et al. (2016) concern issues beyond formal schooling, bridging over to lifelong learning, such as, for instance, what it means to learn in a smart city context. Despite the importance that the learning sciences community has bestowed to informal learning in recent years (National Research Council, 2009), much of the research in the related fields of educational technology (ET), educational psychology (EP), and the learning sciences (LS) has focused on formal education. Koh et al.’s (2014) analysis of the research trends from

10

Technology-Enhanced Learning: A Learning Sciences Perspective

235

5187 journal articles in the fields of ET, EP, and LS from 12 publications between the years 2003 and 2012 indicates that research in informal learning settings has been miniscule with only a small percentage of the total number of articles being reported as focusing in such settings (ET = 2%, EP = 1%, and LS = 3%). A strand titled “Learning Outside of School” was created in the Journal of the Learning Sciences to indicate the desire of learning scientists to explore and report on informal learning research. According to the editors, this strand concluded its course in 2017, as the diversity of contexts in which learning was studied had increased substantially, as judged by the submissions to the journal. However, the small number of published articles identified in comparison to other areas of research indicates that the area of learning in informal settings is still under-explored. Considering an estimation that only about 18% of a person’s time during his or her lifespan is spent in formal schooling, one can easily grasp that researching the full ecology of learning, and, especially, learning that happens in contexts such as family activities and interactions, visits to museums and parks, and learning in the workplace, should be, but is not, yet, an important priority for research (Banks et al., 2007). Another related concept, which is still underexplored empirically, is the connection between formal and informal learning, and how to create productive synergies. Chan et al. (2006) very appropriately used the term “seamless learning” when describing the opportunities afforded by mobile technologies to emphasize the point that advances in ubiquitous computing can lift the artificial constraints on learning as a process that happens within formal contexts only and support learning anytime and anywhere. In seamless learning, technology may adapt to the learner needs without the learner being necessarily aware of this adaptation, and modernday advances in network connectivity ensure that access to the information and technological infrastructure to support learning activities is constant (Milrad et al., 2013). The wide access to technological developments enabling communication between devices, and increased wireless connectivity capabilities, has minimized older connectivity challenges and has, thus, opened new paths for connected, ubiquitous, and personalized learning. This rather nascent field still needs to grow and attend to issues how to best foster learning in informal contexts. As technology evolves, one can witness more powerful ways in which TEL environments can support deep learning in informal settings, affording pedagogical practices and processes which are aligned with the predominant learning theories. Learning outside of formal contexts, such as learning in the workplace or learning in museums, has been particularly enabled with the advent of mobile, personal and personalized computing. Evidence from our own work (e.g., Efstathiou, Kyza, & Georgiou, 2017) as well as work from other colleagues (e.g., Looi et al., 2010) indicate the affordances and positive impact of learning with mobile devices and augmented reality technologies outside of school, even at the early, elementary school years. At the same time, this work also points to new research directions that should explore how to support extended engagement, collaboration, and informal learning in such settings. For instance, Efstathiou et al. (2017) demonstrated how scaffolded, location-based, augmented reality technologies helped 3rd grade students gain historical empathy and increase their conceptual

236

E. A. Kyza

understanding of why Neolithic people chose their settlement site during a visit to an archaeological site. Subsequent analyses of similar interventions examined students’ collaboration patterns, revealing that different approaches to the design of the learning environment and scaffolding the students’ work may be necessary to increase students’ collaboration. Such studies indicate how work to understand the conditions that foster informal learning in the various domains and social contexts is still in early stages.

Using Big Data Big data, defined as datasets whose size supersedes the ability of conventional software to analyze them, are increasingly seen as potentially powerful allies in learning and teaching. However, and despite their promise, there is still considerable work that needs to be achieved to reach maturity and make analytics a truly useful feedback mechanism. The increased capabilities of technologies that can offer a continuous, real-time stream of data on learning processes have led to what Drachsler et al. (2012) have termed as “data-supported technology-enhanced learning.” Drachsler and colleagues identify four challenges associated with big data sets and propose that there is a need to think about: (a) frameworks to analyze such datasets and frameworks to support the reuse of datasets for investigating and validating theory; (b) how data-driven TEL can be used to assess learning and help reduce the decline in interest and participation by providing personalized learning opportunities; (c) ethical issues concerning privacy, data security, and surveillance, which require new legislatures but also a conscious and proactive consideration of practices around big data at the short- and long-term horizon; and (d) designing user-friendly tools that report back to participants (e.g., learners, teachers) in an informative and educationally meaningful way. The ethical dimensions of data-driven technology-enhanced learning are increasingly important due to rapid technological change, the massive uptake of technology in everyday life, and the lack of comprehensive legal frameworks and educational policies to address issues concerning privacy and ethics. From one point of view, technology, especially social media and web 2.0 applications, have enabled wider societal participation in technology-mediated communication and has, thus, augmented the opportunities for informal learning. It is now the norm to anticipate access to wireless connectivity and information and communications technologies in most contexts of everyday personal and professional life. Learning is not, anymore, confined to the walls of a classroom but is distributed across time, media, and participants. This new reality brings with it affordances and challenges. From one point of view, there are multiple opportunities for engaging in learning using digital technologies, such as learning at a museum using augmented reality technologies. On the other hand, the wide accessibility to technology also raises issues of privacy in the information age and protection of personal data, requiring increasing attention to the ethical dimensions of the use of technology.

10

Technology-Enhanced Learning: A Learning Sciences Perspective

237

Learning analytics is one such exciting field of study enabled by big data; it offers unprecedented possibilities for gathering information on learning processes which can then be used to monitor and guide learning as it happens. At the same time, learning analytics is one such case where there is a conflict between the opportunities offered by dynamic assessment technologies, which can provide rich trails of user data often linked with personal information about these users, and the need to protect the users’ privacy. Similar concerns are raised when discussing seamless learning across devices. How does one ensure that digital traces of learners’ performances are not misused either now or in the future, as such information aggregates? How can we balance between the need to collect as much information as possible, in the hope of identifying rich opportunities for learning, and the need to protect sensitive user data?

Equity, Democratic Participation and Technology-Enhanced Learning The idea of digital technologies as a catalyst for self-improvement, personal, and social empowerment is not new, but not all research efforts seek to intervene in favor of such changes, as learning scientists often do. The social agenda of the learning sciences research is revealed in the recurrent discussions on the potential of learning as an empowering and democratizing force; the design of learning environments that promote such goals, and design-based research that iteratively attempts to address them, is characteristic of the work in the learning sciences community (Gutiérrez & Jurow, 2016). Despite the seemingly omnipresent state of technology, issues of access and equity especially by traditionally marginalized groups seem to persist, even in western nations and more so in other parts of the world, such as many African and Asian nations. Such realizations emphasize a need to develop the infrastructure to support disadvantaged populations, attend to issues of social justice, and especially attend to how to support youth engagement with digital technologies to promote critical thinking and creativity. This is important in the light of reports that access, even in technologically advanced nations such as the USA, is not uniform and that such problems severely constrain the opportunities for using technologies to learn in both formal and informal settings (Warschauer & Matuchniak, 2010). Research on diminishing inequalities through design research has included work with marginalized populations (e.g., minorities, immigrants, refugees, etc.), stakeholders in urban communities, whose opinions are not always valued, and stereotypes relating to gender, race, or social class. The latter are just a few of the examples. Maker spaces, digital fabrication labs (also known as fab labs), and spaces for promoting the Do-It-Yourself (DYI) culture are example of efforts to counter act problems of equity using digital technologies. For instance, the work by Kafai, Fields, and Searle (2014) on girls’ use of electronic textiles shows how the redesign of an activity (computing) by attending to gender issues can promote girls’ participation in what has traditionally been viewed as an area privileged for boys. Such

238

E. A. Kyza

spaces for learning are exciting; nonetheless, despite their appeal, more empirical evidence is required to be amassed regarding the conditions that foster learning in such settings. The attempt to address equity imbalances is also seen in contexts that involve participatory design, with the latter being an important aspect of many design approaches in the learning sciences but one that has received less attention from a research perspective. Learning scientists’ efforts to design innovations that are authentic, usable, and sustainable over time, include teachers working with researchers and developers of learning technologies as codesigners (Gomez, Kyza, & Manevice, 2018). Giving voice to the people who are being researched through their participation in designing the innovations can lead to increased feelings of ownership and successful adoptions of the designs. Even though we have many examples of codesign endeavors, reports on codesign roles, tensions between researchers and practitioners, who should be included in the design team, and the overall nature of the codesign effort are still few and do not address all relevant questions. For instance, much emphasis has been given on teachers as codesigners recently; considerably less work has been published on codesign work with other stakeholders, such as students or parents, as participants of codesign teams. To sum up, seen from an agentic perspective, the learning sciences community has a responsibility and an opportunity to investigate contexts that support greater equity (Nasir, Rosebery, Warren, & Lee, 2014) and propose ways in which the design of learning environments can contribute to diminishing such imbalances and maximizing the potential of learning technologies for all.

Conclusion Technology is a big driver for social and economic change; therefore, technologyenhanced learning is important for supporting the acquisition of twenty-first century competencies to enable greater and more equal participation in societal and personal decision-making. This chapter has discussed the topic of technology-enhanced learning (TEL) using a learning sciences perspective, which emphasizes attention to the coupling of developing authentic innovative learning environments that can be used for real-world learning and the concurrent development of instructional theories and theories about how people learn. As technology becomes more pervasive and ubiquitous, the opportunities for learning are increasing; at the same time, issues of access, equity, and responsible use of digital technologies are also extremely important. These issues have consequences for all stakeholders, including students, teachers, researchers, and policy makers, who need to attend to the multiple dimensions of learning, which in the twenty-first century are not only cognitive but are also social, ethical, and affective. Such approaches situate technology-enhanced learning as means to cultivate democracy and empower learners in achieving their utmost potential.

10

Technology-Enhanced Learning: A Learning Sciences Perspective

239

References Amiel, T., & Reeves, T. C. (2008). Design-based research and educational technology: Rethinking technology and the research agenda. Educational Technology & Society, 11(4), 29–40. Anderson, T., & Shattuck, J. (2012). Design-based research a decade of progress in education research? Educational Researcher, 41(1), 16–25. Baker, M., & Lund, K. (1997). Promoting reflective interactions in a computer-supported collaborative learning environment. Journal of Computer Assisted Learning, 13, 175–193. Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1), 1–26. Banks, J. A., Au, K. H., Ball, A. F., Bell, P., Gordon, E. W., Gutiérrez, K., . . . Nasir, N. I. S. (2007). Learning in and out of school in diverse environments: Life-long, life-wide, life-deep. Seattle, WA: The LIFE Center and the Center for Multicultural Education, University of Washington. Barab, S. A. (2006). Methodological toolkit for the learning scientist. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 151–170). New York, NY: Cambridge University Press. Barab, S. A., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1–14. Barab, S. A., Squire, K. D., & Dueber, W. (2000). A co-evolutionary model for supporting the emergence of authenticity. Educational Technology Research and Development, 48(2), 37–62. Bielaczyc, K., & Ow, J. (2014). Multi-player epistemic games: Guiding the enactment of classroom knowledge-building communities. International Journal of Computer-Supported Collaborative Learning, 9(1), 33–62. Bransford, J., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press. Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2, 141–178. Brown, A. L., & Campione, J. C. (1998). Designing a community of young learners: Theoretical and practical lessons. In N. M. Lambert & B. L. McCombs (Eds.), How students learn: Reforming schools through learner-centered education (vol. xiv, pp. 153–186). Washington, DC: American Psychological Association. Buckley, B. C., Gobert, J. D., Kindfield, A. C., Horwitz, P., Tinker, R. F., Gerlits, B., . . . Willett, J. (2004). Model-based teaching and learning with BioLogica™: What do they learn? How do they learn? How do we know? Journal of Science Education and Technology, 13(1), 23–41. Chan, C. K., & Aalst, J. (2008). Collaborative inquiry and knowledge building in networked multimedia environments. In International handbook of information technology in primary and secondary education (pp. 299–316). Chan, T.-W., Roschelle, J., Hsi, S., Kinshuk, Sharples, M., Brown, T., . . . Norris, C. (2006). One-toone technology-enhanced learning: An opportunity for global research collaboration. Research and Practice in Technology Enhanced Learning, 1(01), 3–29. Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O. Shea (Eds.), New directions in educational technology (pp. 15–22). Berlin, Germany: Springer. Collins, A., & Halverson, R. (2009). Rethinking education in the age of technology: The digital revolution and schooling in America. New York, NY: Teachers College Press. Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making thinking visible. American Educator, 15(3), 6–11. Cuban, L. (1990). Reform again, again, and again. Educational Researcher, 19(1), 3–13. Cuban, L. (1993). Computers meet classroom: Classroom wins. Teachers College Record, 95(2), 185–210. Daniels, H. (2011). Vygotsky and psychology. In U. Goswami (Ed.), Blackwell handbook of childhood cognitive development (pp. 673–696). Chichester, NH: Wiley-Blackwell Publishing Ltd.

240

E. A. Kyza

Davis, E. A. (2003). Prompting middle school science students for productive reflection: Generic and directed prompts. The Journal of the Learning Sciences, 12(1), 91–142. de Jong, T. (2006). Technological advances in inquiry learning. Science, 312, 532–533. Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. https://doi.org/10.3102/ 0013189X032001005 Dewey, J. (1903). Democracy in education. The Elementary School Teacher, 4(4), 193–204. Donnelly, D. F., Linn, M. C., & Ludvigsen, S. (2014). Impacts and characteristics of computerbased science inquiry learning environments for precollege students. Review of Educational Research. https://doi.org/10.3102/0034654314546954 Drachsler, H., & Greller, W. (2012). The pulse of learning analytics understandings and expectations from the stakeholders. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 120–129). Vancouver, BC: ACM. Drachsler, H., Verbert, K., Manouselis, N., Vuorikari, R., Wolpers, M., & Lindstaedt, S. (2012). Preface [special issue on dataTEL – Data supported research in technology-enhanced learning]. International Journal Technology Enhanced Learning, 4(1/2), 1–10. Eberle, J., Lund, K., Tchounikine, P., & Fischer, F. (Eds.). (2016). Grand challenge problems in technology-enhanced learning II: MOOCs and beyond. Cham, Germany: SpringerBriefs in Education. https://doi.org/10.1007/978-3-319-12562-6_1 Edelson, D. C. (2002). Design research: What we learn when we engage in design. The Journal of the Learning Sciences, 11, 105–121. https://doi.org/10.1207/S15327809JLS1101_4 Efstathiou, I., Kyza, E. A., & Georgiou, Y. (2017). An inquiry-based augmented reality mobile learning approach to fostering primary school students’ historical reasoning in non-formal settings. Interactive Learning Environments, 1–20. https://doi.org/10.1080/ 10494820.2016.1276076. Elmore, R. F. (1990). Restructuring schools: The next generation of educational reform. San Francisco, CA: The Jossey-Bass Education Series. Engeström, Y. (2011). From design experiments to formative interventions. Theory & Psychology, 21(5), 598–628. Engle, R. A., & Conant, F. R. (2002). Guiding principles for fostering productive disciplinary engagement: Explaining an emergent argument in a community of learners classroom. Cognition and Instruction, 20(4), 399–483. Feyzi-Behnagh, R., Azevedo, R., Legowski, E., Reitmeyer, K., Tseytlin, E., & Crowley, R. S. (2014). Metacognitive scaffolds improve self-judgments of accuracy in a medical intelligent tutoring system. Instructional Science, 42(2), 159–181. Fishman, B. J., Penuel, W. R., Allen, A. R., Cheng, B. H., & Sabelli, N. (2013). Design-based implementation research: An emerging model for transforming the relationship of research and practice. National Society for the Study of Education, 112(2), 136–156. Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: A meta-analysis. Review of Educational Research, 82 (3), 300–329. https://doi.org/10.3102/0034654312457206 Gago, J. M., Ziman, J., Caro, P., Constantinou, C. P., Davis, G., Parchmann, I., . . . Sjoberg, S. (2004). Europe needs more scientists: increasing human resources for science and technology in Europe. Report of the high level group on human resources for science and technology in Europe. [Online]. http://ec.europa.eu/research/conferences/2004/sciprof/pdf/final_en.pdf Gomez, K., Kyza, E. A., & Manevice, N. (2018). So this is going to be a collaboration? Teachers, researchers, and co-design. In F. Fischer, C. Hmelo-Silver, S. R. Goldman, & P. Reimann (Eds.), International handbook of the learning sciences. New York, NY: Routledge. Granger, E., Bevis, T., Saka, Y., Southerland, S., Sampson, V., & Tate, R. (2012). The efficacy of student-centered instruction in supporting science learning. Science, 338(6103), 105–108. Gulikers, J. T., Bastiaens, T. J., & Martens, R. L. (2005). The surplus value of an authentic learning environment. Computers in Human Behavior, 21(3), 509–521.

10

Technology-Enhanced Learning: A Learning Sciences Perspective

241

Gutiérrez, K. D. (2016). 2011 AERA presidential address: Designing resilient ecologies social design experiments and a new social imagination. Educational Researcher, 45(3), 187–196. Gutiérrez, K. D., & Jurow, A. S. (2016). Social design experiments: Toward equity by design. Journal of the Learning Sciences, 25(4), 565–598. Hannafin, M. J., & Land, S. M. (1997). The foundations and assumptions of technology-enhanced student-centered learning environments. Instructional Science, 25(3), 167–202. https://doi.org/ 10.1023/a:1002997414652 Herrington, J., & Oliver, R. (2000). An instructional design framework for authentic learning environments. Educational Technology Research and Development, 48(3), 23–48. Hiebert, J., Carpenter, T. P., Fennema, E., Fuson, K., Human, P., Murray, H., . . . Wearne, D. (1996). Problem solving as a basis for reform in curriculum and instruction: The case of mathematics. Educational Researcher, 25(4), 12–21. Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99–107. Hoadley, C., & Van Haneghan, J. (2011). The learning sciences: Where they came from and what it means for instructional designers. In Trends and issues in instructional design and technology (3rd ed.pp. 53–63). New York, NY: Pearson. Jackson, S. L., Stratford, S. J., Krajcik, J., & Soloway, E. (1994). Making dynamic modeling accessible to precollege science students. Interactive Learning Environments, 4(3), 233–257. Järvelä, S., Häkkinen, P., Arvaja, M., & Leinonen, P. (2004). Instructional support in CSCL. In J. W. Strijbos, P. A. Kirschner, & R. L. Martens (Eds.), What we know about CSCL (pp. 115–139). New York, NY: Kluwer Academic Publishers. Jeong, H., & Hmelo-Silver, C. E. (2016). Seven affordances of computer-supported collaborative learning: How to support collaborative learning? How can technologies help? Educational Psychologist, 51(2), 247–265. Joseph, D. (2004). The practice of design-based research: Uncovering the interplay between design, research, and the real-world context. Educational Psychologist, 39(4), 235–242. Kafai, Y., Fields, D., & Searle, K. (2014). Electronic textiles as disruptive designs: Supporting and challenging maker activities in schools. Harvard Educational Review, 84(4), 532–556. Kali, Y., & Linn, M. C. (2008). Technology-enhanced support strategies for inquiry learning. In Handbook of research on educational communications and technology (pp. 145–161). New York, NY: Lawrence Erlbaum Associates. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86. Kirschner, P. A., Beers, P. J., Boshuizen, H. P., & Gijselaers, W. H. (2008). Coercing shared knowledge in collaborative learning environments. Computers in Human Behavior, 24(2), 403–420. Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hämäläinen, R., Häkkinen, P., & Fischer, F. (2007). Specifying computer-supported collaboration scripts. International Journal of Computer-Supported Collaborative Learning, 2(2), 211–224. Koh, E., Cho, Y. H., Caleon, I., & Wei, Y. (2014). Where are we now? Research trends in the learning sciences. In J. L. Polman, E. A. Kyza, D. K. O’Neill, I. Tabak, W. R. Penuel, A. S. Jurow, . . . L. D’Amico. (Eds.), Proceedings of the international conference of the learning sciences (ICLS) 2014 (Part 1) (pp. 535–542). Boulder, CO: International Society of the Learning Sciences. Kollar, I., Fischer, F., & Slotta, J. D. (2007). Internal and external scripts in computer-supported collaborative inquiry learning. Learning and Instruction, 17(6), 708–721. Kolodner, J. L. (2004). The learning sciences: Past, present, future. Educational Technology, 44(3), 34–40.

242

E. A. Kyza

Kozulin, A., Gindis, B., Ageyev, V. S., & Miller, S. M. (Eds.). (2003). Vygotsky’s educational theory in cultural context. Learning in doing: Social, cognitive, and computational perspectives. Port Chester, NY: Cambridge University Press. Kuhn, D. (2007). Is direct instruction an answer to the right question? Educational Psychologist, 42(2), 109–113. Kyza, E. A. (2009). Middle-school Students’ reasoning about alternative hypotheses in a Scaffolded, software-based inquiry investigation. Cognition and Instruction, 27(4), 277–311. Kyza, E. A., Constantinou, C. P., & Spanoudis, G. (2011). Sixth Graders’ co-construction of explanations of a disturbance in an ecosystem: Exploring relationships between grouping, reflective scaffolding, and evidence-based explanations. International Journal of Science Education, 33(18), 2489–2525. https://doi.org/10.1080/09500693.2010.550951 Linn, M. C., Clark, D., & Slotta, J. D. (2003). WISE design for knowledge integration. Science Education, 87(4), 517–538. Looi, C. K., Seow, P., Zhang, B., So, H. J., Chen, W., & Wong, L. H. (2010). Leveraging mobile technology for sustainable seamless learning: A research agenda. British Journal of Educational Technology, 41(2), 154–169. Lowyck, J. (2014). Bridging learning theories and technology-enhanced environments: A critical appraisal of its history. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 3–20). New York, NY: Springer. Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H., & Koper, R. (2011). Recommender systems in technology enhanced learning. In Recommender systems handbook (pp. 387–415). Boston, MA: Springer. Mayer, R. E. (2003). Theories of learning and their application to technology. In H. F. O’Neil Jr., R. S. Perez, & H. F. O’Neil (Eds.), Technology applications in education: A learning view (pp. 127–157). New York, NY: Routledge. Merriënboer, J. J. G., & Bruin, A. B. H. (2014). Research paradigms and perspectives on learning. In M. J. Spector, D. M. Merrill, J. Elen, & J. M. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 21–29). New York, NY: Springer. Miller, M., & Hadwin, A. (2015). Scripting and awareness tools for regulating collaborative learning: Changing the landscape of support in CSCL. Computers in Human Behavior, 52, 573–588. Milrad, M., Wong, L.-H., Sharples, M., Hwang, G.-J., Looi, C.-K., & Ogata, H. (2013). Seamless learning: An international perspective on next-generation technology-enhanced learning. In Z. L. Berge & L. Y. Muilenburg (Eds.), Handbook of mobile learning (pp. 95–108). New York, NY: Routledge. Nasir, N., Rosebery, A., Warren, B., & Lee, C. D. (2014). Learning as a cultural process: Achieving equity through diversity. In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed.pp. 489–504). New York, NY: Cambridge University Press. National Research Council. (1996). The National Science Education Standards. Washington, DC: National Academy Press. National Research Council. (2009). Learning science in informal environments: People, places, and pursuits. Committee on learning science in informal environments. In P. Bell, B. Lewenstein, A. W. Shouse, & M. A. Feder (Eds.), Board on science education, Center for Education. Division of behavioral and social sciences and education. Washington, DC: The National Academies Press. Nicolaidou, I., Kyza, E. A., Terzian, F., Hadjichambis, A., & Kafouris, D. (2011). A framework for scaffolding students’ assessment of the credibility of evidence. Journal of Research in Science Teaching, 48(7), 711–744. https://doi.org/10.1002/tea.20420. Penuel, W. R., Cole, M., & O’Neill, D. K. (2016). Introduction to the special issue. Journal of the Learning Sciences, 25(4), 487–496. https://doi.org/10.1080/10508406.2016.1215753.

10

Technology-Enhanced Learning: A Learning Sciences Perspective

243

Quintana, C., Reiser, B. J., Davis, E. A., Krajcik, J., Fretz, E., Duncan, R. G., . . . Soloway, E. (2004). A scaffolding design framework for software to support science inquiry. The Journal of the Learning Sciences, 13(3), 337–386. Reiser, B. J. (2004). Scaffolding complex learning: The mechanisms of structuring and problematizing student work. The Journal of the Learning Sciences, 13(3), 273–304. Reiser, B. J., & Tabak, I. (2014). Scaffolding. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 44–62). New York, NY: Cambridge University Press. Rienties, B., Giesbers, B., Tempelaar, D., Lygo-Baker, S., Segers, M., & Gijselaers, W. (2012). The role of scaffolding and motivation in CSCL. Computers & Education, 59(3), 893–906. Roschelle, J. M., Pea, R. D., Hoadley, C. M., Gordin, D. N., & Means, B. M. (2000). Changing how and what children learn in school with computer-based technologies. The future of children, 76–101. Rose, C. P. (2018). Learning analytics in the learning sciences. In F. Fischer, C. Hmelo-Silver, S. R. Goldman, & P. Reimann (Eds.), International handbook of the learning sciences. New York, NY: Routledge. Säljö, R. (2010). Digital tools and challenges to institutional traditions of learning: Technologies, social memory and the performative nature of learning. Journal of Computer Assisted Learning, 26(1), 53–64. Sawyer, T. (Ed.). (2006). The Cambridge handbook of the learning sciences. New York, NY: Cambridge University Press. Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 97–115). Cambridge, UK: Cambridge University Press. Scheuer, O., Loll, F., Pinkwart, N., & McLaren, B. M. (2010). Computer-supported argumentation: A review of the state of the art. International Journal of Computer-Supported Collaborative Learning, 5(1), 43–102. Schmidt, H. G., Loyens, S. M., Van Gog, T., & Paas, F. (2007). Problem-based learning is compatible with human cognitive architecture: Commentary on Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 91–97. Schwarz, B. B., de Groot, R., Mavrikis, M., & Dragon, T. (2015). Learning to learn together with CSCL tools. International Journal of Computer-Supported Collaborative Learning, 10(3), 239–271. Slotta, J. D., & Najafi, H. (2013). Supporting collaborative knowledge construction with Web 2.0 technologies. In Emerging technologies for the classroom (pp. 93–112). New York, NY: Springer. Stahl, G. (2014). The constitution of group cognition. In L. Shapiro (Ed.), The Routledge handbook of embodied cognition (pp. 335–346). New York, NY: Routledge. Stahl, G., Koschmann, T., & Suthers, D. (2014). Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 479–500). Cambridge, UK: Cambridge University Press. Tabak, I., & Baumgartner, E. (2004). The teacher as partner: Exploring participant structures, symmetry, and identity work in scaffolding. Cognition and Instruction, 22(4), 393–429. Tabak, I., & Reiser, B. J. (2008). Software-realized inquiry support for cultivating a disciplinary stance. Pragmatics & Cognition, 16(2), 307–355. Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning a second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4–28. Walkington, C. A. (2013). Using adaptive learning technologies to personalize instruction to student interests: The impact of relevant contexts on performance and learning outcomes. Journal of Educational Psychology, 105(4), 932. Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5–23.

244

E. A. Kyza

Warschauer, M., & Matuchniak, T. (2010). New technology and digital worlds: Analyzing evidence of equity in access, use, and outcomes. Review of Research in Education, 34(1), 179–225. Wiliam, D., & Black, P. (1996). Meanings and consequences: A basis for distinguishing formative and summative functions of assessment? British Educational Research Journal, 22(5), 537–548. Williams, M., & Linn, M. C. (2002). WISE inquiry in fifth grade biology. Research in Science Education, 32(4), 415–436. Wood, D., Bruner, J. S., & Ross, G. (1976). Role of tutoring in problem-solving. Journal of Child Psychology and Psychiatry and Allied Disciplines, 17(2), 89–100. Yang, Y., & van Aalst, J. C. W. (2015). Assessment and collaborative inquiry: A review of assessment-based interventions in technology-enhanced K-14 education. In O. Lindwall, P. Hakkinen, T. Koschmann, P. Tchounikine, & S. Ludvigsen (Eds.), Exploring the material conditions of learning: The computer supported collaborative learning (CSCL) conference 2015 (vol. 1, pp. 190–196). Gothenburg, Sweden: The International Society of the Learning Sciences.

Dr. Eleni A. Kyza is an Associate Professor in Information Society at the Cyprus University of Technology, where she coordinates the Media, Cognition, and Learning Research Group. Her work examines the design and research of technology-enhanced learning, with recent work focusing augmented reality technologies at the elementary and secondary school level. She has worked extensively with science education teachers interested in integrating new technologies to support inquiry-based learning and teaching. Using the STOCHASMOS web-based platform, Dr. Kyza has been involved in design-based research in close collaboration with teacher design teams seeking to understand how technology mediates and supports collaborative learning in authentic classroom environments. Dr. Kyza holds a Ph.D. degree from the Learning Sciences program at Northwestern University, with a specialization in Cognitive Science, a master’s degree in Technology in Education from the Harvard Graduate School of Education, a B.Sc. in Education, summa cum laude, with a concentration in Educational Media and Technology from Boston University, and a Teacher’s Diploma, from the Pedagogical Academy of Cyprus. Her undergraduate, graduate, and postdoctoral studies were funded by CASP/Fulbright scholarships, the Cyprus Research Promotion Foundation and an IRG Marie Curie Fellowship from the European Commission. She can be reached at [email protected].

Self-Determined Learning: Designing for Heutagogic Learning Environments

11

Lisa Marie Blaschke

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heutagogy, or Self-Determined Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Principles of Heutagogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Designing for Heutagogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Elements of Heutagogic Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Strategies for Transitioning to and Sustaining Heutagogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heutagogy Across the Learner Life Cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K-12 Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vocational Education and Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lifelong Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

246 246 249 252 252 253 259 259 259 261 261 262 263

Abstract

Heutagogy, or the study of self-determined learning, has been rapidly gaining interest within the field of education as a response to the market demand for creative and competent employees who can adapt quickly to continuously changing, complex workplace environments. Heutagogy, which can be viewed as extension of pedagogy and andragogy, is based on the principles of human agency (learner-centeredness), capability, self-reflection and metacognition (double-loop learning or learning to learn), and nonlinear teaching and learning. When combined with today’s technology, heutagogy offers a holistic framework for teaching and learning that supports development of self-determined, autonomous learners and provides a basis for creating holistic, learner-centered

L. M. Blaschke (*) Center for Lifelong Learning, Carl von Ossietzky University of Oldenburg, Oldenburg, Germany e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_62

245

246

L. M. Blaschke

education environments. This chapter describes heutagogy, its principles, elements, and theoretical basis, as well as provides a review of the research and applications of heutagogy within different educational levels, from grade school to lifelong learning. In addition, the chapter gives guidance for instructors who want to design for heutagogy in the classroom and provides examples for integrating technological tools that support self-determined learning. Keywords

Heutagogy · Self-determined learning · Lifelong learning · Learner-centered design

Introduction Globalization, the rise of the white-collar worker and the knowledge economy, and rapidly changing technology have all contributed to the growing complexity of today’s work environments. A university or vocational degree is no longer the final threshold of learning, and employers expect their employees to continuously learn in order to remain productive and relevant entities within the organization. Learning has become a lifelong endeavor. Technology is on the forefront of leading this change, and the steady rise and expansion of today’s technologies have made knowledge readily accessible and further opened new avenues of learning, as well as influenced the ways in which learners learn. Demand for new forms of education that better prepare students for lifelong learning is on the rise (Ackoff & Greenberg, 2008; Little & Ellison, 2015; Sharpe, Beetham, & de Freitas, 2010). As a result of this increasing demand, educational theories such as heutagogy – or self-determined learning – have become even more relevant. Heutagogy provides a holistic framework for organizing and conducting learning and teaching within formal education, and also creates a foundation for practicing informal learning throughout one’s lifetime. This chapter will discuss the basic tenets of heutagogy, its fundamental principles and concepts, and underlying theories, as well as describe ways in which the theory can be applied within the classroom.

Heutagogy, or Self-Determined Learning Heutagogy was first defined by Stewart Hase and Chris Kenyon (2000), both from Australia, as the study of self-determined learning. The theory applies a holistic, humanistic approach to developing learner capacity and capability and makes learners “the major agent in their own learning, which occurs, as a result of personal experience” (Hase & Kenyon, 2007, p. 112). In self-determined learning, learners not only define what they will learn but how they will learn it – and are given full agency of their learning environment, content, and process.

11

Self-Determined Learning: Designing for Heutagogic Learning Environments

247

Heutagogy is grounded in earlier learner-centered educational theories and concepts, itself a theory that has emerged over time, evolving through a process much like that described by Albert Einstein (1938): Creating a new theory is not like destroying an old barn and erecting a skyscraper in its place. It is rather like climbing a mountain, gaining new and wider views, discovering unexpected connections between our starting points and its rich environment. But the point from which we started out still exists and can be seen, although it appears smaller and forms a tiny part of our broad view gained by the mastery of the obstacles on our adventurous way up. (Albert Einstein, in Einstein & Infield, 1938, pp. 158–9 as cited in Anderson, 2010, p. 23)

A variety of educational theories have contributed to the development of heutagogy, theories such as humanism (Maslow, 1943; Rogers, 1961), constructivism (Vygotsky, 1978), reflective practice (Schön, 1983), double-loop learning (Argyris & Schön, 1978), andragogy (Knowles, 1975), transformative learning (Mezirow & Associates, 1990), capabilities (Stephenson & Weil, 1992), and self-efficacy (Bandura, 1977). As such, the theory of heutagogy or self-determined learning can be considered as a continuation, or extension, of the theories that have preceded it, a progression of older theories to fit the emergent demands of a global society and the digital age. More specifically, heutagogy is considered to be a continuum of andragogy (selfdirected learning), the study of teaching and learning for adults (Canning, 2010; Knowles, 1975), where learners move through a process from pedagogy to andragogy and then to heutagogy, also known as the PAH continuum (Fig. 1; Blaschke, 2012). As learners become less dependent upon the instructor for

Fig. 1 Progression from pedagogy to andragogy and then to heutagogy (Blaschke (2012)

248

L. M. Blaschke

guidance and structure within the learning process (pedagogy), they are able to advance through the continuum to more autonomous and less structured learning environments. At the first level (pedagogy), instructors are firmly in control of the learning process, working toward motivating students to engage in learning content, for example, by defining specific instructional goals and learning objectives and activities along a structured, linear path. At the next level (andragogy), the instructor begins to cultivate the learner’s ability to self-direct his or her learning, allowing him or her more freedom in directing how learning occurs and providing less structure in the course design. However, the instructor is still the primary agent in the learning process, continuing to scaffold and construct the learning experience while allowing a higher degree of learner autonomy. At the heutagogy level, the learner assumes full control of his or her learning and is granted complete autonomy in deciding what and how he or she will learn. With pedagogy and andragogy, the instructional focus is primarily upon dissemination of the content and learning, which occurs in a linear way with instructor-defined learning outcomes. Heutagogy is a learner-centered theory that places the emphasis on students determining their learning path and on helping students understand how they learn. Learning is active and participatory, driven by the learners, who are proactively involved in the process of learning (e.g., through discovery and reflection, creation of new content/information, and collaboration with others). This form of self-determined learning occurs in a nonlinear manner, giving the learner full agency and following a self-defined learning path not designated by the instructor: from the early stages of learning design to final assessment of how and whether learning has occurred. While the goal of pedagogy, andragogy, and heutagogy is student learning, the approaches used for teaching and learning are different. Table 1 presents a delineation of the critical differences between pedagogy (teacher-directed learning), andragogy (self-directed learning), and heutagogy (self-determined learning). A somewhat opposing view to the PAH continuum is held by Hase and Kenyon (2013), who believe that the ability to be a self-determined learner is innate to humans and exists at a very young age. They argue that “. . .young children are very capable learners. But as we get older our education system seems to suppress our wish to ask questions, by telling us what we need to know” (p. 9). This belief in the basic human ability to be self-determined in learning is also well aligned with the educational approach used, for example, by the Montessori schools. Others argue that both viewpoints are valid, but that there may be those learners who must relearn self-directedness in their learning in order to advance to a state where they can practice self-determined learning (Blaschke, 2014a). Heutagogy should not be confused with self-regulated learning or with selfdetermination theory. In self-regulated learning, “students are self-regulated to the degree that they are meta-cognitively, motivationally, and behaviorally active participants in their own learning process. . .students monitor the effectiveness of their learning methods or strategies and respond to this feedback” (Zimmerman & Schunk, 2001, p. 5); however, with self-regulated learning, instructors continue to direct student learning and what they will learn. Self-determination theory from Deci

11

Self-Determined Learning: Designing for Heutagogic Learning Environments

249

Table 1 Heutagogy as a continuum of andragogy Pedagogy (teacher directed) Some single-loop learning Knowledge transfer and acquisition Linear design of courses/ curriculum and instructordirected learning approach Instructor directed Getting students to learn (content)

Andragogy (self-directed) Stronger emphasis on single-loop learning Competency development Linear design of courses/curriculum with learner-directed learning approach (e.g., organizing his/her learning) Instructor-learner directed Getting students to learn (content)

Heutagogy (selfdetermined) Single- and doubleloop learning Capability development Nonlinear design and learner-determined learning approach Learner determined Getting students to understand how they learn (process)

Based on Blaschke (2012)

and Ryan (2002) places a significant emphasis on the role of motivation in psychological growth and development. Self-regulation and self-motivation are both components of heutagogy, where the learner must be motivated to learn in a selfdetermined way, but these are not the singular aspects of the theory, the principles of which will be described further in the next section.

Principles of Heutagogy Heutagogy is built around the following key principles: human agency (learnercenteredness), capability, self-reflection and metacognition (double-loop learning), and nonlinear teaching and learning (Fig. 2). It is holistic and centered around the learner, with the student defining his/her learning journey and supported by the teacher as guide. In applying the theory, there is a shift to learner-centeredness, moving from traditional pedagogical and andragogical teaching where the instructor is the sage on the stage. In heutagogy, the learner becomes the sage and the instructor the guide on the side.

Human Agency (Learner-Centeredness) Human agency, the ability of humans to make own choices in life, is a central principle of heutagogy, where the learner is the agent or driver of his or her learning. Within a heutagogic environment, learners are given complete responsibility of the learning process and determine what they will learn and the way in which they will learn and ultimately assess the success of their learning (Hase & Kenyon, 2000, 2007, 2013). Learners become the drivers of the learning process, thus requiring them to be highly autonomous, which can then help them feel more competent and in turn promote development of intrinsic, self-motivation (Deci & Ryan, 2002). Giving learners autonomy in defining an own learning path can also empower learners rather than oppress them (Freire, 1970).

250

L. M. Blaschke

Fig. 2 Principles of heutagogy

Capability One of the major goals of heutagogy is to create capable learners who are well equipped for the demands of complex and changing work environments. Stephenson (1996) describes capability as more than skills and knowledge and considers it to be necessary in order for students to succeed outside of formal learning environments. According to Stephenson, three factors have established the relevance of capability – factors that continue to be relevant: “feedback on the quality of graduates; uncertainty and change in society and the work-place; and the growing importance of individual responsibility and interdependence” (p. 3). Competent learners can demonstrate what they have learned, for example, a skill or set of knowledge within a specific context, while capable learners exhibit their capabilities by applying skills and knowledge in new and unfamiliar situations or contexts. While andragogy focuses on the development of skills and competencies, heutagogy takes student learning a step further by placing a focus on building and expanding upon competencies and giving students ownership of learning. This active involvement and ownership of the learning path and process increases learner self-motivation, eventually leading to development of capability. Capability then emerges from a sense of self-efficacy, where learners feel confident in coping with and performing in new and unfamiliar situations and contexts. Other characteristics of the capable learner include creativity, ability to communicate and work with others, and confidence (Stephenson, 1996; Stephenson & Weil, 1992). Self-Reflection and Metacognition (Double-Loop Learning) Additional and related key principles of heutagogy are that of self-reflection and double-loop learning. Having an understanding of how they learn is essential in order for learners to be successful in adopting self-determined learning. This reflection occurs in a holistic way, with learners reflecting on the new knowledge that they

11

Self-Determined Learning: Designing for Heutagogic Learning Environments

251

have acquired, as well as the way in which they have acquired it. Dewey (1997) described this process as one in which the learner suspends his or her assumptions and searches for new information that corroborates or refutes those assumptions or suggestions. In doing so, the learner engages in “systematic and protracted inquiry. . . (which) are the essentials of thinking” (p. 13). In preparation for the complexities of the work environment, Schön (1983) argues that learners – and future employees – must become reflective practitioners and adopt certain practices in order to be able to adapt to the demands of the workplace. He describes the reflective practitioner as one who is able to: • Know in action: Apply what is learned in making decisions. • Reflect in action: Think about an activity as it is being carried out (e.g., “thinking on your feet” or “learning by doing”). • Reflect in practice: Consider the activity that has been carried out and how it has been done so while applying corrective action (e.g., improvements) to current and future activities. Double-loop learning is related to self-reflection, extending the self-reflection process further in that the learner (1) engages in his or her thinking about the ways in which his or her personal belief and values systems align with what has been learned and how it has been learned and then (2) adapts actions and mental models accordingly (Argyris & Schön, 1978; Eberle & Childress, 2009). Double-loop learning is not the same as the single-loop learning characteristic of pedagogy and andragogy, where the learner sets out to find a solution to a problem: first identifying a problem, then potential actions, and finally evaluating outcomes. In double-loop learning, the learner engages in a similar process, but also considers the steps taken to learn and, through self-reflection, how this influences learner beliefs and actions. As a result, double-loop learning engages learners on both behavioral and psychological levels. In undertaking this process, learners fully engage in reflective practice and challenge previously held assumptions, thus opening up opportunities for transformative learning to occur (Mezirow & Associates, 1990).

Nonlinear Teaching and Learning A final fundamental element of heutagogy is nonlinear learning. The learner is responsible for learning and defines the learning pathway; as each learner’s experiences and mental model varies, the path taken can be divergent and unpredictable (Long, 1990). The aspect of nonlinear learning aligns closely with Thorndike’s ideas about connectionism, “the neural connection between stimuli (S) and responses (R)” (Olson & Hergenhahn, 2009, p. 53), and the theory of constructivism, where learners are actively involved in learning as a process of discovery and interpret new information and construct new knowledge based on existing models of understanding and by thinking and reflecting upon what has been learned (Tinkler, 1993). When learners practice nonlinear learning, instruction must also be adapted. Dewey (1997) characterized the processes of teaching and learning “as correlatives or corresponding processes, as much so as selling and buying” (p. 29). In adapting to

252

L. M. Blaschke

Table 2 Designing for heutagogy: elements Element Exploration Creation (creativity) Collaboration Connection (community) Reflection

Assessment Openness (sharing)

Description Nonlinear searching of new paths of learning; creation of a culture of learner discovery and inquiry Development of new content by building upon what has been learned Working with others to build and construct new knowledge and content Connecting with others both inside and outside of the classroom to create new networks for supporting learning; creating personal learning environments for lifelong learning Thinking about what has been learned and how it has been learned, as well as how this process and the new knowledge acquired influences mental models, beliefs, and values Considering how and whether learning has occurred both individually and as a group; establishing the means by which learning will be assessed Sharing of new content with others in the community; showcasing acquisition of skills and competencies

Based on Blaschke and Hase (2015a)

a learner-determined learning path, the role of the instructor in the process then becomes one of mentor and guide of the learning experience, and there is a “transition of the perception of power away from the teacher or facilitator to the learner” (Long, 1990, p. 69). Hase (2014) refers to instructors in this role as learning leaders, exhibiting characteristics such as the ability to handle ambiguity, the capacity to nurture learner engagement and to learn themselves, and the capability of applying open systems thinking (see Table 2).

Designing for Heutagogy First and foremost, heutagogy is centered on the philosophy that learners determine their own learning paths. Thus, the learning environment is entirely designed, defined, and built around and by the individual learner. Instructors and institutions are no longer at the center of the learning experience – learners are. Due to its learner-centered focus and the learner role as self-determined and autonomous, heutagogy creates a new dynamic in education, and designing for heutagogy requires all stakeholders in the system to adapt: from instructors and learners to the institution as a whole.

Elements of Heutagogic Design Table 2 identifies the central elements that should be considered when designing for heutagogy. Human (learner) agency is central to heutagogic design with the learner driving the design process while guided by the instructor.

11

Self-Determined Learning: Designing for Heutagogic Learning Environments

253

Strategies for Transitioning to and Sustaining Heutagogy How do we implement these elements to support development of self-determined learners? Creating an environment that supports self-determined learning is not solely an endeavor of the individual learner. It requires a holistic approach involving instructors and organizational management and leadership and one where all stakeholders have a solid understanding of heutagogy and its principles and how implementation of the approach will impact them. In realizing a heutagogic environment, it is critical that the institution creates and sustains a culture of self-determined learning. In addition, cultural issues must be addressed, for example, when the approach is a new concept and pre-existing expectations of teaching and learning are held by those within the institution (learners, instructors, and institutional management) and externally (stakeholders such as parents, future employers, and society in general) (Long, 1990). To address these issues, a campaign of awareness, can be undertaken, where a clear and shared understanding of and commitment to heutagogy are promoted. There must also be a commitment to self-determined learning by both the learner and the instructor. The learner must understand his or her responsibility for learning and be willing to take on that responsibility. Instructors, too, must understand and embrace their role as guides and mentors of the learning experience. In addition, the institution or organizational structure needs to be a proponent of self-determined learning and provide the necessary infrastructure of support for implementing heutagogy.

Student as Self-Determined Learner The potential for student resistance to self-determined learning cannot be underestimated and should be acknowledged by instructors and institutions from the onset. With heutagogy, students become active participants in their learning, often forced to move out of their comfort zones and experiencing failure before achieving success. Reasons for student resistance can include a fixed expectation of the instructor role, a fear of failure, a lack of traditional and externally accepted measurements of individual learning progress and success, a lack of motivation to learn in a self-directed and self-determined way, and inexperience in and uncertainty about the learning approach (Blaschke, 2014a; Stephenson & Weil, 1992). When encountering student resistance to heutagogy, the following steps can be taken to help learners gain ownership and become agents of their learning: explaining the approach to students and its relevance to the student’s present and future goals and contexts, exposing students to professional practice (e.g., requiring action research and application of new knowledge in familiar and unfamiliar environments), employing peer support, encouraging self-monitoring of progress and providing feedback on student progress, and reporting stories of others’ success in using the approach (Long, 1990). As guidance for learner and instructor collaboration, Andrews (2014) introduces the FACE model, which includes elements of flexible and negotiated curriculum, assessment that is likewise flexible and negotiated, contracts defining learner-

254

L. M. Blaschke

defined pathways, and enquiry-based questions generated by learners. Flexible learning contracts, a two-way process between learner and instructor, are also described and recommended in Stephenson and Weil (1992) and Long (1990) as a means of aligning learner goals with institutional and instructional objectives and for promoting learner self-actualization and reflection on the learning process. These contracts should incorporate learner-centered methods for design and assessment and support a structural (scaffolded) progression through learning material, one that moves students toward more self-managed learning and autonomy (Stephenson & Weil, 1992). In addition, instructors need to promote ongoing reflection on the learning process, for example, by asking learners to relate new information to past experience and to their feelings, values, and perceptions and then having them reevaluate their experience based on the new information acquired (Boud, Keough, & Walker, 1985). Instructors can achieve this by providing learners with “a context and a space to learn, give support and encouragement, listen to the learner, and provide access to particular devices which may be of use” (Boud et al., 1985, p. 38). Techniques that are helpful in realizing reflection include inquiry-based questioning within classroom discussions, autobiographies, and reflective learning journals (Blaschke & Brindley, 2011; Boud et al., 1985). Development of a learner’s sense of self-efficacy can emerge from this process of self-reflection. Encouraging a growth mind-set, where basic qualities are cultivated by personal effort (Dweck, 2006), can also contribute to self-efficacy development and to the success of heutagogy. Failure need not be viewed as a negative result of learning, but rather a desirable stepping-stone to achieving real learning and success. Bandura (1977) states that “To succeed at easy tasks provides no new information for altering one’s sense of self-efficacy, whereas mastery of challenging tasks conveys salient evidence of enhanced competence. . . Thus, people who experience setbacks but detect relative progress will raise their perceived efficacy more than those who succeed but see their performances leveling off compared to their prior rate of improvement.” (p. 201). The more varied these experiences are, the more success a learner will have in developing self-efficacy. Makerspaces are one way in which instructors can help students develop selfefficacy and a growth mind-set, as these hands-on learning activities allow learners to design, create, and collaborate while also experiencing and learning from failure. Gerstein (2015) describes these spaces as giving learners “a can-do attitude and a growth mindset – a belief that your capabilities can be developed, improved and expanded. It’s not just a matter of what you know, it’s a matter of taking risks and perhaps failing and learning from those failures. It’s a matter of being open to exploring new possibilities and developing your full potential.” (para. 20). Making use of open educational resources (OER) is yet another means for supporting self-determined learning. OER not only allow for the free reuse and sharing of educational resources but also the remixing, revising, and redistribution of those resources (Wiley, 2014). The open educational movement supports an open learning culture (Price, 2013) and makes educational resources more freely available to both learners and instructors while also giving them the opportunity and the ability

11

Self-Determined Learning: Designing for Heutagogic Learning Environments

255

to create and collaborate on and share educational resources. Massive open online courses (MOOCs), a technological framework in which open educational resources can be created and distributed, are another way in which learners can explore and engage in self-determined learning. Although students may resist a transition to self-determined learning, once they have “crossed over” to this type of learning, there is less desire to return to more traditional educational forms. Brandt (2013) describes her experience of transitioning to heutagogy – a transition she initially resisted – as empowering and one that led to transformational learning: “Having tasted the freedom of learning and getting the benefit of university credit for it, I wanted more. I wanted to write to the instructors and relate my ideas and tell them where I needed new knowledge. The years of disciplined obedience kept me quiet – grades are important, after all” (p 103).

Instructor as Learning Leader In self-determined learning, the role of the instructor does not become diminished, but rather is enhanced and – as with the student role – empowered. In adopting their new role as guides of learning, instructors become situational leaders, ones who “must adapt his or her behaviour to suit the readiness of an individual for a particular task, function, or activity” (Long, 1990, p. 149). This transition to situational leadership requires that the instructor models desired behaviors, identifies the readiness of the learner for learner-managed learning (willingness, knowledge, and ability), and works to engage and motivate the learner in actively partaking in the learning process (Long, 1990). Empathy and positive reinforcement, for example, in the way of formative assessment and feedback, are instrumental in achieving this goal (Booth, 2014). New skills and attributes become incorporated into the instructor profile as the instructor moves toward heutagogic teaching and develops his/her role as a learning leader (Table 3). To assist instructors in the transition to learner-managed learning environments, Long (1990) recommends starting small in initiating the approach, modeling selforganization and time management skills, practicing teamwork and peer mentoring and collaboration, and providing staff development on the principles of the approach. Creating teams of instructional designers and instructors in designing and creating heutagogic environments is also recommended, as is providing instructors with opportunities (e.g., time and money resources) for the autonomous pursuit of their own learning in embracing their new role (Andrews, 2014). Communities of practice built to provide instructor support and sharing of experiences and practices are also a productive way of assisting instructors during and after the transition (Andrews, 2014; Hexom, 2014; Price, 2014). Although not new to education, flipped classrooms – combined with technology – are also an effective method for creating heutagogic environments. Flipped classrooms allow students to self-direct their learning activities using media such as online videos, chats, and discussion forums when outside of the classroom while

256

L. M. Blaschke

Table 3 Attributes and skills of the learning leader The capacity to accept and manage ambiguity Attributes Low need for control Openness to experience (one of the Big 5 personality traits) Moderate on perfectionism scale (Big 5) High stability (low anxiety) (Big 5) Capability Skills Project management Ability to use social media

The ability to foster engagement Attributes Empathy Optimism Flexibility to change approaches as circumstances change Skills Interpersonal effectiveness Ability to selfregulate Understanding of how to motivate others Ability to foster a shared purpose and vision Maintaining direction Fostering the joy (and rewards) of learning

The ability to learn Attributes Willingness to change own ideas or beliefs Skills Ability to research and learn Being thoroughly on top of one’s subject areas Having wide and accessible networks Ability to share openly with others Knowledge management skills The ability to foster collaborative learning Ability to apply learning Willingness to change own ideas and beliefs

The ability to apply open systems thinking Attributes Willingness to empower others Skills The capacity to frequently scan the external environment Ability to foster participative democracy/collaboration decision-making and process Capacity to work in a team as leader and member Ongoing internal and external analysis of effectiveness (continuous improvement) The ability to filter information (research skills)

Blaschke & Hase (2015a)

using the face-to-face classroom to actively collaborate with peers and engage in discussion, exploration, and hands-on activities.

Institutions as Networks of Support When transitioning to self-determined learning, the institution takes on the role of a supporting network, one that must support both learners and instructors in their new roles. To assist students in the transition, Schön (1983) advises that institutions provide student practicums and real-world, practical examples within the curriculum and to create networks with the professional world from which students can both benefit and learn. To realize their transition to providing a network of support for learners, institutions will need to spread the net wider and work more closely with

11

Self-Determined Learning: Designing for Heutagogic Learning Environments

257

employers in order to gain a better understanding of workplace requirements and demands and to create connections from academia to the professions (Stephenson & Weil, 1992). The role of the institution is no longer strictly accreditation, but one of enabling the networked connections critical to student success in transitioning to and surviving in today’s workplace. To help instructors adapt to the transition, institutions need to provide initial and ongoing staff development and support. When hiring new staff and faculty, it will be critical to hire and train those who value a self-determined approach to learning and teaching. Change within institutions can be challenging, and a move toward heutagogy can push against the status quo. Stephenson and Weil (1992) find that “New approaches inevitably test these structures. Navigating the pathways to change can be unnecessarily burdensome, requiring high commitment and political ingenuity for success” (p. 181). As with any institutional change, it is essential to recognize and support champions of change and obtain higher-level management support.

The Role of Technology In 1983, Schön wrote the following, describing the most desirable environment for reflective practice to occur: A reflective teacher needs a kind of educational technology which does more than extend her capacity to administer drill and practice. Most interesting to her is an educational technology which helps students become aware of their own intuitive understandings, to fall into cognitive confusions and explore new directions of understanding and action. (Schön, 1983, p. 333)

Although Schön could have in no way predicted the revolution brought about in education by rapid technological development, his description is remarkably prescient in that it portrays affordances that are characteristic of today’s educational technologies, specifically the freedom to explore, create, collaborate, connect, and share. This alignment between technological affordances and self-determined learning is particularly relevant when considering the current educational trend toward more learner-centered education. The New Media Consortium’s 2015 Horizon Report on technology in higher education acknowledges this trend, stating that “a student-centered approach to education has taken root, prompting many higher educational professionals to rethink how learning spaces should be configured” (Johnson et al., 2015, p. 18). Heutagogy aligns well with the affordances of current technology, in that the technology supports exploration, learner-determined learning, and personalization of learning; is nonlinear in its design; promotes creation and sharing of information and knowledge; allows for collaboration in co-creation of new information and knowledge; and promotes a network of connectivity that can bridge the gap between academia and the professions while creating personal learning environments (PLEs) and networks for lifelong learning (Blaschke, 2012; McLoughlin & Lee, 2007). According to Price (2013), digital technologies can “accelerate the changes in behaviours, values, and actions, which then transform the way we learn and our capacity to learn” (p. 31). A variety of media and

258

L. M. Blaschke

technologies can be incorporated in support of self-determined learning; a few examples are mentioned here, but are by no means exhaustive, as the technological landscape continues to evolve and expand. A fundamental goal of heutagogy is to promote and sustain lifelong learning as learners acquire competencies and the capability to learn in new and unfamiliar environments. To achieve this goal, learners should be encouraged to establish and build personal learning environments (PLEs). These environments can be internal or external to technological environments, although a mixture of both is recommended. The PLE incorporates not only aspects of formal learning but also informal learning and is based “on the idea that learning will take place in different contexts and situations and will not be provided by a single learning provider” (Atwell, 2007, p. 1). By encouraging and supporting the development of PLEs, instructors can equip students (and themselves) not only for self-determined learning within formal education but also for informal and lifelong learning (Richardson & Mancabelli, 2011). Social media, such as Twitter, LinkedIn, Google Drive, and Facebook, can be helpful technological tools for supporting learners in creation of and collaboration on knowledge artifacts, for sharing resources, and for further expanding their personal, educational, and professional networks (Blaschke & Brindley, 2015). These social networking tools have the potential of improving learner engagement and creating “a new role for the learner as active participant in, rather than passive recipient of, learning experiences” (Facer & Selwyn, 2010, p. 34). Blaschke (2012, 2014a, b) cites a number of examples of how these media can be used for supporting heutagogy, for example, e-portfolios for showcasing acquired knowledge, skills, and competencies; online learning journals and blogs for self-reflection; social networking sites such as LinkedIn, Facebook, and Twitter for creating networks, conducting group work, and for distributing and sharing educational resources and research. Mobile learning in the form of smartphones, pads, and tablets also provides vehicles for supporting a self-determined learning approach in that their usage “facilitate(s) the learning process by encouraging conversations and dialogue between the learner and teacher across authentic learner-generated contexts” (Narayan & Herrington, 2014, p. 153). The heutagogic model developed by Narayan and Herrington (2014) includes elements of participation (collaboration and communication), productivity (creation and consumption), and personalization (learner choice). Cochrane, Antonczak, Gordon, Sissons, and Withell (2012) report on success in using mobile social media for designing heutagogic learning environments that support development of learning communities and communities of practice for learners and instructors, as well as incorporate real-world collaborations and business applications of new knowledge. Learners can also use mobile technology applications to create individual PLEs, for example, choosing the applications that they prefer to use for learning, connecting, collaborating, and networking. The increasing popularity of mobile devices (such as the iPad) also makes this technology a feasible way of supporting ongoing, self-determined professional development, a finding supported by Hexom (2014). Learning analytics also have the potential for use in designing an environment that supports heutagogy. In a recent blog post on learning analytics and double-loop

11

Self-Determined Learning: Designing for Heutagogic Learning Environments

259

learning, Atwell (2016) suggests that learning analytics could provide a framework for guiding learners in the process of reflecting on and better understanding their learning process, thus leading to learner transformation and self-discovery (not discovery of content) and further development of learners’ mental models.

Heutagogy Across the Learner Life Cycle In discussions with educators on the applicability of heutagogy in educational environments, the argument often arises that the approach is not relevant for certain levels of education. The following sections provide examples from the literature of how heutagogy can be realized across the life cycle of education, from kindergarten to 12th grade (K-12 education) through to lifelong learning.

K-12 Education Hase (2013) argues that children are quite capable of self-determined learning, but that the educational system’s approach of lectured teaching and learners’ passive consumption of information suppress the practice of self-determined learning at a young age (see previous discussion of the PAH continuum). St. Paul’s Junior School in Brisbane, Australia, has experienced success in applying the approach in its Junior School (from pre-prep to year 6) and has implemented heutagogy by redesigning its curriculum to become more learner centered, using flexible learning contracts, negotiated (instructor-learner) assessment criteria, and learner-generated inquiry to reach government-mandated educational objectives (Andrews, 2014). Andrews (2014) reports that in realizing the approach, students create portfolios of their learning journey, and instructors provide guidance by mentoring and coaching students along their learning path (according to the individually negotiated contracts). The success of the approach is based on team development of curriculum design (instructors working together with instructional designers); a culture of openness, communication, and trust within the organization; and investment in staff development and resources (e.g., making time for teachers to pursue selfdetermined learning). Much like the Montessori approach to education, St. Paul’s Junior School supports collaboration and learning across grade levels, as well as peer assessment. Blaschke (2014a) also reports on using heutagogy in helping grade school children learn English as a second language, by modeling behavior, supporting collaborative peer learning, and emphasizing play and interaction (to encourage motivation).

Higher Education In a case studies report on using heutagogy for primary school teachers in the UK, Canning (2013) finds that the approach empowered teachers and encouraged

260

L. M. Blaschke

teachers’ reflection in practice and development of teacher self-confidence, resulting in transformative learning, as well as increased teacher motivation. In addition to face-to-face curriculum, the Early Years Professional Status (EYPS) graduate program also incorporated online communities of practice for teachers to discuss and collaborate on course material, thus helping teachers establish a network of connections for current and future practice. Heutagogy as a teaching approach for primary school teachers is also being piloted at a college in Israel, where teachers learn of heutagogy through practice and then model self-determined learning within their classrooms (S. Back & A. Glassner, personal communication, January 19, 2016). In an example of applying heutagogy in his teaching of university-level courses, Dick (2013) describes the use of experiential teams to support action research and action learning and the design principle of “freedom within clear, negotiable limits, high challenge, and high support within the team or group” (p. 51). His approach involves first “crafting the context” of the course, that is, building and expanding community, emphasizing career planning and contact with the profession, and negotiating curriculum with students (Dick, pp. 41–43). Next, he works with students in negotiating the assessment and evaluation processes and criteria. Dick then uses teams for students to engage in action learning, where they have full autonomy in defining and carrying out group projects, with Dick coaching them along the way.

Distance and Online Learning Examples of heutagogic practice can also be found in distance learning environments. In general, online learning has a close affinity with heutagogy due to the high level of learner autonomy required and the role of the instructor as guide (Blaschke, 2012). In applying heutagogy in an online master’s program, Kerry (2013) uses course materials that spark student interest, encourages them to explore topics further, and emphasizes ongoing and supportive tutor guidance and feedback; findings of Kerry’s research showed that students were more reflective and motivated as a result of the course. Within the online graduate program in which she teaches, Blaschke (2014a, b) incorporates social media and learner activities for building competencies and skills, as well as online e-portfolios to showcase abilities and learning journals for self-reflection. Scaffolding support and providing personal guidance are other critical instructor activities that she recommends in order to help students engage in self-determined learning. The largest correspondence distance education provider in the world, the University of South Africa (UNISA), has embarked upon a monumental change for its institution, instructors, and students – transitioning from correspondence education to online learning – and is piloting heutagogy as its model of pedagogy for developing self-directed and self-determined learners (van Schoor & Mischke, 2014; Msila & Setlhako, 2012). By encouraging students to be self-determined in their learning approach, the institution hopes to not only empower learners but also develop learners who are equipped for the complexities of the twenty-first-century workforce. Early results of UNISA’s Signature Course project are promising, despite

11

Self-Determined Learning: Designing for Heutagogic Learning Environments

261

the challenges created by South Africa’s poor technological infrastructure for supporting online learning (van Schoor & Mischke, 2014).

Vocational Education and Training Numerous examples of the application of heutagogy within vocational education and training are available in the literature (Hase & Kenyon, 2013). When conducting workshops, Hase (2013) uses a heutagogic, Socratic approach that allows learners to define their learning objectives by identifying what they want/need to learn, reflecting upon their learning gaps, and then negotiating a path to learning and assessing whether learning has occurred. Although participants are often uncertain and sometimes anxious about the approach, they report that the process is empowering and results in a positive learning experience (Hase, 2013). Kenyon (2014) applies a unique approach to his workshop training, using his Deedeekun© experiential in order to teach participants about the principles of heutagogy. Northcote and Boddey (2014) describe a self-help online resource (Moodle’s Little Helper) that their institution has developed to provide training for instructors on delivering online courses. In implementing its approach, the institution first identified where learning gaps existed among instructors (through researcher reflection journals and faculty surveys) and then stored online data resources, such as best practices, instructions, and tips, within the institution’s learning management system (LMS); faculty could then access the topics on an as-needed basis. Feedback is ongoing and gathered from learning analytics, email, HelpDesk requests, and other sources, which then feeds into the process of resource development for the professional development site and determines which resources are needed by faculty. The approach has helped the institution save time and money on investing in structured face-to-face or online professional development courses, as well as assisted in identifying new areas for development, such as an online community of practice for faculty to collaborate and share best practices.

Lifelong Learning Heutagogy also has applications outside of formal education and in professions that require lifelong learning. For example, heutagogy continues to be highly relevant within the health professions, where lifelong learning is essential. Within nursing education, Ramsey et al. (2013) describe a need for developing learner capability and the ability for self-reflection and find their use of heutagogy has given their students the ability to “unravel the ever-present and inherent uncertainties that define nursing practice” (p. 95). Communities of practice, where individuals join groups for the purposes of learning, are the most common form of heutagogy for lifelong learning. Price (2014) describes these communities emerging in varied ways, for example, through face-to-face meeting events (TeachMeets) and Twitter hashtag (#) meetings.

262

L. M. Blaschke

Characterized by learner autonomy, immediate availability, participant generosity to share and guide, playfulness, respect for one’s colleagues, and the high visibility of the platforms used, these communities allow for professionals to take control of their learning, build upon their professional skills, and expand their network of knowledge and contacts (Price, 2014). Another example of an online community is the Heutagogy Community of Practice (https://heutagogycop.wordpress.com/), which was established by researchers and practitioners of the theory in order to further the discussion and development of heutagogy and from which has emerged numerous research initiatives and scholarly publications, as well as two conferences on heutagogy (Booth, Blaschke, & Hase, 2016). Massive open online courses, or MOOCs, are yet another example of a framework that supports self-determined learning for professional development, as learners can choose the learning topic and then engage and disengage from the MOOC environment as desired.

Conclusion Thus far, the practice of heutagogy has surfaced in pockets of innovation around the world, and interest in heutagogy and self-determined learning continues to rise, as educators and institutions seek out better ways of educating today’s learners. With the advancement of the practice and study of self-determined learning, new areas for research and development have emerged. One of these areas is that of brain research, which Hase (Blaschke & Hase, 2015b) finds further affirms and substantiates the heutagogic practice of inquiry, problem-solving (trial and error), and nonlinear learning. Other areas of research include continuous development of interdisciplinary learning and studies that allow for more learner-designed and self-determined learning, with a stronger focus on problem-solving (Dietz & Eichler, 2013), as well as moving toward learning solutions that support active dialogue, develop learner capacity, and encourage open and ongoing dialogue in community (Snowden & Halsall, 2014). Another interesting development within heutagogy is its role in promoting social justice in learning, which aligns well with Freire’s (1970) call to end pedagogies of oppression. In the African example presented by Msila (2014), he advocates pursuing pedagogies that better reflect the African culture, one that recognizes learners as reasonable people and that supports collaboration and learner empowerment. Enabling learners to be self-determined in the education experience removes the objectification that occurs when humans are no longer allowed to make their own decisions (Freire, 1970), and in this sense, heutagogy is well positioned as an approach that could address the educational needs of learners in both developing and developed countries. Heutagogy’s principles of human agency (learner-centeredness), capability, selfreflection and metacognition, double-loop learning, and nonlinear learning help to create learning environments that encourage a growth mind-set and deeper levels of learning. Given these core principles – combined with the power of today’s

11

Self-Determined Learning: Designing for Heutagogic Learning Environments

263

technologies – and its applicability across all levels of education and disciplines, the theory can be strongly positioned as a holistic educational framework that empowers both learners and instructors and establishes a foundation for lifelong learning.

References Ackoff, R. L., & Greenberg, D. (2008). Turning learning right side up: Putting education back on track. Upper Saddle River/New Jersey, USA: Prentice Hall. Anderson, T. (2010). Theories for learning with emerging technologies. In G. Veletsianos (Ed.), Emerging technologies in distance education. Edmonton, Canada: Athabasca University Press. Retrieved from http://www.aupress.ca/books/120177/ebook/02_Veletsianos_2010-Emerging_ Technologies_in_Distance_Education.pdf. Andrews, J. (2014). From obstacle to opportunity: Using government-mandated curriculum change as a springboard for changes in learning. In L. M. Blaschke, C. Kenyon, & S. Hase (Eds.), Experiences in self-determined learning. USA: Amazon.com. Argyris, C., & Schön, D. (1978). Organizational learning: A theory of action perspective. Reading, MA: Addison Wesley. Atwell, G. (2007). Personal learning environments: The future of eLearning? elearning papers. Retrieved from http://www.informelles-lernen.de/fileadmin/dateien/Informelles_Lernen/ Buecher_Dokumente/Attwell_2007-ple.pdf Atwell, G. (2016). Double-loop learning and learning analytics. Pontydysgu {Blog}. Retrieved from http://www.pontydysgu.org/2016/05/double-loop-learning-and-learning-analytics/ Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. Blaschke, L.M. (2012). Heutagogy and lifelong learning: A review of heutagogical practice and self-determined learning. International Review of Research in Open and Distance Learning, 13 (1), 56–71. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1076/2113 Blaschke, L. M. (2014a). Moving forward in the PAH continuum: Maximizing the power of the social web. In L. M. Blaschke, C. Kenyon, & S. Hase (Eds.), Experiences in self-determined learning. USA: Amazon.com. Blaschke, L.M. (2014b). Using social media to engage and develop online learners in selfdetermined learning. Research in Learning Technology. Retrieved from http://www. researchinlearningtechnology.net/index.php/rlt/article/view/21635/html Blaschke, L., & Brindley, J. (2011). Establishing a foundation for reflective practice: A case study of learning journal use. European Journal of Open, Distance, and E-Learning. Available from http://www.eurodl.org/materials/special/2011/Blaschke_Brindley.pdf Blaschke, L. M., & Brindley, J. (2015). Using social media in the online classroom. In M. Ally & B. Khan (Eds.), The international handbook of e-learning (Vol. 2). Athabasca, Canada: Routledge. Blaschke, L. M., & Hase, S. (2015a). Heutagogy: A holistic framework for creating 21st century self-determined learners. In M. M. Kinshuk & B. Gros (Eds.), The future of ubiquitous learning: Learning designs for emerging pedagogies. Heidelberg, Germany: Springer. Blaschke, L., & Hase, S. (2015b). Heutagogy, technology and lifelong learning: curriculum geared for professional and part-time learners. In A. Dailey-Herbert (Ed.), Transforming processes and perspectives to reframe higher education. New York: Springer. Booth, M. (2014). Assessment as an ongoing act of learning: A heutagogical approach. In L. M. Blaschke, C. Kenyon, & S. Hase (Eds.), Experiences in self-determined learning. USA: Amazon.com. Booth, M., Blaschke, L., & Hase, S. (2016). Practicing the practice: The heutagogy community of practice. In J. McDonald & A. Cater-Steel (Eds.), Communities of practice: Facilitating social learning in higher education. Heidelberg, Germany: Springer.

264

L. M. Blaschke

Boud, D., Keough, R., & Walker, D. (1985). Reflection: Turning experience into learning. London: Kogan Page Ltd. Brandt, B. A. (2013). The learner’s perspective. In S. Hase & C. Kenyon (Eds.), Self-determined learning: Heutagogy in action. London: Bloomsbury Academic. Canning, N. (2010). Playing with heutagogy: Exploring strategies to empower mature learn- ers in higher education. Journal of Further and Higher Education, 34(1), 59–71. Canning, N. (2013). Practitioner development in early years education. In S. Hase & C. Kenyon (Eds.), Self-determined learning: Heutagogy in action. London: Bloomsbury Academic. Cochrane, T., Antonczak, L., Gordon, A., Sissons, H., & Withell, A. (2012). Heutagogy and mobile social media: Post Web 2.0 pedagogy. Retrieved from http://www.ascilite.org.au/conferences/ wellington12/2012/images/custom/cochrane,_thomas_-_heutagogy_and_mobile.pdf Deci, E. L., & Ryan, R. M. (2002). The handbook of self-determination research. Rochester, NY: The University of Rochester Press. Dewey, J. (1997). How we think. Mineola, New York: Dover Publications, Inc. Dick, B. (2013). Crafting learner-centred processes using action research and action learning. In S. Hase & C. Kenyon (Eds.), Self-determined learning: Heutagogy in action. Bloomsbury Academic: London. Dietz, A.S., & Eichler, M.A. (2013). Heutagogy and adults as problem solvers: Rethinking the interdisciplinary graduate degree. Adult education research conference. Paper 15. Retrieved from http://www.adulterc.org/Proceedings/2013/papers/dietz.pdf Dweck, C. S. (2006). Mindset: The new psychology of success. New York: Ballantine Books. Eberle, J., & Childress, M. (2009). Using heutagogy to address the needs of online learners. In P. Rogers, G. A. Berg, J. V. Boettecher, & L. Justice (Eds.), Encyclopedia of distance learning (2nd ed.). New York: Idea Group, Inc. Facer, K., & Selwyn, N. (2010). Social networking: Key messages from the research. In R. Sharpe, H. Beetham, & S. De Freitas (Eds.), Rethinking learning for a digital age: How learners are shaping their own experiences. New York: Routledge. Freire, P. (1970). The pedagogy of the oppressed. London: Penguin Books. Gerstein, J. (2015). Making MAKEing more inclusive (Blog post.) User generated education. Retrieved from https://usergeneratededucation.wordpress.com/2015/05/20/making-makeingmore-inclusive/ Hase, S. (2013). Learner defined learning. In S. Hase & C. Kenyon (Eds.), Self-determined learning: Heutagogy in action. London: Bloomsbury Academic. Hase, S. (2014). Skills for the learning leader in the 21st century. In L. M. Blaschke, C. Kenyon, & S. Hase (Eds.), Experiences in self-determined learning. USA: Amazon.com. Hase, S., & Kenyon, C. (2000). From andragogy to heutagogy. UltiBase. Retrieved from http:// ultibase.rmit.edu.au/Articles/dec00/hase2.htm Hase, S., & Kenyon, C. (2007). Heutagogy: A child of complexity theory. Complicity An International Journal of Complexity and Education, 4(1), 111–119. Hase, S., & Kenyon, C. (2013). Self-determined learning: Heutagogy in action. London: Bloomsbury Academic. Hexom, D. (2014). Heutagogy and the impact on adult learning in higher education. In L. M. Blaschke, C. Kenyon, & S. Hase (Eds.), Experiences in self-determined learning. USA: Amazon.com. Johnson, L., Adams Becker, S., Estrada, V., & Freeman, A. (2015). NMC horizon report: 2015 higher education edition. Austin, TX: The New Medium Consortium. Retrieved from http:// www.nmc.org/publication/nmc-horizon-report-2015-higher-education-edition/ Kenyon, C. (2014). One way of introducing heutagogy. In L. M. Blaschke, C. Kenyon, & S. Hase (Eds.), Experiences in selflf-determined learning. USA: Amazon.com. Kerry, T. (2013). Applying the principles of heutagogy to a postgraduate distance-learning program. In S. Hase & C. Kenyon (Eds.), Self-determined learning: Heutagogy in action. London: Bloomsbury Academic.

11

Self-Determined Learning: Designing for Heutagogic Learning Environments

265

Knowles, M. (1975). Self-directed learning: A guide for learners and teachers. Cambridge, NY: Globe Fearon. Little, T., & Ellison, K. (2015). Loving learning: How progressive education can save America’s schools. New York: W.W. Norton & Company, Inc.. Long, D. (1990). Learner managed learning: The key to life long learning and development. New York: Kogan Page. Maslow, A. H. (1943). A theory of human motivation. Psychological Review, 50, 370–396. McLoughlin, C., & Lee, M.J.W. (2007). Social software and participatory learning: Pedagogical choices with technology affordances in the Web 2.0 era. In Proceedings from ascilite, December 2–5, 2007. Singapore. Retrieved from http://www.ascilite.org.au/conferences/singapore07/ procs/mcloughlin.pdf Mezirow, J., & Associates. (1990). Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning. San Francisco: Jossey-Bass Publishers. Msila, V. (2014). Heutagogy, Africanisation and learning: Experiences from an open and distance learning (ODL) program at the University of South Africa. Mediterranean Journal of Social Sciences, 5(14), 214–220. Retrieved from http://www.mcser.org/journal/index.php/mjss/article/ view/3147 Msila, V., & Setlhako, A. (2012). Teaching (still) matters: Experiences on developing a heutagogical online module at UNISA. Journal of Educational and Social Research, 2(2), 65–71. Retrieved from http://www.sciencedirect.com/science/article/pii/S1877042812053785 Narayan, V., & Herrington, J. (2014). Towards a theoretical mobile heutagogy framework. Proceedings asciilite 2014. Dunedin, New Zealand (pp. 150–160). Retrieved from http://ascilite. org/conferences/dunedin2014/files/fullpapers/138-Narayan.pdf Northcote, M.T., & Boddey, C. (2014). Using the self-determined learning principles of heutagogy to support academic staff who are learning to teach online. Education conference papers. Paper 9. Retrieved from http://research.avondale.edu.au/conferences/9 Olson, M. H., & Hergenhahn, B. R. (2009). An introduction to theories of learning (8th ed.). Upper Saddle River, New Jersey: Pearson/Prentice Hall. Price, D. (2013). Open: How we’ll work, live and learn in the future. (Kindle version.) UK: Crux Publishing Ltd. Price, D. (2014). Heutagogy and social communities of practice: Will self-determined learning rewrite the script for educators? In L. M. Blaschke, C. Kenyon, & S. Hase (Eds.), Experiences in self-determined learning. USA: Amazon.com. Ramsay, M., Hurley, J., & Neilson, G. R. (2013). Workplace learning for nurses. In S. Hase, & C. Kenyon (Eds.), Self-determined learning: Heutagogy in action. London, UK: Bloomsbury Academic. Richardson, W., & Mancabelli, R. (2011). Personal learning networks: Using the power of connections to transform education. Bloomington, IN: Solution Tree Press. Rogers, C. R. (1961). On becoming a person: A therapist’s view of psychotherapy. Boston & New York: Houghton Mifflin Company. Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books, Inc. Sharpe, R., Beetham, H., & de Freitas, S. (2010). Rethinking learning for a digital age: How learners are shaping their own experiences. New York: Routledge. Snowden, M., & Halsall, J. (2014). Community development: A shift in thinking towards heutagogy. International Journal of Multi-Disciplinary Comparative Studies, 1(3), 81–91. Retrieved from http://www.ijmcs-journal.org/IJMCS_December%202014_MICHAEL% 20SNOWDEN%20&%20JAMIE%20HALSALL.pdf Stephenson, J. (1996). Beyond competence to capability and the learning society. Capability, 2(1), 60–62. Stephenson, J., & Weil, S. (1992). Quality in learning: A capability approach in higher education. London: Kogan Page.

266

L. M. Blaschke

Tinkler, D. E. (1993). A “constructivist” theory of acquisition, and its implications for learnermanaged learning. In N. Graves (Ed.), Learner managed learning: Practice, theory and policy. Leeds, England: Higher Education for Capability. Van Schoor, W., & Mischke, G. (2014). From bricks to clicks: A new model for higher education. In A. Kwan, E. Wong, T. Kwong, P. Lau, & A. Goody (Eds.), Research and development in higher education: Higher education in a globalized world (Vol. 37, pp. 304–313). Hong Kong: Higher Education Research and Development Society of Australasia Inc (HERDSA). Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Wiley, D. (2014) The access compromise and the 5th R. (Blog post.) Iterating toward openness. Retrieved from http://opencontent.org/blog/archives/3221 Zimmerman, B. J., & Schunk, D. H. (2001). Self-regulated learning and academic achievement: Theoretical perspectives (2nd ed.). New York/London: Routledge.

Lisa Marie Blaschke is Program Director of the Master of Distance Education and E-Learning (MDE) graduate program at Carl von Ossietzky University of Oldenburg, Germany, as well as an Associate Professor (adjunct faculty) within the MDE at the University of Maryland University College, USA. She is a Vice-President and executive committee member of the European Distance Education and E-Learning Network (EDEN) and an EDEN Fellow. Her research interests are in the areas of lifelong and self-determined learning (heutagogy) and the pedagogical application of technology to create learner-centered educational environments. Before rejoining academia in 2006, Lisa worked within international corporate environments in the software industry, designing, leading, and implementing enterprise-wide knowledge management and training solutions.

Redefining Learning: A Neurocognitive Approach

12

Phillip Harris and Donovan R. Walling

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Origins and Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Formation and Organization of Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ideas and Research on Neurocognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Neurocognitive Reconceptualization of Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Individuation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Motility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Accessibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtuality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rethinking Evaluation: Methods and Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . New Directions and Further Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How Might Units of Measure Be Defined for a Neuro-Cognitive Definition of Learning? . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

268 270 274 279 285 287 291 294 296 299 306 310 311 313

Abstract

How learners, teachers, learning designers, instructional supervisors, education policy makers, and others involved with education institutions and educational enterprises define learning affects understandings about how and what is learned and to what extent learning is accomplished. Such understandings also have broad ramifications for fundamental operations, such as how schools are conceived, from their physical architecture to the organization of learners, classes, subject matter, and so forth, and how learning accomplishments as well as

P. Harris (*) Association for Educational Communications and Technology, Bloomington, IN, USA e-mail: [email protected] D. R. Walling Bloomington, IN, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_63

267

268

P. Harris and D. R. Walling

learners and their teachers are evaluated. The purpose of this work is to explore – and to encourage others to explore – a new definition of learning, a neurocognitive definition, and its ramifications. In the use of neurocognitive, the authors link existing theories of cognition to new research emerging from neuroscience. When cognitivism was proposed in the 1950s, study of the brain was in its infancy. Now, however, scientific understanding of the brain is growing exponentially. Therefore, it is reasonable to explore the link between our growing knowledge of neuroscience and our understanding of cognition.

Keywords

Behaviorism · Cognition · Digital Age · Industrial Age · Neuroscience · Reconceptualism

Introduction Digital Revolution began at the mid-twentieth century and is ongoing, ushering in the current Digital Age, an era fundamentally different from the preceding Industrial Age. However, the Industrial Age has not disappeared nor have values and practices associated with that era been left behind. Quite the opposite is true. The present moment can be characterized by the French term fin de siècle, literally meaning “end of the century.” The term was applied most familiarly to the end of the nineteenth century but is used generally to signal the closing of one cultural era and the onset of another. Often the spirit of fin de siècle is one of degeneration mixed with hope for a new beginning. It can be a chaotic time. Thus the term can be fairly applied to the current period, which is bearing witness to the last throes of the Industrial Age and the burgeoning of the Digital Age. We are prepared to question in full Industrial Age assumptions that undergird education and schooling as we discuss how teaching and learning might be reconceptualized for the Digital Age. To learn seems like a simple verb that embodies a straightforward concept, which involves moving from a state of not knowing something to knowing that thing. However, in actuality learning involves a nuanced and variable set of processes. As Albert Einstein is reputed to have declared: “Any fool can know. The point is to understand.” And understanding is complex. However, it is essential to true learning. The phrase that we and others use – “knowledge and understandings” – embodies this fuller sense of what it means to learn. The purpose of this work is to explore – and to encourage others to explore – a new neurocognitive definition of learning. In the use of neurocognitive, we link existing theories of cognition to new research emerging from neuroscience. When cognitivism was proposed in the 1950s, the scientific study of the brain was in its infancy. Now, however, scientific understanding of the brain is growing exponentially. Therefore, it seems reasonable to explore what neuroscience can contribute to newly emerging knowledge and understandings that should be shaping teaching and learning in the Digital Age.

12

Redefining Learning: A Neurocognitive Approach

269

Our focus will be on learning in prekindergarten through undergraduate studies, or preK-16. Yet the definition of learning we are developing affects education of many kinds, including university graduate studies, medical education, military and corporate training, and other forms of formal, institutional schooling as well as informal learning. We also focus primarily on public education, particularly in the USA; although much of the following discussion will apply to private education and education systems in other countries, particularly Western or Westernized nations. Rather than proceeding from established learning theories, we propose to examine the roots from which learning theory writ large has sprung. To this end we delve into six questions, which compose the main subsections that follow. First, what are the origins and definitions of learning? In other words, what traditional, cultural, social, scientific, and educational history undergirds our current understanding of what it means to learn, how has such understanding changed over time, and how might a new definition of learning fundamentally transform schooling and education in general? Second, how does the definition of learning guide the formation and organization of learning environments? Learning environments are composed of many elements, both intellectual and physical. The existence of a grade sequence, from prekindergarten onward, and how that sequence is segmented – elementary, middle, and high school and undergraduate studies – establish a temporal framework that is seldom modified. School calendars that vary only in minor ways across the country, mandated curricula in certain subjects, and required standardized tests also shape the environment for formal learning. Lesser factors include extracurricular activities, regional or local subject matter (such as locally popular sports or regional history and geography), and a wide variety of other elements that create minor distinctions within a geographic area – all of these elements influence what it means to learn. Third, what ideas and research shape a neurocognitive definition of learning? The prevailing definition of learning today is behavioristic in its origins and implementation, operating in concert with the sensibilities and realities of the Industrial Age and contributing to the development of today’s factory model of schooling. As the Industrial Age is succeeded by the Digital Age, new and rediscovered ideas and recent, relevant research are providing both impetus and foundation for a new definition of learning. Fourth, how might a neurocognitive definition of learning guide the reconceptualization of schooling for the Digital Age? True schools for the Digital Age, schools based on a neurocognitive definition of learning, do not yet exist in a systemic sense; but they will ultimately operate from a very different set of premises than Industrial Age schools based on behaviorist definitions of learning. This is not merely a crystal ball exercise. Rather, we articulate developing characteristics of schools that work toward a profound reformulation of schooling in the public realm. Fifth, what forms or strategies of evaluation can be employed to determine what and how learning is taking place (processes) and when learning goals have been achieved and to what level of depth or sophistication (outcomes)? We begin with defining meaningful measures, a term currently emerging in the evaluation literature but diversely characterized, and then discuss how such meaningful measures can

270

P. Harris and D. R. Walling

inform us about learning and thus be used to shape learning design, curriculum, instruction, and the formulation and implementation of education policy? Sixth, where are examples of teaching and learning that proceed from a neurocognitive definition of learning happening now, how might interested readers and researchers discover more such examples, and what research questions might stimulate new inquiry and research to actualize neurocognitively oriented, learnercentered education? In addition to the text, we make use of two devices intended to complement our work. Both devices are boxed, with questions to ponder in the shaded boxes and supplemental media in the outlined boxes. Questions to Ponder

At various points throughout this work, we pose questions that are designed to allow the reader to ponder further some of the ideas discussed. Readers are invited to add their own questions and to consider how or whether those questions are answered in this text or might lead to additional independent reading or research.

Media Audiovisual media links – for example, to a TEDx talk on YouTube – will lead the reader to supplementary information included to amplify topics discussed in the text. Note: The URLs were active at the time this work was written; however, Internet addresses sometimes are ephemeral.

Origins and Definitions With advances in science and technology, the fundamental constants that govern the laws of nature are being determined with increasing accuracy (see, e.g., Phys.org. 2015). Might not something similar be said of learning, namely, that scientific and technological advances are providing insights into how humans acquire knowledge and understandings? And shouldn’t these new insights require a new definition of learning? In popular usage learning encompasses many forms of knowledge acquisition, from learning how to tie one’s shoes to how to solve quadratic equations. But the verb to learn was far narrower in its origins, some nine hundred years ago. The modern English word traces to Middle English lernen (a cognate of the German verb lernen), from the Old English leornian, meaning to learn, read, or ponder. It is akin to lesan, meaning to glean (a cognate with German lesen, meaning to read). The commonality in these English and German origins is reading – that is, to read is to learn. Reading and learning are virtually synonymous, which bestows on the act of learning, in its original grammatical sense, a connotation of scholarship. To learn means, in this connotation, to acquire knowledge and understandings through

12

Redefining Learning: A Neurocognitive Approach

271

formalized study. In Britain’s universities today, this close connection can still be heard in the phrase “to read,” meaning in the American sense “to study,” as in “I’m reading Economics at Oxford.” How is the definition of learning constrained by the etymology of the word learn? How might the term’s history play a part in helping to think in new ways about what it means to learn? This scholarly definition of learning is hardly egalitarian. Indeed, learning in this sense – largely the operational definition used for formal education across many centuries – was provided for boys and men and was not, in general, offered to women, manual laborers, or slaves. Nowadays the common understanding is that learning in this scholarly sense is available, at least in developed countries, to all, regardless of sex or social status. In reality – particularly with regard to economic status in the USA – distinctions remain. The popular phrase about education for “college and career” subtly retains the notion that there are different definitions of learning that apply to different desired outcomes, particularly with regard to whether learners are prepared to enter university study or to move directly into the workforce following childhood and adolescent schooling. Leaving aside the literal definition, there remains a more fundamental definition of learning to be considered. Looking historically to education in Ancient Greece, often considered an archetype of modern education in the West, it would be a mistake to consider only formal education, which was provided to males of certain classes, usually in the form of a public school or by a private tutor. Learning, in a different, larger sense, also was valued. Girls, for example, received informal education from their mothers on topics ranging from music and dance to housekeeping. Manual laborers and slaves learned trades, such as carpentry, from the masters of various crafts who were their teachers. To none of this informal education could the scholarly connotation of learning as “reading” be applied. Indeed, actual literacy in many earlier eras was not a factor in informal education at all. Yet informal education was vitally important learning in the context of civil society, which was based on a broader, more egalitarian, functional definition of to learn. The dichotomy of definitions of learning – one for formal education, another for informal education – has been stratified through secondary definitions detailing the nature of the learning and how it was acquired, whether by attendance at the academy or through vocational effort. Such stratification continues, marking out elite learning, or scholarship, and the institutions that support it in contrast to vocational learning and its institutions. The lowest stratum is reserved for noninstitutional learning, not because the learning may be inferior but because such learning is not institutionally validated. While educators often extol the virtues of “independent” learning, truly independent learners are seldom esteemed. In past ages scholarly, or formal, learning was not necessarily validated by specific behavioral evidence. Learners in Ancient Greece studied philosophy,

272

P. Harris and D. R. Walling

literature, history, rhetoric, or poetry, for example, not necessarily to become poets or historians but to attain a state of knowledge that would shape good character and support citizens’ engagement in public discourse. This notion of shaping character and fostering citizenship proceeds from an interiorized, or intrinsic, definition of learning – that is, learning is the acquisition of knowledge and understandings that shape how the learner thinks and lives. The contrasting notion, which governed informal education, proceeds from an exteriorized, or instrumental, definition of learning, meaning that the learner’s behaviors in particular instances are formed by experience in those areas. For example, when individuals learn to cook, their behaviors change because of new knowledge and understandings. A novice might sauté onions in butter at too high a temperature and so burn them, but a person who has learned to cook, perhaps through working with a master chef, will choose the correct temperature. With the advent of the American common school in the nineteenth century comes not only formal schooling that is accessible to more segments of the population – notably women and manual laborers – but also schooling that is increasingly formulated to accord with an instrumental definition of learning. The term behaviorism would not be coined until 1913, when John B. Watson published a philosophical manifesto that proposed abandoning the so-called introspectionist focus on consciousness – in other words, intrinsic learning – in favor of focusing on behavioral manifestations of intelligence (Watson, 1913). However, the roots of the twentieth-century behaviorism, which came to be the dominant organizing philosophy of public education in the USA, reach into the ground of the common school and its emphasis not only on egalitarian education – Horace Mann, the “Father of the Common School,” called it “the great equalizer” (Cremin, 1957, p. 65) – but also on practical education. “Introduction to Introspection” provides a brief video overview of introspectionism in psychology. It can be found on YouTube at https://youtu.be/ j1UnYiPwBQ0. The focus on intrinsic, or introspectionist, learning has been preserved at the university level as liberal arts education. But even that iteration has seen a decline over the past century, according to some researchers (e.g., Breneman, 1990; Baker, Baldwin, & Makker, 2012). (For an interesting parsing of the history within psychology of this shift from introspectionism to behaviorism, see Costall, 2006.) Patrick Awuah, co-founder of Ashesi University in Ghana, discusses liberal arts education as critical to forming true leaders in a TED Talk, “How to Educate Leaders? Liberal Arts,” at https://www.ted.com/talks/patrick_awuah_ on_educating_leaders.

12

Redefining Learning: A Neurocognitive Approach

273

In spite of the development of the Digital Age technologies that give us insights into the nature of learning that were heretofore inconceivable, learning as it exists in public education is still defined by what the learner can do as witnessed by others. Such observation may be mediated by some form of testing, which may be as likely to limit or skew understanding as to expand it, but the definition remains locked to the demonstration of observable phenomena. Thus education decisions – and consequently life decisions – are made on this basis, leading to policies and procedures that may actually limit learning. How does emphasizing behavior as a learning outcome shape our collective understanding of what it means to learn? How might that understanding be altered if introspection were emphasized instead of behavior as a learning outcome? Recently modern scientific research has begun to alter the knowledge base by focusing on how learning happens, giving greater credence to cognitivist and related theories than previously had been the case. Cognitivism arose in the 1950s as a psychological theory in direct contrast to behaviorism (Mandler, 2002). Whereas behaviorism identified thinking (learning) as evidenced by externalized behavior, or behavioral change, cognitivism posited that cognition, or thinking, was in itself a behavior within the brain, regardless of external evidence. At the time this theory failed to achieve general acceptance because only limited evidence could be obtained to verify such activity within the brain. Recent strides in neuorscience are now changing that. Today’s researchers can discern activity within the brain using increasingly sophisticated technologies, such as structural magnetic resonance imaging (sMRI) and functional MRI (fMRI). These kinds of noninvasive brain-imaging technologies are, according to researcher William R. Crum (2010), “for the first time offering researchers the ability to directly observe the effect of different types of learning on brain structures and function” (p. 37). Consequently, cognitivism may be seen in a new light, perhaps more accurately termed neurocognitivism, a term we use to recognize the advances in neuroscience now giving new credence to existing cognitivist theory. Fundamentally, these new technological tools of neuroscience are making it possible to redefine learning by moving the threshold of evidence. Teaching and learning that proceed from a behaviorist viewpoint rely on external evidence, such as test scores, to signal, or verify, that learning has occurred. External behavioral change is an evidence threshold. Indeed, strict behaviorists would contend that in the absence of behavioral change, no learning has occurred. This is a false assumption of causality. Neuroscience, by contrast, moves the threshold of evidence to an earlier point in the learning process, showing that activity – therefore some form of learning – is occurring in the brain, prior to or absent evidence in terms of behavioral change. In the Ancient Greek sense, the shaping of good character, for instance, would manifest itself only indirectly. Thus the time between “learning” (some form

274

P. Harris and D. R. Walling

of internal change within the brain) and some sort of behavioral change might be lengthy. No direct assumption of causality would be valid. Indeed, this problem of causality has become associated with so-called educational neuroscience, which has engendered misinformation and “neuromyths.” Consequently, we specifically avoid the term educational neuroscience. Adam Gazzaley, M.D., Ph.D., is an American cognitive neuroscientist, founding director of the Neuroscience Imaging Center and Professor of Neurology, Physiology, and Psychiatry at the University of California, San Francisco. In a January 2013 TEDx talk at the American School of Bombay, a preK-12 school in Mumbai, India, Gazzaley spoke on “Closing the Loop Between the Brain and Education,” https://youtu.be/qJ_-0Q8KIOQ. The new threshold of evidence argues for redefining learning as changes within the learner’s brain, rather than merely changes in the learner’s behavior. This new definition provides an impetus for responding in new and different ways to perceived, or suspected, learning and to the accumulation of evidence, both internal and external, of learning. A neurocognitive definition of learning, in contrast to a behaviorist definition, requires a concomitant rethinking of all aspects of learning design, from structural components, such as traditional grade-level sequencing, to evaluation, which has come to be dominated in the current era by standardized testing. Therefore, for the purposes of this chapter we use the following broad neurocognitive definition of learning: Learning is a multidimensional process that creates a changed state in the brain. This definition echoes prior efforts to characterize learning based on neuroscience. For example, in 1989 Eric Kandel, who would go on to win the 2000 Nobel Prize in Physiology or Medicine, wrote that “learning produces enduring changes in the structure and function of the synapses” (p. 121). Furthermore, Kandel predicted that “in the next decade of research on learning and psychotherapy, we can look forward to using techniques that allow us to follow non invasively the structural changes produced by experience through altered expression of genes” (p. 123). Kandel’s prediction was accurate. How might a new, neurocognitive definition of learning reshape schools and schooling in the future?

Formation and Organization of Learning Environments Learning environments are composed of numerous intellectual and physical elements, which may be considered as broad classes. Within the intellectual class, for example, are theories and evidence for how learning occurs; how it can be stimulated, guided, and evaluated; and ideas about what types of learning happen at which

12

Redefining Learning: A Neurocognitive Approach

275

stages of mental and physical development throughout life. Within the physical class are structural considerations such as settings (campuses, classrooms) and organizational components (curricula, grade sequences). American architect Louis Sullivan (1896) coined the phrase “form follows function,” saying: It is the pervading law of all things organic and inorganic, of all things physical and metaphysical, of all things human and all things superhuman, of all true manifestations of the head, of the heart, of the soul, that the life is recognizable in its expression, that form ever follows function. This is the law. (p. 408)

This dictum, which set a philosophical basis for Modernist architecture, applies to learning. The definition of learning (function) guides, or should guide, how those charged with the formation and organization of learning environments (form) proceed. Both intellectual (or metaphysical, to use Sullivan’s term) elements, such as a curriculum, and physical elements, such as a school schedule, are the embodiment of a particular definition of learning or, perhaps more realistically, a number of generally similar definitions. Learning environments today are iterations of behaviorist definitions of learning. An example is Schenley High School in Pittsburgh, Pennsylvania, which was built adhering to Sullivan’s Modernist sensibilities in 1916, the same year that saw the publication of John Dewey’s Education and Democracy. The architect, Edward Stotz (1868–1948), designed the building in a neoclassical style (see Fig. 1). The school’s blueprint (Fig. 2) reveals the innovative triangular footprint of the building, which rises to four stories. Except for that feature, however, the plan would be recognizable in many school buildings constructed over the last 150 years. Schenley High School opened with 180 rooms for 2800 learners and included a 1600-seat auditorium. The building is on the National Historic Register, although it was closed as a school in 2008. As noted previously, the essential behaviorist definition of learning was formulated in the early twentieth century but from roots stretching back to the early nineteenth century, to the periods of the 1830s and 1840s that gave rise to the Common School Movement. A precursor to this movement was the advent of the Industrial Revolution, which saw a transition to new manufacturing processes during the period roughly from 1760 to 1820. This period marked a turning point in history, as the changes in the workplace affected a large majority of the population. As the nature of work was transformed by the rise of the factory system, so too did the nature of schooling change in response to the accompanying increase in the standard of living and population growth that marked the beginnings of the new Industrial Age. A capstone of the Industrial Age occurred in the early twentieth century with the introduction of the manufacturing assembly line, sometimes referred to as progressive assembly, the origin of which often is mistakenly attributed to Henry Ford. According to Robert W. Domm (2009), however, another automobile maker, Ransom Olds, created the first modern assembly line, which was used to build the first mass-produced automobile, the Oldsmobile Curved Dash, in 1901. Olds patented the assembly-line concept, although there were examples of assembly-line style

276

P. Harris and D. R. Walling

Fig. 1 Exterior view of Schenley High School in Pittsburgh, Pennsylvania (Source: Wikimedia Commons. Public domain)

Fig. 2 The first-floor plan of Schenley High School (Source: Wikimedia Commons. Public domain)

12

Redefining Learning: A Neurocognitive Approach

277

manufacturing in other industries dating back to the beginnings of the Industrial Revolution. Nonetheless, the early twentieth-century assembly line was a game changer and was mirrored in ever more standardized school environments. There was – and still is – a caveat to school standardization in that schools for the most part retain a measure of local independence under broad direction by the various state education departments and the federal government. Despite the diversity that might be evident across the multitude of different local communities throughout the nation, schools across the USA during the twentieth century and continuing today are broadly similar. Public schools in New York City and rural Arizona may be somewhat architecturally distinct, given their geographic differences, but their physical learning spaces (classrooms, libraries, gymnasiums, etc.) and their intellectual organization (curriculum, class composition, and schedules) are likely to be cut from the same industrial model cloth. This functional standardization is what critics of the Industrial Age school model have decried. A well-known example is Alvin Toffler’s criticism in his 1970 book, Future Shock: Mass education was the ingenious machine constructed by industrialism to produce the kind of adults it needed. The problem was inordinately complex. How to pre-adapt children for a new world – a world of repetitive indoor toil, smoke, noise, machines, crowded living conditions, collective discipline, a world in which time was to be regulated not by the cycle of sun and moon, but by the factory whistle and the clock. The solution was an educational system that, in its very structure, simulated this new world. This system did not emerge instantly. Even today it retains throw-back elements from pre-industrial society. Yet the whole idea of assembling masses of learners (raw material) to be processed by teachers (workers) in a centrally located school (factory) was a stroke of industrial genius. The whole administrative hierarchy of education, as it grew up, followed the model of industrial bureaucracy. The very organization of knowledge into permanent disciplines was grounded on industrial assumptions. Children marched from place to place and sat in assigned stations. Bells rang to announce changes of time. The inner life of the school thus became an anticipatory mirror, a perfect introduction to industrial society. The most criticized features of education today – the regimentation, lack of individualization, the rigid systems of seating, grouping, grading and marking, the authoritarian role of the teacher – are precisely those that made mass public education so effective an instrument of adaptation for its place and time. (pp. 354–355)

With the emergence of the Digital Age, the advent of computers in the classroom – now finding new manifestations through tablet technology and BYOD (bring your own device) strategies – has engendered criticism of a different nature. A large measure of this criticism stems from the overselling of computers as a panacea for whatever ills or shortcomings have been perceived in traditional schooling (à la Toffler and others). Proponents of technology in schools tout computers and related devices as curative technology. An early proponent, Seymour Papert (1980), for example, averred, “The computer can be seen as an engine that can be harnessed to existing structures in order to solve, in local and incremental measures, the problems that face schools as they exist today” (p. 186). This solution has not been borne out in practice. The problem lies in the “existing structures.”

278

P. Harris and D. R. Walling

We contend that, rather than view computers and related forms of technology as patches and props to repair an outmoded Industrial Age system of education, new Digital Age technology, proceeding from a neurocognitive definition of learning, should more appropriately be employed to transform schools and schooling writ large. This transformation, or reconceptualization, will mean that true twenty-firstcentury schools and what happens within them will look very different from the twentieth-century schools currently in operation. Do today’s schools embrace the current Digital Age, or do they continue to follow an Industrial Age model with modest incorporations of computer technology? The Industrial Revolution led to an Industrial Age, in which transformed schools adopted an industrial, or factory, model of operation. The Digital Revolution has now given rise to a Digital Age. Should schools again be transformed accordingly? What would such transformation look like? A couple of paragraphs from a New York Times report illustrate some of the issues that lie at the heart of today’s criticism of schools: Advocates for giving schools a major technological upgrade – which include powerful educators, Silicon Valley titans and White House appointees – say digital devices let students learn at their own pace, teach skills needed in a modern economy and hold the attention of a generation weaned on gadgets. Some backers of this idea say standardized tests, the most widely used measure of student performance, don’t capture the breadth of skills that computers can help develop. But they also concede that for now there is no better way to gauge the educational value of expensive technology investments. (Richtel, 2011)

This observation, perhaps inadvertently, pinpoints a problem with simply incorporating new technology into the Industrial Age school model. Digital Age devices could “let students learn at their own pace,” but the Industrial Age model for schools is not geared to this end, nor are today’s increasingly onerous standardized test protocols designed to measure individually paced student learning – something far more often discussed than realized in today’s classrooms, regardless how technologically rich they might be. This type of critique is by no means a recent development, as similar criticism can be seen to proceed from earlier descriptions of the emergence of the Digital Age. The Czech philosopher Radovan Richta, for example, coined the term technological evolution to describe a theory of technology development. Richta posited three stages of such development: (1) the tool (a mechanical advantage in the work of humans), (2) the machine (a more sophisticated tool that can substitute for human physical effort), and (3) automation (a machine that can remove the element of human control by use of an automatic algorithm) (Bloomfield, 1995). Notwithstanding Richta’s 1960s Marxist orientation, his observations are prescient:

12

Redefining Learning: A Neurocognitive Approach

279

In all probability it will take decades for the scientific and technological revolution to become the predominant process in the areas where it does not encounter social obstacles. The revolutionary social changes of recent times, however, [bearing in mind that this article was written during the turbulent period of the civil rights movement in the United States and to various degrees elsewhere] hold out the promise that obstacles can be overcome. (Richta, 1967, p. 67)

The Industrial Age conceptualization of schools as iterations of the factory model transformed schooling over the course of the three centuries, with the most significant changes becoming evident during the nineteenth century. The Digital Age, complemented by increasingly sophisticated technology, has not transformed schooling – yet – because the new, neurocognitive definition of learning that we propose requires a radical reconceptualization of how schools are formed and organized. Such reconceptualization, when it occurs, produces a transformation in schooling that affects both intellectual and physical elements. The Digital Age school should, indeed must, be very different from today’s outmoded Industrial Age schools.

Ideas and Research on Neurocognition As discussed in the preceding sections, behaviorism was well suited to the Industrial Age factory model of schooling. Even so, behaviorism was not universally accepted as an ideal definition of learning upon which to construct both the philosophical and the physical architecture of schooling. Cognitivism offered a counter definition. As outlined above, cognitivism gained recognition beginning in the 1950s (Mandler, 2002). However, what it lacked was behavioral evidence, the “observable” being the gold standard for research-based credibility at that time. At the mid-twentieth century, there was little research that could provide scientists, educators, theorists, and education policy makers with visible evidence of cognition, or thinking. Cognitivism focused on mental processes: perception, thought, problem-solving, and so on – in other words, the tools and processes that enable learning. For cognitivists the evidence was in the internal, in the thinking processes themselves; however, these processes were largely invisible. Consequently, cognitivism was relegated to the status of an interesting theory but not one that could be operationalized with a high degree of confidence in its efficacy, despite the fact that it was, though not in name, the undergirding theory of learning during most pre-Industrial Age periods. Indeed, before behaviorism became dominant, there were other theories of learning that bore similarities to cognitivism. Such theories stretch back to ancient times, from the early twentieth-century introspectionism (Watson, 1913) to Plato and Socrates in Ancient Greece, who posited that physical events (i.e., behaviors) are “shadows” of their perfect or ideal, metaphysical, or theoretical and thus non-visible forms (Plato 360 BCE). González (2013) suggests, with regard to Plato’s ideas:

280

P. Harris and D. R. Walling

The scientific method requires quantifiable evidence. Philosophical truth, more often than not, requires time to flush out fallacious premises. . .. Truth, Plato tells us, is objective and serves as the ground of human reality.

González continues, “Education, in the Platonic sense, cannot exist to merely catalogue the objects, those particulars that populate the sensual world of appearance (phainomena). Instead, education seeks to understand the essence of the timeless, universal principles that rule over human existence.” These “timeless principles” are metaphysical – interior or mental “behaviors,” rather than the exterior, observable behaviors, or “particulars” of the “sensual world of appearance.” In the early 1970s, another movement in educational psychology and philosophy arose that attempted to extend the essential ideas of cognitivism and its various precursors. It was termed reconceptualism, and its primary proponent was curriculum theorist William Pinar. He is perhaps best known for suggesting that curriculum might be thought of as a verb rather than as a noun. Pinar used the term currere (the infinitive form of curriculum) to shift the focus of curriculum theory toward selfreflection as a means of shaping teaching and learning. According to Pinar (2004): The method of currere reconceptualized curriculum from course objectives to complicated conversation with oneself (as a “private” intellectual), an ongoing project of self-understanding in which one becomes mobilized for engaged pedagogical action – as a private-and-public intellectual – with others in the social reconstruction of the public sphere. (p. 37)

Pinar’s emphasis was on educators as curriculum developers and deliverers; however, it is a small step rather than a large leap to apply his ideas to learners as well. (We will discuss this idea in greater detail in a subsequent section on Rethinking Evaluation: Methods and Alignment.) If currere leads to curriculum drawn from self-reflection, rather than prescribed by policy, what then are the cues that must be gained from learners in order to translate “private-and-public intellectual” action into practice? How might this lead to “social reconstruction of the public sphere”? From a psychological point of view, behaviorism is descriptive. Learners (and teachers) exhibit certain behaviors under certain circumstances. Thus it can be fair to say that “many” or “most” kindergarteners, for instance, come to school knowing basic colors, some numbers (perhaps 1 to 10), and at least a few letters. The shift from psychology to education has been one dominated by policy formulations, in which behaviorism is prescriptive. In other words, in order to be considered “ready” for kindergarten, children must know basic colors, some numbers, and so forth. This prescriptive policy formulation is a manifestation of the threshold notion that we suggested in the previous “Origins and Definitions” section. The threshold of evidence that a child is ready for kindergarten, in a prescriptive behaviorist sense, is the presence of certain behaviors.

12

Redefining Learning: A Neurocognitive Approach

281

William Pinar participated in a panel discussion on interdisciplinary teaching and intrinsic learning at the Eighth Annual Teaching and Learning Higher Education Conference in 2014 at the University of KwaZulu-Natal in South Africa. The discussion was live-streamed, and the archived video can be viewed at https://youtu.be/E5iHt72PJ9E. Pinar would suggest that teachers’ self-reflection on their own experiences and education provides a lens through which to view their role as educators, rather than rely on prescriptive notions of curriculum. What is taught, how, and when thereby become lived experiences. According to Pinar (1975): “They must not subordinate the lived present to their abstract ‘selves’” (p. 11). In similar manner, taking this idea a few steps further, we would suggest that by moving the threshold of evidence in accordance with a neurocognitive definition, teaching (thus curriculum) should be guided not by a behaviorist prescription but, rather, by a judgment about each learner. That is, as educators we might better teach by adapting instruction to where the learner is, instead of presuming where the learner should be in terms of knowledge and skills development. Increasingly, neuroscience is providing brainbased evidence to guide teaching and learning. This notion of proceeding from a neurocognitive definition of learning, incidentally, harkens to John Dewey’s admonition in his 1916 classic, Education and Democracy: Were all instructors to realize that the quality of mental process, not the production of correct answers, is the measure of educative growth something hardly less than a revolution in teaching would be worked. (2008/1916, p. 183)

Later, Dewey was one of 34 signers of the 1933 A Humanist Manifesto (Bragg, 1933). The Deweyan focus on humanism defined learning in individualistic terms, which was amplified by others in the progressive education movement of the early twentieth century. Learning as a manifestation of individualism was intended to counter the Industrial Age idea that learning is purely instrumental – that is, learning is training for future work. The sentiment is still heard in the often-repeated phrase today: “college and career readiness.” Public education, however, because of other currents in the USA and global culture – the two World Wars, the Great Depression, the Cold War, and other fraught periods – continued to be seen as the servant of industry and students as merely future workers. The shift in viewpoint from an industrial future to an individual future was then – and remains today – a radical and largely unrealized change in what it means to learn. How might Dewey’s ideas have changed if he were writing in 2016, rather than 1916, given our current research in the areas of neuroscience? Would they have changed?

282

P. Harris and D. R. Walling

It is time, indeed it is well past time, for a Digital Age revolution in teaching à la Dewey, which can be achieved by moving the threshold of evidence using neuroscientific advances as guides. Visser and Visser (2001) take a somewhat different tack from Dewey and Pinar, although a compatible definitional approach, suggesting that learning should be “undefined” – meaning that learning should be redefined in broader terms. They suggest that research currently underway. . .makes it possible to get a better insight into the meaning of learning from the perspective of those who learn, rather than the point of view of those who design or facilitate learning. (p. 1)

The research to which Visser and Visser allude involves “learning stories,” in which an “emphasis on the role of curiosity and challenge as conditions present in people’s most meaningful learning experiences speaks directly to the design of learning environments and instructional materials” (p. 7). They also point to the importance of “constructive and conscious involvement in someone else’s learning” and the “presence of a role model or emotionally significant support” (p. 7). These characteristics echo Pinar’s focus on self-reflection, plus observation, which are variants of a cognitive definition of learning, rather than a behavioral one. Fundamentally, Visser and Visser arrive at this perspective in recognition of the world of the Digital Age: The conditions that prevail in today’s world mark a fundamental change with those that characterized the state of the planet a mere couple of decades ago. This calls for new visions of learning and the re-examination of the conditions that promote and facilitate it. (2000, p. 1)

We would argue that Visser and Visser underestimate the advent of “today’s world” with casual mention of “a couple of decades.” In fact, the Digital Revolution began more than half a century ago, and Digital Age technology – particularly neuroscience – should be shaping a new era in education, a long-overdue transition from factory schools to something else. We will describe the potential characteristics of that “something else” in due course. Those characteristics will be driven by the new definition of learning that we have proposed, which presupposes a grounding in neuroscientific discoveries that are emerging with unimagined speed and complexity. For example, merging innovative instruction with brain science, Carnegie Mellon researchers (Reder, Liu, Keinath, & Popov, 2015) have been able to offer evidence of a phenomenon that educators have long known intuitively: that learners learn more effectively and more easily when new knowledge is mediated through existing knowledge. These researchers examined memory related to recall of Chinese characters and paired English words. Although the findings have specific implications for second language learning, they can be generalized to learning in almost any subject.

12

Redefining Learning: A Neurocognitive Approach

283

David Amen, psychiatrist and brain disorder specialist, discusses single photon emission computed tomography (SPECT) as a diagnostic tool. Amen and his colleagues work primarily in the area of brain rehabilitation, which has broad ramification for teaching and learning. His TEDx talk can be found on YouTube at https://www.youtube.com/watch?v=esPRsT-lmw8&index=3& list=WL. Direct examination of the brain in learning also is giving insights into phenomena of knowledge acquisition that are not visible. In other words, neuroscience is making it possible to move the threshold of evidence of learning back from the behavioral position to one closer to the source: the brain in action. For example, researchers (Jaberzadeh, Bastani, Zoghi, Morgan, & Fitzgerald, 2015) at Monash University in Melbourne, Australia, discovered that noninvasive brain stimulation enhanced brain “excitability,” which could improve physical performance in healthy individuals, such as athletes and musicians. In a summary article for Neuroscience News (2015), Shapour Jaberzadeh, one of the researchers, commented: This treatment, which we called transcranial pulsed current stimulation (tPCS) is a non-constant form of stimulation with “on” and “off” periods – or pulsing – between the two electrodes. . .. We discovered that this new treatment produced larger changes in the brain and that the interval between pulses also had an effect. The shorter the interval between pulses the larger the excitability effect in the brain. . .. When we learn a task during movement training (for example playing the piano), gradually our performance gets better. This improvement coincides with enhancement of the brain excitability. Compared to tDCS [transcranial direct current simulation], our novel technique can play an important role in enhancement of the brain excitability, which may help recipients learn new tasks faster.

Animal studies have long preceded studies in human subjects. But they can be no less revealing. In another recent study, for example, researchers (Cichon & Gan, 2015) at New York University School of Medicine used calcium imaging of neurons in the motor cortex of mice to explore how the brain stores new information (an aspect of learning) without disrupting previously acquired memories (i.e., prior knowledge). According to these researchers, their findings show that “dendriticbranch-specific generation of Ca2+ spikes is crucial for establishing long-lasting synaptic plasticity, thereby facilitating information storage associated with different learning experiences” (p. 1). In humans, according to lead researcher Joseph Cichon, their discoveries could have implications for explaining underlying neural circuit problems that occur in disorders such as autism and schizophrenia. Another study using mice emerged from Australia, where researchers at the Queensland Brain Institute (QBI) demonstrated that noninvasive ultrasound could be used to treat Alzheimer’s disease and restore memory (Leinenga & Götz, 2015). This discovery could potentially be a major breakthrough in treatment of human subjects with Alzheimer’s and possibly other brain disorders.

284

P. Harris and D. R. Walling

While animal studies are important, real strides in understanding brain function in learning also are being made in human studies that involve noninvasive investigative techniques. For example, researchers (Glezer, Kim, Rule, Jiang, & Riesenhuber, 2015) at the Georgetown University Medical Center in Washington, D.C., studied how individuals learn new words, a key factor in learning writ large. Taking their cue from prior studies that have given evidence that reading “engages the left ventral occipitotemporal cortex” (p. 4965), the researchers investigated new word acquisition by studying 25 native English speakers, ages 18–35, using fMRI-RA (functional magnetic resonance imaging-rapid adaptation). Their findings offer several important observations that may influence teaching and learning: It has been proposed that the VWFA [the visual word formation area in the brain] develops with reading acquisition as a result of the “recycling” of visual cortex, resulting in neurons dedicated to orthographic processing. . .. Our study supports the theory that the role of the VWFA in reading is that of an orthographic lexicon in which during word learning, neurons come to be selective for the “objects” of reading, that is, whole words, enabling the rapid recognition of familiar words. These findings have interesting implications for reading remediation in individuals with phonologic processing impairments because they suggest the possibility that these individuals might benefit from visual word learning strategies to circumvent the phonologic difficulties and directly train holistic visual word representations in the VWFA. (p. 4971)

For readers who remember the vigorous debates in reading instruction over phonics approaches versus whole language approaches, especially during the 1980s and 1990s, this study provides one way to consider how neuroscience can move the threshold of evidence from behavioral observations to fundamental observations of phenomena occurring within the brain. Anna Wilson, a researcher whose specialty is dyscalculia and mathematical cognition, is a lecturer in the College of Education at the University of Canterbury in New Zealand. In a talk at the university in 2013, Wilson provided an overview of facts about neuroscience and “neuromyths” titled, “What if. . .neuroscience could change education?” The talk is on YouTube at https://youtu.be/Q96MnaJyaaA. Evidence is rapidly accumulating that will support a transformation from systems of education based on behaviorist definitions of learning to systems that approach teaching and learning based on some form of neurocognitive definition of learning. In the next section, we describe how education might be reconceptualized based on our proposed definition of learning. Our focus is primarily on children and adolescents, what is now termed preK-16, or prekindergarten (preschool) through undergraduate studies. However, the neurocognitive reconceptualization we suggest can be applied to other levels and forms of education, in collegiate or other environments, such as in medical, military, or corporate settings.

12

Redefining Learning: A Neurocognitive Approach

285

How might teaching and learning change if teachers, parents, and perhaps even learners had access to brain-imaging information instead of just observational data and test scores?

Neurocognitive Reconceptualization of Education The Industrial Age factory model of schools in which education is premised on a behaviorist definition of learning is both implicitly and explicitly driven by competition. The term normative is applicable, meaning the process of comparing one learner with his or her peers, one teacher with other teachers, one school with other schools, the schools in one state with those in another, in one country with those in other countries, and so on. The standardized tests required under various federal and state laws are explicitly normative. Implicitly, this model means that some learners, teachers, schools, and so forth will be judged better or worse in comparison to others, rather than against a set of criteria, or indicators of success, achievement, excellence, or some other quality. We will discuss this topic further in the section titled Rethinking Evaluation: Methods and Alignment. How might education be reconceptualized to accord more closely with a Digital Age worldview? This worldview has been propelled by a tectonic collision in educational thought and the seismic advance of new technology, which has shaken the foundations of schooling as we have known them over the past century. Ever-emerging new research findings about how the brain changes with learning are making prescriptive behaviorist notions of schooling increasingly outmoded shadows of a bygone era. Our neurocognitive definition situates within the general domain of cognitive science as explained by Gerrig and colleagues (2008): “The domain of cognitive science occupies the intersection of philosophy, neuroscience, linguistics, cognitive psychology, and computer science (artificial intelligence)” (p. 248) (see Fig. 3). The inclusion of linguistics is especially pertinent within the scope of this work because we must necessarily use the words we know and now must explain or define in new ways, or run the risk of inventing new words that may baffle the reader. Thus we felt it was important to spend time in the first section to parse the familiar definitions of learning before proposing a new definition. We will spend some time, in a similar way, redefining and repurposing terms in this section. Sir Kenneth Robinson, Ph.D., is Professor Emeritus at the University of Warwick in the UK, where he was Professor of Arts and previously Director of the Arts in Schools Project. In a TEDx talk in February 2010 in Long Beach, California, his topic was “Bring on the Learning Revolution!” The talk is available at https://youtu.be/kFMZrEABdw4. He makes the case for a radical shift from standardized schools to personalized learning.

286

P. Harris and D. R. Walling

Fig. 3 Cognitive science brings together – and attempts to integrate and make sense of – inputs from a variety of areas, all of which are important to teaching and learning (Source: Wikimedia Commons. Public domain)

Transformations from the familiar to the unfamiliar in the present context mean moving from conceptions of teaching and learning as we have known them during the Industrial Age to new conceptions as we envision them for the current Digital Age. What we are envisioning is not an evolution of the factory model school but, rather, a revolutionary reconceptualization of teaching and learning that will require a variety of educational environments, both physical and intellectual, that are radically different from the ones we now know. To be clear, underlying our notions about schools for the Digital Age that are consonant with emerging neuroscience is a grounding belief that public education is fundamental to democracy. Thomas Jefferson is often cited for saying, in a letter to James Madison (1787): “Above all things I hope the education of the common people will be attended to, convinced that on their good sense we may rely with the most security for the preservation of a due degree of liberty.” Political theorist Benjamin Barber (1992) echoes Jefferson and expands on this theme in a way that offers the fundamental principles that form the substructure for our view of public education for the Digital Age: In the tradition of Jefferson and Dewey, I believe it is possible to understand all public education as liberal education – teaching liberty – and thus to understand liberal education as democratic education. . .. But public education is general, common, and thus in the original sense “liberal.” This means that public education is education for citizenship. In aristocratic nations, in elitist regimes, in technocratic societies, it may appear as a luxury. . .. But in democracies, education is the indispensable concomitant of citizenship. Where men and women would acquire the skills of freedom, it is a necessity. (p. 15)

So what might Digital Age education look like? The responses to this question are premised on a general conceptual shift from the linear to the nonlinear. The factory model school is sequentially linear in character, and the sequences – school terms, grades, and classes – are essentially static, one predictably following the other with rare exceptions. In contrast, a neurocognitive approach may be characterized as multidimensional and nonlinear within the contexts of interconnected, technologyrich physical and intellectual environments that characterize the Digital Age. We posit four areas in which this shift will be most manifest: individuation, motility,

12

Redefining Learning: A Neurocognitive Approach

287

accessibility, and virtuality. In each of these areas, advances in neuroscience and technology will propel ongoing, and in some aspects quite rapid, change.

Individuation Focus in the Digital Age must shift toward the individual learner. The importance of learner-centered education can be traced to a long line of philosophical and pedagogical ideas, from those of the Enlightenment philosopher Jean-Jacques Rousseau (1712–1778) to the Swiss pedagogue and education reformer Johann Heinrich Pestalozzi (1746–1827) and more recent figures, such as the American John Dewey (1859–1952) and the Swiss Jean Piaget (1896–1980). Learner-centered education is the iteration of Barber’s notion of acquiring “the skills of freedom,” and how better than to facilitate the freedom to choose, with appropriate guidance, one’s own learning? Researchers Tzuo, Yang, and Wright (2011) suggest that placing learner centeredness at the heart of schooling will require teachers and learning designers to focus on “developmentally appropriate practices” that take into account learners’ intellectual strengths, interests, and needs within the social and cultural contexts in which learners live. In particular they refer to incorporating a notion from reconceptualism, which sees “teachers as scholars who continually revise their theories of education as well as their pedagogy based on what they discover in the classroom” (p. 555). In the context of reconceptualizing schooling for the Digital Age, Dimitriadis and Goodyear (2013) provide a useful frame of reference. They cast notions of learnercentered teaching and learning design as “forward oriented.” Their framework rests upon the importance of four elements: 1. Design needs to be understood as having an indirect effect on learning. Learning itself cannot be designed; things can be designed which can have a beneficial effect on learning. 2. Teachers are often essential actors at learntime, since they may intervene with respect to the real-time coordination of classroom events (orchestration, in its strict interpretation). Design methodologies need to be able to take into account the various times and ways in which teachers, as actors with bounded capabilities, can enhance what occurs at learntime. Design methodologies need to be robust and general enough to cope with face-to-face, online, and blended contexts, with synchronous and asynchronous interactions, as well as situations where teachers’ time, skills, or attention are limited and even with situations in which there is no teacher (e.g., in self-study courses). 3. Design for learning needs to find ways of working with the dialectical relationships between structure and agency. Providing structures such as scripts and scaffolding is not antithetical to student autonomy. Design is, par excellence, a discipline for resolving competing forces, including balancing structure and freedom, at various scale levels (whole course, learning episode, infrastructure, tool, etc.).

288

P. Harris and D. R. Walling

4. All design is future oriented, of course. But when we talk about forward-oriented design, we mean something much stronger. It is partly about designing with a sensitivity to the complexities and unpredictability of what happens after a design “goes live.” But in addition to designing with contingencies in mind, forwardoriented design embraces the following sense. Once a design goes live, it is affected by processes that are active in different ways, and to different degrees, during successive phases of the life cycle. Different things happen during configuration, orchestration and evaluation/reflection. . .. The intellectual and physical architecture of the factory model school constrains individuation in favor of linear grades and relatively static class groups. Learners enter the formal education assembly line at a predetermined age and proceed though the next 12 to 16 years in a sequence that is seldom substantially modified. In essence, learners are products to be formed and finished. Schooling by time clock. True individuation can be achieved only by reconceptualizing several common presumptions and related practices that, in the factory model, have been taken for granted for several generations. These presumptions include school readiness, predictable progress, and academic compartmentalization.

School Readiness School readiness presumes that learners, with very few exceptions, should be ready to start formal schooling at predetermined age, which ironically varies according to local or state policy. This geographic variation alone is evidence of the arbitrariness of this presumption. While physical and intellectual development is predictable within an age range, the concept of “range” is essential to understanding that not all children are “ready” for school – whether preschool, high school, or undergraduate school – at an arbitrarily chosen age or sequential point in their educational career. The concept of “readiness” itself is open to debate. Indeed, readiness to enter formal schooling has been the subject of much study, a good deal of it devoted to the question of how to get children ready to start school by the policy-prescribed age, an approach that too often ignores neuroscience findings about early brain development, just as it ignores pre-neuroscience theory and practice findings by education researchers, philosophers, clinicians, and practitioners such as Piaget, Abraham Maslow (1908–1970), and Maria Montessori (1870–1952). The National School Readiness Indicators Initiative adopted a somewhat innovative approach, although it was still situated within the constraints of the factory model school. The initiative involved 17 states in a three-year project to develop sets of indicators of school readiness and to track results for children from birth through age 8. The goal was to inform policymaking, implicitly with regard to getting children ready for school “on time,” rather than altering the meaning of “on time.” The results of the initiative were published in a report titled Getting Ready (2007), which concluded:

12

Redefining Learning: A Neurocognitive Approach

289

Closing the school readiness gap will require attention to the multiple influences on early child development, including the contributions of family and neighborhood, home environments related to language and literacy, parenting practices, health status, health behaviors, child care and early education. The growing racial, ethnic, linguistic, and cultural diversity of young children requires that health, mental health, early childhood and education programs periodically reassess their appropriateness and effectiveness for the wide variety of families they serve. (p. 39)

While these are vital considerations, they raise a question with regard to imposing a readiness deadline. Instead of labeling beginners as “ready” or “not ready” at some arbitrary school starting age, true individuation would suggest that flexibility with regard to starting school is essential in order to recognize that children become “ready” at different ages. Some of this differentiation is the result of environmental factors, health, and so forth, characterized as “indicators”; however, simple human diversity also accounts for much of such differentiation. This diversity can now be identified through neuroscience, for example, using various forms of brain imaging, as discussed previously. Allowing learners to start school at whatever age they are judged to be “ready” – whether that is age 4 or age 8, for instance – would enact a more learner-centered school starting policy. Public education policy guided by neuroscience research and a neurocognitive definition of learning would recognize at minimum four factors articulated in a policy brief from the Wisconsin Council on Children and Families (2007): With the neuroscience of brain development unfolding, we now know that (1) the way a brain develops hinges on the complex interplay between the genes a person is born with and the experiences a person has from birth on; (2) it actually takes up to 12 years for the brain to become fully organized, with parts of the cortex still to become organized through the later teen years; (3) the quality of an infant’s relationship with his or her primary caregivers has a decisive impact on the architecture of the brain, affecting the nature and extent of adult capabilities; and (4) early interactions directly affect the way the brain is “wired,” and do not merely create a context for development. (p. 1)

Predictable Progress The notion of “readiness” at a policy-determined age or sequential point ripples along the length of the school assembly line and lies at the heart of various other target-point evaluations that can ultimately affect learners’ future learning and success in life. These target points are manifestations of a presumption of predictable progress – in other words, that learners will learn according to a behaviorally prescribed timetable. This timetable does not – in fact, cannot – account for human variations in brain development and consequently imposed sequences, such as grade progression, do not foster effective teaching and learning for exceptional students, whether they learn slower, faster, or simply differently from their peers. An example is the Indiana Reading Evaluation and Determination, or IREAD-3, assessment that imposes an arbitrary deadline for third-grade students to attain certain reading skills. Failure can mean that students are held back, rather than allowed to move into fourth grade, which is problematic in several ways, from likely ineffectiveness of remediation tied to simple repetition of third grade to the increased

290

P. Harris and D. R. Walling

risk that retained students will drop out before graduation. This arbitrary sorting according to the results of a standardized reading test of questionable value is an unnecessary gateway that can stifle learning for students whose brain development does not fit within the prescribed range (see Richmond, 2012). If teachers and learning designers set aside the constraints of predictable progress, how might their thinking about curriculum, lesson structure, and assessment of learning need to change? Predictable progress is the philosophical underpinning of the assembly line. When learners behave contrary to the predictable progress framework, they are seen as the problem rather than the school structure being the problem. The round holes of the factory school cannot well accommodate students who are square pegs. A famous example is Thomas Edison, who was age 7 in 1854. After Edison spent several weeks in a one-room schoolhouse, Edison’s teacher grew frustrated with the child’s constant questions and apparently self-centered behavior. Edison, who had not learned to talk until age 4, was deemed to be “addled,” although today’s education psychologists probably would have labeled him ADHD (attention deficit hyperactivity disorder) and prescribed Ritalin. In frustration, Edison’s mother Nancy withdrew the boy to be homeschooled. Today’s homeschooling proponents point to Edison’s subsequent education to tout the merits of home teaching, but the central operant factor was individuation. After leaving school Edison was not constrained within the Industrial Age model to predictable progress. Rather, through his mother’s instruction, independent reading in his father’s extensive library, and free rein to explore in his own manner subjects ranging from history and poetry to chemistry and physics, Edison’s education became the epitome of learner centered (Beals, 1999). And so the question should be: How can schools be reconceptualized to nurture exceptionality to whatever minor or major degree that can be accounted for in terms of human variation across the broad spectrum of brain development, learning, and behavior? In other words, how can Digital Age schools succeed with all students, rather than constraining or pushing out those who do not conform? One response certainly must be to discard the notion of predictable progress.

Academic Compartmentalization Edison also provides an example of the power of interdisciplinary learning, free from the predetermined constraints of academic disciplines maintained artificially discrete. While certain areas of the brain function in identifiable ways (see, e.g., Glezer et al., 2015), the real power of learning rests in making, or facilitating, connections – among information known in different ways, among the known and the newly learned, and so on. The umbrella term is neuroplasticity,

12

Redefining Learning: A Neurocognitive Approach

291

which refers to changes in the brain that may be caused by any number of factors, a key one for all human beings’ learning. William Safire (2009) put it this way: Today, neuroscientists. . .are delving into the connectivity among the brain’s universe of neurons. . .. Because cognition is rooted in the Latin word for knowledge, educators also have a great stake in the idea of circuits. In great universities and in elementary classrooms, the constricted “stovepipe” departments of the past have given way to interdisciplinary approaches. Such connectivity in teaching gives memorable context to learning; equally important, it spurs student creativity. Subjects cross over each other, transferring skills and knowledge, figuratively as they do in the brain. (p. 1)

Interdisciplinary learning is not a new idea, but it gains power when combined with other strategies for individuation.

Motility If teaching and learning are truly focused on the individual learner, then it follows that different environments – different teachers, pedagogies, curricula, physical settings – will be required at different times, as the learner matures both physically and intellectually. Rather than compelling learners to stay in static school situations, movement – between grades, subjects, and learning spaces – will need to be encouraged and valued, rather than resisted not only by policy makers, educators, and parents but also, in many cases, by learners themselves who have become habituated to the factory model school. Peter Hutton, an Australian public high school principal, offers a firsthand perspective on the question, “What If Students Controlled Their Own Learning?” His TEDxMelbourne talk can be viewed on YouTube at https://youtu.be/ nMxqEkg3wQ0. Another viewpoint can be found in the Charles Tsai report, “If Students Designed Their Own Schools. . .,” filmed at Monument Mountain Regional High School in Great Barrington, Massachusetts, which can be viewed on YouTube at https://youtu.be/RElUmGI5gLc. One manifestation of individuation is termed personalized learning, which can be defined as placing the learner at the center of the learning experience. According to a Center for Digital Education (2015) report: “What really matters in a personalized model is that students learn a concept, apply it in real life, and demonstrate mastery of it before moving on to something else. Learning stays constant, while time becomes the variable” (p. 10). This notion of time as the variable is a central factor in determinations of “readiness,” but for true individuation, time must be coupled to

292

P. Harris and D. R. Walling

place. Without this coupling, time is still a significant constraint. Motility really means that learners not only may learn at a self-determined pace but also that to enable such learning may require systemic changes in educational environments, both the sequential nature of grade progression that divides students into age-similar facilities (elementary schools, middle schools, and so on) as well as architectural structures in which spatial determinations segregate subject matter (classrooms for history, laboratories for science, and so on). What might schools look like if education planners and architects asked students to design spaces where they would like to learn? However creative the architecture of Schenley High School (see Figs. 1 and 2) was in 1916, like its less innovative counterparts built during the twentieth century (and many still being designed and constructed today), it is an embodiment of the factory model of sequentially static, compartmentalized teaching and learning. The classrooms of Schenley may align in a triangle, but they are still boxes that serve to contain and constrain, rather than to liberate and personalize. Architecture presents a bias of longevity. Schools built fifty, even one hundred years ago, are still being used today and consequently act as a constraint on innovation and change with regard to what takes place within these structures. Therefore, one challenge to actualizing environments for the Digital Age initially will be repurposing existing structures until new structures can be constructed. What might these new structures look like? Forward-looking facilities designers can take cues from new architectural designs in instances when design and Digital Age schooling have cross-pollinated to produce truly innovative structures. An example is Ørestad College (Ørestad Gymnasium), a high school for 16- to 19-year-olds in Denmark. According to the architectural firm 3XN that designed the school: The college is interconnected vertically and horizontally. Four boomerang shaped floor plans are rotated to create the powerful super structure which forms the overall frame of the building – simple and highly flexible. Four study zones occupy one floor plan each. Avoiding level changes makes the organizational flexibility as high as possible, and enables the different teaching and learning spaces to overlap and interact with no distinct borders. The rotation opens a part of each floor to the vertical tall central atrium and forms a zone that provides community and expresses the college’s ambition for interdisciplinary education. (3XN, 2016)

The innovative spaces that support learner-centered teaching and learning lie behind a deceptively bland four-story façade that, while employing colorful window treatments, is conventional in comparison to its highly unconventional interior (see Fig. 4). Learning spaces flow seamlessly from one configuration to another, allowing for learners to engage in individual, independent, small-group, or large-group learning activities (see Fig. 5). This school – from the flow of its physical spaces to the flow of ideas within them – offers an exemplar of motility.

12

Redefining Learning: A Neurocognitive Approach

293

Fig. 4 Ørestad College, a Danish high school in Copenhagen at Ørestads Boulevard 75, offers a conventional exterior appearance that belies its innovative interior floor plans (Source: Palnatoke (https:// commons.wikimedia.org/ wiki/User:Palnatoke); public license under Wikimedia Commons)

Fig. 5 This interior architectural rendering of Ørestad College illustrates the dynamic interconnectedness of the free-flowing learning spaces (Source: Demos Helsinki (https://www.flickr.com/ photos/demoshelsinki/); public license under Wikimedia Commons)

294

P. Harris and D. R. Walling

The Council on Educational Facility Planners International (CEFPI), established in 1921, hosts a competition for students to design future schools using new concepts and technology. “School of the Future – Teeland 2012” is a video showcasing the winning entry developed by students from Teeland Middle School in Wasilla, Alaska. The video is available on YouTube at https:// youtu.be/S2BTmYcE0JU.

Accessibility Resistance to technology, innovation, and change in general often seemed to be hallmarks of the factory model school. In the Digital Age, change, often rapid and rolling, is consistent with the relentless march of technological advance. In 1979 – ancient times by technology standards – Christopher Evans, in his landmark book, The Micro Millennium, described the rapid pace of change in computer technology with an analogy to the automobile industry. If automobile manufacturing had developed at the same rate as computers, then today’s consumers would be able “to buy a Rolls-Royce for $2.75, it would do three million miles to the gallon, and it would deliver enough power to drive the Queen Elizabeth II. And if you were interested in miniaturization, you could place half a dozen of them on a pinhead” (p. 76). That was nearly four decades ago. The pace has not slowed. Technology – particularly portable devices, such as smartphones and tablet computers – are facilitating greater accessibility to information than ever before in the history of human learning. The Digital Revolution in accessibility began when the advent of personal computers was coupled to the development of the Internet. Personal computing moved accessibility to information-processing into the hands of nonscientists, everyone from students to shopkeepers; the Internet grew exponentially into a vast network of information that could be tapped by ordinary people who had little or no access to physical resources, such as libraries or bookstores. How do you use computers (any sort) and the Internet to acquire information and build knowledge and understandings? How might self-reflection on personal technology-mediated learning help to shape interactions with students and technology to enhance teaching and learning? As schools integrated these resources into teaching and learning, use at first was limited. Teachers often used computers for housekeeping – attendance taking, grade recording – rather than to enhance teaching. Learner-used computers were segregated into multiuse computer labs, where use for learning was selective, scheduled, and limited. This was the standard mode of “technology-mediated” teaching and learning from roughly the decade of the 1980s through the next 20 years (and in many cases, much longer).

12

Redefining Learning: A Neurocognitive Approach

295

The introduction and rapid deployment and improvement of laptop computers during the 1990s – Apple’s first PowerBook came onto the market in 1991 – began a gradual movement toward more learner-centered computer use, as schools moved from lab-based computing to more classroom-based uses. This movement was accompanied by an uptick in the use of computers by teachers to enhance teaching through the design and delivery of lessons developed using computer technology. Similarly, the proliferation of laptops allowed the introduction in some schools of one-to-one (1:1) “laptop classrooms,” in which learners were able to work at a dedicated computer, sometimes independently, sometimes at their own pace, though always within the confines of a linear, sequence-driven curriculum structured to accommodate a linear progression of age-grade placements. The next phase of the technology revolution arrived with tablet computers and smartphones. Apple launched the first-generation iPad in April 2010 and sold some 300,000 units that month alone. Other computer manufacturers raced to compete, and now consumers can choose from a wealth of tablet devices. Within a short time, “tablet classrooms” were beginning to proliferate, with many schools transitioning from laptops to tablets. The advantages were several. According to Walling (2014): Tablet computers offer even greater portability than typical laptops, and their smaller size is matched by a smaller price tag, making them more affordable than traditional computers. For schools that want 1:1 computer capability for students, tablets can be a good fit for tight budgets. (p. 3)

Smartphones – mobile, or cell, phones with personal computer operating systems added to the communication (telephone) function – actually came onto the technology scene in advance of tablets. Apple, again the leader, introduced the first iPhone in 2007; however, most schools initially saw smartphones in the same way as cell phones, as distractions and nuisances. It was not until tablets were recognized as having educative potential that smartphones were recognized as being, in essence, miniature tablet computers that just happened to have phone capability as well. This realization sparked a concurrent movement to integrate Internet-capable computer technology into teaching and learning that usually is referred to as BYOD, or bring your own device. “Classrooms of Tomorrow” received an Honorable Mention at the 2012 Edmonton Catholic Schools Film Festival. This short film offers a glimpse into the ubiquity of personal computing devices in education. It can be viewed on YouTube at https://youtu.be/iA18DsuyaFc. BYOD provides a truly learner-centered option in that the computer belongs to the learner and can be used anywhere, whereas even in schools committed to 1:1 tablets those devices might or might not be allowed to travel with the learner, whether from class to class or from school to home.

296

P. Harris and D. R. Walling

When learners possess a device, regardless of ownership, that they can use whenever and wherever they choose, then learning can occur not only in the classroom but in school corridors or the lunch room, on the school bus, at home, or wherever. The possibilities are endless. Moreover, such accessibility means that subject matter can be more readily integrated and need not remain compartmentalized according to schedules, classes, and curricula. Accessibility to information, and thus to increasing (often independently) knowledge and understandings – i.e., learning – is a force toward reconceptualizing what we mean by school. Some researchers refer to this in terms similar to those we have used, namely, redefinition, or, as Visser and Visser (2000) would term it, undefinition: While there are good reasons for things to be defined, there is no reason to keep clinging to particular definitions. When established definitions get in the way of continued development in a field of intellectual pursuit and practice, i.e. when they become “too narrow to comprehend new [and thus also envisioned] experiences” (Bohr, 1987, p. 67), there is an urgent need to undefine them. (p. 9)

This has become the case with accessibility, which has moved from going to a resource (physical or electronic) to find a particular piece of information to using multiple, readily obtained resources through the interconnected information of the Internet to acquire integrated knowledge and understandings that break down the traditional compartmentalization of times, places, and teaching/learning sequences.

Virtuality Perhaps more than in any other area, teaching and learning within technologymediated “virtual” environments will probably propel the most dramatic changes in what educators, learners, and their parents are used to thinking of as school. As a forerunner, in the 1970s the open classroom movement was cast as philosophically learner centered and environmentally freeing. Larry Cuban (2004), professor emeritus at Stanford University and a proponent of open classrooms, points out, “The fact is that no single best way for teachers to teach and for children to learn can fit all situations.” Many schools embraced the movement and took out walls between existing classrooms only to replace the walls or put up partitions a few years later because other structural elements of both teaching and schooling did not change in the same manner. The open classroom movement came to be seen as a fad. Cuban, however, suggests that “while the open classroom has clearly disappeared from the vocabulary of educators, another variation of open education is likely to reappear in the years ahead.” While new architectural approaches are, in fact, creating schools with open classrooms (see Figs. 4 and 5), technology, in various ways, can create virtual open classrooms. In isolation from the other areas, this expanded vision of the role of technology may be seen as a particularly compelling challenge, one imbued with an aura

12

Redefining Learning: A Neurocognitive Approach

297

of futurism or science fiction, while in practice the inertia of “business as usual” will blunt any real change. As international advisor on education, Sir Kenneth Robinson (2011) observed, “It is an interesting feature of cultural change that, for a period of time, new technologies tend to be used to do the same old thing” (p. 204). That is the current status of educational technology in a myriad of conventional education settings, from early childhood to graduate school. For example, many e-books (and increasingly eTextbooks) are simply digitized versions of their paper counterparts. Likewise, many readers tend to read e-books in the same manner as traditional books, that is, in a linear progression from front to back. But some publishers, including textbook publishers (although many are still figuring out how to navigate these new technological waters) are distinguishing their electronic books by including interactive media and extended research tools, such as sophisticated search functions that allow readers to dip into the book in nonlinear ways as well as to move out of the book and link digitally to other resources. Multifunctionality is an essential feature of Digital Age educational technology, which cannot be merely an electronic version of “the same old thing.” In what ways might teaching and learning change if learning designers, teachers, and students collaborated to develop eTextbooks, rather than using commercially prepared texts, whether traditional paper or electronic? What are the pros and cons? Perhaps more to the point, virtual “book” technology is allowing end users – learning designers, teachers, and students – to create their own resources. “Studentspecific data can now be used to customize curricula and suggest resources to students in the same way that businesses tailor advertisements and offers to customers” (Johnson et al., 2013). If we take data in this observation at its broadest connotation, then it is possible to envision teaching and learning environments in which learner-centered projects require learners to curate their own resources, meaning that they will assemble their own virtual tools, such as multifunctional eTextbooks and digital libraries. Walling (2014) elaborates: Today, more and more learning designers are creating their own eTextbooks, often in collaboration with students and other teachers, by gathering the electronic resources and courseware they need to address particular curricula. Resources include articles found online, simulations, and audio and video files. These may be supplemented by lesson plans shared by other educators. Although building an eTextbook in this manner can be labor intensive and time consuming, the result can be a uniquely suitable, targeted, and welltailored learning resource. (p. 59)

Creating digital resources will require everyone concerned in the enterprise of schooling to think outside the traditional idea of a book or textbook. The complementary requirement will be for everyone – especially educators and policy makers –

298

P. Harris and D. R. Walling

to think outside the traditional idea of a school or classroom. We have already discussed notions about altering physical sequences and spaces, but what about virtual spaces? “The Virtual Classroom: Online Learning” provides an overview of virtual teaching and learning in this video from Edutopia (http://edutopia.org). It can be viewed on YouTube at https://youtu.be/DQ-1zhFXiJU. Virtual schooling has moved from sci-fi to reality with the advance of the Digital Age. While a virtual school environment might be a suitable alternative to a brickand-mortar environment (e.g., as a form of distance learning for underserved populations) and certainly could be fashioned around the concept of learner-centered or learner-directed learning, the more versatile model is likely to be a hybrid that mixes the physical and the virtual. This hybrid model often is a manifestation of so-called blended learning, which connotes some form of learning design that combines, or blends, face-to-face interaction with one or more types of online learning, which may range from simply watching an instructional video to using conferencing media to interact with peers in some other location, whether down the hall, across town, or around the globe. An example of the latter might be a group of learners in one location collaborating on a project with learners in another location. For instance, researchers at the University of Michigan School of Information established a Collaboratory on Technology-Enhanced Learning Communities (Cotelco): Using a suite of commercially available Web-based collaboration tools, Cotelco brings together faculty, staff, and students from the University of Michigan and American University in the United States and the University of the Witwatersrand and the University of Fort Hare in South Africa to develop and conduct collaborative research, share data, engage in distributed research team meetings, and to deliver a semester-long weekly, geographically distributed graduate seminar entitled “Globalization and the Information Society: Information Systems and International Communications Policy,” known as the Globalization Seminar at each of the participating institutions. (Cogburn & Levinson, 2003, p. 35)

Similar ventures are taking place at all levels. While this US-South Africa project involves university students, others include younger students. For example, Global Nomads Group (http://gng.org) gave students in California “a virtual glimpse of the chaos and carnage endured by civilians caught up in the Syrian civil war” before they videoconferenced with Syrian students living as refugees in Jordan (Berdik, 2015). The Digital Age might also be termed the Age of the Individual. Much has been written about the nature of community – i.e., the traditional classroom – as a societal construct, as a representation of face-to-face relationships. With the emergence of Digital Age technology that allows for coordination, cooperation, and collaboration virtually, the idea of community has expanded into a new form, which has been called by some “networked individualism” (see, e.g., Wellman, 2002; Miller, 2011). In summary, a full reconceptualization of education – hopefully a new normal of public education in the Digital Age – will require attention to all aspects of

12

Redefining Learning: A Neurocognitive Approach

299

schooling. Collating these aspects into four categories – individuation, motility, accessibility, and virtuality – implies a conceptual tidiness that belies a maelstrom of change. Little of the transition from the Industrial Age to the Digital Age in education is likely to be tidy, but what is revolution?

Rethinking Evaluation: Methods and Alignment Over the past several decades, there has been a vast increase in the use of standardized tests at all levels of preK-16 schooling, but especially in the K-12 years, to evaluate learners’ acquisition of knowledge. These tests have mainly focused on so-called core subjects, such as reading and mathematics. For example, the No Child Left Behind (NCLB) Act mandated annual tests of reading and math in grades 3 through 8 and once in high school (Klein, 2015). Many US states added their own mandated standardized tests to this federal requirement (see Indiana’s IREAD3, discussed in the previous section). Under NCLB states were required to bring all learners to the “proficient level” by the 2013–2014 school year an unrealistic, all-ornothing goal. The successor to NCLB and currently the federal law, the Every Student Succeeds Act (ESSA) of 2015, does little to modify this mandate, although it incorporates a requirement for states to include at least one “nonacademic” measure to judge school performance and reduces some of NCLB’s rigidity. However, education blogger Mercedes Schneider (2015) accurately points out that still, ESSA is a test-centered bill, including the expectation that test results will be part of state accountability systems; Title I is worth billions (and states will bow to those billions), and so, the stage is set for a child’s public school education to (continue to) be increasingly devoted to prep for high-stakes tests. . .. Yes, of late, the Obama administration has not pulled NCLB waivers and instituted punishments for states with large opt-out numbers. And yes, ESSA nullifies NCLB waivers. But the problem is that on its face, ESSA pushes for that 95-percent-test-taker-completion as a condition of Title I funding and leaves states at the mercy of the US secretary of education to not cut Title I funding in the face of parents choosing to refuse the tests.

Fundamentally, according to best-selling economist and statistician Charles Wheelan (2013), “Any evaluation of teachers or schools [or students] that is based solely on test scores will present a dangerously inaccurate picture” (p. 51). Part of the problem can be laid at the doorstep of the notion of normative assessment, usually meaning tests that compare one test-taker to his or her peers with the expectation that test scores will follow a “normal” distribution, usually illustrated as a bell curve (see Fig. 6). In a “normal” distribution, most test-takers score somewhere in the middle and a lesser number score somewhat lower or higher. Standardized, norm-referenced tests, in themselves, harm many learners because the tests do not accurately or adequately capture a true portrait of individual learners’ knowledge, understandings, or abilities. When a behaviorist overlay of prescriptive “normality” is imposed, the results are even less reliable as indicators of, well, anything.

300

P. Harris and D. R. Walling

Fig. 6 The so-called normal curve actually derives from a misinterpretation of the Gaussian function (named for Carl Friedrich Gauss, 1777–1855), a distribution in probability theory never intended for prescribing human learning behavior

Fig. 7 The Gaussian bell curve line is shown in comparison to the shaded Paretian distribution

From a neurocognitive viewpoint, learner-centered education can more effectively rely on nonnormative evaluation strategies that focus not on prescribed “normality” but on describing individual learning. Researchers O’Boyle and Aguinis (2012), for example, studied the performance of individuals involved in four broad areas of human endeavor: academics writing papers, athletes at the professional and collegiate levels, politicians, and entertainers. Their findings challenge the “‘norm of normality’ where individual performance follows a normal distribution and deviations from normality are seen as ‘data problems’ that must be ‘fixed.’” O’Boyle and Aguinis suggest, alternatively, that distributions of individual performance – such as the learning of students at various levels of schooling – do not follow a Gaussian distribution but, rather, a Paretian distribution (see Fig. 7). Named for Italian economist Vilfredo Pareto (1848–1923), this “power law” distribution, sometimes referred to as the “80/20 rule,” was originally used to describe the allocation of wealth in Italian society – i.e., eighty percent of the wealth generally rests in the hands of 20% of the population. The distribution has broader applicability. The 80/20 rule is shorthand, not a fixed distribution or a prescription; but it is consistent over many activities involving large groups of people and often

12

Redefining Learning: A Neurocognitive Approach

301

fairly describes smaller groups as well. For example, in a given classroom, a small percentage of students often is responsible for achieving a large percentage of the top marks or, say, on a sports team, where a small percentage of players often is responsible for garnering a large percentage of goals or points. In education contexts, the so-called Pareto principle, rather than prescribing how students ought to perform, can be used to help students monitor their own learning (O’Boyle & Aguinis, 2012). (For a more extensive discussion of this topic, see Walling, 2013.) We contend that standardized tests reduce knowledge and understandings – i.e., learning – to a quantification, a number or set of numbers that provides merely the illusion of certainty but, in fact, offers little insight into actual learning and none with regard to improving teaching and learning. Standardized tests are a politically expedient mechanism for sorting (comparing) students and increasingly teachers, schools, and communities, rather than improving education. Furthermore, such comparisons are biased by nonschool factors, such as poverty or affluence, language, family background and formal education attainment, geography, access to learning resources, and so on. In the words of Harris, Smith, and Harris (2011), these outside-of-school influences tend to flock together like the proverbial birds of a feather. Students whose parents didn’t graduate from high school or don’t speak English in the home tend not to live in wealthy suburbs or in faculty enclaves near universities. Many of them live at or below the poverty level. The scores of the schools these children attend also reflect the influence of these extraschool factors, and in this case, they are likely to depress the scores. (p. 45)

Imagine if standardized tests were eliminated. What “measures” would be meaningful to federal and state policy makers? What are their policy information needs, and how might evaluative measures be constructed to meet them without taking away from time and funds that would be better spent on teaching and learning? Rather than rely on normative standardized testing to evaluate students’ learning, we believe that schools need to develop meaningful measures that are appropriate to specific and highly varied contexts in which learning is desired, expected, or anticipated. Consistent with our contention that education writ large – but also specifically public preK-16 schooling – should be focused on individuals as learners, we direct our attention to this point, rather than to the policymaking needs of state and federal governments. To discuss non-comparative evaluation in this learner-centered framework, it is necessary, first, to define what we mean by meaningful measures and, second, to define what we mean by contexts. Our general definition is that a meaningful measure is a body of information designed to inform the learner about his or her learning progress. There are several key words and phrases in this definition that

302

P. Harris and D. R. Walling

articulate the notion of “meaningful.” To whom? In learner-centered schooling, it is the learner to whom information about learning accomplishments should be most meaningful. The teacher is a secondary recipient, in order to facilitate learning and to design instruction that complements the learner’s learning goals. Other interested parties, from parents to public officials, are tertiary receivers. “Measures” are intentionally open to interpretation. In the current normativeoriented late manifestation of the factory model school, measures have invariably been construed as tests – whether constructed by teachers for classroom use or by textbook publishers or testing corporations for wide-scale applications. However, most such measures are not meaningful at the school or individual level of teaching and learning. After all, standardized tests are not matched to individualized learning goals – or even specific teacher-developed goals – and they are administered, scored, and reported in a manner that ensures that the results will not be available to shape teaching and learning for those who actually took the tests. Standardized test results customarily are reported to schools months later, if not the following year, after a given test has been administered – and usually the results are merely numerical scores that give the learner no useful information to further refine his or her own learning strategies. Meaningful measures, therefore, must fulfill a goal of immediacy. The term formative is applicable, because in order to be meaningful, the measures must be capable of being used to form, or shape, ongoing teaching and learning activities. This point is related to the importance of context. Meaningfulness comes from measures that inform the learner about what he or she is learning at that moment and within a context of skills and subject matter targets, or criteria, not for some time in the past that is no longer relevant except as a record of prior learning. For the learner, this sort of record has limited meaning, because past learning – already acquired knowledge and understandings – has been internalized and is an integral component of present-moment context. The limited meaning it may hold is related almost solely to affect, affirming progress in learning by comparing past and present in the individual’s own development path. In learnercentered practice, incidentally, this is the only valid comparison. Notice that this form of comparison of present to past performance by the individual learner is very different from the concept of comparing learners to each other, which is a form of competition. Learner centeredness, as we conceive it, focuses on the individual learner’s growth in knowledge and understandings toward the learner’s own chosen goals. What measures, then, might prove to be meaningful? If the subject matter and goal contexts are largely developed by the learners themselves, then it follows that evaluation also should be learner centered and learner driven. Consequently, the most useful evaluation to inform future learning will come from the learner’s selfevaluation. In essence this is Pinar’s reconceptualist idea of currere applied to learners. Is this really a radical notion? We don’t believe so. For example, Ross (2006) conducted a review of research evidence on student self-assessment and found that

12

Redefining Learning: A Neurocognitive Approach

303

(1) self-assessment produces consistent results across items, tasks, and short time periods; (2) self-assessment provides information about student achievement that corresponds only in part to the information generated by teacher assessments; (3) self-assessment contributes to higher student achievement and improved behavior. The central finding of this review is that (4) the strengths of self-assessment can be enhanced through training students how to assess their work and each of the weaknesses of the approach (including inflation of grades) can be reduced through teacher action.

McMillan and Hearn (2008) believe that the key to stronger motivation and higher achievement lies in student self-assessment. A central goal of learner centeredness is to increase the individual’s “ownership” of his or her own learning and achievements. “Correctly implemented,” McMillan and Hearn aver, “student self-assessment can promote intrinsic motivation, internally controlled effort, a mastery goal orientation, and more meaningful learning” (p. 40). These qualities are precisely aligned with the shift away from a constrained, bureaucracy-dominated factory model of schooling that depends on extrinsic control of learning and proceeds from a behaviorist reliance on external evidence, such as test scores, to signal, or verify, that learning has occurred. Neuroscience – the underpinning of our proposed definition of learning moves the threshold of evidence to an earlier point in the learning process, showing that activity – therefore some form of learning – is occurring internally (i.e., within the brain). The operant terms, for us, in McMillan and Hearn’s contention are intrinsic and internally controlled. As we discussed earlier, the internal evidence increasingly can be found in ever more sophisticated understandings of brain activity. We make a distinction between self-evaluation and self-assessment, which are often conflated in the literature. We define self-evaluation as an overarching, holistic process, of which self-assessment is one part along with goal setting, selfmonitoring, and reflecting on one’s learning. In Fig. 8 we illustrate a way to understand the cycle of self-evaluation in terms of these four phases: 1. Self-identification of learning goals and criteria. The learner identifies what he or she wants, needs, and intends to learn and the criteria that will identify whether the learning goals have been achieved. 2. Self-monitoring. During learning activities, the learner attempts (and may initially be guided) to be actively aware of his or her own learning and whether such learning directly, indirectly, or peripherally relates to the learning goals – or, indeed, is unrelated but perhaps useful in some other, unintended way. 3. Self-Assessment. At one or more reasonable points during the learning activities, the learner stops to take stock, matching activities and achievement to goals and criteria to assess whether his or her learning is successful and on track. 4. Self-Reflection. In this phase the learner thinks about his or her progress, concluding that the goals have been met or that continued effort, perhaps through revised learning strategies, must be made. If this is an interim point, then the cycle continues with revised goals and criteria. If this is an end point, then a new project can be undertaken and therefore new goals and criteria for that project will be set.

304

P. Harris and D. R. Walling

Self-Identification of Learning Goals and Criteria Initial or revised Iearning targets

Self-Reflection Thinking about progress and revising learning strategies

LEARNING DESIGN AND TEACHER FACILITATION

Self-Monitoring Active awareness of thoughts and activities

Self-Assessment Matching activities and achievement to goals and criteria

Fig. 8 The Learner Self-Evaluation Cycle

As indicated in the figure, learning design and teacher facilitation influence each phase of the cycle. Learner self-evaluation does not come without guidance in the process. Self-identification of goals and learning criteria, self-monitoring, selfassessment, and self-reflection are learned behaviors facilitated by learning designers and teachers. Andrade and Valtcheva (2009) suggest that one way to facilitate effective self-evaluation is through the use of criteria that align with the goals and contexts of learning. When students use criteria-referenced self-assessment, according to these researchers, “The effect can be both short-term, as when selfassessment influences student performance on a particular assignment, as well as long-term, as students become more self-regulated in their learning” (p. 17). What this cycle implies for the learning designer and the teacher, who may be one and the same, is that they also should, as an iteration and extension of currere, move through a similar self-oriented evaluation of their teaching. What goals and criteria should they have for guiding or facilitating the learning of their students? How should they maintain active awareness as learners engage with them and work independently or collaboratively with other students on their projects? Do the guidance and activities match the goals and criteria? When they think about what they have observed and the actions they have taken, how might improvements be made, or is the process complete and it’s time to move on?

12

Redefining Learning: A Neurocognitive Approach

305

Elsewhere we have written: Learning designers approach their work in different ways that resonate with their own education, technical knowledge, learning styles, instructional preferences, and artistic sensibilities. [cf. Pinar’s articulation of currere.] Their work contexts also differ and so too must their approaches to learning design as they work within those contexts. They may or may not be able to articulate the learning theories that ground their practice. Maybe that’s not necessary. Their leadership story may be subtle, implicit, and nuanced in different ways than we hear the leadership stories of others. The learning designer, now and for the future, must be willing to be flexible in practice, building learning designs contextually. . .. (Harris & Walling, 2013, p. 41)

Rolheiser, Bower, and Stevahn (2000) believe that self-assessment engenders greater achievement resulting from self-confidence in individual learning through the “learning goals that students set and the effort they devote to accomplishing those goals. An upward cycle of learning results when students confidently set learning goals that are moderately challenging yet realistic, and then exert the effort, energy, and resources needed to accomplish those goals” (p. 35). When learning designers incorporate self-evaluation and teachers instruct students in how to use the selfevaluation cycle – and use it themselves – then learner-centered practice increases positive affect and cognition for everyone involved. In each phase the learning designer and teacher are instrumental in developing learners’ understandings with regard to the self-evaluation cycle components and in facilitating their use. This process begins with teacher-guided self-regulation of learning and then, as learners gain experience with directing their own learning, becomes more independent self-regulation. Documentation of each phase of the selfevaluation cycle can be maintained through learner and teacher narratives, studentdeveloped texts (such as reports and journals) and other artifacts (such as physical products, videos, and so forth), records of collaboration, and other evidence, which is collected and collated in physical or electronic portfolios. These portfolios become a basis for longer-term self-evaluation by learners as well as for summative evaluation by their teachers. In ESSA, the recent update to federal education law, there is a requirement for states to include at least one nonacademic measure in judging school performance. This requirement is still open to interpretation. However, a well-curated collection of learner portfolios could prove to be not only a fairer measure of student performance but also a far more informative one, as it would incorporate both affective and cognitive indicators of learning and development. Janet Rooney, an award-winning instructor at Manchester Municipal College, discusses “Peer Evaluation and Self-Evaluation” in this Good Practice Exchange video from the Centre for Excellence in Learning and Teaching (CELT) in the UK. The video is available on YouTube at https://youtu.be/ dHm7V-SKFlo.

306

P. Harris and D. R. Walling

New Directions and Further Research Our purposes in this section are, first, to cite a few places where educational practice for the Digital Age is being shaped by the considerations we have discussed and, second, to raise questions that might lead to further inquiry and research about the nature and practice of education for the Digital Age that proceeds from a neurocognitive definition of learning. One direction for future research, for example, might focus on how individuation and motility might be realized in practice. “Just because you have a classroom full of learners who are about the same age doesn’t mean they are equally ready to learn a particular topic, concept, skill, or idea,” writes researcher Margaret SemrudClikeman of the University of Minnesota Medical School (2016). What might result from abandoning same- or similar-age grouping and, instead, using neuroscientific indicators of brain maturity, demonstrated interests or abilities, and other factors in combination when it is necessary to group learners? What if learner groups were fluid, forming and changing according to the needs of learner-driven projects, rather than static and teacher-determined? Multiage and open school/open classroom models have a rich history in both experimentation and, in some places, standard use over many years. What new research linking such concepts to neuroscience might help to frame a reconceptualization of education and to move schooling from the Industrial Age factory model to a new Digital Age model – or cluster of models? One radical innovation dating back nearly a century is the free school movement, taking its impetus from the work of Scottish educator A.S. Neill, best known for his Summerhill School, which was founded in 1921 and still operates, now in Leiston, Suffolk, in the UK (http://www.summerhillschool.co.uk/contact-us.php). Free schools operate on the principle of democracy, in that learners decide among themselves through regular meetings how the school will operate and determine individually what and how they will learn. Neill commented on the founding of Summerhill that “we had one main idea; to make the school fit the child – instead of making the child fit the school” (Neill, 1992, p. 9). A number of free schools based on A.S. Neill’s model can be found in the USA, such as Sudbury Valley School in Massachusetts (http://www.sudval.com) and elsewhere in the UK. Are there aspects of the free school that might be adapted from the small, private school setting to the larger, public school environment? “Free School” is a video overview of the free school movement, Summerhill School, and Sudbury Valley School. It can be found on YouTube at https:// youtu.be/dtSvPWcY5_g. For a different approach to learner-centered schools, Geoff Mulgan, director of the Young Foundation, a center for social innovation, provides “A Short Intro to the Studio School,” which can be found online from TEDGlobal 2011 at https://www.ted.com/talks/geoff_mulgan_a_short_ intro_to_the_studio_school.

12

Redefining Learning: A Neurocognitive Approach

307

Another touch point is architectural innovation. Previously we cited 3XN’s open, flowing architectural plan for Ørestad College, a Danish high school in Copenhagen, as an example of school architecture matching a Digital Age school philosophy closely related to our redefined, neuroscience-based idea of learning. In what other ways might architectural concepts of space intersect with factors such as those we posited as necessary to move toward a neurocognitive reconceptualization of education, namely, individuation, motility, accessibility, and virtuality? How might form follow function? For some starting points to examine school architecture for the Digital Age, readers may want to consider the collection of preK-12 schools at NAC Architecture (http://www.nacarchitecture.com/portfolio/k12-schools.html). NAC has offices in Los Angeles, California, and Seattle and Spokane, Washington. Another group of starting points can be found among the projects of Fielding Nair International, particularly their design for the International School of Brussels High School (http://www.fieldingnair.com/projects/international-school-of-brussels-high-school/). Yet another set of starting points can be found in the projects of Leddy Maytum Stacy Architects, headquartered in San Francisco, California. Their Nueva School at Bay Meadows in San Mateo, California (http://www.lmsarch.com/projects/nueva-schoolbay-meadows), is an example. However, even the most innovative architecture can go only as far as the client’s vision. When that vision is limited by the Industrial Age factory model, then no matter how striking the exterior and interior views of a new construction might be, it will ultimately still be a collection of boxes. Form follows function. How must function change in order for architects to respond with plans that are a match to schools reconceptualized to actualize a neurocognitive definition of learning? Israeli designer and architect Neri Oxman is working at the intersection of computational design, additive manufacturing, materials engineering, and synthetic biology, pioneering a symbiosis between microorganisms, human bodies, products, and architecture. What if educators conceptualized Oxman’s “design” in terms of designing for teaching and learning? Oxman’s TED talk, “Design at the Intersection of Technology and Biology,” can be found the TED website at https://www.ted.com/talks/neri_oxman_design_at_the_intersec tion_of_technology_and_biology. Technology is another touch point for future research. One of our new local coffee shops, like many small startups, relies exclusively on tablet computer technology in place of the traditional cash register. The barista enters the patron’s order on the tablet, then flips it around so that the patron can decide whether to add a tip or not. One touch by the patron – that’s it. If the patron has been there in the past, the swipe of the credit or debit card triggers the tablet to remember the patron’s email address, and a digital receipt of the transaction hits the patron’s online mailbox before the

308

P. Harris and D. R. Walling

coffee even starts to cool. What if this type of technology were broadly incorporated into schools? The location/GPS technology would allow a coordinating teacher to let the computer “take attendance” and, because in learner-centered and learner-directed learning the learner might be anywhere, also locate a given learner at any time – and communicate with the learner electronically by text message to prompt regarding a project, to answer a question, or to coordinate a group meeting. Right now, in many schools, computers are being used to record grades, to send messages to learners and their parents, and to plan instruction; but those applications barely scratch the surface potential of technology-mediated teaching and learning. How might current and emerging or anticipated technologies be employed in reconceptualizing teaching and learning in the Digital Age? Meron Gribetz, CEO at Meta (https://www.metavision.com), envisions “A Glimpse of the Future Through an Augmented Reality Headset,” merging virtuality and neuroscience, in a TED Talk that can be found on YouTube at https://youtu.be/koYLJOyevIE. How might augmented reality change teaching and learning? Learner centeredness encompasses concepts of adaptation, whether through learner-directed projects that match activities to learners’ needs and interests or through adaptive technology that matches device functionality to learners’ psychological/physical learning preferences or requirements. For example, Northwestern University researcher Nina Kraus works in the area of auditory neuroscience and has found that individuals who actively play music tend to hear better in noisy settings and are better able to distinguish target sounds from background sounds. A similar finding notes that bilingual individuals also hear better than individuals who speak only one language. In both cases, the key seems to be auditory discrimination training, whether in music or language, that makes the difference. Such research has implications for teaching and learning with students generally but perhaps, more significantly, with students who evidence language processing-related disorders. Kraus’ work has shown that sound processing in the brain, as evidenced through noninvasive monitoring technology, can be a neurological marker for autism, dyslexia, and other languagerelated learning problems (Dovey, 2015). Proceeding from a general neurocognitive definition of learning as a multidimensional process that creates a changed state in the brain, what types of research might further contribute to our understanding of specific changed states in the brain that signal how some forms of learning are occurring and thus might help learning designers and teachers construct projects and environments to match learning activities to effective brain changes? Matthew Peterson, Ph.D., co-founder of MIND Research Institute, describes the institute’s programs and successes with instructional software using nonlanguage approaches to teach mathematics in a TEDxOrangeCoast talk. It can be found at https://youtu.be/2VLje8QRrwg.

12

Redefining Learning: A Neurocognitive Approach

309

To close this section, we pose a few questions that we believe merit inquiry and possibly serious research as the Digital Age unfolds. The questions are in no particular order. We invite readers to add their own questions, and we hope that researchers, practitioners, and policy makers in the future will find answers – and more questions – to move forward from isolated instances of neurocognitively informed, learnercentered education to broad-scale systemic reconceptualization of public education. 1. What fundamentals of learner centeredness must be in place to constitute a reconceptualization of schooling for the Digital Age? In other words, what’s the tipping point for transformation from the factory model school to a neurocognitively defined, learner-centered model of schooling? 2. What policy changes and bureaucratic reconfigurations will support a largescale shift in public education to the Digital Age model we have posited? 3. If self-evaluation were fully implemented, how might evaluation at the federal and state policy levels be reenvisioned? What roles, if any, might traditional testing corporations and textbook publishers play in such a reconceptualization? 4. Competition and comparison are ingrained in the factory model; consequently, what research agendas are necessary to support and to actualize a non-comparative model of education? 5. As neuroscience continues to develop, do findings support or refute existing theories of human development (e.g., Piaget’s theory of cognitive development)? 6. How must teacher education and the training of learning designers change in order to facilitate a neurocognitively based, learner-centered model of teaching and learning? 7. What communication strategies and community education will be needed for parents and others to understand and fully embrace learner-centered schooling? 8. The shift in the public education model from the Industrial Age to the Digital Age is both massive and intricately nuanced. How can a common vision be shaped that affirms the common good of public education in a democracy? 9. The notion of an open curriculum resides within our reconceptualization of Digital Age education. How open is open, in contrast to the prescriptive curricula of the factory model? To what extent, if at all, might a curriculum still need to be prescribed? 10. Rolling education reform initiatives have been attempted throughout the modern history of education, with recent emphasis on “standards,” meaning mainly content standards. What standards make sense for Digital Age schooling? 11. What is the place of joy in learning? In neurocognitively based, learner-centered schools should educators and policy makers consider affective as well as cognitive goals and standards? 12. How might a national research agenda be constructed to support a purposedriven education system guided by advances in neuroscience? 13. What opportunities can be developed for multinational and cross-cultural collaboration in the creation of educational environments and teaching and learning strategies that resonate with the advance of neuroscience knowledge and understandings internationally?

310

P. Harris and D. R. Walling

14. In science there is a periodic table of elements; should there be a table of educational “elements” that can be identified through the intersections of learning theory, educational practice, and neuroscientific discovery? This is not a comprehensive list of potential research questions, but we hope it is sufficient to stimulate inquiry and even more questions that may prove useful as those who believe that public education is inextricably woven into the fabric of a democratic society attempt to construct education for today and the future.

Conclusion The purpose of this work has been to explore – and to encourage others to explore – a new neurocognitive definition of learning. We posited that this definition could be stated as follows: Learning is a multidimensional process that creates a changed state in the brain. We are mindful that this neurocognitive definition has long roots, stretching deeply into the soil of learning at least to the historical depth of Ancient Greece. The “neuro” element recognizes today’s capacities for examining actions within the brain that we call learning and that, if considered holistically, would lead educators and policy makers to a fundamental reconceptualization of schools and schooling. This reconceptualization can be marked out in certain ways, which we have described as individuation, or essential learner/child centeredness; motility, or systemic fluidity that accommodates physically and intellectually driven mobility within the educational structure; accessibility, or the capacity of learners and educators to use Digital Age technologies for both fundamental and extended learning opportunities; and virtuality, or the seamless integration of computer-mediated learning environments into the education system at all levels. What we envision as this reconceptualization of education from a neurocognitive definition of learning will not be achieved overnight, nor do we have a crystal ball with which to predict exactly its shape. But we can point to other examples in which visions of the future have, over the course of time, been realized. For instance, Leonardo da Vinci, in the late fifteenth and early sixteenth centuries, envisioned airplanes and helicopters that would not be realized as actual working technologies until the twentieth century. In more recent times, Albert Einstein propounded his theory of general relativity in 1915, which among other things predicted the existence of gravitational waves, a phenomenon finally observed in late 2015 and reported in early 2016 by a team of scientists that heard and recorded the sound of two black holes colliding a billion light-years away (Overbye, 2016). In this work we likewise stand on the shoulders of education philosophers, scholars, and researchers – such as Plato, Dewey, Kandel, Pinar, and many, many others – who envisioned learning and schooling focused on learners and how they acquire knowledge and understandings that create internal transformations – our “changed state in the brain” – that are not necessarily immediately manifested in externally observable behavioral changes. We invite readers to ponder the ideas

12

Redefining Learning: A Neurocognitive Approach

311

expressed in this work and to consider further how teaching and learning might be reconceptualized for the Digital Age as it is now and as it stretches forth into the future, rather than perpetuate the outmoded education system of the Industrial Age, which has now passed into history.

How Might Units of Measure Be Defined for a Neuro-Cognitive Definition of Learning? The following story circulated on social media recently: Albert Einstein’s great breakthrough came when he put known measures to one side. The notion that time and space were regular and linear was entrenched in science, and had led to an impasse which prevented it from making sense of the universe. By seeing that time and space might flex led to the Theory of Relativity, and led Einstein into a realisation that philosophical steps must be taken if breakthroughs were to be made. (Quoted by Ravitch 2016)

In “Redefining Learning: A Neuro-Cognitive Approach” we point out that new digital age technologies are now providing insights into the nature of learning that were heretofore inconceivable. But we are as yet stymied when it comes to measuring learning in ways consistent with our neuro-cognitive definition, namely, that learning is a multidimensional process that creates a changed state in the brain (Harris and Walling 2016). In 2015, an international conference of physicists and metrologists convened a workshop on the determination of fundamental constants to share research aimed at better defining an array of “fundamental constants,” which in turn will “aid in the effort to redefine several standard scientific units, including the kilogram and the Kelvin, by 2018” (Phys.org 2015). The need for redefinition was explained as follows: Fundamental constants describe a variety of physical properties in the world around us. Planck’s constant, for example, governs the relationship between energy and frequency. The fine-structure constant explains the strength of electromagnetic interaction between charged particles. Fundamental constants such as these underlie the development of much of today’s technology, from atomic clocks to GPS systems. They are also linked to the International System of Units (SI), the standard measurement system used throughout the scientific community and in most countries around the world. By defining units like the meter in terms of fixed fundamental constants such as the speed of light, we ensure that they remain the same over time. However, some SI units, like the kilogram, still rely on a physical standard—in this case, a platinum-iridium cylinder housed in France. Now that scientific research is carried out across the globe, relying on a single physical standard is somewhat limiting, as mass standards in other countries must be periodically calibrated against the original. In addition, the standard itself is subject to changes in mass over time. To make the system more consistent and accessible, the international metrology community plans to redefine all SI units in terms of fundamental constants by 2018. (Phys.org 2015)

312

P. Harris and D. R. Walling

We are trying to discover the physical properties of learning. Just as advances in science and technology argue for redefining standard units of measure, so too do similar advances in how we define learning offer impetus to redefine units of measure that pertain to learning. A couple of possibilities may be suggested, each possessing both potential and limitations. For example, if – and this is a very big if – neuroscience were to advance to a point that researchers using non-invasive brain imaging technologies could actually identify the type or nature of learning taking place in real time, then it might be possible to construct a system of measures that would refine our definitional “change” taking place in the brain. For the present, however, this possibility remains in the realm of science fiction, not science fact. Brain-imaging technologies, and human understanding of the complexities of the brain and how it functions, simply are not sufficiently advanced and do not seem likely to reach this point in the foreseeable future. Another, more realistic possibility, might be to reexamine behaviorist notions of evidence of learning and to reconceptualize them on the basis of our neuro-cognitive definition. In the section our paper titled, Rethinking Evaluation: Methods and Alignment, we argue that learning currently is “measured” according to the results of various standardized tests, many of which are mandated by federal and state laws. Thus, the units of measure of learning in the current era have been reduced to a hodgepodge of numerical scores, conforming to the notion of normative assessment – that is, comparing one test-taker to his or her peers with the expectation that a body of test scores will follow the so-called normal distribution, or bell curve. This is problematic. We quote statistician Charles Wheelan (2013), who said, “Any evaluation of teachers or schools [or students] that is based solely on test scores will present a dangerously inaccurate picture” (p. 51), to which we add that when a behaviorist overlay of prescriptive “normality” is imposed, the results are even less reliable as indicators – or measures – of, well, anything. What if, as an alternative to prescriptive normality and numerical test scores being perceived as “units of measure,” a system of measurement could be devised based on a framework of meaningful measures? In “Redefining Learning: A Neuro-Cognitive Approach” we intentionally left “measures” open to interpretation, with the following proviso: In learner-centered schooling it is the learner to whom information about learning accomplishments should be most meaningful. The teacher is a secondary recipient, in order to facilitate learning and to design instruction that complements the learner’s learning goals. Other interested parties, from parents to public officials, are tertiary receivers.

We also suggest that meaningful measures must fulfill a goal of immediacy. That is, they must be formative measures, capable of being used to form, or shape, ongoing teaching and learning. In many contexts, units of measure are numerical. However, numbers are an abstraction, further removing the “measure” from its narrative definition. What if units of measure were conceived as narrative descriptions of learning? How might

12

Redefining Learning: A Neurocognitive Approach

313

such units be framed to reflect learners’ self-evaluation, which we would contend is the most meaningful evaluation in a learner-centered context? One might turn, for instance, to a hierarchy of learning for starting points to formulate narrative units of measure. Two such theoretical hierarchies come readily to mind: Bloom’s taxonomy of the cognitive domain (Bloom et al. 1956) and Gagnè’s conditions, or levels, of learning (Gagné 1965). Both theories of learning have undergone revisions, made by the originators as well as by other researchers and scholars. What if we used one of these hierarchies, or a similar theory, to help shape new units of meaningful measure for learning? Gagnè, for example, posits that learning tasks can be organized according to a hierarchy of complexity: • • • • • • • • •

Gaining attention (reception) Informing learners of the objective (expectancy) Stimulating recall of prior learning (retrieval) Presenting the stimulus (selective perception) Providing learning guidance (semantic encoding) Eliciting performance (responding) Providing feedback (reinforcement) Assessing performance (retrieval) Enhancing retention and transfer (generalization)

Each of these features of the hierarchy might be reframed to reflect a learner’s self-evaluation, from “attending to the learning task” (reception) and “stating my learning objectives” (expectancy) in the beginning stages of the hierarchy to “assessing my performance” (retrieval) and “generalizing what I learned to other contexts” (generalization) in the completion stages. Such narrative units of measure – even without further amplification – would provide to teachers and policy makers more information about the substance and quality of learning than a numerical test score. This suggestion offers merely one possibility for defining units of measure. Our purpose in this postscript to “Redefining Learning: A Neuro-Cognitive Approach” is not to arrive at definitive units of measure for a neuro-cognitive definition of learning. Rather, we raise the question: How might such units of measure be defined? We invite other scholars, researchers, and practitioners to respond to this question.

References 3XN. (2016). Ørestad College, Copenhagen, Denmark, 2007. Accessed March 9, 2016, at http://www.3xn.com/#/architecture/by-year/78-%F8restad-college Andrade, H., & Valtcheva, A. (2009). Promoting learning and achievement through selfassessment. Theory into Practice, 48(1), 12–19. Accessed March 30, 2016, at http://dx.doi. org/10.1080/00405840802577544. Baker, V. L., Baldwin, R. G., & Makker, S. (2012, Summer) Where are they now? Revisiting Breneman’s study of liberal arts olleges. Liberal Education 93. Accessed November 10, 2015, at http://www.aacu. org/publications-research/periodicals/where-are-they-now-revisiting-brenemans-study-liberal-arts

314

P. Harris and D. R. Walling

Barber, B. R. (1992). An aristocracy of everyone: The politics of education and the future of America. New York, NY: Ballantine. Beals, G. (1999). The biography of Thomas Edison. Accessed March 9, 2016, at http://www. thomasedison.com/biography.html Berdik, C. (2015, November 19). Can online exchange programs really help kids learn about the world? Slate. Accessed March 21, 2016, at http://www.slate.com/articles/technology/future_tense/2015/11/ online_global_education_initiatives_are_expanding_the_classroom_and_connecting.html Bragg, R. B. (1933). Humanist manifesto I. American Humanist Association. Accessed February 3, 2015, at http://americanhumanist.org/Humanism/Humanist_Manifesto_I Breneman, D. W. (1990). Are we losing our liberal arts colleges? AAHE Bulletin, 43(2), 3–6. Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company. Bloomfield, M. (1995). The automated society: What the future will be and how we will get it that way. Canoga Park, CA: Masefield Books. See also http://massebloomfield.com. Bohr, N. (1987). Unity of knowledge. In The philosophical writings of Niels Bohr: Volume II – essays 1932–1957 on atomic physics and human knowledge. Woodbridge, CO: Ox Bow Press. Center for Digital Education. (2015). Personalized learning. (Issue 4). Folsom, CA: e.Republic. Accessed March 8, 2016, at http://www.centerdigitaled.com/paper/Personalized-Learning-Creating-a-RelevantLearning-Culture-for-the-Next-Generation-8132.html?promo_code=CDE_web_library_list Cichon, J., & Gan, W.-B. (2015). Branch-specific dendritic Ca2+ spikes cause persistent synaptic plasticity. Nature, 520, 180–185. Accessed February 18, 2016, at http://www.nature.com/nature/ journal/v520/n7546/full/nature14251.html#author-information. Cogburn, D. L., & Levinson, N. S. (2003). U.S.-Africa virtual collaboration in globalization studies: Success factors for complex, cross-national learning teams. International Studies Perspectives, 4, 34–51. Accessed March 21, 2016, at https://www.researchgate.net/publication/227759655_USAfrica_Virtual_Collaboration_in_Globalization_Studies_Success_Factors_for_Complex_CrossNational_Learning_Teams. Costall, A. (2006). ‘Introspectionism’ and the mythical origins of scientific psychology. Consciousness and Cognition, 15, 634–654. Accessed March 23, 2016, at http://cspeech.ucd.ie/Fred/docs/ historyOfPsychology.pdf. Cremin, L. (1957). The republic and the school: Horace Mann on the education of free men. New York, NY: Teachers College Press. Crum, W. R. (2010). What can neuroimaging tell us about learning in higher education? Higher Education Research Network Journal, 1, 37–47. Accessed November 10, 2015, at https://kclpure. kcl.ac.uk/portal/en/publications/what-can-neuroimaging-tell-us-about-learning-in-higher-education (415943e7-d05a-45c6-a071-be600e5c2b31).html. Cuban, L. (2004). The open classroom. Education Next, 4(2). Accessed February 23, 2016, at http://educationnext.org/theopenclassroom/. Dewey, J. (2008, 1916). Democracy and education. (Reprint). Carbondale, IL.: Southern Illinois University Press. Dimitriadis, Y., & Goodyear, P. (2013). Forward-oriented design for learning: Illustrating the approach. Research in Learning Technology 21. Accessed March 28, 2016, at http://dx.doi. org/10.3402/rlt.v21i0.20290 Domm, R. W. (2009). Michigan yesterday & today. Minneapolis, MN: Voyageur. Dovey, D. (2015, December 18). Do you hear what I hear? Experiences shape the brain and what you hear may sound different to someone else. Medical Daily. Accessed April 2, 2016, at http://www.medicaldaily.com/do-you-hear-what-i-hear-experiences-shape-brain-and-what-youhear-may-sound-different-366034 Evans, C. C. (1979). The micro millenium. New York, NY: Viking. Gagné, R. M. (1965). The conditions of learning and theory of instruction. New York: Holt, Rinehart & Winston.

12

Redefining Learning: A Neurocognitive Approach

315

Gerrig, R. J., Zimbardo, P. G., Campbell, A. J., Cumming, S. R., & Wilkes, F. J. (2008). Psychology and life (Australian ed.). Sydney: Pearson Education Australia. Getting Ready. (2007, February). Providence, R.I.: Kids Count. Available at http://www. GettingReady.org Glezer, L. S., Kim, J., Rule, J., Jiang, X., & Riesenhuber, M. (2015). Adding words to the brain’s visual dictionary: Novel word learning selectively sharpens orthographic representations in the VWFA. Journal of Neuroscience, 35(12), 4965–4972. Accessed February 18, 2016, at http://www.jneurosci.org/content/35/12/4965.full.pdf+html. González, P. B. (2013). Human nature, allegory, and truth in Plato’s republic. Accessed January 25, 2016, at http://www.kirkcenter.org/index.php/bookman/article/human-nature-allegory-andtruth-in-plato-republic/ Harris, P., Smith, B. M., & Harris, J. (2011). The myths of standardized tests: Why they don’t tell you what you think they do. Lanham, Md: Rowman & Littlefield. Harris, P., & Walling, D. R. (2013, September/October). The learning designer: Merging art and science with educational technology. TechTrends, 57(5), 35–41. Harris, P., & Walling, D. R. (2016). Redefining learning: A neuro-cognitive approach. In Learning, design, and technology: An international compendium of theory, practice, and research. AECT/ Springer. Jaberzadeh, S., Bastani, A., Zoghi, M., Morgan, P., & Fitzgerald, P. B. (2015, July 15). Anodal transcranial pulsed current stimulation: The effects of pulse duration on corticospinal excitability. PLOS one. Accessed February 18, 2016, at http://journals.plos.org/plosone/article?id=10. 1371/journal.pone.0131779. Jefferson, T. (1787). Letter to James Madison. In Paul Leicester Ford (Ed.) The writings of Thomas Jefferson (10 Vols., pp. 1892–1899). New York: G.P. Putnam’s Sons. Accessed March 28, 2016, at http://famguardian.org/Subjects/Politics/thomasjefferson/jeff1350.htm Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Ludgate, H. (2013). NMC horizon report: 2013 (K-12th ed.). Austin, TX: New Media Consortium. Kandel, E. (1989). Genes, nerve cells, and the remembrance of things past. Journal of Neuropsychhiatry, 1(2), 103–125. Klein, A. (2015, April 10). No child left behind: An overview. Education Week. Accessed March 24, 2015, at http://www.edweek.org/ew/section/multimedia/no-child-left-behind-overview-defi nition-summary.html Leinenga, G., & Götz, J. (2015). Scanning ultrasound removes amyloid-ß and restores memory in an Alzheimer’s disease mouse model. Science Translational Medicine, 7(278), 278–233. Accessed March 21, 2016, at http://stm.sciencemag.org/content/7/278/278ra33. Mandler, G. (2002). Origins of the cognitive (r)evolution. Journal of the History of the Behavioral Sciences, 38, 339–353. McMillan, J. H., & Hearn, J. (2008, Fall). Student self-assessment: The key to stronger student motivation and higher achievement. Educational Horizons, 89(1), 40–49. Accessed March 30, 2016, at https://www.jstor.org/stable/42923742?seq=1#page_scan_tab_contents Miller, V. (2011). Understanding digital culture. Thousand Oaks, CA: Sage. Neill, A. S. (1992). Summerhill school: A new view of childhood. Original edition: Summerhill (1960). New York: St. Martin’s Press. Neuroscience News. (2015, July 15). Learning could be revolutionized by noninvasive brain stimulation technique. Author. Accessed February 18, 2016, at http://neurosciencenews.com/ tacs-learning-brain-stimulation-2259/ O’Boyle, E., Jr., & Aguinis, H. (2012). The best and the rest: Revisiting the norm of normality of individual performance. Personnel Psychology, 65(1), 79–119. Accessed April 1, 2016, at http://onlinelibrary.wiley.com/doi/10.1111/j.1744-6570.2011.01239.x/full. Overbye, D. (2016, February 11). Gravitational waves detected, confirming Einstein’s theory. New York Times. Accessed March 7, 2016, at http://www.nytimes.com/2016/02/12/science/ ligo-gravitational-waves-black-holes-einstein.html?_r=0

316

P. Harris and D. R. Walling

Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. New York, NY: Basic Books. Phys.org. (2015, July 14). Advances in determination of fundamental constants to guide redefinition of scientific units. Accessed September 29, 2015, at http://phys.org/news/2015-07-advancesfundamental-constants-redefinition-scientific.html Pinar, W. F. (1975). Method of Currere. Paper presented at the Annual Meeting of the American Educational Research Association, Washington, D.C. (ED 104 766). Accessed February 17, 2016, at http://files.eric.ed.gov/fulltext/ED104766.pdf Pinar, W. F. (2004). What is curriculum theory? Mahwah, NJ: Lawrence Erlbaum Associates. Plato. (360 BCE). The republic. Available at http://classics.mit.edu/Plato/republic.html Phys.org. (2015, July 14). Advances in determination of fundamental constants to guide redefinition of scientific units. http://phys.org/news/2015-07-advances-fundamental-constants-redefinitionscientific.html. Accessed 29 Sept 2015. Ravitch, D. (2016, November 11). James Perry: Just Because It Can Be Measured, Does It Matter? Diane Ravitch’s Blog. http://dianeravitch.net/ Reder, L. M., Liu, X. L., Keinath, A., & Popov, V. (2015). Building knowledge requires bricks, not sand: The critical role of familiar constituents in learning. Psychonomic Bulletin & Review, 23(1), 271–277. Richmond, E. (2012, February 14). Third grade again: The trouble with holding students back. The Atlantic. Accessed March 8, 2016, at http://www.theatlantic.com/national/archive/2012/02/ third-grade-again-the-trouble-with-holding-students-back/253065/ Richta, R. (1967). The scientific and technological revolution. Australian Left Review, 1(7), 54–67. Accessed February 10, 2016, at http://ro.uow.edu.au/alr/vol1/iss7/11/. Richtel, M. (2011, September 3). In classroom of future, stagnant scores. New York Times. Accessed January 21, 2016, at http://www.nytimes.com/2011/09/04/technology/technology-in-schoolsfaces-questions-on-value.html?_r=1 Rolheiser, C., Bower, B., & Stevahn, L. (2000). The portfolio organizer: Succeeding with portfolios in your classroom. Alexandra, VA: American Society for Curriculum Development. Ross, J. A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment, Research & Evaluation, 11(10), 1–13. Accessed March 30, 2016, at http://pareonline.net/getvn. asp?v=11&n=10. Robinson, K. (2011). Out of our minds: Learning to be creative. Chichester, UK: Capstone. Safire, W. (2009). The circuits of neuroeducation: A prolegomenon. In M. Hardiman, S. Magsamen, G. McKhann, & J. Eilber (Eds.), Neuroeducation: Learning, arts, and the brain (pp. 1–3). New York, NY: Dana Press. Schneider, M. (2015, December 2). Every Student Succeeds Act (ESSA) passes House 359–64. Deutsch29. Accessed March 27, 2016, at https://deutsch29.wordpress.com/2015/12/02/everystudent-succeeds-act-essa-passes-house-359-64/ Semrud-Clikeman, M. (2016). Research in brain function and learning: The importance of matching instruction to a child’s maturity level. American Psychological Association. Accessed March 21, 2016, at http://www.apa.org/education/k12/brain-function.aspx Sullivan, L. H. (1896). The tall office building artistically considered. Lippincott’s Magazine (March 1896): 403–409. Accessed January 5, 2016, at https://archive.org/details/tallofficebuildi00sull Toffler, A. (1970). Future shock. New York, NY: Random House. Tzuo, P. W., Yang, C. H., & Wright, S. K. (2011). Child-centered education: Incorporating reconceptualism and poststructuralism. Educational Research and Reviews, 6(8), 554–559. Accessed March 28, 2016, at http://www.academicjournals.org/ERR. Visser, J., & Visser, Y. L. (2000). On the difficulty of changing our perspectives about such things as learning. Paper presented at the Association for Educational Communications and Technology Annual International Convention, Denver, CO. Accessed February 17, 2016, at http://www. learndev.org/dl/DenverVisserVisser.PDF Visser, J., & Visser, Y. L. (2001). Undefining learning: Implications for instructional designers and educational technologists. Educational Technology 12. Accessed February 17, 2016, at https://

12

Redefining Learning: A Neurocognitive Approach

317

www.researchgate.net/publication/234770466_Undefining_Learning_Implications_for_Instruc tional_Designers_and_Educational_Technologists Walling, D. R. (2013). The curse of the bell curve. Unpublished paper. Available at https://www. academia.edu/7488772/The_Curse_of_the_Bell_Curve Walling, D. R. (2014). Designing learning for tablet classrooms: Innovations in instruction. New York, NY: Springer. Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158–177. Accessed September 30, 2015, at http://psychclassics.yorku.ca/Watson/views.htm. Wellman, B. (2002). Little boxes, glocalization, and networked individualism. In M. Tanabe, P. van den Besselaar, & T. Ishida (Eds.), Digital cities II: Computational and sociological approaches (pp. 10–25). Berlin: Springer. Wheelan, C. (2013). Naked statistics. New York: W.W. Norton. Wisconsin Council on Children and Families. (2007, Winter). Brain development and early learning. Quality matters: A policy brief series on early care and education (Vol. 1). Madison, WI: Author. Accessed March 8, 2016, at https://larrycuban.files.wordpress.com/2013/04/brain_dev_and_early_ learning.pdf

Phillip Harris is Executive Director of the Association for Educational Communications and Technology. He previously was Director of the Center for Professional Development at Phi Delta Kappa International, the association for professional educators, and was a member of the faculty of Indiana University for 22 years, serving in both the Department of Psychology and the School of Education. Harris is working actively to preserve the public education system in the USA and is currently working on developing alternative strategies to counter the high-stakes testing movement. His most recent book, co-authored with Bruce M. Smith and Joan Harris, is The Myths of Standardized Tests: Why They Don’t Tell You What You Think They Do, published by Rowman & Littlefield. Donovan R. Walling is an independent scholar, writer, and editorial consultant. He is a senior consultant for the Center for Civic Education and for 13 years was Director of Publications for the education association Phi Delta Kappa International. Walling is the author or editor of 17 books. His most recent book is Designing Learning for Tablet Classrooms: Innovations in Instruction, published by Springer in 2014. Other recent titles include Why Civic Education Matters, Writing for Understanding: Strategies to Increase Content Learning, Visual Knowing: Connecting Art and Ideas Across the Curriculum, and Public Education, Democracy, and the Common Good. He also has contributed numerous articles to professional journals and encyclopedias.

Psychological Framework for Quality Technical and Vocational Education and Training in the Twenty-First Century

13

F. K. Sarfo

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Some of Basic Concepts and Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cognitive Psychology and the Development of Expertise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cognitive Processes of Technical and Vocational Expertise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Knowledge and Skills of Technical and Vocational Expertise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning Processes for Development of TVE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning Theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behavioral Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cognitive Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Situated Cognition Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Constructivist Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Integration of the Learning Theories Toward TVET Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications for Competency-Based Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

320 322 323 324 326 328 328 329 329 333 334 336 338 341 341

Abstract

To cope with the rapid changes in the real world of work and to prepare the workforce to enter the twenty-first century, there is a paradigm shift in the modern Technical and Vocational Education and Training (TVET) toward competencybased training (CBT) or vocational pedagogy. However, there is the lack of literature/research in TVET education that reveals the fundamental understanding and roles of contemporary learning theories in instructional psychology for creating successful competency-based learning environments for quality TVET teaching and learning. To contribute to the solution, this chapter argues that there

F. K. Sarfo (*) Department of Educational Leadership, University of Education, Winneba, Winneba, Ghana e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_65

319

320

F. K. Sarfo

are significant relationships between (1) knowledge and skills and their associated learning processes required of modern TVET and (2) learning outcomes and their associated learning processes of the contemporary learning theories. In accordance with this assertion, an integrated learning theories for quality TVET education is proposed. The logic is that, supported by consistent empirical evidence as discussed in this chapter, the conditions and instructional methods of the learning theories could be successfully used to design powerful competency-based learning environments to promote the development of technical and vocational expertise in TVET teaching and learning for the twenty-first century.

Keywords

Integrated learning theories · TVET · Model of technical and vocational expertise · Expertise development · Competency-based learning environment · 4C/ID model · Competency-based training

Introduction In the world of rapid changes in knowledge and technology, how to facilitate education effectively to promote learning to meet the requirements of the world of work attracts a lot of attention. Technical and Vocational Education and Training (TVET) is recognized as the educational system that seeks to develop competencies in the relevant technical and vocational subjects for the world of work. TVET has been endorsed by most governments and international development agencies (e.g., the United Nations Educational, Scientific and Cultural Organization (UNESCO), Organization for Economic Cooperation and Development (OECD), World Bank, African Development Bank (AfDB)) as being capable of providing the needed workforce for the building and the development of sub-Saharan Africa and other countries worldwide. It has been recognized that the vast majority of the labor force worldwide, including knowledge workers, requires technical and vocational knowledge and skills throughout life (UNESCO 2004). It is therefore affirmed that skills development leading to age-appropriate TVET should be integrated into education at all levels and should no longer be regarded as optional or marginal (UNESCO 2004). However, in the emerging technological and knowledge-driven society, both the nature and the requirements of work in the real world are undergoing tremendous changes (Schleicher 2016; Lucas 2014; van Merrienboer and Kirschner, 2007; Kirschner, Carr, and van Merrienboer 2002): • More and complex occupations are emerging due to fast changes of technological and societal needs. • Machines are taking over the routine tasks, and humans are expected to take over the nonroutine tasks.

13

Psychological Framework for Quality Technical and Vocational. . .

321

• Workers are expected to learn fast on the job, regulate their learning processes, and think critically to help their organizations stand competitive. • Workers are supposed to be creative (innovative and inventive). • Workers are expected to manage their emotion and relate well with others from different cultures. • Changes in availability of resources and equipment required for work. • Changes in workers’ interest and abilities. Changes in the world of work call for vast changes in preparation of TVET teaching and learning. TVET teachers or instructors should be equipped with standard requirements of the world of work, extensive knowledge and skills in their subject areas and methodology to deliver their subject areas. More importantly, to cope with the fast changes in the real world of work, there is a paradigm shift in the modern TVET education toward competency-based training (CBT) or competency-based learning (CBL) or competency-based education and training (CBET) or vocational pedagogy (Kirschner et al. 2002; Lucas 2014). Competency-based training was originated from the United States of America (USA) in the 1960s and grounded in behaviorism and systems approach (Hodge 2007). It gradually extended to the United Kingdom (UK), Australia, Germany, France, Japan, and other countries. The concept of competency-based training/learning or vocational pedagogy in TVET is well discussed in some countries, while in some countries, such as Ghana, it is less discussed (Lucas 2014). Research findings by Sarfo and Elen (2008) showed that there is learning problem in TVET education. This is confirmed by the 2012 Education for All (EFA) Global Monitoring Report (UNESCO 2012) that progress on access to TVET has been improved but that the quality of learning is still low. Among other factors, low quality of learning might stem from the argument (e.g., Kuijpers and Gundy 2011) that behaviorism which is recognized as theoretical model for CBT in TVET education is not sufficient for successful design of competency-based learning environments to cope with TVET learning in the twenty-first century. However, no comprehensive empirical learning theories have been identified in the literature on CBT and TVET education. This observation has been supported and endorsed by UNESCO-UNIVOC (2014) and Lucas (2014) by the assertion that CBT or “vocational pedagogy is under-researched and undertheorised” (Lucas 2014, p. 2). To expand the theoretical basis, and to contribute to the solution of the learning problem as well as the lack of literature on CBT in TVET education, this chapter is aimed at activating discussions on the potentials of contemporary learning theories for designing competency-based learning environments for successful quality TVET education in the twenty-first century. The main intention is to propose integrated learning theories for designing competency-based learning environment (CBLE) for effective and quality TVET teaching and learning at all levels of education. First, the discussion starts with the definitions of the concepts of “TVET,” “expertise,” “competency,” “technical and vocational expertise” (TVE), and “competency-based learning environment.” This is followed by the development of expertise from cognitive psychology perspective. Under this, based on research on

322

F. K. Sarfo

vocational and technical education, cognitive processes and the nature of knowledge and skills of technical and vocational expertise will be identified and described. The identified knowledge and skills for development of TVE are termed as the model of TVE; the paper then describes learning processes that facilitate the development of the identified knowledge and skills of TVET education. Fourth, based on research on learning and instructional sciences, the paper discusses four basic and contemporary learning theories and their associated learning processes related to the model of TVE. This will follow by the discussion on integrating the four learning theories and their associated learning processes for the development of TVE. Sixth, the paper then discusses the implications of the integrated learning theories for (1) designing CBLE to facilitate the development of TVE in TVET and (2) research. This is followed by the conclusion.

Some of Basic Concepts and Definitions Over the years, different terms have been used to describe the elements of the field that is now conceived as TVET. Some of the terms include vocational training (VT), vocational education (VE), vocational education and training (VET), technical education (TE), technical education and training (TET), technical and vocational education (TVE), and professional and vocational education and training (PVET). Recently, UNESCO defined TVET as a comprehensive term referring to those aspects of the educational process involving, in addition to general education, (1) the study of technologies and related sciences and (2) the acquisition of practical skills, attitudes, understanding, and knowledge relating to occupations in various sectors (del Mar 2011). According to del Mar (2011), TVET is an integral part of general education, a means of preparing for occupational fields and for effective participation in the world of work, and an aspect of lifelong learning and preparation for responsible citizen. This implies that, and also in the context of this chapter, any discipline at various levels of education which aimed at preparing students directly for the work of world is classified as TVET. However, it is important to note that TVET programs vary from country to country and they reflect specific socioeconomic requirement. Based on the above definition of TVET (del Mar 2011), it is understood that the main purpose of TVET is to educate and train people to develop expertise to work effectively and efficiently in the world of work. The literature on expertise portrays general consensus about what expertise is. Experts have a great deal of knowledge and skills in their area(s) of specialization. Expertise is knowledge and skills that enable one to function intelligently and smoothly in work situations or everyday tasks (Bereiter and Scardamalia 1993). Expertise manifests itself in many domains (e.g., architectural engineering, civil engineering, mechanical engineering, catering and hospitality, medicine, teaching, nursing, etc.). People who specialize in these areas are normally referred to as technicians, medical doctors, nurses, teachers, caterers, building designers, engineers, etc.

13

Psychological Framework for Quality Technical and Vocational. . .

323

Competencies, according to Kirschner et al. (2002), “can be construed as abilities that enable learners to recognise and define new problems in their domain of study and future work as well as solve these problems” (p. 86). The acquired competencies enable learners to apply these skills and attitudes in a variety of situations and over an unlimited time. Competence is a related term of expertise; competence and expertise are often used interchangeably (refer to the next section). Technical and vocational expertise (TVE) can be described as knowledge and skills that enable workers to function intelligently and smoothly in their work settings. In this chapter, competency-based learning environment is an approach to teaching and learning that helps TVET students to acquire TVE needed to work in the world of work to cope with fast technological and societal changes.

Cognitive Psychology and the Development of Expertise The nature of expertise and the study of expertise development in cognitive psychology in the 1980s could be explained from the work of Chi, Glaser, and Farr (1988). These researchers propose that if experts and novices in a chosen domain are compared, the qualities exhibited by experts but not novices become the basis for explaining expertise. Based on this proposition, various expert/novice research studies have been conducted to investigate problem solving in participants of different ages and examined cognitive mechanisms in various areas including medical diagnoses, mathematics, nursing, mechanical engineering, catering, building design, etc. In this regard, consistent and reliable features of expertise, across the various areas, have been documented (Alexander and Murphy 1998) suggesting that experts: • • • •

Possess extensive, rich, and well-structured domain knowledge Are effective at recognizing the underlying structure of domain knowledge Select and apply appropriate problem-solving procedures for the problem at hand Can retrieve relevant domain knowledge and strategies with minimal cognitive effort

Furthermore, various studies conducted in cognitive psychology suggest that expertise can only be gradually acquired (Bereiter and Scardamalia 1993; van Merrienboer 1997; Flavell 1979) with intentional efforts or deliberate practice (Ericsson 1993). Alexander (2003) further conducted research studies on development of expertise and proposed a model of domain learning (MDL) as a theory of expertise development. The MDL is based on quantitative and qualitative methodologies and crosssectional and longitudinal studies involving students from elementary school through university. Domain investigated included social studies, astrophysics, engineering, technology, etc. According to MDL, in developing expertise in schools, attention should focus on (1) domain knowledge, (2) strategic processes, and (3) interest (individual interest and situational interest). Strategic processes include

324

F. K. Sarfo

surface-level processing strategies and deep-level processing strategies. The three components described should interplay with three stages (Alexander 2003): acclimation, competency, and proficiency/expertise. The acclimation stage is an initial stage of domain expertise. Within this stage, learners have limited and fragmented knowledge; they use surface-level strategies, and their individual interests have limited chance to take form. The next stage termed “competency” is distinguished by a body of knowledge that is cohesive and principled in nature. They use a mix of surface-level strategies and deep processing strategies; their personal interest in the domain increases and their situational interest decreases. The third stage is labeled proficiency/expertise. The knowledge base within this stage is both broad and deep. Experts use deep processing strategies and they have a very high interest in the domain and the reliance in the situational interest levels off. Components of domain knowledge, strategic processes, and interest configure differently as an individual progresses from an acclimation stage to competence stage and to proficiency or expertise stage. Alexander (2003) explicitly describes the cognitive and noncognitive features of expertise across various domains.

Cognitive Processes of Technical and Vocational Expertise In relation to cognitive assumptions of expertise development, Lindekens, Heylighen, and Nueckermans (2003) conducted an empirical study in which four architects – two novices and two expert designers – were asked to develop a concept for the reorganization of and the extension to an architectural school. The subjects were asked to “think aloud” while designing. During the session, all actions of the designers were recorded. The intention was to reveal the cognitive processes of building designers. The results of the analysis revealed that: • Experts reason on the concepts and principles of building drawing continuously until the very end of the session. • (Expert) designers refer to the basic principles of architectural design (e.g., materials, symbols, economic, volume) when designing. • Expert building designers display four categories of strategies: (1) analysis, (2) synthesis, (3) evaluation (the designer switches between these three categories of strategies), and (4) explicit strategies (organization of tasks before design starts, examining how they should cope with different tasks and how they should continue the design). • While sketching/drawing, the designers’ decisions and choices are based on the problem brief; some of them are based on the basic principles or their own preconceptions. • Decisions are sometimes very clear and architects do not seem to doubt their choice. At times they suggest a solution for part of the design and continue this

13

Psychological Framework for Quality Technical and Vocational. . .

325

line of thought and find out whether it also offers a solution for other parts. If so, they continue their proposal. If not, it is rejected and another proposal is chosen for evaluation. Still at other times, different possibilities are considered simultaneously. In a similar direction, Casakin (2004) conducted an empirical study to investigate the use of visual analogical reasoning by novice and expert architectural designers during the design process. Twenty-six architectural designers participated in the experiment: eleven expert architects and fifteen novice architects. On the one hand, the analysis of the cognitive processes revealed that (1) during drawing/ designing, novice designers reproduced almost exact copies of the source provided and focused on surface properties which did not lead to a successful solution; (2) novice designers failed to retrieve a structured principle and establish an analogy with problem. On the other hand, analysis of the cognitive processes also revealed that (1) while sketching and drawing/designing, expert designers did not copy exactly what was provided; instead they manage to activate their memory and retrieve knowledge related to the row house organization, and (2) while designing, expert designers decided to add further constraints than those that were required in the original goals. They refined their sketches or drawings where necessary. In addition, to describe practice behaviors or cognitive processes of expert technicians, Cross (2004) reports three empirical studies that reveal the cognitive processes of three successful/expert designers from three different domains of design: bicycle luggage carrier, sewing machine, and racing car. Comparative review of the three studies indicates that the cognitive processes of TVE are similar. The above research findings (e.g., Alexander 2003, Lindekens et al. 2003, Casakin 2004, and Cross 2004) provide empirical evidence that while cognitive processes of expert technical and vocational workers are similar in different domains, cognitive processes of expert technical and vocational workers (or students) are qualitatively different from cognitive processes of novice vocational and technical workers (or students). Table 1 presents expert (technical and vocational workers/students) cognitive features that are absent in novices. The research findings further reveal that most of the cognitive activities of expert technical and vocational workers occur simultaneously. Table 1 Experts’ cognitive features that are absent in novices Expert technical and vocational workers/students Use basic structured concepts, rules, and principles of domain knowledge Use conceptual and functional reasoning on the domain knowledge Use rules of thumb, reflective strategies, and problem-solving strategies when solving problems Use basic domain principles/rule-based behaviors (e.g., application of standards and symbols) and reflective strategies simultaneously (expert building designers reflect on client needs or problem brief while drawing/designing a building plan) Use analysis, evaluation, synthesis, and explicit strategies

326

F. K. Sarfo

Knowledge and Skills of Technical and Vocational Expertise As discussed in the previous section, the execution of a task by expert technical and vocational workers is a highly complex activity that requires execution of varied and integrated set of knowledge and skills. According to van Merrienboer (1997), the body of knowledge that constitutes integrated set of knowledge and skills, also called complex cognitive skills (or complex vocational and technical skills), consists of nonrecurrent skills and recurrent skills. Figure 1 presents a model of technical and vocational expertise that highlights this. For instance, in building design, the nonrecurrent aspects pertain to reasoning on the conceptual and functional principles of building drawing, reflective practice, and the use of rules of thumb by expert building designers; and the recurrent aspects pertain to the use of rule-based behavior as well as application of symbols, dimensions, procedures, and other routines by expert building designers. The nonrecurrent skills can be described in terms of cognitive schemata, and the recurrent skills can be described in terms of cognitive rules or automated schemata (van Merrienboer, Clark, and de Crook 2002). Cognitive schemata consist of highly structured domain declarative knowledge (mental model), cognitive strategies (van Merrienboer et al. 2002), and metacognitive strategies as shown in light gray color background in Fig. 1. Cognitive schemata direct problem-solving behavior and allow for reasoning in the domain. A highly structured domain declarative knowledge is mental model in which the nodes may be facts, concepts, plans, or principles that are related to each other non-arbitrarily. For instance, a highly structured declarative knowledge about causeeffect relationships of different kinds of soil and types of foundation enables an expert building designer to choose the right foundation. Cognitive strategies can be described as general strategies of solving problems (Derry 1990). They are strategies employed by learners in a particular learning situation to facilitate the acquisition of knowledge and skills or to carry out a

Technical and Vocational Expertise

Non-recurrent skills and recurrent skills

Well- structured and organised knowledge and skills Procedural Knowledge

Declarative Knowledge

Fig. 1 Model of technical and vocational expertise

Cognitive strategies

Metacognitive skills

13

Psychological Framework for Quality Technical and Vocational. . .

327

complex task. Cognitive strategies consist of systematic approaches of problem solving (SAPS) and rules of thumb or heuristics (van Merrienboer et al. 2002). Experts apply cognitive strategies on the domain knowledge when solving a problem. For instance, in designing an electrical circuit, the expert electronic has to think about and identify the goal of the design, think of the appropriate solutions, select the right solutions, and then execute the solution. Or sometimes, if a learner is solving a question, he has to think and choose the correct mental tactics that he thinks will enable him to solve the problem. The exploration of cognitive processes of vocational and technical expert (e.g., architectural) designers indicated the presence of self-reflection, self-monitoring, selfevaluation, and self-regulatory skills – metacognitive skills – in problem-solving behavior of expert building designers. Metacognitive knowledge and skills were originally described by Flavell (1979). Metacognitive knowledge is described as learners’ awareness and knowledge of their own learning processes (cognitive strategies); and metacognitive skills are described as learners’ abilities to control these learning processes during learning/problem solving. For instance, in the course of aspirating joint fluid with large bore needle, by employing metacognitive knowledge and skills, the learner or an expert medical doctor may reflect on his cognitive schemata and the goal of the problem and realize that he is not using the right method to achieve the goal and therefore adjust his selection of method. The cognitive schemata acquired in former problem-solving situations (e.g., in classroom context) may help to solve the non-familiar aspects of current problem situation (van Merrienboer 1997). The body of knowledge of technical and vocational expertise that pertains to the recurrent aspects of the constituent skills is termed as automated schemata or cognitive rules or procedural knowledge structure (van Merrienboer 1997). The procedural knowledge structure as shown in white background color in Fig. 1 links particular characteristics of the problem situation (condition) to particular actions. Experts may reach a level of practice where they execute recurrent skills or routines automatically without investing any mental power or cognitive effort. For instance, an expert building designer may display open symbols and use room dimensions automatically (without conscious control). Automated schemata acquired in former problem-solving situations (e.g., in a classroom context) help to solve the familiar aspects of current problem situation (van Merrienboer 1997). According to Fairey (1960), expert vocational and technical workers have extensive knowledge and skills (both recurrent and nonrecurrent skills) in their mother tongue that enable them to organize, direct, instruct, and interact with others effectively in the domain. To summarize, the answer to the question “what are the knowledge and skills that must be mastered by students in technical and vocational education in order to become an expert?” is recurrent and nonrecurrent skills that comprise: • Well-structured and organized domain-specific knowledge (declarative and procedural) • Cognitive strategies • Metacognitive skills

328

F. K. Sarfo

Experts apply cognitive strategies and metacognitive skills on domain declarative and procedural knowledge when solving problems. The directions of the arrows in Fig. 1 highlight this. As discussed above (e.g., Lindekens et al. 2003) when expert technical and vocation workers are solving problems, they execute most of these knowledge and skills simultaneously. Therefore, they should be developed concurrently (van Merrienboer 1997).

Learning Processes for Development of TVE Various psychological functions must be performed if learning is to be effective. These functions are termed as learning processes (Shuell 1988). In order to understand what students must do in order to be successful at learning from instruction (competency-based learning environment), certain characteristics of learning processes need to be considered (Elen 1995). Nonrecurrent and recurrent constituent skills are qualitatively different in nature in the sense that they perform different functions in expert problem solving (e.g., designing a building plan). In this regard they are qualitatively different in desired exit behaviors. Different but simultaneous (because experts often apply nonrecurrent and recurrent skills simultaneously) learning processes may be responsible for the acquisition. Elaboration and induction are the main learning processes that promote schema construction for development of nonrecurrent skills (van Merrienboer et al. 2002). Moreover, rule automation leads to the development of procedural knowledge structures (cognitive rules), which are responsible for development of recurrent constituent skills (or reproductive skills) that involve rule-based behavior. Automation is mainly a function of the amount and quality of practice that is provided to the learners and eventually leads to automated rules or cognitive rules that directly control behavior (van Merrienboer et al. 2002). Restricted encoding, chunking/ compilation, and strengthening are the main learning processes that promote rule automation (van Merrienboer 1997). In addition, from a cognitive information processing point of view, students can only process seven plus or minus two bits or chunks of new information (Miller 1956). This is due to limited attentional resources and limited memory capacity (Anderson 1983; van Merrienboer 1997). The learning of technical and vocational skills is constrained by this limited amount of mental capacity. The next section describes learning theories and their associated learning outcomes and processes which are in line with the proposed model of technical and vocational expertise and their related learning processes.

Learning Theories Psychology is a field or discipline which provides intellectual, philosophical, scientific, and practical descriptions and prescriptions of learning, how people learn and what can be done to help people learn to acquire technical and vocational expertise.

13

Psychological Framework for Quality Technical and Vocational. . .

329

Learning can be defined as relatively outward change of external capabilities which are constructed internally (in the mind) as the individuals engaged in both mental and social activities (Sarfo 2011). Learning theories explain how learning occurs and what can be done to promote the development of learning goals/outcomes. Over the past years, there had been several dominant theories of learning. Different learning theories have different descriptions and prescriptions for (1) how different learning outcomes are acquired/developed, (2) learning processes that facilitate the development of different learning outcomes, and (3) what can be done to facilitate different learning processes to promote the development of different learning outcomes in students. In the following section, the basic and contemporary learning theories such as behavioral, cognitive, situated cognition, and constructivist learning theories in line with their learning goals and processes related to the acquisition of different learning outcomes (technical and vocational expertise) are described. Table 3 highlights this.

Behavioral Learning Theory Behavioral learning theorists (e.g., Skinner 1974, 1958) proposed that psychology of learning is better understood by exploring the functional relationships between environmental variables and behavior (Skinner 1974, 1958). Skinner (1958) identified two types of reinforcement: positive reinforcement and negative reinforcement. Reinforcer (either negative or positive) according to Skinner is anything that increases the likelihood of a behavior (learning) happening again. Reinforcement is central learning process to Skinner’s behavioral learning theory, and it always results in behavior increase. After new behavior has been learned and strengthened by behavioral principles of reinforcement such as shaping and chaining, schedules of reinforcement are useful and effective for maintaining such behaviors (refer to Driscoll 2005 for more on behaviorist learning theory). For behaviorists, learners exhibit or learn predetermined desirable observable behaviors by the society (e.g., teacher, industry, organizations) which are reinforced and rewarded. Behavioral reinforcement learning principles are effective for teaching learners to reassemble weapons (Driscoll 2005). As depicted in Table 2, principles of reinforcement promote the development and acquisition of procedural and declarative knowledge, recurrent skills (rule-based or practice-based behaviors or reproductive skills or automated rules) for instance in medicine, building drawing, electronics, catering, teaching, chemical engineering, etc., in TVE.

Cognitive Learning Theory Cognitive learning theory derived from Gestalt psychology is based on the notion that true understanding occurs only through the reorganization of ideas and

330

F. K. Sarfo

Table 2 Learning theories and their associated learning processes Learning theory Behavioral learning theory

Cognitive learning theories Information processing Meaningful learning Schema theory Situated cognition learning theory

Constructivist learning theory Social constructivism Social constructionist Cognitive constructivist

Learning outcome Domain knowledge Declarative knowledge (mastery of facts, concepts, principles) Procedural knowledge Domain knowledge Declarative knowledge Procedural knowledge Meta knowledge/skills Cognitive strategies, learning strategies, problem-solving strategies, metacognitive strategies or control strategies

Domain knowledge Declarative and procedural (contextual) knowledge Meta knowledge/skills Heuristic strategies, control strategies, learning strategies Domain knowledge Declarative and procedural (contextual) knowledge Meta knowledge/skills Pro-social skills, interpersonal skills, control strategies, transferable skills, creative skills Problem-solving strategies, critical thinking skills, rules of thumbs, personal inquiry skills

Body of knowledge Automated rules

Cognitive schemata Automated schemata

Cognitive schemata Automated schemata

Cognitive schemata Automated schemata

Learning process Reinforcement, shaping, chaining, schedules of reinforcement, strengthening

Attention, pattern recognition, rehearsal, chunking, encoding retrieval, selection, organization, integration, Correlative subsumption, superordinate, combinatorial Accretion (elaboration), tuning, restructuring (induction) Enculturation, LPP, the process of interpreting signs

Structuring and restructuring of knowledge, dynamic nature of knowledge, learning by design

perceptions, not through memorization and conditions. This section describes information processing learning theory, meaningful learning and schema theory under cognitive learning theory.

Information Processing Learning Theory Cognitive information processing (CIP) learning theory is focused on how stimulus (information) or inputs from the environments are perceived, processed, stored, and retrieved or manipulated (in the mind) to solve (complex) tasks. According to Atkinson and Shiffrin (1968) during learning, the learners perceive the stimuli (information from the environments) through their sensory receptors. Attention causes the information stored in the sensory register to be passed along to the short-term memory (STM) through pattern recognition. The short-term memory

13

Psychological Framework for Quality Technical and Vocational. . .

331

Table 3 Integrated learning theories for quality TVET education Learning theories Behavioral Declarative knowledge (facilitated by reinforcement, schedules of reinforcement, strengthening) Procedural knowledge (facilitated by chaining, shaping, reinforcement)

Cognitive Cognitive schemata Declarative knowledge, cognitive strategies, learning strategies, problem-solving strategies, metacognitive knowledge and skills, control strategies (facilitated by attention, rehearsal, encoding, integration, elaboration, tuning restructuring, etc.) Automated schemata Procedural knowledge (facilitated by encoding, chunking, strengthening, etc.)

Situated cognition Cognitive schemata Declarative knowledge, heuristic strategies, control strategies, learning strategies (facilitated by legitimate peripheral practice, enculturation, sign interpretation) Automated schemata Procedural knowledge (facilitated by enculturation and LPP)

Constructivist Cognitive schemata Declarative knowledge Pro-social skills, interpersonal skills, control strategies, transferable skills, creative skills, problem-solving strategies, critical thinking skills, rules of thumbs, personal inquiry skills (facilitated by structuring, restructuring, interpretation, elaboration, induction) Automated schemata procedural knowledge

Model of technical and vocational expertise Recurrent and Recurrent and nonrecurrent skills nonrecurrent skills Declarative and procedural knowledge Declarative and Cognitive strategies procedural Metacognitive strategies (facilitated by elaboration, rehearsal, attention, knowledge chunking, induction, compilation, strengthening, accretion, encoding, (facilitated by correlative subsumption, elaboration, interpretation of signs LPP, reinforcement and/or enculturation, etc.) strengthening)

can only store seven plus or minus two numbers at a time (Miller 1956). However, the working memory capacity may be increased through creating large bits, known as “the process of chunking” (Driscoll 2005, p. 87). If the information in the shortterm memory is not rehearsed within 15–30 s, it would decay. The rehearsed information in the STM gets stored in the long-term memory (LTM) through semantic encoding (Gagne 1985). Encoding is described as the process of relating incoming information to concepts and ideas already in memory in a way that the new material is more memorable (Gagne 1985; Mayer 2002). The LTM, unlike the STM and sensory memory, has unlimited capacity. The outcome of successful cognitive processing of information is the construction of a mental model or coherent mental representation of declarative and procedural knowledge structure (Mayer 2002). Through the process of retrieval, the encoded information previously stored in the LTM may be returned to STM also known as working memory either for the purposes of combining with other information to bring new learning or for making a response.

332

F. K. Sarfo

In line with the limited capacity of human cognitive architecture, Miller (1956) and Sweller and Chandler (1994) have distinguished between intrinsic and extraneous source of cognitive load. The limited cognitive resources force learners or problem solvers to make decisions during active learning (Mayer 2002). These decisions include which pieces of incoming information to pay attention to and the degree to which the learners should build connection between selected pieces of information and the existing knowledge. Mayer (2002) asserts that metacognitive strategies are techniques for allocating, monitoring, coordinating, and adjusting these limited cognitive resources. Gagne (1985) conceives metacognitive strategies as cognitive strategies, and he indicates that executive control structure governs the use of cognitive strategies. (For more on this, read Mayer 2002; Driscoll 2005; Gagne 1985.)

Meaningful Learning Meaningful learning also known as reception learning is a component of cognitive learning theory developed by David Ausubel (1963). Ausubel connected the idea of conceptual scheme by Piaget to his explanation of how learners acquire knowledge. Ausubel (1960) subsumption theory asserts that learner’s existing knowledge is the principal and basic factor influencing the learning and retention of meaningfully new materials. The subsumption theory describes the need to relate new information to learner’s existing cognitive structure before the new information is presented. This proposition of Ausubel is directly associated with the advanced organizer developed by Ausubel (1960). Ausubel (1963) proposed four learning processes through which learning occurs. They are derivative subsumption, correlative subsumption, superordinate learning, and combinatorial learning (Table 2 depicts this). Derivative subsumption is described as the situation in which the new information learners learn is an instance or example of a concept that learners have already learned. Correlative subsumption involves the alteration of the new concepts to include more instances. Superordinate learning is a situation whereby the learners knew a lot of examples of a concept, but they did not know the concept until it was taught. Combinatory learning process is a learning process by which the new concept is derived from another concept that comes from learners’ previous knowledge in a different but related branch. Learning by analogy is an example of combinatory learning process. The four learning processes are internal learning processes through which new information is incorporated into cognitive structure by associating it to anchoring ideas. With respect to model of knowledge, Ausubel proposed cognitive structure as learners’ overall memory structure. The main learning outcomes for meaningful learning are organized conceptual knowledge and skills of the domain that involves meaningful understanding. Schema Theory Schema theory by Rumelhart and Norman (1976) is a more polished elaborate and modern form of meaningful learning (Driscoll 2005).

13

Psychological Framework for Quality Technical and Vocational. . .

333

According to Rumelhart and Norman (1976), schema is described as memory structure that contains the records of human experiences or information. Schema consists of a network of interrelations among its constituent parts which themselves are schemata (Rumelhart and Norman 1976). Schema theory is a theory of how schemata are acquired and presented in our memory and how the presentation facilitates the use of knowledge in a particular way. Schemata are active in promoting learners’ interpretation of events and problem-solving processes, and as such they are also known as mental model (Driscoll 2005). Mental models provide a basis for reason. The learning processes that facilitate schema construction and automation are accretion/elaboration, tuning, and restructuring/induction (Rumelhart and Norman 1976). Learning by accretion is described as learning by adding new information to the existing schema (database of memory), following the organization of alreadypresent schema. Tuning is learning by developing new schemata, based on the existing schemata, by minor change. It involves a modification of facts/concepts about a topic under study and marks acquisition of new conceptualization. Restructuring is learning by erecting or creating entirely new schemata to replace or incorporate the existing schemata to deal with “troublesome” information (Rumelhart and Norman 1976). In this case restructuring, also, sometimes involves tuning. Activating prior knowledge which is similar to advanced organizer by Ausubel has significant influence on schema construction and activation. Cognitive learning processes such as attention, rehearsal, encoding, combinatory, comparative, accretion (elaboration), tuning, and restructuring (induction) (refer to Table 2) promote schema construction and the development of declarative and procedural knowledge, metacognitive skills, and cognitive strategies in catering, building design, medicine, teaching, nursing, fashion design, civil engineering, etc., in TVE in TVET education.

Situated Cognition Learning Theory Brown, Collins, and Duguid (1989) were among those who made situated cognition receive important attention in the community of instructional psychology. Lave and Wenger (1991) also made significant contribution to situated cognition learning theory. Brown et al. (1989) proposed that most of the traditional teaching practices result in the inability of students to use what they know in relevant situation (world of work). They argue that students in traditional schools learn knowledge in decontextualized way. Brown et al. (1989) challenge the separation of what is learned from how it is learned and used. They assert that the activity in which knowledge is developed and deployed is not separable from learning and cognition. Activity, content (concepts), and culture are interdependent. Learning and cognition are fundamentally situated. It is useful to consider conceptual knowledge, to some great extent, as similar to a set of tools (Brown et al. 1989). Tools can only be fully understood through use. And the

334

F. K. Sarfo

users have to change their views of the world and adapt their belief system of the culture in which the tools are used. It is believed that unlike other learning theories, in situated cognition theory, cognition takes place within the world not in the minds of the individual learners (Whitson 2005; Driscoll 2005). Therefore, cognitive from situated perspective is a semiosis (Whitson 2005). According to Whitson (2005), semiosis is described as continuously dynamic and productive of signs (a sign is anything that stands for something else). In other words, semiosis is any form of activity, conduct, and process that involves signs. Situated cognitive learning theories emphasize three learning processes (Table 2 highlights this): learning as a process of enculturation (Brown et al. 1989), learning as a process of legitimate peripheral practice (Lave and Wenger 1991), and learning as process of interpretation of signs (Driscoll 2005; Whitson 2005). Learning by the process of enculturation indicates that the culture and the use of a tool act simultaneously to determine the way the practitioner views the world. In this respect to learn to use tools as practitioners use them, a student as apprentice must enter the community and its culture. Lave and Wenger (1991) assert that learning as a situated activity has its central defining learning process referred to as legitimate peripheral participation (LPP). In addition to legitimate peripheral practice, Lave and Wenger (1991) identified that learning could occur in the form of apprenticeship. Situated cognitive learning theories and their related learning processes are effective and efficient for the development of (1) declarative and procedural knowledge (domain knowledge), (2) heuristic strategies and control strategies, and (3) learning strategies in teaching, nursing, catering and hospitality, civil engineering, medicine, etc., for development of TVE in TVET. Learning as a process of enculturation and LPP furthermore depicts the significance of “workplace experience learning (WEL),” “industrial attachment,” “off-campus teaching practice,” and “internship program” in TVET education.

Constructivist Learning Theory Constructivism applies to both philosophy and learning. Since the focus of this chapter is psychological framework for instruction and learning of TVET, constructivist learning is more considered. In this section constructivist learning theory is described in terms of social constructivist learning, social constructionist learning, and cognitive constructivist learning.

Social Constructivist Learning Theory Learning theorists (e.g., Bruner 1964; Vygotsky 1978; Collins et al. 1989) propose that learning is considered as both social and cognitive activities. Learning is a social enterprise in that learners construct ideas, social tools, and language concepts cognitively through the interaction of individual and culture. Learning is largely mediated by social interaction between students and more knowledgeable others

13

Psychological Framework for Quality Technical and Vocational. . .

335

(teachers, parents, coaches mentors, peers, experts, etc.) as well as internalization of socially mediated understanding to become personal knowledge (Vygotsky 1978; Bruner 1966).

Social Constructionist Learning Theory Bruner (1964) proposes that discovery as a learning process involves an expectation of finding regularities and relationships in the environments. In the course of discovery, the learner should be able to connect the symbolic (information) to the enactive and iconic models meaningfully. This led to the origination of constructionism (Driscoll 2005; Brunner 1966). Constructionist learning theory (Papert 1991) is recently used in the literature on instructional psychology. Constructionism suggests that new ideas are most likely to be created as learners are actively building some type of external artifact (e.g., car engine, table, building plan) that they can reflect upon and share with others (Papert 1991). Constructionist learning emphasizes the value of learning through (or by) creating, programming, or participating in other forms of designing that will result in the development of internal artifacts. Cognitive Constructivist Learning Theory The constructivist conception that thought is embodied is in line with cognitive constructivism which is focused on the thinking (mental) activities of the learners by Piaget. Regardless of what is being learned, constructive processes operate and learners form, elaborate, and test their mental structure until a satisfactory one emerges (Perkins 1991). In line with cognitive constructivism, Spiro, Vispoel, Schmitz, Samarapungavan, and Boerger (1987) propose cognitive flexibility theory which postulates that knowledge is not simple and orderly as it is thought of. Simple representation of knowledge will miss important facet of complex concepts. Knowledge that will have to be used in many ways (in real-life situations) has to be learned, represented, tried, and applied in many ways. Therefore, multiple representations are very useful in understanding complex individual concepts (Spiro et al. 1987). Cognitive flexibility involves learners’ ability to select and use knowledge to adaptively fit the needs of understanding and decision-making in a particular domain. Cognitive flexibility promotes the development of personal inquiry skills, skill for solving ill-structured problem, and knowledge transferability (Spiro et al.). Constructivist learning theorist, Cunningham (as cited in Driscoll 2005), declares that there is no particular organizational structure of cognitive models in the memory of learners. The models or knowledge in the minds of the learners constantly changes shape, and at every point, it appears to be connected with every other part (Driscoll 2005). Constructivist learning theories and their associated learning processes promote schema construction as well as the development of highly structured procedural and declarative knowledge, cognitive strategies, and metacognitive strategies in, for instance, pharmacy, teaching, mechanical engineering, and business administration for acquisition of VTE in TVET.

336

F. K. Sarfo

Integration of the Learning Theories Toward TVET Learning With regard to their functions in the development of TVE, some of the learning theories have important relationships with others, some are consistent with others, and there are differences among some of them. But each theory has a learning process(es) that is related to the model of TVE (refer to Table 3). This therefore implies that all the learning theories are potential for the development of some or all the component(s) of the TVE depending on the interest, the goal, and the level of a given TVET program. For instance, there is established research evidence (e.g., Anderson 1983; Ericsson 1993; van Merrienboer 1997) that repetition and practice with feedback based on the principles of behavioral and cognitive learning theories facilitate the compilation/chunking or restricted encoding and strength of skills and hence promote the development of automatic or reproductive skills in TVE. However, unlike the principles of cognitive learning theories, principles of behavioral learning theories do not directly contribute to acquisition and execution of complex thinking skills and creativity in TVE. But as processing of skills becomes more automatic, the requirements for operating space in the memory diminish, allowing for more storage space (Case 1984) for the performance of complex thinking in the development of TVE. This indicates that behavioral learning theories and their related learning processes directly and indirectly contribute to the development of TVE. Situated cognition and constructivist (e.g., social constructivist and constructionist) learning theories have direct and strong positive significant impact on acquisition of contextual knowledge and skills (both domain and meta knowledge and skills) (Collins et al. 1989, Papert1991; Vygotsky 1978). Unlike other learning theories, they directly and importantly contribute to the transferability of knowledge and skills in real-life situation. Therefore their learning processes are strongly advocated for schema construction and schema automation for the development of TVE in TVET education. Social constructionism (Papert 1991) is focused on the artifacts that are created or constructed through the social interaction of a group, while social constructivism (Vygotsky 1978) is focused on an individual’s learning (construction of knowledge and skills) that takes place because of their interactions in a group. It is argued that social constructionist and social constructivist functions in development of VTE are closely related. This is because learners work together to construct technical and vocational expertise (an integrated set of knowledge, attitudes, and skills – recurrent and nonrecurrent skills (Fig. 1 and Table 3)) individually in the mind to enable them solve (real life) problems and also create artifacts that are observable and directly useful (to reduce human discomfort) in real-life situation. Enculturation, LPP, learning by design, interpretation of signs, structuring, and restructuring learning processes of situated cognition and constructivist learning theories (refer to Table 3), as indicated already, facilitate workplace experience learning (WEL), internship and attachment programs, and off-campus teaching practice in TVET education. Even though the transferability of knowledge and skills is critically important, it depends upon a true understanding of concepts, principles, and facts (Driscoll 2005).

13

Psychological Framework for Quality Technical and Vocational. . .

337

Among the learning theories, cognitive constructivist learning theories and more especially cognitive learning theories (e.g., meaningful learning and schema theories) and their associated learning processes more effectively contribute to a true understanding of concepts, principles, and facts (Driscoll 2005; Ausubel 1960 Rumelhart and Norman 1976). Conceptual understanding and cognitive flexibility play a very significant role in the development of technical expertise (Sarfo 2011). For instance, in a study conducted by Balasubramanian and Wilson (2007), students indicated that in learning by creating artifacts in real-life (or authentic) situation as advocated by social constructivist and constructionist learning theories, putting stuff together was easy; they did not have to think as much, not have to write as much, and just have to pay attention instead of to read a lot of stuff. This demonstrates that in learning by creating artifacts in social constructivist and constructionist contexts, students either disregard their cognitive activities or do not apply the domain conceptual knowledge; and this may lead to inadequate conceptual understanding (Mayer 2004; Sarfo 2011). And this can be interpreted to mean that social constructivist and constructionist learning alone might support some learners to become merely traditional craftsmen instead of competent problem solvers or reflective and creative practitioners (technical and vocational experts). Cognitive, situated cognition, and constructivist learning theories contribute to the acquisition of meta knowledge and meta skills (learning strategies, heuristic strategies, problem-solving skills, critical thinking skills, self-management skills, control strategies, personal inquiry skills, reflective skills, social skills, etc.). All these skills are directly related to the model of TVE (Fig. 1) as well as the qualities of the workforce of the twentyfirst century (Schleicher 2016; Lucas 2014). Cognitive constructivist learning processes and cognitive learning processes (e.g., elaboration, induction, accretion restructuring, reinterpretation, chunking compilation, etc.) are directly related to the learning processes of the model of TVE and directly and effectively contribute to schema acquisition and automation in development of VTE (recurrent and nonrecurrent skills) (Fig. 1, Tables 2 and 3 highlight on this). All the four learning theories described or some aspects of the four learning theories have relatively different positive impact and significant contributions to promote the different learning processes of the various components of the model of TVE. More importantly it is strongly argued that all the components of the model of TVE and their respective learning processes, as discussed earlier, are covered by the learning outcomes and the learning processes of the four learning theories (refer to Table 3). Also taking into consideration the requirements of workforce of the twentyfirst century, it might not be possible to concentrate on the principles of one specific learning theory for designing competency-based learning for quality TVET education. Therefore, the description and discussion above, as summarized, indicated, and shown in Table 3, are strongly recommended as the integrated learning theories for quality TVET education in the twenty-first century. Thanks to behavioral, cognitive, situated cognition, and constructivist learning theories.

338

F. K. Sarfo

Implications for Competency-Based Learning First, in the context of the proposed integrated learning theories, the learning outcomes and their associated learning processes of the four learning theories have direct relationships with the model of TVE (Fig. 1 and Table 3) and its associated learning processes. For this reason it is strongly argued that the learning principles or conditions of learning and methods of instruction of the four learning theories can be used to fully design competency-based learning environments (CBLE) to facilitate all the different learning processes for the development of TVE for quality TVET education in the twenty-first century. But it is important to note that, as has already been discussed, a well-designed competency-based learning environment for acquisition of TVE will not aim at students/learners gaining each of these knowledge and skills separately but will instead try to achieve learner acquisition of the ability to use all of the knowledge and skills in a coordinated and integrated fashion while doing real-life task. With respect to this and in accordance with the literature on instructional sciences, from the perspective of the principles of the above learning theories and processes, a well-designed competency-based learning environments for acquisition of TVE can be acquired in an authentic realistic context (Brown et al. 1989) and more particularly in learning environments which (1) are task oriented, (2) activate students prior knowledge, (3) demonstrate what is to be learned, (4) encourage learners to integrate the new knowledge to their everyday life, (5) are application oriented, and (6) consider the fact that students learn in different ways. Learning environments with these features can be promoted by using 4C/ID model (van Merrienboer 1997; Sarfo and Elen 2007, 2008) or ten steps for complex learning (Van Merrienboer and Kirschner 2007). Four-component instructional design model (4C/ID model) presents a blueprint for complex learning which is based on four different components (learning tasks, supportive information, procedural information, and part-task information) of learning processes and associated instructional methods (van Merrienboer 1997). Competency-based learning environments designed in accordance with specifications of 4C/ID model for acquisition of TVE have been tested and found effective and usable in a true and vivid context of traditional classrooms of secondary and technical schools (Sarfo and Elen 2007, 2008). Second, the development of recurrent skills or rule-based behaviors as already been indicated is better facilitated by repetition and practice with feedback (e.g., Anderson 1983; Ericsson 1993; van Merrienboer 1997) under principles of behavioral and cognitive learning theories; the development of nonrecurrent or productive skills is better facilitated by principles of cognitive, situated cognition, and constructivist learning theories. This implies that instructional practices (e.g., prescriptive instruction) based on principles of behavioral and cognitive learning theories are potential for designing competency-based learning environment to promote the development of recurrent skills of TVE in the TVET education, while instructional practices (e.g., ill-structured instruction) based on the principles of cognitive, situated cognition, and constructivist learning theories are potential for designing CPLE to promote the development of nonrecurrent skills of VTE in TVET. Both

13

Psychological Framework for Quality Technical and Vocational. . .

339

instructional practices can be considered simultaneously or separately when designing CPLE for acquisition of VTE depending on the goal and interest of a particular TVET program. Third, the key ingredients of behavioral learning theories are reinforcers or stimuli from the environment manipulated by the instructor or the teacher to produce desired response or behaviors in learners. Teacher’s ability to select the appropriate stimuli which would lead to a particular response in learners is very important. The teacher or instructor is certain that with appropriate stimuli, learner would achieve predicted desirable behavior (Gagne and Dick 1983). This suggests that the effectiveness or success of learning environment is, to some great extent, determined by the teacher or the external institutions but not the learners. Therefore, teacher-related factors such as teacher’s ability to (1) select appropriate behavioral objective (2) design appropriate stimulus materials and (3) select appropriate reinforcers and reward the desirable behaviors of students should be seriously considered when designing CBLE, based on behavioral learning principles, to promote the development of VTE in TVET. In relation to this, it is argued (e.g., Kuijpers and Gundy 2011) that reproductive learning or learning that the learner cannot attribute personal meaning to the stimulus materials is inappropriate for career development as well as the development of TVE in TVET education. This is confirmed by Schleicher (2016), OECD Education Director, that we are in the fast-changing world and reproductive learning will not meet the requirements of the twenty-first century. This supports the earlier claim that principles of behavioral learning theory alone are not adequate for designing CBLE to promote successful learning in TVET education in the present era. On the other hand, the focus of cognitive, situated cognition, and constructivist learning theories is learners’ social and mental activities/process. Learners’ cognitive factors mediate between the external learning processes and the internal/external learning products (e.g., construction of cognitive schemata). These mediating variables include learners’ individual differences, learning styles, interest, motivation, instructional conceptions, prior knowledge, and others. As a result of these mediating variables, the success of learning environments related to the principles of these learning theories to some great extent is not determined by the teacher or the instructor (Lowyck and Elen 1994). This indicates that learners’ related cognitive and noncognitive variables should be seriously considered when designing CBLE based on principles’ cognitive, situated cognition, and constructivist learning to promote the development of TVE in TVET. TVET teachers/instructors should be flexible and adapt the instruction/teaching to meet the learners’ learning needs (e.g., career development (Kuijpers and Gundy 2011)) and the social contexts of learning. Students in the present era should develop capacity and capability to live in a multi-faceted world as an active citizen. They manage their learning processes, and this should shape the role of education in the twenty-first century (Schleicher 2016). In this regard, it is argued that CBLE based on constructivist, cognitive, and situated cognition learning principles might promote acquisition of TVE in TVET better than CBLE based on behaviorism. Fourth, the above discussions on behavioral, cognitive, situated cognition, and constructivist learning principles further imply that front-end analyses

340

F. K. Sarfo

such as behavioral task analysis, cognitive task analysis, as well as learners’ analysis are basic requirements for designing competency-based learning environments for quality TVET education. The design of CBLE should also consider the limited processing capacity of learners’ cognitive architecture as well as cognitive load. Fifth, in most of the developing countries, the medium of instruction for TVET teaching is not the mother tongue (L1) but different. However, careful consideration of the principles and learning processes of sociocultural and cognitive learning theories by Collins et al. (1989) and Vygotsky (1978) together with the proposition of Fairey (1960) suggests that teaching TVET in the mother tongue is of critical importance. Or at least the mother tongue may be used to support the medium of instruction in TVET if the medium of instruction is different from the mother tongue. Sixth, as already discussed, the basic goal of developing learners’ VTE in TVET education is to enable students to perform real-life tasks constituting both recurrent and nonrecurrent skills to the required level of expertise. This indicates that both formative assessment and summative assessment of VTE in school context should focus on (1) learners’ ability to perform a skill, (2) learners’ conceptual understanding, (3) learners’ ability to design an artifact which is functional, and (4) learners’ ability to solve real-life problems in the domain of interest to the required standard and level of expertise. The formative assessment should aim at helping learners to improve their learning processes toward the achievement of the required standard and level of desired expertise; the summative assessment should aim at making a decision as to whether or not the learners have achieved the required standard and level of expected expertise (van Merrienboer and Kirshner 2007). Students’ internship, WEL, students’ attachment program, and microteaching should be designed to contribute to formative and summative assessments of TVE in TVET education. These activities should be supported by students’ portfolios that show their progress and achievement (Driscoll 2005). Finally, as it has been indicated, there is research/grounding evidence that learners’ cognitive and noncognitive variables might facilitate or handicap the effectiveness of instructional interventions (Lowyck and Elen 1994) in the context of principles of cognitive and constructivist learning theories. Conversely research findings (e.g., Sarfo and Elen 2007) show that secondary technical students’ instructional metacognitive knowledge has no moderating effect on the impact of instructional interventions in the context of principles of cognitive, situated cognition, and constructivist learning theories. This indicates that there are mixed findings about the effect of learners’ cognitive and noncognitive factors on the impact instructional interventions such as CBLE for promoting the development of TVE in TVET. In this respect there is a need for better clarifications in order to make an empirically sound and cost-effective recommendations of instructional practice, based on principles of situated cognition, cognitive, and constructivist learning theories for quality TVET education. It is therefore suggested that further research studies should investigate the moderating and mediating effects of learners’ cognitive and noncognitive variables on the effect of CBLE based on principles of cognitive, situated cognition, and constructivist learning theories for development of VTE in TVET education.

13

Psychological Framework for Quality Technical and Vocational. . .

341

Conclusion The chapter exposes instructional practitioners of TVET education to cognitive processes of novice and expert technical and vocational workers and students based on empirical research on cognitive psychology and technical and vocational education. The chapter further stimulates discussions on theoretically sound integrated learning theories and their related practically relevant instructional guidelines for designing powerful CBLE to facilitate productive and quality learning in TVET education. The intentions are to extend and add new insight to the existing traditional theoretical grounding of CBT to cope with the challenges of TVET learning in the twenty-first century. It is argued that in the context of the proposed integrated learning theories, the successful implementation of competency-based learning environments for promoting quality TVET teaching and learning requires great deal of (1) knowledge and skills from TVET practitioners in (1) prescriptive instructional interventions based on principles of behavioral and cognitive learning theories; (2) flexible instructional intervention based on principles of cognitive, situated cognition, and constructivist learning theories to meet learners’ learning needs, learning contexts, and limited cognitive resources; and (3) principles of instructional design. TVET educational practitioners/instructional designers should further conduct research on effectiveness of the above instructional interventions for effective learning at various levels of TVET education. In conclusion, successful implementation of competency-based learning environments, based on the proposed integrated learning theories, to promote quality TVET education in the present era, requires consistent training and support from TVET educational policy makers and other TVET stakeholders to equip TVET instructional practitioners and researchers. It requires entire systemic and systematic change of the TVET education.

References Alexander, P. A. (2003). The development of expertise: The journey from acclimation to proficiency. Educational Researcher, 32(8), 10–14. Alexander, P. A., & Murphy, P. K. (1998). The research base for APA’s learner-centered principles. In N. M. Lambert & B. L. McCombs (Eds.), Issues in school reforms: A sampler of psychological perspectives on learner-centered schools (pp. 25–60). Washington, DC: The American Psychological Associations. Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press. Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control processes. In K. W. Spence & J. T. Spence (Eds.), The psychology of learning and motivation (pp. 89–195). New York, NY: Academic Press. Ausubel, D. P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267–272. Ausubel, D. P. (1963). The psychology of meaningful verbal learning. New York, NY: Grune & Stratton. Balasubramanian, N., & Wilson, B. G. (2007). Learning by design: Teachers and students as co-creators of knowledge. In K. Kumpulainen (Ed.), Educational technology: Opportunities and challenges (pp. 30–51). Oulu, Finland: University of Oulu Retrieved October 23, 2011, from http://herkules.oulu.fi/isbn9789514284069/isbn9789514284069.pdf.

342

F. K. Sarfo

Bereiter, C., & Scardamalia, M. (1993). Surpassing ourselves: An inquiry into the nature and implication of expertise. La Selle, IL: Open Cort. Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42. Bruner, J. S. (1964). The course of cognitive growth. American Psychologist, 19, 1–15. Bruner, J. S. (1966). Toward a theory of instruction. Cambridge, MA: Belknap. Casakin, H. (2004). Visual analogy as a cognitive strategy in the design process. Expert versus novice performance. The Journal of Design Research. Retrieved September 15, 2011, from http://jdr.tudelft.nl/articles/issues2004.02/Art6.html Case, R. (1984). The processing of stage transition: A neo-Piagetian view. In R. J. Sternberg (Ed.), Mechanism of cognitive development (pp. 171–246). New York, NY: Freeman. Chi, M. T. H., Glaser, R., & Farr, M. J. (1988). The nature of expertise. Mahwah, NJ: Lawrence Erlbaum Associates. Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning and instruction: Essays in honor of Robert Glasr (pp. 453–494). Hillsdale, NJ: Lawrence Erlbaum. Cross, N. (2004). Creative thinking by expert designers. The Journal of Design Research. Retrieved September 11, 2012, from http://jdr.tudelft.nl/articles/issues2004.02/Art3.html del Mar, V. (2011). Introducing UNESCO’s technical and vocational education and training (TVET) definition and strategy. Retrieved February 15, 2016, from http://www.uis.unesco.org/ Stat Derry, J. S. (1990). Learning strategies for acquiring useful knowledge. In B. F. Jones & L. Idol (Eds.), Dimensions of thinking and cognitive instruction (pp. 347–375). Hillsdale, NJ: Lawrence Erlbum Associates. Driscoll, M. P. (2005). Psychology of learning for instruction (3rd ed.). Boston, MA: Pearson Allyn and Bacon. UNESCO. (2012). Education for all global monitoring report. Retrieved November 10, 2015, from unesdoc.unesco.org Elen, J. (1995). Blocks on the road to instructional design prescriptions: A methodology for I.D. research exemplified. Leuven: Leuven University press. Ericsson, K. A. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363–406. Fairey, F. (1960). Relationship between technical and vocational education and training. A paper presented at the Regional Workshop Seminar on Vocational and Technical Education on 28th March to 9th April, 1960 in Accra, Ghana. Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive developmental inquiry. American Psychologist, 34, 906–911. Gagne, R. M. (1985). The conditions of learning (4th ed.). New York, NY: Holt, Rinehart, & Winston. Gagne, R. M., & Dick, W. (1983). Instructional psychology. Annual Review of Psychology, 34, 261–295. Hodge, S. (2007). The origins of competency-based training. Australian Journal of Adult learning, 47(2), 1–31 Retrieved October 9, 2016, from http://files.eric.ed.gov/fulltext/EJ797578.pdf. Kirschner, P., Carr, C., & van Merrienboer, J. (2002). How expert designers design. Performance Improvement Quarterly, 15(4), 86–104. Kuijpers, M., & Gundy, C. (2011). The relationship between learning environment and career competencies of students in vocational education. Journal of Vocational Behaviour, 78(2011), 21–30. Lave, J.& Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York: Cambridge University Press. ISBNN 0-521-42374-0. Lindekens, J., Heylighen, A., & Nueckermans, H. (2003). Understanding architectural Re-design. In G. Aouad, & L. Ruddock (Eds.), Proceedings of the 3rd International Postgraduate Research

13

Psychological Framework for Quality Technical and Vocational. . .

343

Conference in the Built and Human Environment, ESAI (pp. 671–681). Salford, UK: University of Salford. Lowyck, J., & Elen, J. (1994). Students’ instructional metacognition in learning environment (SIMILE). Leuven: K.U. Leuven, C.I.P.& T. Lucas, B. (2014). Vocational pedagogy: What is it, why it matters and what we can do about it. Retrieved March 20, 2016, from http://www.unevoc.unesco.org/fileadmin/up/vocational_peda gogy_bill_lucas_unesco-unevoc_30april.pdf Mayer, R. E. (2002). Multimedia Learning. Cambridge, UK: University Press. Mayer, R. (2004). Should there be a three-strike rule against pure discovery learning? The case for guided methods of instruction. American Psychology, 59, 14–29. Miller, G. A. (1956). The magical number seven, plus or minus two: Some units on our capacity for processing information. Psychological Review, 63, 81–97. Papert, S. (1991). Situating constructionism. In I. Harel & S. Papert (Eds.), Constructionism (pp. 1–12). Norwood, NJ: Ablex Publishing. Perkins, D. N. (1991). What constructivism demands of the learner. Educational Technology, 31(9), 19–21. Rumelhart, D. E., & Norman, D. A. (1976). Accretion, Tuning, and restructuring: Three modes of learning (Report No.7602). Retrieved October 16, 2010, from http://www.eric.ed.gov/PDFS/ ED134902.pdf Sarfo, F. K. (2011). Learning by design. In N. Seed (Ed.), Encyclopedia of the Sciences of learning (pp. 1817–1821). New York, NY: Springer p. Sarfo, F. K., & Elen, J. (2007). Developing technical expertise in secondary technical schools: The effect of 4C/ID learning environments. Learning Environments Research, 10(3), 207–221. Sarfo, F. K., & Elen, J. (2008). The moderating effect of instructional conceptions on the effect of powerful learning environment. Instructional Science, 36, 137–153. Shuell, T. J. (1988). The role of the student in learning from instruction. Contemporary Educational Psychology, 13, 276–295. Skinner, B. F. (1958). Teaching machines. Science, 128, 969–977. Skinner, B. F. (1974). About behaviourism. New York, NY: Alfred A. Knopf. Schleicher, A. (2016). The case for 21st –century learning: Report of OECD. Retrieved October 20, 2016 from http://www.oecd.org/general/thecasefor21st-centurylearning.htm Spiro, R. J., Vispoel, W. L., Schmitz, J., Samarapungavan, A., & Boerger, A. (1987). Knowledge acquisition for application: Cognitive flexibility and transfer in complex content domains. In B. C. Britton & S. Glynn (Eds.), Executive control processes. Hillsdale, NJ: Lawrence Erlbaum Associates. Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and Instruction, 12, 185–233. UNESCO. (2004). Education for all: The quality imperative. Retrieved October 17, 2015, from http://unesdoc.unesco.org/ima UNESCO UNEVOC. (2014). Vocational pedagogy: What it is, why it matters and how to put it into practice. Report of the UNESCO-UNEVOC Virtual Conference. 12–26 May, 2014. Retrieved October 20, 2015, from http://www.unevoc.unesco.org van Merrienboer, J. J. G. (1997). Training complex cognitive skills: A Four-Component Instructional Design model for technical training. Englewood Cliffs, NJ: Educational Technology Publications. van Merriënboer, J. J. G., Schuurman, J. G., de Croock, M. B. M., & Paas, F. (2002). Redirecting learners’ attention during training: Effects on cognitive load, transfer test performance, and training efficiency. Learning and Instruction, 12, 11–37. van Merriënboer, J. J. G., & Kirschner, P. A. (2007). Ten steps to complex learning: A systematic approach to four-component instructional design. Lawrence Erlbaum. Vygotsky, L. S. (1978). Mind in Society. Cambridge, MA: Harvard University Press.

344

F. K. Sarfo

Whitson, J. (2005). Cognition as semiotic process: From situated mediation to critical reflective transcendence. Retrieved January 10, 2016, from http://www1.udel.edu/educ/whitson/files/ WhitsonCogSem.pdf

Frederick Kwaku Sarfo is an Associate Professor of Instructional Technology and a dean of faculty at University of Education, Winneba – Kumasi Campus, Ghana. He teaches courses in educational technology, general principles and methods of teaching, and models and strategies of curriculum development at undergraduate and postgraduate levels. His research interest is focused on (1) integration of ICT into education, (2) instructional design for learning in difficult situation, (3) instructional conceptions, and (4) designing powerful learning environment for the development of expertise in technical and vocational education. He is a visiting scholar at the Catholic University of Leuven, Belgium, and a member of AECT Advisory Board on the 6th Edition of the Handbook of Research on Educational Communications and Technology. He is a member of research team on a project entitled “Using Moodle for Teaching and Learning at University of Education, Winneba.” He teamed up with a representative from ETC, the Netherlands, to train vocational and technical institution (VTI) teachers in competency-based training (CBT). He has been working as a consultant/resource person in various institutions to train workers and instructors in curriculum, instructional design, CBT, and the use of audiovisuals in teaching and learning.

Future Trends in the Design Strategies and Technological Affordances of E-learning

14

Begoña Gros and Francisco J. García-Peñalvo

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Concept of E-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Evolution of the Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-Learning Generations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pedagogical Approaches in E-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning Ecosystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

346 347 347 349 352 357 361 362

Abstract

E-learning has become an increasingly important learning and teaching mode in recent decades and has been recognized as an efficient and effective learning method. The rapidly rising number of Internet users with smartphones and tablets around the world has supported the spread of e-learning, not only in higher education and vocational training but also in primary and secondary schools. E-learning and traditional distance education approaches share the emphasis on “any time, any place” learning and the assumption that students are at a distance from the instructor. The design of the initial e-learning courses tended to replicate existing distance education practice based on content delivery. However, long textual lectures were clearly not suitable for the online environment. These early insights guided the development of e-learning (technical and pedagogical) and emphasized the need for communication and interaction.

B. Gros (*) Universidad de Barcelona, Barcelona, Spain e-mail: [email protected] F. J. García-Peñalvo Universidad de Salamanca, Salamanca, Spain e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_67

345

346

B. Gros and F. J. García-Peñalvo

E-learning describes learning delivered fully online where technology mediates the learning process, teaching is delivered entirely via Internet, and students and instructors are not required to be available at the same time and place. E-learning practices are evolving with the mutual influence of technological e-learning platforms and pedagogical models. Today, the broad penetration and consolidation of e-learning needs to advance and open up to support new possibilities. Future e-learning should encompass the use of Internet technologies for both formal and informal learning by leveraging different services and applications. The purpose of this chapter is to provide a general analysis of the evolution and future trends in e-learning. The authors intend to summarize findings from contemporary research into e-learning in order to understand its current state and to identify the main challenges in the technological and pedagogical affordances of e-learning. Keywords

E-learning development · E-learning technology · E-learning models · Learning digital ecosystems

Introduction Advances in educational technology and an increasing interest in the development of asynchronous spaces influenced the rise of the term e-learning in the mid-1990s as a way to describe learning delivered entirely online where technology mediates the learning process. The pedagogical design and technology behind e-learning have gradually evolved to provide support and facilitate learning. E-learning has become an increasingly important learning and teaching mode, not only in open and distance learning institutes but also in conventional universities, continuing education institutions and corporate training, and it has recently spread to primary and secondary schools. Moreover, greater access to technological resources is providing e-learning not only in formal education but also in informal learning. The evolution of e-learning has evolved from instructor-centered (traditional classroom) to student-centered approaches, where students have more responsibility for their learning. This evolution has been made possible due to the technological platforms that support e-learning. Learning management systems (LMS) provide the framework to handle all aspects of the e-learning process. An LMS is the infrastructure that delivers and manages instructional content, identifies and assesses individual and organizational learning or training goals, tracks progress toward meeting those goals, and collects and presents data to support the learning process. It is also important to stress the influence of social media on users’ daily habits, as this has led to increased demand for learning personalization, social resources to interact with peers, and unlimited access to resources and information (Siemens, 2014). Moreover, e-learning is also being called on to offer flexibility in the way and place people learn and permit a natural and necessary coexistence of both formal and

14

Future Trends in the Design Strategies and Technological Affordances. . .

347

informal learning flows. Thus, the “traditional” e-learning platforms, despite their extensive penetration and consolidation, need to evolve and open themselves up to supporting these new affordances to become another component within a complex digital ecosystem. This, in turn, will become much more than a sum of its independent technological components due to the interoperability and evolution properties orientated to learning and knowledge management, both at institutional and personal levels. The continued growth and interest in e-learning have raised many questions related to learning design and technology to support asynchronous learning: What are the best instructional models in online settings? How have the roles of instructors and learners evolved? What are the most appropriate forms of interaction and communication? How can formal and informal learning be combined? What is the most appropriate technology to support e-learning? The main goal of this chapter is to describe the evolution of e-learning and to analyze the current situation and future trends in the design strategies and technological affordances of e-learning. The chapter is divided into four sections. Firstly, we describe the meaning of the term e-learning and its evolution from the early 1990s until today. In the second part, we focus on the evolution of pedagogical approaches in e-learning. The third part analyzes learning technologies with particular emphasis on the development of the learning ecosystem as a technological platform that can provide better services than traditional LMS. Finally, in the fourth part, based on the resulting analysis, the authors offer some general remarks about the future of e-learning.

The Concept of E-Learning In this section we analyze the meaning of the term e-learning in relation to other similar terminologies (distance education, online learning, virtual learning, etc.) and the evolution of e-learning generations from the early 1990s until today.

Evolution of the Concept A major confusion in the discourse on e-learning is its blurring with distance education: e-learning and distance education are not synonymous. Distance education can be traced back to ancient times, whereas e-learning is a relatively new phenomenon associated with the development of the Internet in the 1990s. However, it is undeniable that the origins of e-learning lie in distance education and share the idea that the use of media can support massive learning without face-to-face interaction. The first documented example of training by correspondence (as distance education was known for many years) dates back to 1828, when Professor C. Phillips published an advertisement in the Boston Gazette offering teaching materials and tutorials by correspondence. In 1843, the Phonographic Correspondence Society was founded, which could be considered the first official distance education

348

B. Gros and F. J. García-Peñalvo

institution as it would receive, correct, and return shorthand exercises completed by students following a correspondence course. The idea that technology such as radio and television could be used to bring education to a wide audience began to surface as long ago as the 1920s, but it was not until the early 1960s that the idea gained momentum, with the landmark creation of the Open University in the UK, with a manifesto commitment in 1966 that became a reality in 1971 when this university started to accept its first students. The e-learning concept has evolved alongside the evolution of its supporting technology, from the early concept linked to the introduction of personal computers up to today’s distributed systems, which have favored learning networks and the roots of connectivism (Siemens, 2005). However, the most outstanding and important event in the history of e-learning is the emergence of the Web, after which the evolution of the e-learning model has been inextricably linked to the evolution of the Web (García-Peñalvo & Seoane-Pardo, 2015). When a time approach is used to classify e-learning models according to their technological evolution, the most suitable metaphors are generations (Downes, 2012; García-Peñalvo & Seoane-Pardo, 2015; Garrison & Anderson, 2003; Gros et al., 2009) or timelines (Conole, 2013), as opposed to other taxonomies that use variables such as centrality (Anderson, 2008) or the pedagogical model (Anderson & Dron, 2011). Garrison and Anderson (2003) refer to five stages, or generations, of e-learning, each with its own theoretical model. The first is based on a behaviorist approach; the second appears as a result of the influence of new technologies and an increasing acceptance of the cognitive theory, including strategies focused on independent study; the third generation is based on constructivist theories and centers on the advantages of synchronous and asynchronous human interaction; the fourth and fifth generations have no theoretical background, and the authors considered that their main characteristics were not yet present in training programs, but they would be based on a huge volume of content and distributed computer processing to achieve a more flexible and intelligent learning model. Gros et al. (2009) present three generations, each with a different e-learning model. The first generation is associated with a model focused on materials, including physical materials enriched with digital formats and clearly influenced by the book metaphor. The second generation is based on learning management systems (LMS) inspired by the classroom metaphor, in which huge amounts of online resources are produced to complement other educational resources available on the Internet known as learning objects (Morales, García-Peñalvo, & Barrón, 2007; Wiley, 2002). In this generation the interaction dynamics start through messaging systems and discussion forums. The third generation is characterized by a model centered on flexibility and participation; the online content is more specialized and combines materials created both by the institution and the students. Reflectionorientated tools, such as e-portfolios and blogs (Tan & Loughlin, 2014), and more interactive activities, such as games (Minović, García-Peñalvo, & Kearney, 2016; Sánchez i Peris, 2015), are also introduced to enrich the learning experience with a special orientation toward the learning communities model (Wenger, 1998). In

14

Future Trends in the Design Strategies and Technological Affordances. . .

349

addition, web-based solutions are expanded to other devices which leads to the development of mobile learning training activities (Sánchez Prieto, Olmos Migueláñez, & García-Peñalvo, 2014). Stephen Downes (2012) starts with a generation zero based on the concept of publishing multimedia online resources with the idea that computers can present content and activities in a sequence determined by the students’ choices and by the results of online interactions, such as tests and quizzes. This foundational basis is the point of departure for all subsequent developments in the field of online learning. Generation one is based on the idea of the network itself, with tools such as websites, e-mail, or gopher to allow connection and virtual communication through specialized software and hardware. Generation two takes place in the early 1990s and is essentially the application of computer games to online learning. Generation three places LMS at the center of e-learning, connecting the contents of generation zero with the generation one platform, the Web. Generation four is promoted by the Web 2.0 concept, which in online education is known as e-learning 2.0 (Downes, 2005). One of the most significant characteristics of e-learning 2.0 is the social interaction among learners, changing the nature of the underlying network where the nodes are now people instead of computers. This social orientation also causes a real proliferation of mobile access and the exploitation of more ubiquitous approaches in education and training (Casany, Alier, Mayol, Conde, & García-Peñalvo, 2013). Generation five is the cloud-computing generation (Subashini & Kavitha, 2011) and the open-content generation (García-Peñalvo, García de Figuerola, & Merlo-Vega, 2010; McGreal, Kinuthia, & Marshall, 2013; Ramírez Montoya, 2015). Finally, generation six is fully centered on Massive Open Online Courses (MOOC) (Daniel, Vázquez Cano, & Gisbert, 2015; SCOPEO, 2013). Gráinne Conole (2013) presents a timeline to introduce the key technological developments in online education over the last 30 years (see Fig. 1).

E-Learning Generations Based on the generation metaphor presented above, García-Peñalvo and SeoanePardo (García-Peñalvo & Seoane-Pardo, 2015) reviewed the e-learning conceptualization and definition according to three different generations or stages that are consistent with the broad proposals of the different authors and particularly with Stephens Downes’ idea that generations are not replaced but coexist, and the maturity of the first brings the evolution of the following and the emergence of new generations (Downes, 2012). In fact, the term “e-learning” have been used as a teaching and learning method but also as a learning and teaching approach. The first generation is characterized by the emergence of online learning platforms or LMS as the evolution of a more generic concept of the virtual learning environments that were set up after the Web appeared, with the broad (and poor) idea that e-learning is a kind of teaching that uses computers (Mayer, 2003). These learning environments are too centered on content and overlook interaction. The technological context is more important than the pedagogical issues. The classic

350 Fig. 1 The e-learning timeline adapted from Conole, 2013

B. Gros and F. J. García-Peñalvo

80s

Multimedia resources

93

The Web

94

Learning objects

95

Learning Management Systems

98

Mobile devices

99

Learning Design

00

Gaming technologies

01

Open Educational Resources

04

Social and participatory media

05

Virtual worlds

07

eBooks and smart devices

08

Massive Open Online Courses

10

Learning Analytics

definitions of e-learning are generally associated with this e-learning generation. For example, Betty Collis (1996) defines tele-learning as “making connections among persons and resources through communication technologies for learning-related purposes.” Marc Rosenberg (2001) confines e-learning to the Internet as the use of Internet technologies to deliver a broad array of solutions that enhance knowledge and performance. He bases his idea on three fundamental criteria: (1) networked, (2) delivered to the end user via a computer using standard Internet technology, and (3) focused on the broadest view of learning. García-Peñalvo (2005) defines e-learning with a perspective focused on interaction, a characteristic of the next generation, “non-presential teaching through technology platforms that provides flexible access any time to the teaching and learning process, adapting to each student’s skills, needs and availability; it also ensures collaborative learning environments by using synchronous and asynchronous communication tools, enhancing in sum the competency-based management process.” The second generation underlines the human factor. Interaction between peers and communication among teachers and students is the essential elements for highquality e-learning that seeks to go beyond a simple content publication process. Web 2.0, mobile technologies, and open knowledge movement are significant factors that help this e-learning generation to grow. Based on this, LMS evolved to support socialization, mobility, and data interoperability facilities (Conde et al., 2014). Examples of e-learning definitions that are congruent with these second generation principles include: “training delivered on a digital device such as a smart phone or a laptop computer that is designed to support individual learning or organisational performance goals” (R. C. Clark & Mayer, 2011) or “teaching-to-learning process aimed at obtaining a set of skills and competences from students, trying to ensure the

14

Future Trends in the Design Strategies and Technological Affordances. . .

351

highest quality in the whole process, thanks to: predominant use of web-based technologies; a set of sequenced and structured contents based on pre-defined but flexible strategies; interaction with the group of students and tutors; appropriate evaluation procedures, both of learning results and the whole learning process; a collaborative working environment with space-and-time deferred presence; and finally a sum of value-added technological services in order to achieve maximum interaction” (García-Peñalvo, 2008). The third and last generation of e-learning is characterized by two symbiotic aspects. The first is technological: the LMS concept as a unique and monolithic component for online education functionality is broken (Conde-González, GarcíaPeñalvo, Rodríguez-Conde, Alier, & García-Holgado, 2014). Since the emergence of Web 2.0 and social tools, the e-learning platform has become another component in a technological ecosystem orientated toward the learning process (GarcíaHolgado & García-Peñalvo, 2013), transcending the mere accumulation of trending technology. This learning ecosystem should facilitate interaction and offer greater flexibility for any educational teaching. The second aspect implies a loss of verticality in the e-learning concept to become a broader and more transverse element that is at the service of education in its wider sense. Both from an intentional (formal and informal) and unintentional (informal) view, learning ecosystems are at the service of people involved in teaching and learning processes or in self-learning. Thus, e-learning is integrated into educational designs or learning activities in a transparent way. It reveals the penetration of technology into people’s everyday lives, making it easier to break down the barriers between formal and informal learning (Griffiths & García-Peñalvo, 2016). Technological learning ecosystems facilitate this globalization of the e-learning notion, either to support an institutional context (García-Holgado & GarcíaPeñalvo, 2014; García-Peñalvo, Johnson, Ribeiro Alves, Minovic, & CondeGonzález, 2014; Hirsch & Ng, 2011) or a personal one through the concept, more metaphorical than technological, of the personal learning environment (PLE) (Wilson et al., 2007). Nevertheless, technological learning ecosystems are supporting other approaches to using technology in the classrooms, such as flipped teaching (Baker, 2000; Lage, Platt, & Treglia, 2000). Flipped teaching methodology is based on two key actions: moving activities that are usually done in the classroom (such as master lectures) to the home and moving those that are usually done at home (e.g., homework) into the classroom (García-Peñalvo, Fidalgo-Blanco, Sein-Echaluce Lacleta, & CondeGonzález, 2016). The Observatory of Education Innovation at the Tecnológico de Monterrey (2014) has also detected a tendency to integrate inverted learning with other approaches, for example, combining peer instruction (Fulton, 2014), selfpaced learning according to objectives, adaptive learning (Lerís López, Vea Muniesa, & Velamazán Gimeno, 2015), and the use of leisure to learn. Thus, the flipped teaching model is based on the idea of increasing interaction among students and developing their responsibility for their own learning (Bergmann & Sams, 2012) using virtual learning environments as supported tools. These virtual environments allow students to access learning resources, ask questions, and share material in

352

B. Gros and F. J. García-Peñalvo

forums, as it is mandatory for students to have help available while studying at home (Yoshida, 2016). In this last stage, the MOOC concept has broken out strongly, perhaps with no new e-learning approach, but with sufficient impact to make institutions reflect on their e-learning processes and conceptions. The term MOOC appeared for the first time in 2008 to describe the connectivism and connected knowledge course by George Siemens and others (http://cckno8. wordpress.com). This course gave rise to cMOOCs, where “c” means that the course is based on the connectivist approach (Siemens, 2005). A second type of MOOC appeared in 2011 under the name xMOOC, which is based on digital content and individualized learning as opposed to cMOOCs, which are more related to collaborative learning. There is currently a great deal of interest in MOOCs among the e-learning community. Other proposals for improving MOOCs have introduced the use of associated learning communities (Alario-Hoyos et al., 2013), adaptive capabilities (Fidalgo-Blanco, García-Peñalvo, & Sein-Echaluce Lacleta, 2013; SeinEchaluce Lacleta, Fidalgo-Blanco, García-Peñalvo, & Conde-González, 2016; Sonwalkar, 2013), and gamification capabilities (Borrás Gené, Martínez-Nuñez, & Fidalgo-Blanco, 2016). However, the existing dichotomy between cMOOCs and xMOOCs is questioned by different authors due to its limitations. Thus, Lina Lane (2012) proposes the sMOOC (skill MOOC) as a third kind of MOOC based on tasks; Stephen Downes (2013) suggests four criteria to describe an MOOC’s nature, autonomy, diversity, openness, and interactivity; Donald Clark (2013) defines a taxonomy with eight types of MOOC, transferMOOC, madeMOOC, synchMOOC, asynchMOOC, adaptiveMOOC, groupMOOC, connectivistMOOC, and miniMOOC; and finally Conole (2013) provides 12 dimensions to classify MOOCs, openness, massivity, multimedia usage, communication density, collaboration degree, learning path, quality assurance, reflection degree, accreditation, formality, autonomy, and diversity. With regard to the core elements that define this third generation, García-Peñalvo and Seoane-Pardo (2015, 5) propose a new definition of e-learning as “an educational process, with an intentional or unintentional nature, aimed at acquiring a range of skills and abilities in a social context, which takes place in a technological ecosystem where different profiles of users interact sharing contents, activities and experiences; besides in formal learning situations it must be tutored by teachers whose activity contributes to ensuring the quality of all involved factors.”

Pedagogical Approaches in E-Learning In the previous section, we described the evolution of e-learning and noted the existence of different educational approaches over time. In this section, we focus on the evolution of e-learning, taking into account the pedagogical approach. Pedagogical approaches are derived from learning theories that provide general principles for designing specific instructional and learning strategies. They are the

14

Future Trends in the Design Strategies and Technological Affordances. . .

353

Fig. 2 A theory-based design framework for e-learning (Source: Dabbagh (2005, p. 32))

mechanism to link theory with practice. Instructional strategies are what instructors or instructional designers create to facilitate student learning. According to Dabbagh (2005, p. 32), “there are three key components working collectively to foster meaningful learning and interaction: (1) pedagogical models; (2) instructional and learning strategies and, (3) pedagogical tools or online learning technologies (i.e., Internet and Web-based technologies). These three components form an iterative relationship in which pedagogical models inform the design of e-learning by leading to the specification of instructional and learning strategies that are subsequently enabled or enacted through the use of learning technologies” (see Fig. 2). Due to the fact that learning technologies have become ubiquitous and new technologies continue to emerge bringing new affordances, pedagogical practices are continuously evolving and changing. This does not mean that some designs and pedagogical practices have disappeared. As we have mentioned, generations of e-learning coexist. For example, some instructive models based on the transmission of knowledge are still used but, sometimes, they incorporate new strategies such as gamification. Conole (2014) divided pedagogies of e-learning into four categories: 1. Associative – a traditional form of education delivery. Emphasis is on the transmission of theoretical units of information learning as an activity through structured tasks, where the focus is on the individual, with learning through association and reinforcement. 2. Cognitive/constructivist – knowledge is seen as more dynamic and expanding rather than objective and static. The main tasks here are processing and understanding information, making sense of the surrounding world. Learning is often task orientated.

354

B. Gros and F. J. García-Peñalvo

E-training Drill & practice

Experiential, problem-based, role play

Inquiry learning Resource-based

Associative

Constructivist

Focus on individual Learning through association and reinforcement

Building on prior knowledge Task-orientated

Situative

Connectivist Learning in a networked environment

Learning through social interaction Learning in context

Reflective & dialogical learning, Personalised learning

Fig. 3 The pedagogies of e-learning. Source: teachertrainingmatters.com/blog-1/2015/12/19/learn ing-theories-in-practice

3. Situative – learning is viewed as social practice and learning through social interaction in context. The learner has a clear responsibility for his/her own learning. This approach is therefore “learner centered.” 4. Connectivist – learning through a networked environment. The connectivist theory advocates a learning organization in which there is not a body of knowledge to be transferred from educator to learner and where learning does not take place in a single environment; instead, it is distributed across the Web and people’s engagement with it constitutes learning. Each of these theories has a number of approaches associated with it which emphasize different types of learning (Fig. 3). For example, the associative category includes behaviorism and didactic approaches, the cognitive/constructivist category includes constructivism (building on prior knowledge) and constructionism (learning by doing), etc. The development of the first e-learning platforms supported an instructional design based on the associative/behaviorist approach. The design process follows a sequential and linear structure driven by predetermined goals, and the learning output is also predefined by the learning designer. The designers organize the content and tasks and break them down from simple to complex. Information is then delivered to the learner from the simplest to the most complex depending on the learner’s knowledge.

14

Future Trends in the Design Strategies and Technological Affordances. . .

355

This type of approach has major limitations because it is not really suited to the needs of the learner. The evolution of technology allows the development of approaches that accommodate constructivist and connectivist perspectives that engage learners and give them more control over the learning experience. Choosing the pedagogical approach is obviously related to what we want to achieve. However, it is important to establish a clear difference between designing face to face or e-learning. Many of the studies into the effectiveness of e-learning (Noesgaard & Ørngreen, 2015) have employed a comparative methodology. This means that the effectiveness of e-learning is based on the comparison between traditional face-to-face teaching and online learning. Along these lines, Noesgaard and Ørngreen (2015, p 280) ask “should different modalities have the same measures of performance, or should we consider e-learning to be a unique learning process and thus use different definitions of effectiveness?” This question is important because the effectiveness of e-learning can be analyzed in different ways. For instance, we can design e-learning to improve learning retention, work performance, or social collaboration. The measure to assess effectiveness will be different in each case. However, what is clear is that there are still some research gaps regarding the impact of e-learning on educational and training environments, as well as insufficient studies on cost-effectiveness and long-term impact. Research on e-learning design points out that one of the most significant requirements for further adoption of e-learning is the development of well-designed courses with interactive and engaging content, structured collaboration between peers, and flexible deadlines to allow students to pace their work (Siemens, 2014). Certainly, every aspect of such a design can be interpreted in different ways. Nevertheless, research shows that structured asynchronous online discussions are the most prominent approach for supporting collaboration between students and to support learning. Darabi et al. (2013) consider that the greatest impact on student performance is gained through “pedagogically rich strategies” that include instructor participation, interaction with students, and facilitation of student collaboration as well as continuous monitoring and moderating discussions. A promising approach to developing self-regulatory skills using externally facilitated scaffolds is presented in Gašević, Adescope, Joksimović, and Kovanović’s (2015) study. Their research shows that meaningful student-student interaction could be organized without the instructor’s direct involvement in discussions. There is a significant effect of instructional design that provides students with qualitative guidelines on how to discuss, rather than setting quantitative expectations only (e.g., number of messages posted) (Gašević et al., 2015). The provision of formative and individualized feedback has also been identified as an important challenge in e-learning (Noesgaard & Ørngreen, 2015). In addition to support from the theories of learning, we can also find e-learning models that provide specific support for designing effective learning experiences for students participating in online courses. Bozkurt et al. (2015) provide a content analysis of online learning journals from 2009 to 2013. In their study, they found that the Community of Inquiry model has been particularly relevant to the successful implementation of e-learning.

356

B. Gros and F. J. García-Peñalvo

In the Community of Inquiry model (Garrison, Anderson & Archer, 2003), learning is seen as both an individual and a social process, and dialogue and debate are considered essential for establishing and supporting e-learning. The Community of Inquiry model defines a good e-learning environment through three major components: 1. Cognitive presence: the learners’ ability to construct knowledge through communication with their peers 2. Social presence: the learners’ ability to project their personal characteristics and identities in an e-learning environment 3. Teaching presence: defined as the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes Teaching presence provides the necessary structures for a community’s formation, social presence fosters a community’s development by introducing students and instructor to each other, and cognitive presence ensures the community’s continuing usefulness to its participants. After undertaking an extensive review of the literature on online interactions and communities, Conole (2014) developed a new Community Indicators Framework (CIF) for evaluating online interactions and communities. Four community indicators appear to be common: participation, cohesion, identity, and creative capability. Participation and patterns of participation relate to the fact that communities develop through social and work activity over time. Different roles are evident, such as leadership, facilitation, support, and passive involvement. Cohesion relates to the way in which members of a community support each other through social interaction and reciprocity. Identity relates to the group’s developing self-awareness and in particular the notion of belonging and connection. Creative capability relates to how far the community is motivated and able to engage in participatory activity. The Community Indicators Framework (CIF) provides a structure to support the design and evaluation of community building and facilitation in social and participatory media. Research shows that structured asynchronous online discussions are the most prominent approach for supporting collaboration between students and to support learning. The approaches described are based on a conception of the use of e-learning in formal learning contexts. However, the broad penetration of e-learning prompts the need to develop designs that allow formal and informal settings to be linked. In this sense, we maintain that an ecological approach can be useful to support the systemic perspective needed to integrate formal and informal processes. Brown (2000) uses the term ecology as a metaphor to describe an environment for learning. “An ecology is basically an open, complex adaptive system comprising elements that are dynamic and interdependent. One of the things that makes an ecology so powerful and adaptable to new contexts is its diversity.” Brown further describes a learning ecology as “a collection of overlapping communities of interest (virtual), cross-pollinating with each other, constantly evolving, and largely

14

Future Trends in the Design Strategies and Technological Affordances. . .

357

self-organizing.” The ecology concept requires the creation and delivery of a learning environment that presents a diversity of learning options to the student. This environment should ideally offer students opportunities to receive learning through methods and models that best support their needs, interests, and personal situations. The instructional design and content elements that form a learning ecology need to be dynamic and interdependent. The learning environment should enable instructional elements designed as small, highly relevant content objects to be dynamically reorganized into a variety of pedagogical models. This dynamic reorganization of content into different pedagogical models creates a learning system that adapts to varying student needs. Barron (2006) defines personal learning ecologies as “the set of contexts found in physical or virtual spaces that provide opportunities for learning. Each context is comprised of a unique configuration of activities, material resources, relationships and the interactions that emerge from them” (Barron, 2006, p. 195). From this perspective, learning and knowledge construction are located in the connections and interactions between learners, teachers, and resources and seen as emerging from critical dialogues and enquiries. Knowledge emerges from the bottom-up connection of personal knowledge networks. Along these lines, Chatti, Jarke, and Specht (2010, p. 78) refer to the learning as a network (LaaN) perspective. “Each of us is at the centre of our very own personal knowledge network (PKN). A PKN spans across institutional boundaries and enables us to connect beyond the constraints of formal educational and organisational environments. Unlike communities, which have a start-nourish-die life cycle, PKNs develop over time.” Knowledge ecologies lie at the heart of the LaaN perspective as a complex, knowledge-intensive landscape that emerges from the bottom-up connection of personal knowledge networks. The value of the ecological perspective is that it provides a holistic view of learning. In particular, it enables us to appreciate the ways in which learners engage in different contexts and develop relationships and resources. The emphasis is on self-organized and self-managed learning. The learner is viewed as the designer and implementer of their own life experience. The important question here is whether we are using the appropriate technology in e-learning to support an ecological approach. In the next section, we analyze the use of learning management systems (LMS) and propose new technological innovations and solutions to improve e-learning.

Learning Ecosystems There are very few technological innovations that reach a sufficient level of maturity to be considered as consolidated technologies in the productive sector. It is also true that some of these technologies arrive on the scene surrounded by a halo of fascination that leads to the creation of different ad hoc practices, often resulting in

358

B. Gros and F. J. García-Peñalvo

unfulfilled expectations and eventually the complete disappearance of said technology. In e-learning, LMS are a paradigmatic case. They are a fully consolidated educational technology, although the educational processes in which they are involved could improve substantially. E-learning platforms are well established in the higher education area and enjoy very significant adoption in other educational levels and the corporate sector. Although LMS are very complete and useful as course management tools, they are too rigid in terms of communication flow, limiting participants’ interaction capabilities too much. For this reason, teachers and students tend to complement e-learning platforms with other tools, thereby creating personal learning networks (Couros, 2010). It would seem that LMS have lost their appeal as a trending or research topic due to their known limitations, while different approaches and technologies are appearing in the education sector to claim the apparently empty throne. Various reports on educational technology trends underline topics such as MOOCs (SCOPEO, 2013), gamification (Lee & Hammer, 2011), learning analytics (Gómez-Aguilar et al. 2014), adaptive learning (Berlanga & García-Peñalvo, 2005), etc., but none of these proposed technologies, by themselves, have achieved the disruptive effect that allows them to substantially improve or change teaching and learning processes. Consequently, LMS can no longer be regarded as the only component of technological/educational innovation and corporate knowledge management strategy (García-Peñalvo & Alier, 2014). Nevertheless, these platforms should be a very important component of a new learning ecosystem in conjunction with all the existing and future technological tools and services that may be useful for educational purposes (Conde-González et al., 2014). Technological ecosystems are the direct evolution of the traditional information systems orientated toward supporting information and knowledge management in heterogeneous contexts (García-Peñalvo et al., 2015). Recently, there has been a fundamental change of approach in debates on innovation in academic and political systems toward the use of ecologies and ecosystems (Adkins, Foth, Summerville, & Higgs, 2007; Aubusson, 2002; Crouzier, 2015). The European Commission has adopted these two concepts as regional innovation policy tools according to the Lisbon Declaration, considering that a technological ecosystem has an open software component-based architecture that is combined to allow the gradual evolution of the system through the contribution of new ideas and components by the community (European Commission, 2006). In fact, the technological ecosystem metaphor comes from the field of biology and has been transferred to the social area to better capture the evolutionary nature of people’s relationships, their innovation activities, and their contexts (Papaioannou, Wield, & Chataway, 2009). It has also been applied in the services area as a more generic conceptualization of economic and social actors that create value in complex systems (Frow et al., 2014) and in the technological area, defining Software

14

Future Trends in the Design Strategies and Technological Affordances. . .

359

Ecosystems (SECO) (Yu & Deng, 2011) inspired by the ideas of business and biological ecosystems (Iansiti & Levien, 2004). These software ecosystems may refer to all businesses and their interrelations with respect to a common product software or services market (Jansen, Finkelstein, & Brinkkemper, 2009). Also, from a more architecture-orientated point of view, a technological ecosystem may be studied as the structure or structures in terms of elements, the properties of these elements, and the relationships between them, that is, systems, system components, and actors (Manikas & Hansen, 2013). Dhungana et al. (2010) state that a technological ecosystem may be compared to a biological ecosystem from resource management and biodiversity perspectives, with particular emphasis on the importance of diversity and social interaction support. This relationship between natural and technological is also presented by other authors who use the natural ecosystem concept to support their own definition of technological ecosystems (Chang & West, 2006; Chen & Chang, 2007). Although there are various definitions of natural or biological ecosystems, there are three elements that are always present in all of them: the organisms, the physical environment in which they carry out their basic functions, and the set of relationships between organisms and the environment. Thus, the technological ecosystem may be defined as a set of software components that are related through information flows in a physical medium that provides support for these flows (García-Holgado & García-Peñalvo, 2013). The ecosystem metaphor is suitable for describing the technological background of educational processes because the ecosystem may recognize the complex network of independent interrelationships among the components of its architecture. At the same time, it offers an analytic framework for understanding specific patterns in the evolution of its technological infrastructure, taking into account that its components may adapt to the changes that the ecosystem undergoes and not collapse if they cannot assume the new conditions (Pickett & Cadenasso, 2002). On the other hand, the users of a technological ecosystem are also components of the ecosystem because they are repositories and generators of new knowledge, influencing the complexity of the ecosystem as artefacts (Metcalfe & Ramlogan, 2008). From the learning technologies perspective, the past has been characterized by the automation that spawned the development of e-learning platforms. The present is dominated by integration and interoperability. The future challenge is to connect and relate the different tools and services that will be available to manage knowledge and learning processes. This requires defining and designing more internally complex technological ecosystems, based on the semantic interoperability of their components, in order to offer more functionality and simplicity to users in a transparent way. Analyses of the behavior of technological innovations and advances in cognitive and education sciences indicate that the (near) future use of information technology in learning and knowledge management will be characterized by customization and adaptability (Llorens, 2014). The learning ecosystem as a technological platform should be organized into a container, the architectural framework of the ecosystem, and its functional components (García-Holgado & García-Peñalvo, 2016).

360

B. Gros and F. J. García-Peñalvo

Fig. 4 Ecosystem architecture

The framework should involve the integration, interoperability, and evolution of the ecosystem components and a correct definition of the architecture that supports it (Bo, Qinghua, Jie, Haifei, & Mu, 2009). The current status and technical and technological evolution of technological ecosystems show very pronounced parallelism with all the technology developing around the Internet and cloud services. More specifically, the evolution in data collection, analysis procedures, and decisionmaking drink from the same fountain as certain types of emerging technologies such as the Internet of things, the processes that extract concepts from business intelligence, or data mining processes applied to knowledge management. Figure 4 presents the essential architecture of a learning ecosystem, distinguishing the framework and a set of basic components for analytics, adaptive knowledge management, gamification, and evidence-based portfolios. The interconnection of platforms, tools, and services requires communication protocols, interfaces, and data and resource description standards that enable data to be entered and transmitted with minimal quality requirements that allow its meaning and context to be preserved. Interconnection protocols and data collection rely on platform interoperability, on the possibility of using sensors and other ways of gathering evidence of learning, on open data with standard semantic content, and even on descriptors and evidence linked to knowledge acquisition processes (Retalis, Papasalouros, Psaromiligkos, Siscos, & Kargidis, 2006). The current state of development of e-learning ecosystems and their extension to different learning methodologies and paradigms pinpoints the relevance of this research area for the process,

14

Future Trends in the Design Strategies and Technological Affordances. . .

361

because data is the raw material (U.S. Department of Education - Office of Educational Technology, 2012) for designing the learning cycle (data-driven design), assessing learning tasks and activities (learning analytics), and even as a means of providing real-time feedback (data-driven feedback) and tailoring the learning environment to the learner’s needs. The most outstanding characteristic of these learning ecosystems is that they are a technological approach but they are not an end in themselves. Instead, they serve the pedagogical processes that teachers want to organize in the technological contexts they provide, masking the internal difficulty of the technology itself.

Concluding Remarks In the 1990s, student profiles in e-learning were similar to those of classic distance education: most learners were adults with occupational, social, and family commitments (Hanson et al., 1997). However, the current online learner profile is beginning to include younger students. For this reason, the concept of the independent adult, who is a self-motivated and goal-orientated learner, is now being challenged by e-learning activities that emphasize social interaction and collaboration. Today’s online learners are expected to be ready to share their work, interact within small and large groups in virtual settings, and collaborate in online projects. According to Dabbagh (2007, p. 224), “the emerging online learner can be described as someone who has a strong academic self-concept; is competent in the use of online learning technologies, particularly communication and collaborative technologies; understands, values, and engages in social interaction and collaborative learning; possesses strong interpersonal and communication skills; and is self-directed.” Stöter, Bullen, Zawacki-Richter, and von Prummer (2014) identify a similar list to Dabbagh and also include learners’ personality traits and disposition for learning, their selfdirectedness, the level of motivation, time (availability, flexibility, space) and the level of interaction with their teachers, the learning tools they have at their disposal, and the level of digital competency, among many other characteristics. The research into learner characteristics identifies behaviors and practices that may lead to successful online learning experiences for learners. However, it is important to emphasize that due to today’s greater diversity of profiles, there are many influences on students’ individual goals and success factors that are not easy to identify. As Andrews and Tynan (2012) pointed out, part-time online learners are a very heterogeneous group. Due to this diversity of e-learners, it is not appropriate to privilege a particular pedagogical model, instead it is very important to design learning environments that take learners’ needs and the context into account. Providing formative, timely, and individualized feedback has also been identified as an important challenge in the online learning environment. Likewise, more recent studies have also highlighted the importance of timely, formative, effective, and individualized feedback in order to efficiently support learning. As Siemens (2014) argues, there is also a great opportunity for further research to examine how (and whether) institutions are redesigning online courses based on the

362

B. Gros and F. J. García-Peñalvo

lessons learned from MOOCs. Moreover, another potential line of research might be investigating how universities position online learning with respect to on-campus learning. Finally, current research also shows that higher education has been primarily focused on content design and curriculum development. However, in order to develop personalization, adaptive learning is crucial.

References Adkins, B. A., Foth, M., Summerville, J. A., & Higgs, P. L. (2007). Ecologies of innovation: Symbolic aspects of cross-organizational linkages in the design sector in an Australian innercity area. American Behavioral Scientist, 50(7), 922–934. https://doi.org/10.1177/ 0002764206298317. Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado-Kloos, C., Parada, H. A., Muñoz-Organero, M., & Rodríguez-de-las-Heras, A. (2013). Analysing the impact of built-in and external social tools in a MOOC on educational technologies. In D. Hernández-Leo, T. Ley, R. Klamma, & A. Harrer (Eds.), Scaling up learning for sustained impact. 8th European conference, on technology enhanced learning, EC-TEL 2013, Paphos, Cyprus, September 17–21, 2013. Proceedings (Vol. 8095, pp. 5–18). Berlin Heidelberg: Springer. Anderson, T. (2008). Toward a theory of online learning. In T. Anderson (Ed.), Theory and practice of online learning (2nd ed., pp. 45–74). Edmonton, AB: AU Press, Athabasca University. Anderson, T., & Dron, J. (2011). Three generations of distance education pedagogy. The International Review of Research in Open and Distance Learning, 12(3), 80–97. Aubusson, P. (2002). An ecology of science education. Int J Sci Educ, 24(1), 27–46. https://doi.org/ 10.1080/09500690110066511. Andrews, T., & Tynan, B. (2012). Distance learners: Connected, mobile and resourceful individuals. Australasian Journal of Educational Technology, 28(4), 565–579. Baker, J. W. (2000). The ‘Classroom Flip’: Using web course management tools to become the guide by the side. In J. A. Chambers (Ed.), Selected papers from the 11th international conference on college teaching and learning (pp. 9–17). Jacksonville, FL: Community College at Jacksonville. Barron, B. (2006). Interest and self-sustained learning as catalysts of development: A learning ecology perspective. Human development, 49(4), 193–224. Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. New York: Buck Institute for International Society for Technology in Education. Berlanga, A. J., & García-Peñalvo, F. J. (2005). Learning technology specifications: Semantic objects for adaptive learning environments. International Journal of Learning Technology, 1 (4), 458–472. https://doi.org/10.1504/IJLT.2005.007155. Bo, D., Qinghua, Z., Jie, Y., Haifei, L., & Mu, Q. (2009). An E-learning ecosystem based on cloud computing infrastructure. In Ninth IEEE International Conference on Advanced Learning Technologies, 2009 (pp. 125–127). Riga: Latvia. Borrás Gené, O., Martínez-Nuñez, M., & Fidalgo-Blanco, Á. (2016). New challenges for the motivation and learning in engineering education using gamification in MOOC. International Journal of Engineering Education, 32(1B), 501–512. Bozkurt, A., Kumtepe, E. G., Kumtepe, A. T., Aydın, İ. E., Bozkaya, M., & Aydın, C. H. (2015). Research trends in Turkish distance education: A content analysis of dissertations, 1986–2014. European Journal of Open, Distance and E-learning, 18(2), 1–21. Brown, J. S. (2000). Growing up: Digital: How the web changes work, education, and the ways people learn. Change: The Magazine of Higher Learning, 32(2), 11–20. Casany, M. J., Alier, M., Mayol, E., Conde, M. Á., & García-Peñalvo, F. J. (2013). Mobile learning as an asset for development: Challenges and oportunities. In M. D. Lytras, D. Ruan,

14

Future Trends in the Design Strategies and Technological Affordances. . .

363

R. Tennyson, P. Ordoñez de Pablos, F. J. García-Peñalvo, & L. Rusu (Eds.), Information systems, E-learning, and knowledge management research. 4th World Summit on the Knowledge Society, WSKS 2011, Mykonos, Greece, September 21–23, 2011. Revised Selected Papers (Mykonos, Greece, 21–23 September 2011) (Vol. CCIS 278, pp. 244–250). Berlin/ Heidelberg: Springer . Chang, E., & West, M. (2006). Digital ecosystems a next generation of the collaborative environment. In G. Kotsis, D. Taniar, E. Pardede, & I. K. Ibrahim (Eds.), Proceedings of iiWAS'2006 The Eighth International Conference on Information Integration and Web-based Applications Services, 4–6 December 2006, Yogyakarta, Indonesia (pp. 3–24): Austrian Computer Society. Chatti, M. A., Jarke, M., & Specht, M. (2010). The 3P learning model. Educational Technology & Society, 13(4), 74–85. Chen, W., & Chang, E. (2007). Exploring a digital ecosystem conceptual model and its simulation prototype. In Proceedings of IEEE international symposium on industrial electronics, 2007 (ISIE 2007) (pp. 2933–2938). Spain: University of Vigo. Clark, D. (2013). MOOCs: Taxonomy of 8 types of MOOC. Retrieved from http://donaldclarkplanb. blogspot.com.es/2013/04/moocs-taxonomy-of-8-types-of-mooc.html Clark, R. C., & Mayer, R. E. (2011). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning (3rd ed.). San Francisco, USA: Pfeiffer. Collis, B. (1996). Tele-learning in a digital world. The future of distance learning. London: International Thomson Computer Press. Conde, M. Á., García-Peñalvo, F. J., Rodríguez-Conde, M. J., Alier, M., Casany, M. J., & Piguillem, J. (2014). An evolving learning management system for new educational environments using 2.0 tools. Interactive Learning Environments, 22(2), 188–204. https://doi.org/ 10.1080/10494820.2012.745433. Conde-González, M. Á., García-Peñalvo, F. J., Rodríguez-Conde, M. J., Alier, M., & GarcíaHolgado, A. (2014). Perceived openness of learning management Systems by students and teachers in education and technology courses. Computers in Human Behavior, 31, 517–526. https://doi.org/10.1016/j.chb.2013.05.023. Conole, G. (2013). Digital identity and presence in the social milieu. Paper presented at the Pelicon conference, 2013, 10–12th April, Plymouth. Conole, G. (2014). Learning design: A practical approach. London: Routledge. Couros, A. (2010). Developing personal learning networks for open and social learning. In G. Veletsianos (Ed.), Emerging technologies in distance education (pp. 109–127). : Athabasca: Canadá Athabasca University Press/Edmonton. Crouzier, T. (2015). Science Ecosystem 2.0: How will change occur? Luxembourg: Publications Office of the European Union. Dabbagh, N. (2005). Pedagogical models for E-Learning: A theory-based design framework. International Journal of Technology in Teaching and Learning, 1(1), 25–44. Dabbagh, N. (2007). The online learner: Characteristics and pedagogical implications. Contemporary Issues in Technology and Teacher Education, 7(3), 217–226. Daniel, J., Vázquez Cano, E., & Gisbert, M. (2015). The future of MOOCs: Adaptive learning or business model? RUSC. Universities and Knowledge Society Journal, 12(1), 64–73 https://doi. org/10.7238/rusc.v12i1.2475. Darabi, A., Liang, X., Suryavanshi, R., & Yurekli, H. (2013). Effectiveness of online discussion strategies: A meta-analysis. American Journal of Distance Education, 27(4), 228–241. Dhungana, D., Groher, I., Schludermann, E., & Biffl, S. (2010). Software ecosystems vs. natural ecosystems: Learning from the ingenious mind of nature ECSA '10 Proceedings of the Fourth European Conference on software architecture: Companion Volume (pp. 96–102). New York, NY: ACM. Downes, S. (2005). E-learning 2.0. eLearn Magazine (October). Downes, S. (2012). E-Learning generations. Retrieved from http://halfanhour.blogspot.be/2012/ 02/e-learning-generations.html

364

B. Gros and F. J. García-Peñalvo

Downes, S. (2013). Week 2: The quality of massive open online courses. Retrieved from http:// mooc.efquel.org/week-2-the-quality-of-massive-open-online-courses-by-stephen-downes/ European Commission. (2006). A network of digital business ecosystems for Europe: Roots, processes and perspectives. Brussels/Belgium: European Commission, DG Information Society and Media Introductory Paper. Fidalgo-Blanco, Á., García-Peñalvo, F. J., & Sein-Echaluce Lacleta, M. L. (2013). A methodology proposal for developing adaptive cMOOC. In F. J. García-Peñalvo (Ed.), Proceedings of the First International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM’13) (pp. 553–558). New York: ACM. Frow, P., McColl-Kennedy, J. R., Hilton, T., Davidson, A., Payne, A., & Brozovic, D. (2014). Value propositions: A service ecosystems perspective. Marketing Theory, 14(3), 327–351. https://doi. org/10.1177/1470593114534346. Fulton, K. P. (2014). Time for learning: Top 10 reasons why flipping the classroom can change education. Thousand Oaks, CA: Corwin Press. García-Holgado, A., & García-Peñalvo, F. J. (2013). The evolution of the technological ecosystems: An architectural proposal to enhancing learning processes. In F. J. García-Peñalvo (Ed.), Proceedings of the First International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM’13) (Salamanca, Spain, November 14–15, 2013) (pp. 565–571). New York: ACM. García-Holgado, A., & García-Peñalvo, F. J. (2014). Knowledge management ecosystem based on drupal platform for promoting the Collaboration between public administrations. In F. J. GarcíaPeñalvo (Ed.), Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM’14) (Salamanca, Spain, October 1–3, 2014) (pp. 619–624). New York: ACM. García-Holgado, A., & García-Peñalvo, F. J. (2016). Architectural pattern to improve the definition and implementation of eLearning ecosystems. Science of Computer Programming, 129, 20–34. https://doi.org/10.1016/j.scico.2016.03.010. García-Peñalvo, F. J. (2005). Estado actual de los sistemas E-Learning. Education in the Knowledge Society, 6(2). García-Peñalvo, F. J. (Ed.) (2008). Advances in E-learning: Experiences and methodologies. Hershey, PA, USA: Information Science Reference (formerly Idea Group Reference). García-Peñalvo, F. J., & Alier, M. (2014). Learning management system: Evolving from silos to structures. Interactive Learning Environments, 22(2), 143–145. https://doi.org/10.1080/ 10494820.2014.884790. García-Peñalvo, F. J., Fidalgo-Blanco, Á., Sein-EchaluceLacleta, M., & Conde-González, M. Á. (2016). Cooperative micro flip teaching. In P. Zaphiris & I. Ioannou (Eds.), Proccedings of the Learning and collaboration technologies. Third international conference, LCT 2016, held as part of HCI international (Toronto, ON, Canada, July 17–22, 2016) (pp. 14–24). Cham, Switzerland: Springer International Publishing. García-Peñalvo, F. J., García de Figuerola, C., & Merlo-Vega, J. A. (2010). Open knowledge: Challenges and facts. Online Information Review, 34(4), 520–539. https://doi.org/10.1108/ 14684521011072963. García-Peñalvo, F. J., Hernández-García, Á., Conde-González, M. Á., Fidalgo-Blanco, Á., SeinEchaluce Lacleta, M. L., Alier-Forment, M., ... Iglesias-Pradas, S. (2015). Learning servicesbased technological ecosystems. In G. R. Alves & M. C. Felgueiras (Eds.), Proceedings of the third international conference on technological ecosystems for enhancing multiculturality (TEEM’15) (Porto, Portugal, October 7–9, 2015) (pp. 467–472). New York: ACM. García-Peñalvo, F. J., Johnson, M., Ribeiro Alves, G., Minovic, M., & Conde-González, M. Á. (2014). Informal learning recognition through a cloud ecosystem. Future Generation Computer Systems, 32, 282–294 https://doi.org/10.1016/j.future.2013.08.004. García-Peñalvo, F. J., & Seoane-Pardo, A. M. (2015). Una revisión actualizada del concepto de eLearning. Décimo Aniversario. Education in the Knowledge Society, 16(1), 119–144 https:// doi.org/10.14201/eks2015161119144.

14

Future Trends in the Design Strategies and Technological Affordances. . .

365

Garrison, D. R., & Anderson, T. (2003). E-Learning in the 21st century: A framework for research and practice. New York: RoutledgeFalmer. Garrison, D. R., Anderson, T., & Archer, W. (2003). A theory of critical inquiry in online distance education. In M. G. Moore & W. G. Anderson (Eds.), Handbook of distance education (pp. 113–127). Mahwah, NJ: Lawrence Erlbaum Associates. Gašević, D., Adesope, O., Joksimović, S., & Kovanović, V. (2015). Externally-facilitated regulation scaffolding and role assignment to develop cognitive presence in asynchronous online discussions. The Internet and Higher Education, 24, 53–65. Gómez-Aguilar, D. A., García-Peñalvo, F. J., & Therón, R. (2014). Analítica Visual en eLearning. El Profesional de la Información, 23(3), 236–245. Griffiths, D., & García-Peñalvo, F. J. (2016). Informal learning recognition and management. Computers in Human Behavior, 55A, 501–503. https://doi.org/10.1016/j.chb.2015.10.019. Gros, B., Lara, P., García, I., Mas, X., López, J., Maniega, D., & Martínez, T. (2009). El modelo educativo de la UOC. Evolución y perspectivas (2nd ed.). Barcelona, Spain: Universitat Oberta de Catalunya. Hanson, D., Maushak, N. J., Schlosser, C. A., Anderson, M. L., Sorensen, C., & Simonson, M. (1997). Distance education: Review of the literature (2nd ed.). Bloomington, IN: Association for Educational Communications and Technology. Hirsch, B., & Ng, J. W. P. (2011). Education beyond the cloud: Anytime-anywhere learning in a smart campus environment. In Proceedings of 2011 International Conference for Internet Technology and Secured Transactions (ICITST) (pp. 718–723). Abu Dhabi, United Arab Emirates: Conference on IEEE. Iansiti, M., & Levien, R. (2004). Strategy as ecology. Harvard Business Review, 82(3), 68–78. Jansen, S., Finkelstein, A., & Brinkkemper, S. (2009). A sense of community: A research agenda for software ecosystems. In 31st International Conference on Software Engineering - Companion Volume (pp. 187–190). Vancouver/Canada: ICSE-Companion 2009. Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education, 31(1), 30–43. Lane, L. (2012). Three Kinds of MOOCs. Retrieved from http://lisahistory.net/wordpress/2012/08/ three-kinds-of-moocs/. Lee, J. J., & Hammer, J. (2011). Gamification in education: What, how, why bother?. Academic Exchange Quarterly, 15(2), 146. Lerís López, D., Vea Muniesa, F., & Velamazán Gimeno, Á. (2015). Aprendizaje adaptativo en Moodle: Tres casos prácticos. Education in the Knowledge Society, 16(4), 138–157 https://doi. org/10.14201/eks201516138157. Llorens, F. (2014). Campus virtuales: De gestores de contenidos a gestores de metodologías. RED, Revista de Educación a distancia, 42, 1–12. Manikas, K., & Hansen, K. M. (2013). Software ecosystems – A systematic literature review. Journal of Systems and Software, 86(5), 1294–1306 https://doi.org/10.1016/j.jss.2012.12.026. Mayer, R. E. (2003). Elements of a science of e-learning. Journal of Educational Computing, 29(3), 297–313. https://doi.org/10.2190/YJLG-09F9-XKAX-753D. McGreal, R., Kinuthia, W., & Marshall, S. (Eds.). (2013). Open educational resources: Innovation, research and practice. Vancouver: Commonwealth of Learning and Athabasca University. Metcalfe, S., & Ramlogan, R. (2008). Innovation systems and the competitive process in developing economies. The Quarterly Review of Economics and Finance, 48(2), 433–446. https://doi. org/10.1016/j.qref.2006.12.021. Minović, M., García-Peñalvo, F. J., & Kearney, N. A. (2016). Gamification in engineering education. International Journal of Engineering Education (IJEE), 32(1B), 308–309. Morales, E. M., García-Peñalvo, F. J., & Barrón, Á. (2007). Improving LO quality through instructional design based on an ontological model and metadata. Journal of Universal Computer Science, 13(7), 970–979. https://doi.org/10.3217/jucs-013-07-0970.

366

B. Gros and F. J. García-Peñalvo

Noesgaard, S. S., & Ørngreen, R. (2015). The effectiveness of e-learning: An explorative and integrative review of the definitions, methodologies and factors that promote e-learning effectiveness. Electronic Journal of e-Learning, 13(4), 278–290. Observatory of Educational Innovation of the Tecnológico de Monterrey. (2014). Flipped learning. Retrieved from Monterrey, México: http://observatorio.itesm.mx/edutrendsaprendizajeinve rtido. Papaioannou, T., Wield, D., & Chataway, J. (2009). Knowledge ecologies and ecosystems? An empirically grounded reflection on recent developments in innovation systems theory. Environment and Planning C: Government and Policy, 27(2), 319–339. https://doi.org/10.1068/c0832. Pickett, S. T. A., & Cadenasso, M. L. (2002). The Ecosystem as a multidimensional concept: Meaning, model, and metaphor. Ecosystems, 5(1), 1–10. https://doi.org/10.1007/s10021-0010051-y. Ramírez Montoya, M. S. (2015). Acceso abierto y su repercusión en la Sociedad del Conocimiento: Reflexiones de casos prácticos en Latinoamérica. Education in the Knowledge Society (EKS), 16(1), 103–118 https://doi.org/10.14201/eks2015161103118. Retalis, S., Papasalouros, A., Psaromiligkos, Y., Siscos, S., & Kargidis, T. (2006). Towardsnetworked learning analytics—A concept and a tool. In Proceedings of the fifth international conference on networked learning (pp. 1–8). UK: Lancaster. Rosenberg, M. J. (2001). E-learning: Strategies for delivering knowledge in the digital age. New York: McGraw-Hill. Sanchez i Peris, F. J. (2015). Gamificación. Education in the Knowledge Society, 16(2), 13–15. Sánchez Prieto, J. C., Olmos Migueláñez, S., & García-Peñalvo, F. J. (2014). Understanding mobile learning: Devices, pedagogical implications and research lines. Education in the Knowledge Society, 15(1), 20–42. SCOPEO. (2013). MOOC: Estado de la situación actual, posibilidades, retos y futuro. Retrieved from Salamanca, Spain: http://scopeo.usal.es/wp-content/uploads/2013/06/scopeoi002.pdf. Sein-Echaluce Lacleta, M. L., Fidalgo-Blanco, Á., García-Peñalvo, F. J., & Conde-González, M. Á. (2016). iMOOC Platform: Adaptive MOOCs. In P. Zaphiris & I. Ioannou (Eds.), Proceedings of the learning and collaboration technologies. Third international conference, LCT 2016, held as part of HCI international 2016 (Toronto, ON, Canada, July 17–22, 2016) (pp. 380–390). Cham, Toronto, Canada: Springer International Publishing. Siemens, G. (2005). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2(1), 3–10. Siemens, G. (2014). Digital Learning Research Network (dLRN). Retrieved from http://www. elearnspace.org/blog/2014/11/18/digital-learning-research-network-dlrn/. Sonwalkar, N. (2013). The First Adaptive MOOC: A case study on pedagogy framework and scalable cloud architecture—Part I. MOOCs Forum, 1(P), 22–29. https://doi.org/10.1089/ mooc.2013.0007. Stöter, J., Bullen, M., Zawacki-Richter, O., & von Prümmer, C. (2014). From the back door into the mainstream: The characteristics of lifelong learners. In O. Zawacki-Richter & T. Anderson (Eds.), Online distance education: Towards a research agenda. Athabasca, Canada: Athabasca University Press. Subashini, S., & Kavitha, V. (2011). A survey on security issues in service delivery models of cloud computing. Journal of Network and Computer Applications, 34(1), 1–11. Tan, E., & Loughlin, E. (2014). Using ‘Formally’ informal blogs to reate learning communities for students on a teaching and learning programme: Peer mentoring and reflective spaces. In F. J. García-Peñalvo & A. M. Seoane-Pardo (Eds.), Online tutor 2.0: Methodologies and case studies for successful learning (pp. 163–175). Hershey: IGI Global. U.S. Department of Education - Office of Educational Technology. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. Retrieved from Washington, D.C.: https://tech.ed.gov/wp-content/uploads/2014/03/edm-la-brief.pdf. Wenger, E. C. (1998). Communities of practice: Learning, meaning, and identity. New York: Cambridge University Press.

14

Future Trends in the Design Strategies and Technological Affordances. . .

367

Wiley, D. A. (2002). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In D. A. Wiley (Ed.), The instructional use of learning objects. Bloomington, Indiana: Agency for Instructional Technology. Wilson, S., Liber, O., Johnson, M., Beauvoir, P., Sharples, P., & Milligan, C. (2007). Personal learning environments: Challenging the dominant design of educational systems. Journal of e-Learning and Knowledge Society, 3(3), 27–38. Yoshida, H. (2016). Perceived Usefulness of “Flipped Learning” on instructional design for elementary and secondary education: With focus on pre-service teacher education. International Journal of Information and Education Technology, 6(6), 430–434. https://doi.org/10.7763/ IJIET.2016.V6.727. Yu, E., & Deng, S. (2011). Understanding software ecosystems: A strategic modeling approach. In S. Jansen, J. Bosch, P. Campbell, & F. Ahmed (Eds.), IWSECO-2011 Software Ecosystems 2011. Proceedings of the Third International Workshop on software ecosystems. Brussels, Belgium, June 7th, 2011 (pp. 65–76). Aachen, Germany: CEUR Workshop Proceedings.

Begoña Gros obtained her Ph.D. in Pedagogy from the University of Barcelona in 1987. Currently, she holds the academic position of Professor at the University of Barcelona. She was Vice-rector of Research and Innovation at the Open University of Catalonia (2007–2012). She is the Director of the research group Environments and Materials for Learning (EMA). Her research activities are in the area of the use of ICT in education, digital games for learning, learning design, and innovation. In recent years she has also focused on emergent technologies for advanced education purposes. She is the author of more than 100 publications in the area of ICT use in education. She has coordinated and participated in national and international projects funded by the European Union. She is an associated editor of the journal “Cultura y Educación.” Further information: https://www.researchgate.net/profile/Begona_Gros

Francisco José García-Peñalvo completed his undergraduate studies in Computing at the University of Salamanca and University of Valladolid and his Ph.D. at the University of Salamanca. Dr. García-Peñalvo is the head of the GRIAL research group (InterAction and eLearning Research Group). His main research interests focus on eLearning, Computers and Education, Adaptive Systems, Web Engineering, Semantic Web, and Software Reuse. He has led and participated in over 50 research and innovation projects. He was Vice Chancellor for Innovation at the University of Salamanca between March 2007 and December 2009. He has published more than 300 articles in international journals and conferences. He has been the guest editor of several special issues of international journals (Online Information Review, Computers in Human Behavior, Interactive Learning Environments, etc.). He is also a member of the program committee of several international conferences and reviewer for a number of international journals. At present, he is the editor-in-chief of the International Journal of Information Technology Research and the Education in the Knowledge Society Journal. He is also the coordinator of the multidisciplinary Ph.D. program on Education in the Knowledge Society.

The Theory of Totally Integrated Education (TIE)

15

Theodore W. Frick

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . An Overview of TIE Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Big Picture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Emotions Structure Memories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Why TIE Theory? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TIE Theory Development Through Retroduction and Deduction . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction to TIE Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Twelve Kinds of Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Basic Predictions in TIE Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cases of Two Unique Education Systems that Illustrate Elements of TIE Theory . . . . . . . . . . . . SUNY Cobleskill . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bloomington Montessori School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What is TIE Theory Good for? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . New Research Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Designing New and Improved Curriculum for Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Task-centered Instruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary of the Theory of Totally Integrated Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix A: Defined and Undefined Terms in TIE Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Definitions of Basic Terms in TIE Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Undefined Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . More Definitions of Terms in TIE Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

370 371 371 372 373 373 377 377 379 387 387 389 392 392 393 394 395 396 396 396 397 397 399

T. W. Frick (*) Department of Instructional Systems Technology, School of Education, Indiana University Bloomington, Bloomington, IN, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_69

369

370

T. W. Frick

Abstract

The theory of totally integrated education (TIE) predicts that mental structures formed by learners are expected to be stronger when “knowing that one,” “knowing how,” and “knowing that” are integrated with learner emotions and intentions. Such whole, completely connected mental structures are expected to be less vulnerable to forgetting. TIE theory builds on seminal work of John Dewey, Charles Sanders Peirce, Maria Montessori, Elizabeth Steiner, George Maccia, Stanley Greenspan, Kenneth Thompson, Myrna Estep, Eric Kandel, David Merrill, and Jeroen van Merriënboer. Two unique extant cases of education systems are described which illustrate parts of TIE theory. A further strategy for improving curriculum is recommended, which is based on sequencing authentic, whole learning tasks from simple to complex. Most importantly, these learning tasks are expected to help students integrate nine kinds of cognition with emotions and intentions: recognitive, acquaintive, appreciative, protocolic, adaptive, creative, instantial, relational, and criterial. A variety of teaching methods can be used to implement such an improved curriculum. TIE theory does not prescribe specific instructional methods or practices; rather it provides a set of principles which can be used to evaluate curriculum itself. To the extent these principles are present in curriculum, TIE theory predicts that students are more likely to achieve curriculum goals. Keywords

Integration of cognition, emotion, and intention · Holistic teaching and learning · Definition of kinds of learning · Educational theory · System theory

Introduction The impetus for the theory of Totally Integrated Education (TIE) began six decades ago. Near the end of my first semester of college in 1966, I specifically remember walking through a beautiful wooded area of campus along a small stream. I still retain a clear mental image of that particular location on a bleak, cloudy, snowless wintry day. I was feeling very disillusioned, disappointed, and wondering: “This is higher education? There has got to be a better way to learn.” I dropped out of college in my sophomore year for about seven months and worked in factory jobs, questioning the worth of my education and why I should continue. I did return to the university, but about six years later I dropped out again – this time from my doctoral program and instead worked as a research associate in higher education. What was bothering me had become very clear by 1973 – how can I really know something? Not just to parrot other people’s words, but to understand deeply and to be clearly grounded in what I knew. I told my friends at the time, and later students, that I dropped out of school in order to “get an education.” I had come to the

15

The Theory of Totally Integrated Education (TIE)

371

realization that much of what I believed I did not really know. I felt incredibly ignorant that I needed to start my learning from scratch. I also had come to the realization that many other people were like me. I concluded that too many people just really did not know what they were talking or writing about. I vowed at that time to suspend belief about what I read or heard until I could verify things myself – in other words, to come to know, to really learn. To do this required that I discontinue my formal schooling for about seven more years to “get an education” through doing authentic tasks with real-world consequences. I have continued to learn this way throughout my life and career as a college professor. Fast-forward to 2010 when, during my final sabbatical, I began to tie together the numerous pieces of my previous work on systems theory, epistemology, learning theory, and instructional theory – coupled with decades of teaching experience. That was when these ideas became more formalized as the theory of totally integrated education (TIE). That theoretical work has been intertwined with my evolving work on educology (available at http://educology.indiana.edu and explicated in a different volume and chapter of this Master Reference Works series). What follows in this chapter is essentially a relatively quick tour of TIE theory. Full explication will require a much larger venue.

An Overview of TIE Theory The Big Picture I have taken classes in school where I was constantly asking myself: Why do I need to learn this stuff? This is so boring. I wish I were doing something else. Every once in a while, I was lucky to take a class that was terrific. My teacher was excellent. I was totally involved, completely immersed. I could not learn enough. This learning was very important to me. So, what was the difference between those boring classes I took and ones that were really great? In this chapter, I explain why we learn a lot more in some situations compared with others. First, emotion is important for learning and memory. Many people remember highly emotional experiences. They can usually tell you in great detail about a particularly thrilling, highly stressful, or a frightening experience – even if it occurred a long time ago. For example, I can still vividly recall when I was about 6 years old; I was riding my bicycle along the right side of the state highway on my way home. A boy next door yelled across the road, asking me to come over to his house to play. I started to turn left across the road, but quickly turned back to the side berm when I heard brakes screeching right behind me. A big truck stopped suddenly just in time to keep from running over me. And then several cars going too fast and following too closely behind the truck crashed into each other – a chain-reaction collision.

372

T. W. Frick

I was really scared. I could have been run over and crushed by the semi. A policeman later came to my house and asked me questions about what happened. I was afraid he would arrest me and that I would go to jail because I caused an accident. But he did not. I still clearly remember this traumatic event, more than 60 years ago.

Emotions Structure Memories Emotions organize and give meaning to experience. “They can, therefore, serve as the architect or orchestra leader for the mind’s many functions” (Greenspan & Shanker, 2004, location 658). Stanley Greenspan, a medical doctor with decades of clinical experience in treating autism, repeatedly observed that human emotion while engaging in an activity or while interacting with another person creates the architecture of that person’s mental structure. The dual coding of sensations and emotions arising from those experiences organize one’s mental structure. There is further evidence from molecular biology that supports Greenspan and Shanker’s claim. Kandel (1989), a Nobel-prize winning neuroscientist, claimed that “evidence suggests that learning produces enduring changes in the structure and function of synapses. . .” (p. 121). He recommended further study on “the power of experience in modifying brain function by altering synaptic strength. . .” (p. 123, italics added). According to Eagleman (2015), when we are born we each have approximately 100 billion neurons in our nervous system. During the first 2–3 years of life, our body creates trillions of connections among those neurons. As we further grow, develop, and learn, our individual experiences literally prune those connections, so that the remaining connections form a unique mental structure, the unique long-term memories we each have. Certain connections are strengthened through those experiences throughout our lives, and other connections are weakened. Emotions arising during life experience apparently strengthen certain connections (synapses) among neurons, forming the architecture of each of our minds. Greenspan and Benderly (1997) have noted that since the ancient Greek philosophers, the rational or cognitive aspect of mind has often been viewed as developing separately from emotion. They argue that this view has blinded us to the role of emotion in how we organize what we have learned: “In fact, emotions, not cognitive stimulation, serve as the mind’s primary architect” (p. 1). Greenspan and Benderly (1997) identify the importance of emotion during human experience: “. . . each sensation . . . also gives rise to an affect or emotion. . .. It is this dual coding of experience that is the key to understanding how emotions organize intellectual capacities . . .” (p. 18). The theory of totally integrated education (TIE) builds on these fundamental premises.

15

The Theory of Totally Integrated Education (TIE)

373

Why TIE Theory? Why is it that the majority of US high school students are bored every day in school? Yazzie-Mintz (2007) summarizes results from a survey of 81,499 students in 110 high schools across 26 US states. Approximately two out of three students said that they were bored in class every day. When asked why they were bored, the top reasons were that learning materials were uninteresting, irrelevant, and not challenging enough. Yazzie-Mintz cited one student who stated, “Our school needs to be more challenging. Students fall asleep because the classes aren’t really that interesting.” Another said, “School is easy. But too boring. Harder work or more is not the answer though. More interesting work would be nice” (p. 10). Students who considered dropping out of school indicated that the main reasons are dislike of their school and teachers. Sixty percent further said, “I didn’t see the value in the work I am asked to do” (p. 5). For those who stay in school, the primary reason they do so is to get their high school diploma, so that they can go on to college. Likewise, many of us have experienced taking classes in our formal education, or even on the job, during which we were thinking to ourselves, “Who cares about this subject? This is so boring. I am wasting my time.” Every once in a while, we may have been fortunate to take that rare class or course that was terrific. Our teacher was so inspiring. We spent hours absorbed completely, unaware of the passage of time. So, what was the difference between the former and latter experiences, from the point of view of the student – an utter waste of time versus experiencing elation, flow, and wanting to learn more? The theory of totally integrated education (TIE) aims to explain why. More importantly, TIE theory should help parents, teachers, curriculum developers, and instructional designers to create student learning experiences which more often result in the latter situation. Students will be thankful. Especially those who have to go to a place called school or college. Also, students who are learning on the job outside of a formal educational setting called school will be thankful.

TIE Theory Development Through Retroduction and Deduction Thompson (2006b) emphasized that the value of new theory is to predict the unknown: . . . the purpose of a theory is to provide the means to develop mathematical, analytical, or descriptive models that predict counterintuitive, nonobvious, unseen, or difficult-to-obtain outcomes. When all we are testing are outcomes that are preconceived, then we are missing the very purpose of scientific inquiry – to determine what it is that we do not know, rather than that which we have just not yet confirmed, or patterns that we have just not yet discerned. Confirmation of a hypothesis may be interesting and of limited value, but to call a body of knowledge that does nothing more than confirms perceptions of known events

374

T. W. Frick

is to trivialize the notion of theory to the point where any proclamation becomes a theory. (p. 16)

Thompson (2006b) provides an example of how quantum mechanics theory in physics has predicted unexpected, counterintuitive outcomes: Possibly the best example of theory development and results comes from quantum mechanics that has predicted so many counterintuitive events. The Josephson Effect, a quantummechanical effect in superconductors, is a specific example from physics. Holding two superconductors close to each other, there is a coupling of the quantum mechanical wave functions between them. The equations governing the theory of superconductivity predicted this coupling and laboratory testing quickly confirmed the prediction. . .. Voltage standards, highly-sensitive microwave detectors, high-density computer circuits and nanotechnologies, generally, have been developed with reliance on the Josephson Effect. Here, the theory predicted non-obvious outcomes, the very purpose of a theory. (p. 16)

New theory is not developed in a vacuum. Previous knowledge, experience, and observations do play a role, but the use of other extant theories via retroduction is also needed. Steiner (1988) argued that development of new theory cannot be done by inductive logic alone. Inductive reasoning is important when verifying theory, but retroduction is needed for creating new theory. Steiner (1988, p. 97) cites Peirce (1934) who specified the logic of retroductive inference: 1. The surprising phenomenon, C, is observed; and 2. but if A were true, C would be a matter of course; 3. hence, there is a reason to suspect that A is true. (Collected Papers, 5.189)

Steiner referred to the “theory models” approach with respect to retroductive inference. To illustrate the significance of retroduction and its application, Maccia and Maccia (1966) developed new educational system theory through use of the SIGGS theory model which was retroduced initially from set, information, graph, and general system theories. Through retroduction and deduction, the SIGGS theory model was used to devise new educational theory that consisted of 201 hypotheses. Thompson (2006a, b; 2008a, b) has further developed Axiomatic Theory of Intentional Systems (ATIS), that built upon the SIGGS theory model. The website at https://www.indiana.edu/~tedfrick/edutheo.html provides definitions of terminology in the SIGGS theory model with illustrations and examples. ATIS is explicated at http://educology.indiana.edu/Thompson/index.html. So, where did the theory of totally integrated education come from? There were several key sources. First, the idea that creation of new “mental structures” is the result of learning was critical (Greenspan & Benderly, 1997; Kandel, 1989, 2001; Squire & Kandel, 1999; Steiner, 1988). Structures can be represented through digraph theory (e.g., Brandes & Erlebach, 2005; Maccia & Maccia, 1966; Thompson, 2008b). Kandel (1989, 2001) demonstrated that when learning occurs, new connections are formed in the

15

The Theory of Totally Integrated Education (TIE)

375

nervous system via chemical strengthening of synapses. Long-term memory is enabled through these strengthened synaptic connections. The nervous system can be represented by a digraph, i.e., the unique network of connections among nerve cells. System theory was also a very important model that heavily influenced TIE theory. The notion of affect relations (connections among system components) and properties of affect-relation sets in systems was central (from SIGGS and ATIS). The structural properties “strongness,” “wholeness,” and “integration” were key ideas. In the SIGGS theory model (Maccia & Maccia, 1966), 22 structural properties are defined, for example: • “Complexness is the number of connections” (p. 62). • “Flexibleness is different subgroups of components through which there is a channel between two components with respect to affect relations” (p. 59). • “Strongness is not complete connectedness and every two components are channeled to each other with respect to affective relations” (p. 54, italics added). • “Complete connectionness is every two components directly channeled to each other with respect to affect relations” (p. 54, italics added). • “Wholeness is components which have channels to all other components” (p. 58, italics added). • “Integrationness is [maintenance of] wholeness under system environmental change” (p. 58, brackets added). • “Vulnerableness is some connections when removed produce disconnectivity with respect to affect relations” (p. 56). • “Disconnectionness is not either complete connectedness or strongness or unilateralness or weakness and some components are not connected with respect to affect relations” (p. 55, italics added). (See https://www.indiana.edu/~tedfrick/ siggs.html for definitions and examples of 77 system properties in the SIGGS Theory Model (Maccia & Maccia, 1966). Building on SIGGS, Thompson (2006a, b; 2008a, b) has further developed, refined and defined Axiomatic Theories of Intentional Systems (ATIS). See http://www.indiana.edu/~aptac/glossary/ and http://www.indiana.edu/~aptac/glossary/atisTheory.html.) In particular, the notion of increasing system “complexness” is the foundation for the definition of “learning” [and consistent with Kandel’s (1989) neurological findings]. The notion of “forgetting” was retroduced as decreasing system complexness, i.e., some connections are broken and no longer exist. Most importantly, the central ideas in TIE theory are system “wholeness” and greater “flexibility.” Deductively, when each component has two or more connections to every other component, it follows that such a network is less vulnerable when compared to a weakly or unilaterally connected network. That is, when some connections are broken, “wholeness” of long-term memory can still be maintained. Also, through deduction, “completely-connected” component sets should provide

376

T. W. Frick

greater flexibleness, i.e., because many different pathways exist between individual components. As an analogy, think of a spider’s web – when nodes are highly interconnected, small tears in a few places will not destroy the whole web, and allow a spider to make quick repairs. Therefore, to the extent wholeness among component sets is maintained or increased, then vulnerableness should decrease – forgetting of what has been learned is less likely. Thus, the retroductive prediction is that learning activities which increase wholeness in student mental structures should be more effective with respect to student learning (and not forgetting). Totally integrated education should maximize strong connections among kinds of knowing (cognition), intention (conation), and feelings (emotion). Student learning which occurs during totally integrated education should result in more stable long-term memory, less likely to be forgotten as time passes. A further key idea in TIE theory came from epistemological theory, in particular, Maccia’s identification of qualitative, quantitative, and performative knowing as being distinctive kinds of cognition (Estep, 2006; Frick, 1997; Maccia, 1986, 1987, 1988; Steiner, 1988). Again, this stimulated the idea that, if these three kinds of knowing are interconnected, this greater integration among kinds of knowing would further contribute to increased wholeness of mental structures. And finally, TIE theory predicts that integration of emotion, intention, and cognition through teaching and learning activities is expected to contribute to increased interconnectivity (i.e., strongness) of mental structures. Moreover, the roles of emotion and intention as critical elements to strengthening mental structures, according to the work of Greenspan and Benderly (1997), have contributed to the retroduction of TIE theory. When taken together these theoretical ideas have led to the definitions of terms and schemas represented in section “Introduction to TIE Theory” (Figs. 1–6). To my knowledge, this leads to a new prediction that has not been empirically tested in educational research. TIE theory predicts that, to the extent all these elements are integrated in teaching and learning activities, student mental structures will be stronger and less vulnerable to forgetting as time passes. A series of experiments can be designed, where disconnectivity is systematically increased (as illustrated in Figs. 18, 19, and 20, respectively) and learning outcomes could be measured. Learning achievement is predicted in TIE theory to be greatest under conditions illustrated in Fig. 18 below, and least in Fig. 20. If these predictions are confirmed, TIE theory has highly significant implications for improving our current systems of education (which typically are more like Fig. 20, when compared to Fig. 18). These are nonobvious and unseen outcomes, and demonstrate the value of TIE theory (see Steiner, 1988; Thompson, 2006b). In a similar vein, Einstein’s theory of relativity predicted the bending of light from distant stars by the gravitational pull from large masses (e.g., such as our sun). This bending of light waves was a nonobvious, counterintuitive, and unseen event prior to Einstein’s theory early in the twentieth century. Einstein’s theoretical prediction was nonetheless later empirically confirmed by new observations during a full solar eclipse (see Thompson 2006b).

15

The Theory of Totally Integrated Education (TIE)

377

While there are examples which implement parts of TIE theory (described in section “Cases of Two Unique Education Systems That Illustrate Elements of TIE Theory”), I am unaware of any existing education system that is designed to maximize total integration as described in TIE theory. When researchers and practitioners seek to make systemic change in education, TIE theory provides a foundation for predicting the effectiveness of newly designed education systems.

Introduction to TIE Theory When developing descriptive theory, we must have well-defined terms. Otherwise, theory lacks clarity. TIE theory makes clear distinctions between 12 kinds of learning – especially those kinds of learning that are within the domain of education and those that are not (see Fig. 1).

Twelve Kinds of Learning First, education is not the same as learning. Education is a subset of all learning. Fig. 1 illustrates the relationship between learning and education. Learning can occur without guidance – i.e., without teaching. To be education, however, learning must be both guided and intended (Steiner, 1988). Furthermore, not all education is effective or worthwhile. Some education can be ineffective. Some education can

Fig. 1 Venn diagram representation of kinds of learning and education (graphic design by Colin Gray and Ted Frick)

378 Fig. 2 Accidental learning: neither intended nor guided (Type 1)

Fig. 3 Guided learning (Type 2)

Fig. 4 Intended learning (Type 3)

T. W. Frick

15

The Theory of Totally Integrated Education (TIE)

379

Fig. 5 Conducive learning (education): intended and guided (Type 4)

Fig. 6 Ineffective education: neither instrumentally good nor intrinsically good (Type 5)

be effective but not worthwhile. Thus, these distinctions are made in the Venn diagram in Fig. 1. Key concepts from which definitions of types of learning are derived from this Venn diagram are further illustrated by specific shadings in Venn diagrams in Figs. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, and 14.

Basic Predictions in TIE Theory The theory of totally integrated education (TIE) describes what is necessary for worthwhile education (Type 7 learning, Fig. 8). Learning is the formation of new mental structures. Complexity of the connectedness of structures increases. TIE

380

T. W. Frick

Fig. 7 Effective education: instrumentally good (Type 6)

Fig. 8 Worthwhile education: instrumentally and intrinsically good (Type 7)

theory further predicts that mental structures will be strongest when student willing, feeling, and thinking are working in concert as she or he engages in learning tasks: • S intends to learn X, and • S feels strongly while learning X, and • S forms new mental structures for X (i.e., S learns X) where X is the integration of knowing that, knowing how, and knowing that one and S is the student. Strongness is a system structural property, as well as integration and wholeness. These properties of connectedness were described above, as well as in Axiomatic Theory of Intentional Systems (ATIS), developed by Thompson (2006a, 2008b). ATIS has further served as a theory model for development of TIE theory, and, in

15

The Theory of Totally Integrated Education (TIE)

381

Fig. 9 Discovery learning: intended but unguided (Type 8)

Fig. 10 Disciplined inquiry (research): discovery learning that is regulated by criteria (Type 9)

turn, set theory, di-graph theory, and general system theory have served as theory models for ATIS. Theory models are used in retroductive reasoning as part of new theory development (see Peirce, 1932; Steiner, 1988; Thompson, 2006b, 2008b). In contrast, deductive reasoning is used for creating logical implications derived from initial postulates and subsequent theorems. Finally, inductive reasoning is used for evaluating theorems and their deduced implications via empirical research in science and praxiology. See Steiner (1988), Peirce (1932), and Thompson (2006b). According to Greenspan and Benderly (1997), the emotion arising through engagement in a learning task creates the architecture of the student’s mental structure. This structure increases in complexity – there are more connections in a person’s mental structure than before, which is the definition of learning in TIE

382

T. W. Frick

Fig. 11 Compelled learning: guided but unintended (Type 10)

Fig. 12 Induced learning: guided but initiallyunintended (Type 11)

theory. The dual coding of sensations and emotions from that experience organizes the mental structure. Figure 15 illustrates graphically the desired relationship among thinking, willing, and feeling during the learning process. Cognition, intention, and emotions are temporally connected, rather than being at odds with each other (disconnected). Ideally the learning task is something that the student intends to do, his or her thinking is focused on the learning task, and she or he feels strongly about this activity. From a biological perspective, synapses that connect neurons in long-term memory are chemically strengthened under these conditions (see Kandel 2001). The theory of totally integrated education (TIE) predicts that when three kinds of knowing are integrated (i.e., “knowing that one,” “knowing how,” and “knowing that” are connected so as to remain whole), and when student cognition, intention, and feelings are temporally connected, then students will form stronger mental

15

The Theory of Totally Integrated Education (TIE)

383

Fig. 13 Effective bad education: instrumentally good but not intrinsically good (Type 12)

Fig. 14 All learning: accidental or discovery or conducive or compelled

structures. Strongly connected mental structures are less vulnerable to subsequent disconnection in long-term memory. Such structures are less vulnerable to forgetting. For example, in Fig. 16, the boy recognizes his unique dog, Rover (“knows that one”). He also “knows that” Rover is an instance of the class of dogs, and he “knows how” to give Rover a bath. A practical implication of TIE theory is that for education to be most effective, teachers should choose multiple learning tasks that result in student formation of strong connections among these kinds of knowing for each educational objective. Furthermore, these learning tasks should be authentic (i.e., selected from the existing culture in which students and teachers live), so that students can see the relevance of the tasks to their personal lives. If students see the relevance and purpose of the tasks, then they are more likely to be motivated to engage in the learning tasks (Keller, 1983). In the Yazzie-Mintz (2007) study, students who were

384

T. W. Frick

Fig. 15 Schema for desired connections among a student’s cognition (thinking), intention (willing), and emotion ( feeling) during a learning activity (graphic by Colin Gray)

Fig. 16 Illustration of three kinds of knowing (drawings by Elizabeth Boling)

considering dropping out of school said, “I didn’t see the value of the work I am expected to do” (p. 5). If educators design or choose learning tasks that are authentic, it will help students appreciate the value of those tasks. Appreciation is a type of qualitative knowing (“knowing that one”). Figures 17 and 18 illustrate further details and relationships among kinds of knowing. In particular, Fig. 18 shows full integration of the nine kinds of cognitive relations – they are completely connected, which is represented by the arrows between the rounded rectangles. From digraph theory and ATIS, complete connectedness is a system structural property. Furthermore, the embedding of the rounded rectangles illustrates the set theoretical relationship within each kind of knowing. For example, appreciation is a subset of acquaintance and acquaintance is a subset of recognition. Both recognition and acquaintance are necessary (but not sufficient) for appreciation. Thus, if a learning task connects appreciation of “that-one,” creative “know how,” and criterial “know that,” then it follows via deductive reasoning that

15

The Theory of Totally Integrated Education (TIE)

385

Fig. 17 Further explication of kinds of knowing (graphic by Colin Gray)

Fig. 18 Illustration of completely-connected knowing where student cognition, intention and emotions are in harmony – i.e., Fig. 15 is superimposed on Fig. 17 to result in this figure (graphic by Colin Gray)

the nine kinds of knowing must be connected since the other six are necessary for these three. Figure 19 illustrates mental structures that are not completely connected. Note the gaps (white space, lack of shaded areas): Criterial relationships are missing with

386

T. W. Frick

Fig. 19 Illustration of partially-connected mental structures (graphic by Colin Gray). The lack of shading in the rounded rectangles represents absence of connectivity – hence cognition and thinking, willing and feeling are not totally integrated

Fig. 20 Disconnected mental structures (graphic by Colin Gray). “Knowing how” and “knowing that one” are disconnected from “knowing that.” Student intention and emotion are disconnected from “knowing that”

respect to cognition, intention, and feeling. Relational (theoretical) connections are missing with respect to intention and feeling. Appreciative relationships are missing with respect to cognition, intention, and feeling. Creative relationships are missing with respect to intention and feeling; and adaptive relationships are missing with respect to feeling. Too often, the picture is even more empty – e.g., attempting to bring students to “know that,” without connecting “knowing that” to “knowing how” or “knowing

15

The Theory of Totally Integrated Education (TIE)

387

that one.” Furthermore, if students do not intend to learn (i.e., are compelled – Type 10 learning, Fig. 11), and their feelings are likewise out of sync (temporally disconnected), then what is illustrated in Figs. 18 and 19 largely disappears. Such a graphic would be mostly empty, with very little to show (Fig. 20). Under these conditions, TIE theory predicts that such disconnected learning – resulting in lack of wholeness of mental structures – will be highly vulnerable to forgetting. This is the kind of disconnected learning that students often experience in school when they are bored. They are required to learn facts and concepts they do not care about, which have no perceived practical value and which are disconnected from unique elements in their culture. TIE theory contains numerous well-defined terms that constitute an important part of the descriptive theory. These defined and undefined terms are provided in Appendix A, not because they are unimportant – they are vitally important – but because they will likely make more sense to readers after some real-world examples, which are discussed next. These and other defined terms, many of which have examples that elaborate definitions, are available on the Educology website: http:// educology.indiana.edu.

Cases of Two Unique Education Systems that Illustrate Elements of TIE Theory SUNY Cobleskill The State University of New York (SUNY) at Cobleskill has been recently implementing new cross-disciplinary programs that are attempting to integrate different kinds of subject matter and the three kinds of knowing that are part of TIE theory. Feldman (2016) describes new programs which include: • Food Systems and Technology • Fermentation Science and Applied Fermentation • Graphic Design In the Food Systems and Technology program, a student who had visited Puerto Rico evidenced “appreciative knowing that one” who said: After meeting the workers and managers, seeing the terrain, and feeling the weather, I can now pick up a humble banana and marvel at the journey it took to reach the local grocery store. This is a view of the modern food system that the vast majority of American consumers will never get to see. (p. 3)

Feldman (2016) further writes: The program pulls together four academic cores – sustainability, food policy and law, food production and science, and food business management – to teach students how to efficiently

388

T. W. Frick

and effectively produce, process and distribute food. The interdisciplinary approach also aims to instill the innovative mindset that will be needed to feed an ever-growing population. . .. Students in the program will get plenty of hands-on experience in the College’s livestock and dairy facilities, greenhouses, alternative energy labs and culinary facilities. (p. 3)

This new program illustrates student learning activities that help make connected mental structures among “knowing that,” “knowing how,” and “knowing that one” as described in TIE theory. In particular, the student visiting Puerto Rico has been learning about “that one” and evidences “appreciative knowing that one.” Furthermore, students in this program are getting hands-on experience, which illustrate connections between “knowing how” and “knowing that one” as they learn in the on-campus dairy facilities, greenhouses, etc. Through more traditional course work, students create mental structures for “knowing that” and “knowing how,” for example, in “ag[riculture] business, culinary arts, animal science, plant science, ag engineering and more” (Feldman, 2016, p. 3). The new program at SUNY Cobleskill on Fermentation Science and Applied Fermentation appears to be organized in a similar interdisciplinary approach: The programs are notable in their breadth, focusing on fermentation as it applies to fields like food and beverage production as well as pharmaceuticals, industrial manufacturing, environmental conservation, even renewable energy. . .. In using microbial processes to help turn waste into energy, the programs address sustainability and conservation; in growing fruits, vegetables or grains for a top-to-bottom farm brewery, they draw on agriculture; in producing foods like sauerkraut, kimchi, summer sausages, tea and coffee, they bring in the College’s Culinary Arts program. They even include business courses, a rarity for fermentation programs. (Feldman, 2016, p. 5)

An old building on the SUNY Cobleskill campus was recently converted into a new Design Center, . . . a place where design and the visual arts all come together under one roof. . .. The centralization was driven by the introduction of a four-year Graphic Design program. On the second floor of the building today, art studios and gallery spaces have been carved out of what used to be a large gymnasium. . .. The labs offer easy, reliable access to the kind of technology students will encounter in the work environment. . .. The student design club, Logos, has experienced a revival, as well. Previously a place for students to share ideas and take design-related field trips, the club now functions as a student-run design agency for other clubs and community organizations. Club members design logos and flyers as well as offering other creative design solutions on a real-world commission model. (Feldman, 2016, pp. 8–9)

What exemplifies TIE theory is that “creative knowing how” and “appreciative knowing that one” are interconnected through student learning in the Graphic Design program. Graphic Design students in the Logos club presumably are applying concepts and theories they have learned to their real-world project, and are likely applying “criterial knowing that” when evaluating and choosing design solutions during those projects with clients in the local community. Moreover, the connection

15

The Theory of Totally Integrated Education (TIE)

389

of intentionality, emotion, and cognition is likely formed through these projects. See Figs. 2–5 in section “Introduction to TIE Theory” above. In effect, what appears to be happening at SUNY Cobleskill is the integration of three kinds of knowing as illustrated in section “Introduction to TIE Theory,” Fig. 17 above. It is unclear from Feldman’s description the extent to which student cognition, intention and emotion are interconnected (Fig. 15). Nor is it clear if the new program is organized according to task-centered instruction – a series of whole, authentic tasks that are arranged from simple to complex. (See discussion in section “What is TIE Theory Good for?” below about First Principles of Instruction and the 4C/ID Model.) I currently do not know whether or not faculty and administrators at SUNY Cobleskill have been explicitly aware of TIE theory and consciously applying its principles. Descriptions of this emerging TIE theory have been available on the Web since 2011. See: • http://educology.indiana.edu/Frick/index.html • https://www.indiana.edu/~tedfrick/research.html#major%20presentations In any case, it appears that recent new programs on SUNY Cobleskill campus illustrate practical implementations of TIE theory, as their faculty are creating new curriculum alongside more traditional courses and programs. It further appears that SUNY Cobleskill exemplifies the process of systemic change in education at the post secondary level.

Bloomington Montessori School The Bloomington Montessori School (BMS) began as a private preschool program in 1968 for 3- to 6-year-old children in Bloomington, Indiana. The BMS has grown to include elementary programs for students ages 6–9 and 9–12. Students in the BMS upper-elementary mixed-age program are similar in age to those in grades 4 through 6 in more traditional public schools. However, the Montessori approach is very different in ways in which student learning is structured, as well as in the curriculum resources, and how they are used to support student learning. Each classroom in the BMS is run by a head teacher certified by the American Montessori Society. Koh and Frick (2010) conducted a case study of the BMS upper-elementary classroom, led by a male head teacher with 32 years of Montessori experience and two assistants, where 28 students were enrolled, ages 9–11 at the time of the study. The classroom’s physical appearance was notably different from a typical public school classroom in that the space was filled with rolling shelves containing “works” from the Montessori curriculum, stationary bookshelves containing a large number of books (not textbooks) for student use in doing research, science apparatus for conducting experiments, a few cages with birds and small animals and some potted plants, and appropriately sized tables and chairs spread out in nooks and crannies (some with desktop computers on them). Separate boys’ and girls’ restrooms were immediately

390

T. W. Frick

adjacent to the classroom. There was significant personal space in a further adjacent room for teachers to do preparatory work. The space, about the size of a large walk-in closet, contained private storage in cabinets and counter space for working. There was a further adjacent small observation booth with one-way glass windows for small groups of parents and visitors to observe the classroom (accessible only from a commons area outside the classroom). Most of the classroom was carpeted; many students would often do their work on the floor by using small mats to sit or lay on. Compared with public school classrooms, there were abundant curriculum resources stored in the upper-elementary classroom itself that were immediately available to students. There was also a massive attic area on the second floor above the classroom, where works were stored that were not in current use. Teachers would bring works into the classroom, depending on what students were learning at the time, and return works to the attic area that were no longer needed (typically when class was not in session). There was also a separate library nearby the upperelementary classroom in the same building, where students could also work independently, and which contained not only books but also more desktop computers. The BMS is incorporated as a nonprofit organization run by a volunteer board, who are elected by parents of children who attend. It is a relatively autonomous private school, not under control by the state as is the public education system. The BMS school building that included new elementary programs was originally designed in the early 1980s through a cooperative endeavor among teachers and parents working with an architect and constructed by a local builder. New classrooms of similar configurations have been more recently added to the original building to handle growing demand and enrollments. Central to the Montessori philosophy is that students are able to make their own individual choices about what to learn, when, and how long, in a carefully prepared and orderly environment which contains curriculum and hands-on materials readily available to students. For example, mathematics activities included works for arithmetic, fractions and decimals that are typical subjects taught at the elementary school level and also included works for more advanced students such as those covering set theory, algebra, and geometry – subjects typically learned at the high school level. There was a wide range of reading materials at various levels of reading difficulty. These books (not school textbooks) were relevant to student research projects and report writing – students did a large amount of research and writing activities in this upper-elementary class. Other works were available for geography with numerous maps and cultural information; and science works included apparatus for doing chemistry and physics experiments. Lillard (2008) has described many more aspects of the philosophy, methods, and curriculum resources that are part of Montessori’s method. These are not discussed here due to space limitations. With respect to organization of time at the BMS upper-elementary classroom Koh and Frick (2010) reported in their case study that the first hour of each day was typically spent doing “Head Problems” and the next 3 h was the “morning work period” before a break was taken for lunch. Each day the whole class would work on a Head Problem created by a teacher from current events or from other cultural artifacts in the community. For example, one of the head problems was “A Sunkist soda contains X amount of

15

The Theory of Totally Integrated Education (TIE)

391

caffeine. How many Hershey bars contain X amount of caffeine” (Koh & Frick, 2010, p. 9). Students not only needed to read and decode food nutrition labels on soda cans and candy bar wrappers, but also needed to understand measures of quantity such as ounces, grams, or milligrams and convert measures from one to another. They may have needed to consult other resources in the process and then use some basic algebra to answer the question. Each student was required to justify her or his conclusion by explaining how they arrived at facts and their reasoning process. Many head problems were based on local current events and illustrated how mathematics could be practically used. Students worked at their own pace individually or in small groups as they tried to solve that morning’s head problem. With respect to TIE theory, this head problem activity illustrated the connection of “knowing that,” “knowing how,” and “knowing that one.” Unlike much of the rest of the day, all students in the class worked on the same head problem during the first 30–45 min of the day. When a student finished the head problem, she or he individually moved on to their morning work period, typically lasting about 3 h. During this time interval, students chose individually the Montessori works that they wanted to do. When finished with one work, they would return it to the shelf where they got it and then would begin another work. What is notable is that, at any given time, a student was usually working on something different from what other students were doing in the classroom. Feedback was built into many works so students could judge themselves how well they were doing. Teachers were available to help students as needed, and to introduce new works to students when they were ready for them. But teachers tended to mostly be observing what students were doing and provided individualized student feedback as needed on the work each was attempting. Teacher-led, grouppaced activities were seldom observed during the morning work period. Koh and Frick (2010) noted: Montessori education is established upon the philosophy of helping each child attain selfmastery and independence. It emphasizes that students be given autonomy to engage freely with their learning environment. This case study of an upper elementary classroom found that the Montessori philosophy of education guided how teachers used autonomy-supportive strategies. Teachers supported student organizational autonomy by allowing them choice in terms of school work and work partners. They fostered cognitive autonomy by encouraging student independent thinking, encouraging self-initiation, and honoring students’ voice. . . Students surveyed [using the Academic Self-Regulation instrument] rated themselves highly in terms of intrinsic motivation for school work. (p. 1)

From the perspective of TIE theory, in this Montessori upper-elementary classroom student, intention, emotion, and cognition were connected through learning activities they pursued, as illustrated in Fig. 15 in section “Introduction to TIE Theory.” This contrasts with many traditional elementary schools where students have little choice of what to learn, when, and how long to spend on it. Teachers typically organize work periods by subjects (spelling, arithmetic, reading, science, etc.) which are group paced and teacher led – often based on a “sage on the stage” approach. This contrasts with typical Montessori classrooms, where teachers spend much of their time as “guides on the side.”

392

T. W. Frick

From the perspective of TIE theory, what appeared to be often missing is directly connecting what students were learning in the Montessori classroom to unique elements in the local community and culture (at least as observed in the case study by Koh & Frick, 2010). It should be noted that in Montessori’s curriculum beyond the upper-elementary level, students are expected to do more of their learning activities outside the classroom in their local communities (see A. Lillard, 2008; P. Lillard, 1996). The challenge, of course, in educating younger children is the need for adults to supervise and especially to address student safety concerns. Field trips into the local community setting provide logistical challenges that make such events relatively rare, when compared with the time spent in school classrooms during school hours. This contrasts with the mostly young adults who attend SUNY at Cobleskill, where more field work and learning are possible and are incorporated into their programs.

What is TIE Theory Good for? New Research Studies As mentioned in section “An Overview of TIE Theory,” the value of new theory is to predict unexpected, nonobvious, unseen, and counterintuitive outcomes. Does TIE theory do that? Empirical research studies can be designed which use Figs. 15–20 as a basis of designing different kinds of learning environments. Learning achievement – particularly long-term achievement gains compared with short-term gains – can be investigated by manipulating components of TIE theory. For example, Fig. 20 illustrates typical classroom learning in elementary, secondary, and postsecondary schools, where “knowing that” is disconnected from “knowing that one” and “knowing how.” TIE theory predicts that, under these conditions, student mental structures will be weaker, more disconnected, and more vulnerable to forgetting, especially if relationships illustrated in Fig. 15 are missing (i.e., lack of intention to learn and lack of emotional involvement in learning activities). On the other hand, Fig. 18 illustrates completely-connected cognition, intention, and emotion with “knowing that,” “knowing how” and “knowing that one.” These two kinds of contrasting systems (illustrated by Figs. 18 and 20) could be empirically compared on a number of dimensions – student motivation and satisfaction, attitude toward learning, mastery of expected learning outcomes, teacher satisfaction, and so on. Modern technologies in neuroscience could also be used to study brain activity under various conditions of learning, as alluded to by Eagleman (2015). TIE theory has further implications for schools without walls, that is, education systems which include local community and culture as integral parts of the education system as content for learning. This contrasts with exclusion of content about what happens outside classrooms in the local community. In other words, if students are learning in real-world contexts (i.e., literally through hands-on learning activities), would they be better able to connect “knowing that” and “knowing how” with authentic parts of their culture (with “knowing that one”)? After all, for tens of

15

The Theory of Totally Integrated Education (TIE)

393

thousands of years, humankind did learn in real-world contexts prior to more recent attempts via formal schooling in the twentieth and twenty-first centuries where students have largely been sequestered inside buildings – e.g., see examples of alternatives in Frick (1991).

Designing New and Improved Curriculum for Education One implication of TIE theory is that extant curriculum for education can be significantly improved. For example, see Figs. 19 and 20, which illustrate missing connections. Traditional curriculum in P-16 schools has been carved up to the point that it is often difficult for students to see the relevance and value of learning activities in those subject areas. This is one of the primary reasons students are bored and unmotivated to learn (Yazzie-Mintz, 2007). Metaphorically speaking, Humpty Dumpty has been broken into many small pieces; the whole of Humpty Dumpty has been lost in traditional curriculum subjects such as algebra, geography, science, history, reading, writing, spelling, grammar, and arithmetic. A century ago Dewey (1916) also observed this problem in education: . . . the bonds which connect the subject matter of school study with the habits and ideals of the social group are disguised and covered up. The ties are so loosened that it often appears as if there were none; as if subject matter existed simply as knowledge on its own independent behoof. . . irrespective of any social values. (p. 181) The subject matter of the learner is not . . . identical with the formulated, the crystallized, and systematized subject matter of the adult. . .. [which] enters into the activities of the expert and the educator, not that of a beginner, the learner. Failure to bear in mind the difference in subject matter from the respective points of teacher and student is responsible for most of the mistakes made in the use of texts [books] and other expressions of preexistent knowledge. (pp. 182–183)

In TIE theory, the goal is to help learners create mental structures that are whole by attempting to completely connect the kinds of knowing with intention and emotion. Learning tasks which are authentic and whole are needed. The criterion of authenticity requires that the tasks should be selected from what people in the social system and culture actually do and which contribute to that social system. Students need to learn to participate in that social system and to be productive members of society, contributing to overall well-being of the social system. The primary goal of education should be the transmission of good culture. This transmission has been vital to the advancement of human civilization. Dewey (1916) recommended the same approach as I now do in TIE theory: engage students in meaningful learning through direct hands-on experience. Dewey started a laboratory school at the University of Chicago, which still exists, in order to test his ideas. Although his notion of progressive education never caught on in mainstream education, more recent educational approaches as set out in Ten Steps to Complex Learning (van Merriënboer & Kirschner, 2013) and First Principles of

394

T. W. Frick

Instruction (Merrill, 2013) are consistent with fundamental ideas that Dewey originally promoted, and provide clear elaborations of important ideas.

Task-centered Instruction While it is useful to have current examples of meaningful education, the point here is not to advocate particular approaches such as the Bloomington Montessori School or SUNY Cobleskill as the answer to education’s problems. Rather, the goal is to use the principles from TIE theory to design and develop curriculum and instruction organized by authentic tasks. Noteworthy is that Merrill, Barclay and van Schaak (2008) have articulated the difference between task-centered instruction and topic-centered instruction. For a specific example, see the report by Mendenhall et al. (2006). The same goals of topic-centered instruction (subject matter to be learned) can be achieved by taskcentered instruction. A significant challenge is to design and sequence such authentic whole tasks without generating student cognitive overload. Too much cognitive load will interfere with learning (see van Merriënboer, Kirschner & Kester, 2003). One instructional design approach shows promise in how to sequence tasks for complex learning while managing student cognitive load: the four-component instructional design (4C/ID) model, further explicated by Ten Steps to Complex Learning (van Merriënboer & Kirschner, 2013). They recommend that instructional designers begin by identifying an authentic, whole, real-world task as the goal. Since such a complex task would be overwhelming for a novice, the essential relationships of the whole task are identified, so that versions of the whole task can be arranged from simple to complex. Each version of the whole task is called a task class. Within each task class, variations of the task are created, along with supportive information, justin-time procedural information, and part-task practice (if needed). As the learner completes variations of the whole task in the task class, the amount of support (e.g., teacher feedback, coaching, scaffolding) is gradually faded until the student can perform the entire task independently. Then the student moves to the next task class, which is a little more complex, and the whole cycle is repeated. Students proceed through task classes in this manner until they can successfully perform the original authentic, whole, and complex task that was identified as the goal. What is noteworthy about Dewey’s philosophy, Bloomington Montessori School, SUNY Cobleskill, First Principles of Instruction and the Ten-Steps-to-ComplexLearning model is that students are expected to engage in authentic, purposeful tasks or projects. Education can be organized by these tasks/projects, rather than by traditional subjects, and the levels of complexity of these tasks can be matched to what students are capable of doing at their current level of development. The same educational goals can be achieved with respect to traditional curriculum standards, but the means of accomplishing these goals would be different.

15

The Theory of Totally Integrated Education (TIE)

395

When instructional and learning activities are grounded in authentic tasks, a further benefit is that students will come to “know that one” in the process of learning. As Estep (2006) observed, tasks for “knowing how” are grounded in particulars – i.e., such authentic tasks are performed in a unique context. Furthermore, many kinds of “knowing how” benefit from “knowing that.” In other words, students can apply generalizable concepts, relations, and criteria as they carry out a specific task. Performance of these tasks can help students connect their mental structures for “know how” with “know that one” and “know that.” Mental structures are literally tied together.

Summary of the Theory of Totally Integrated Education Totally integrated education results in completely connected student knowing. Completely connected knowing results in holistic student mental structures. That is, students should form interconnected mental structures for: • Knowing that one (right opinions: recognizing, becoming acquainted with, and appreciating authentic uniques in one’s culture) • Knowing how to do (effective and ethical action: protocolic, adaptive, and creative conduct) • Knowing that (true beliefs: understanding concepts, explicating theories, and applying rational criteria for judgment) Students are expected to form mental structures for right opinions, effective and ethical doings, and true beliefs – that is, the goal should be worthwhile education. Students should not form mental structures for wrong opinions, for ineffective or unethical doings, or for false beliefs. When teachers select or create learning activities that help students to appreciate unique elements of their culture, to be creative in their doings, and to rationally judge kinds of objects and their relationships according to norms, then students are predicted to form more completely-connected mental structures, as illustrated in Figs. 17 and 18. Such strong mental structures are predicted to be less vulnerable to forgetting. When emotion, intention, and thinking are temporally connected through learning tasks in which students are engaged, mental structures are more likely to be strengthened (see section “Introduction to TIE Theory,” Figs. 15, 16, 17, and 18). For further explication of these ideas, and in particular a glossary of terms used in TIE theory, see the Educology (2017) website: http://educology.indiana.edu/ index.html. The acronym, TIE, expresses that the central idea is literally to tie together ideas in our minds through interconnected mental structures formed through intentional, emotional, and cognitive experiences.

396

T. W. Frick

Conclusion This journey started during a walk along a small stream in a beautiful wooded area on campus when I was a college freshman feeling disillusioned and disappointed about my educational experience. Over 50 years later, I have identified and articulated what was lacking at that time. Organized now, TIE theory emerged from educology. I believe that TIE theory can inform our attempts to improve our current education systems. Rather than proceeding by the trial-and-error method of previous times, TIE theory will help us describe and evaluate instrumentally and intrinsically valuable education.

Appendix A: Defined and Undefined Terms in TIE Theory Definitions of Basic Terms in TIE Theory In order to explicate theory, it is necessary to define terms. Steiner (1988) states it this way: . . . when one sets forth the terms of the theory and their definitions, descriptive metaphysics is presented. . .. Descriptive metaphysics is a division of the phenomena which are the object of theorizing – the system – so that a set of descriptors characterizing the system’s properties emerges. To do this, the metaphysician must provide a set of class terms for characterizing each and every component of the system. . .. Therefore, classification is basic to descriptive metaphysics. However, classification always involves definition. A class term denotes all the particulars to which the term is applicable (the extension of the term) and connotes the characteristics that a particular must have in order for the term to be applicable to it (the intension of the term). (Steiner, 1988, p. 64)

Steiner provides criteria for evaluating descriptive theory: exactness, exclusivity, exhaustiveness, external coherence, extendibility, equivalence, chaining and substitution (pp. 64–74). Descriptive theory is necessary for building a foundation before explanatory theory can be explicated. Fundamental to TIE theory are the following defined terms (“=Df ” is read as “is defined as”) (These and other terms are defined at http://educology.indiana.edu. This website provides definitions of these terms and more. It is easier to follow the chains of definitions on the website by clicking on the hyperlinks.): • Mental structures = Df affect-relations which constitute intelligence (Certain terms are defined elsewhere by Thompson (2008a). See http://www.indiana. edu/~aptac/glossary/. These terms, defined in Axiomatic Theories of Intentional Systems (ATIS) can also be viewed at http://educology.indiana.edu. (Those defined terms include: affect-relations, complexity, system environment, intentional system, and complete connectedness). Mental structures can be formed for

15

The Theory of Totally Integrated Education (TIE)

397

right and wrong opinions, for effective, ineffective, ethical and unethical conduct, and for true or false beliefs.) • Learning = Df increasing of complexity of a person’s mental structure (for Types 1–12) • Learner = Df person whose volition is learning • Forgetting = Df decreasing of complexity of a person’s mental structure I have been discussing ‘mental structure’ above, and now I must be more precise. I take some definitions here from general system theory, and in particular, Axiomatic Theories of Intentional Systems (Thompson, 2006a, b; 2008a, b). “Affect-relations” are the connections among components of a system, and “complexity” is the number of connections. Thus, learning is defined as increasing the number of connections in a one’s mental structure. This is consistent with what Kandel (1989) has concluded on a biological level, claiming that long-term memory is “associated with growth in synaptic connections [among neurons]” (p. 115) and that “learning produces enduring changes in structure and function of synapses” (p. 121). The biological explanation of changes in the human nervous system is not part of TIE theory. TIE theory asserts that humans form mental structures as they learn. To use Steiner’s criterion, there is external coherence. This definition of learning in TIE theory has external coherence with biological knowledge.

Undefined Terms Some terms in a theory must remain undefined (Steiner, 1988). Definitions could go on ad infinitum if there are no primitive terms. This is to avoid circularity in definitions, as well as infinite regress. Undefined terms in TIE theory follow: intelligence, think, feel, intend, believe, perceive, guide, person, good, object (thing), course of action (conduct), end (goal).

More Definitions of Terms in TIE Theory The domain of human learning is shown as a Venn diagram in Fig. 1, which illustrates defined terms that include “intended learning,” “guided learning,” “education,” “effective education,” and “worthwhile education.” Figures 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, and 14 illustrate via shadings in the Venn diagram how these terms are related but yet distinct: • Accidental learning = Df learning which is neither guided nor intended (see Fig. 2) • Discovery learning = Df learning which is intended but unguided (see Fig. 9) • Compelled learning = Df learning which is not intended but guided (see Fig. 11) • Conducive learning = Df education = Df learning which is both intended and guided (see Fig. 5) • Student = Df a person who intends to learn content with a teacher

398

T. W. Frick

• Teacher = Df a person who intends to guide another person’s learning • Teaching = Df a teacher guiding another person’s learning (see Fig. 3) • Sign = Df representamen = Df “something which stands to somebody for something in some respect or capacity. . .. every representamen being thus connected with three things, the ground, the object, and the interpretant” (see Peirce, 1932, 2.228) – Interpretant = Df a sign derived by a person as a mental construct that is a representamen of the equivalent external sign, which relates to an object • Content = Df objects and signs of objects selected for student learning • Context = Df system environment of teacher and student that contains content • Education system = Df intentional system consisting of at least one teacher and one student in a context • Knowing = Df mental structures which consist of warranted beliefs, right opinions, and capabilities for performance (C. S. Peirce (1877) discussed four methods of fixating belief: tenacity, authority, agreeableness to reason, and science. Scientific method (or more generally disciplined inquiry) means that any rational agent can repeat the same method and should come to the same conclusion. (see Figs. 15, 16 and 17). Other mental structures can result from learning, such as beliefs that are unwarranted by the method of science, such as authority or agreeableness to reason. Learning can also create mental structures for wrong opinion and for ineffective and unethical conduct.) – Knowing that one: mental structures for right opinion • Recognitive: select the unique Q (Q is the unique object of knowing.) from not-Q and not-Q from Q. • Acquaintive: identify relations determinate of the unique Q. • Appreciative: identify relations appropriate of the unique Q. – Knowing how: mental structures for effective performance • Protocolic: take one path to goal. • Adaptive: take alternative paths to goal, choosing or combining paths based on specific conditions. • Creative: innovate or invent a new way to reach an existing or new goal. – Knowing that: mental structures for beliefs warranted by disciplined inquiry • Instantial: classification of objects of the same kind. • Relational: rational explanation of relationships between kinds of objects. • Criterial: rational judgment of kinds of objects and their relations according to a norm. • Knowledge = Df record of knowing = Df preservation of signs that represent what is known in some medium external to knower. • Disciplined inquiry = Df rigorous research = Df learning which is regulated by criteria for creating scientific, praxiological, and philosophical knowledge. (Of course, persons who are called teachers can work together with students in

15

• • • • • •

The Theory of Totally Integrated Education (TIE)

399

disciplined inquiry. In this case they are both intending to learn something that is unknown to either. In this sense, the teacher is not acting as a guide because he or she does not know their destination. Rather they are exploring together – attempting and intending to learn something new. The process of disciplined inquiry is regulated by criteria. This is different from when a teacher is leading a student to a known outcome, such as repeating an experiment that has already been done – e.g., by dropping a feather and a golf ball in a vacuum, to “discover” that their acceleration is the same. The student might learn something new in this case, but not the teacher. Isaac Newton did not have a teacher to lead him to discover the laws of gravity. Rather, he did this through disciplined inquiry.) (See Fig. 10.) Instrumentally good = Df means that are good for an end (goal) – Means = Df course of action, a way to reach an end (goal) Intrinsically good = Df means or ends that are good in themselves, not with respect to their instrumental goodness Effective Education = Df education that is instrumentally good (Steiner, 1988, pp. 16–17) (see Fig. 6) Effective Bad Education = Df education that is instrumentally good but not intrinsically good (see Fig. 13) Worthwhile Education = Df education that is both instrumentally and intrinsically good (Steiner, 1988, p. 17) (see Fig. 8) Totally Integrated Education = Df education that results in student completely connected knowing, intention and feeling (see Fig. 18)

References Brandes, U., & Erlebach, T. (Eds.). (2005). Network analysis: Methodological foundations. Heidelberg, Germany: Springer. Dewey, J. (1916). Democracy and education. New York, NY: The Free Press. Eagleman, D. (2015). The brain. New York, NY: Pantheon Books. Educology. (2017). Knowledge of education. Retrieved January 27, 2017 from: http://educology. indiana.edu Estep, M. (2006). Self-organizing natural intelligence: Issues of knowing, meaning and complexity. Dordrecht, The Netherlands: Springer. Feldman, J. (Ed.). (2016). SUNY Cobleskill magazine. Albany, NY: Fort Orange Press. Frick, T. W. (1991). Restructuring education through technology. Bloomington, IN: Phi Delta Kappa Education Foundation. Frick, T. W. (1997). Artificially intelligent tutoring systems: What computers can and can’t know. Journal of Educational Computing Research, 16(2), 107–124. Greenspan, S. I., & Benderly, B. L. (1997). The growth of the mind and the endangered origins of intelligence. Reading, MA: Addison-Wesley. Greenspan, S. I., & Shanker, S. G. (2004). The first idea: How symbols, language, and intelligence evolved from our primate ancestors to modern humans. Cambridge, MA: Da Capo Press (Kindle edition).

400

T. W. Frick

Kandel, E. R. (1989). Genes, nerve cells, and the remembrance of things past. Journal of Neuropsychiatry, 1(2), 103–125. Kandel, E. R. (2001). The molecular biology of memory storage: A dialogue between genes and synapses. Science, 294, 1030–1038. Keller, J. M. (1983). Motivational design of instruction. In C. M. Reigeluth (Ed.), Instructionaldesign theories and models: An overview of their current status (pp. 383–434). Mahwah, NJ: Lawrence Erlbaum Associates. Koh, J., & Frick, T. (2010). Implementing autonomy support: Insights from a Montessori classroom. International Journal of Education, 2(2:E3), 1–15. Lillard, A. S. (2008). Montessori: The science behind the genius. New York: Oxford University Press. Lillard, P. P. (1996). Montessori today: A comprehensive approach to education from birth to adulthood. New York, NY: Schocken Books. Maccia, E. S. & Maccia, G. S. (1966). Development of educational theory derived from three educational theory models. Washington, DC: Final Report, Project No. 5–0638, U.S. Department of Health, Education, and Welfare. Maccia, G. S. (1986). Right opinion and Peirce’s theory of signs. Paper presented at the Semiotic Studies Faculty Seminar. Retrieved January, 27, 2017 from: http://educology.indiana.edu/ Maccia/RightOpinionAndPeircesTheoryOfSigns_GSMaccia1987.pdf Maccia, G. S. (1987). Genetic epistemology of intelligent natural systems. Systems Research, 4(3), 213–218. Retrieved January, 27, 2017 from: http://educology.indiana.edu/Maccia/Correspon dence_GeneticEpistemologyOfIntelligentNaturalSystems1987.pdf Maccia, G. S. (1988). Genetic epistemology of intelligent natural systems: Propositional, procedural and performative intelligence. Paper presented at Hangzhou University, China. Retrieved January, 27, 2017 from: http://educology.indiana.edu/Maccia/GeneticEpistemologyOfIntelligen tSystems_propositionalProceduralPerformativeIntelligence1988.pdf Merrill, M. D., Barclay, M., & van Schaak, A. (2008). Prescriptive principles for instructional design. In J. M. Spector, M. D. Merrill, J. van Merriënboer, & M. F. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 173–184). New York: Lawrence Erlbaum Associates. Mendenhall, A., Buhanan, C. W., Suhaka, M., Mills, G., Gibson, G. V., & Merrill, M. D. (2006). A task-centered approach to entrepreneurship. Tech Trends, 50(4), 84–89. Peirce, C. S. (1932). Collected papers, Vol. II, Elements of logic (C. Hartshorne & P. Weiss, Eds.). Cambridge, MA: Harvard University Press. Peirce, C. S. (1934). Collected papers, Vol. V, Pragmatism and pragmaticism (C. Hartshorne & P. Weiss, Eds.). Cambridge, MA: Harvard University Press. Squire, L. R., & Kandel, E. R. (1999). Memory: From mind to molecules. New York, NY: Henry Holt and Co. Steiner, E. (1988). Methodology of theory building. Sydney: Educology Research Associates. Thompson, K. R. (2006a). “General system” defined for predictive technologies of A-GSBT (Axiomatic General Systems Behavioral Theory). Scientific Inquiry Journal, 7(1), 1–11. Thompson, K. R. (2006b). Axiomatic theories of intentional systems: Methodology of theory construction. Scientific Inquiry Journal, 7(1), 13–24. Thompson, K. R. (2008a). ATIS glossary. Retrieved January, 27, 2017 from http://www.indiana. edu/~aptac/glossary/ Thompson, K. R. (2008b). ATIS graph theory. Columbus, OH: Systems Predictive Technologies. Retrieved January, 27, 2017 from: https://www.indiana.edu/~aptfrick/overview/reports/ 11ATISgraphtheory.pdf van Merriënboer, J. J., Kirschner, P. A., & Kester, L. (2003). Taking the load off a learner’s mind: Instructional design for complex learning. Educational Psychologist, 38(1), 5–13.

15

The Theory of Totally Integrated Education (TIE)

401

van Merriënboer, J. J., & Kirschner, P. A. (2013). Ten steps to complex learning: A systematic approach to four-component instructional design. New York, NY: Routledge. Yazzie-Mintz, E. (2007). Voices of students on engagement: A report on the 2006 high school survey of student engagement. Retrieved January, 27, 2017 from http://www.eric.ed.gov/PDFS/ ED495758.pdf

Theodore W. Frick is Professor Emeritus in the School of Education at Indiana University Bloomington where he taught and mentored graduate students in Instructional Systems Technology for 29 years. His primary research and development interests include Analysis of Patterns in Time (APT), computer adaptive testing, computerized classification testing, systems thinking, designing usable websites, computer simulations and tutorials, and evaluating instructional effectiveness. In addition to APT, a research methodology he invented, his most important theory development includes educology and Totally Integrated Education (TIE).

Roles and Competencies of Educational Design Researchers: One Framework and Seven Guidelines

16

Susan McKenney and Saskia Brand-Gruwel

Contents Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conducting Design Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multiple Phases of Design Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multiple Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Crosscutting Competencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developing Design Researcher Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design Researcher Learning Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Situated and Whole-Task Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Guidelines for Design Researcher Learning Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

404 405 405 408 411 414 414 414 418 422 424

Abstract

Design research is a genre of inquiry in which the iterative development of solutions to problems in practice provides the setting for scientific inquiry. Design researchers and practitioners collaborate to analyze the problems being tackled and to develop and refine solutions, which are informed by (formative) evaluation along the way. In these studies, the function of the investigator is typically multifaceted, including the roles of consultant, designer, and researcher. While most design researchers are afforded formal opportunities to develop their research skills (e.g., through seminars and courses on research design, interview S. McKenney (*) ELAN, Department of Teacher Professional Development, Faculty of Behavioral, Management and Social Sciences, University of Twente, Enschede, The Netherlands e-mail: [email protected] S. Brand-Gruwel Faculty of Psychology and Educational Sciences, Open University of the Netherlands, Heerlen, The Netherlands e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_123

403

404

S. McKenney and S. Brand-Gruwel

techniques, data analysis, etc.), the consultant and designer skills receive far less explicit attention and tend to be learned informally, at best. If design research is to realize its potential contribution to the field of learning and instruction, then explicit attention must be given to holistically developing design researcher capacity. This chapter first discusses design research, with attention to the goals, nature, and processes of this approach, how each role is relevant to each process, and foundational competencies that are required to enact the roles. Then, the chapter turns toward developing design researcher capacity. First, a framework for design researcher learning is introduced, followed by consideration of how that learning takes place, and culminating in guidelines for developing design researcher learning trajectories. The chapter concludes with discussion of these ideas in light of educational research capacity in general. Keywords

Design research · Design-based research · Competencies

Rationale Educational research increasingly focuses on conducting practice-based inquiry to get more insight in and understanding of instructional practices in specific settings and how to take further actions to improve education and, ultimately, foster student learning. Conducting practice-based educational research is also seen as a vehicle for teacher professionalization. While conducting this research, teachers in schools often work together with educational researchers from universities or institutes for higher education. Increasingly, research conducted in collaboration with educational practitioners can be characterized as educational design research. Design research is an important genre of research in the field of learning and instruction. In design research, practitioners and researchers work together to produce meaningful change in contexts of practice (DBRC, 2003). Through the collaborative process, empirical investigation takes place, and valuable insights are gained for the development of learning theories as well as learning resources (Hoadley, 2004). Commensurate with its twin goals of meaningful change in practice and deriving theoretical understanding, design research communities are characterized with “innovativeness, responsiveness to evidence, connectivity to basic science, and dedication to continual improvement” (Bereiter, 2002, p. 321). The “social design” of educational research in general (cf. Wagner 1997) and design research in particular (cf. Barab et al. 2007) plays an important, if not determining, role in shaping design research activities (Ormel, Pareja Roblin, McKenney, Voogt, & Pieters, 2012). Researchers and practitioners take on multiple roles during design studies, and these shift over time (McKenney, 2017). However, one can question if researchers and teachers are well prepared to conduct this type of research. Most academic master programs and teacher training programs do not incorporate the skills needed to conduct educational design research in their

16

Roles and Competencies of Educational Design Researchers: One. . .

405

curriculum. And while doctoral researchers are afforded formal opportunities to develop their research skills (e.g., through seminars and courses on research design, interview techniques, data analysis, etc.), the development of other competencies required for this kind of research receives far less explicit attention. Most of the time, researchers and teachers develop design research skills informally, through experience. Further, crosscutting and foundational competencies underpin the skill set affiliated with each role. The purpose of this chapter is to create awareness that doing educational design research is a complex skill and to highlight how training, support, and guidance can be given to develop design researcher capacity. Specifically, this chapter conceptualizes design researcher learning that stands to benefit collaboration with practitioners and ultimately contribute to the learning of teachers and their students. In the first half of the chapter, we discuss design research and what it requires. First, drawing on nearly two decades of experience in conducting and mentoring design research, as well as literature on both design research and the design and implementation of instructional innovations, the tasks undertaken in each core design research process are related to three main roles: consultant, designer, and researcher. Second, each role is described, and research-based factors known to contribute to the performance of each role are explained. Third, in relation to the roles, four crosscutting design researcher competencies are described: empathy (e.g., fostered when exploring (un)shared goals or becoming exposed to the incentives, motives, and reward structures in different settings), orchestration (e.g., developed by simultaneously attending to research framing, data collection, solution design, implementation, infrastructure woes, and stakeholder ownership), creative and analytical flexibility (e.g., learned while optimizing the human and material resources available in ways that remain aligned with instructional goals), and social competence, including robust repertoire of interaction strategies (e.g., developed largely through exposure). Building on this, the second half of the chapter focuses on developing design researcher capacity. It begins with a framework that articulates crucial design researcher capacities in relation to each phase of the process. Next, principles of situated and whole-task learning are described. Then, we offer guidelines to design researchers and their mentors for creating learning trajectories that foster educational design research capacity. In the conclusion of the chapter, we reflect on the significance of these ideas for other forms of research and for design researchers in particular.

Conducting Design Research Multiple Phases of Design Research Despite the rich variation in approaches to design research, several characteristics of this genre are defining and universal. First, design research features twin goals of deriving new scientific understanding and addressing real-world problems in practice. The scientific understanding produced through design research can be used to

406

S. McKenney and S. Brand-Gruwel

describe, explain, or predict specific phenomena. Sometimes the findings of design research are used for more normative purposes, such as the design principle database with research-based guidelines for technology-enhanced learning in science (Kali, 2006). Design research yields varied kinds of interventions to address problems in practice, including programs, processes, products, and/or policies. Second, to achieve these goals, design studies share certain characteristics. Specifically, design studies are (McKenney & Reeves, 2012) theoretically oriented (building on as well as producing theoretical understanding), interventionist (integrated in research and development efforts to render productive change in practice), collaborative (working with practitioners and other stakeholders), responsively grounded (steered by empirically based insights), and iterative (featuring successive cycles of investigation over time). Third, while specific processes vary greatly, several key processes are present across design research endeavors. Shown in Fig. 1, McKenney and Reeves (2012) identify four key phases: analysis and exploration, design and construction, evaluation and reflection, and – concurrent with each – implementation and spread. As discussed in the remainder of this section, each phase features different core tasks and thus requires a diverse set of researcher competencies.

Analysis and Exploration The analysis and exploration phase yields a better understanding of the problem to be addressed. After initial orientation to the main issues, literature review is conducted to understand and frame investigation of the problem, context, and other relevant issues. Field study is conducted to understand the root causes of the problem(s), identify element issues worth tackling, and portray any affordances and limitations that should be taken into consideration during design (e.g., stakeholder concerns). Networking and site visits are undertaken to explore other settings in which similar problems have been tackled. The process of reaching out to practitioners, experts, and researchers begins to create a network of “critical friends” who may be able to inform the research. This phase yields a descriptive and

Fig. 1 Generic model for conducting educational design research (McKenney & Reeves, 2012)

16

Roles and Competencies of Educational Design Researchers: One. . .

407

explanatory definition of the problem to be tackled and a long-range goal. In addition, initial notions about potential solutions (e.g., constraints, imperatives, possibilities) may be generated. For example, Boschman, McKenney, and Voogt (2014) described an investigation during this phase, which yielded a better understanding of the intuitive decisions teachers make when designing technologyrich learning environments. Depending on the problem, context, and stakeholders involved, quest for understanding the existing situation involves the problem owners (typically practitioners) and often experts.

Design and Construction Interventions to address the problem are explored and mapped out during design, then built and refined during construction. The processes of design and construction are systematic and intentional, but they also include inventive creativity, application of emerging insights, and openness to serendipity. Throughout this phase, ideas about how to address the problem tend to start off rather large and vague; and gradually they become refined, pruned, and operationalized. The work is guided by theory, as well as local expertise and inspiring examples. During design, potential solutions are explored by generating ideas, considering each, and checking the feasibility of ones that seem the most promising. Once a limited number of options have been identified, potential solutions are gradually mapped from a skeleton design to detailed specifications. Then, the solution is constructed, usually through a process of prototyping. Early prototypes tend to be incomplete; sometimes several are tested. Later versions are usually more detailed and functional. Often, the design and/or construction processes lead to new insights, prompting new cycles (e.g., revisiting the setting for additional context analysis). Two main types of outputs emerge from this phase: products describing design ideas for the intervention (e.g., key characteristics of learning activities) and products embodying design ideas for the intervention (e.g., learning activity work sheets). Edelson, Gordin, and Pea (1999) offer both in their paper on inquiry-based learning through technology and curriculum design, which provides key design principles (describing the design) as well as specific examples from their own work (embodying the principles). In some projects, practitioners are more active in this phase (e.g., leading or collaborating during creation), but in many projects, the role of practitioners is more reactive (e.g., providing comments on initial ideas). Evaluation and Reflection Initial ideas, partial prototypes, and full designs are the objects of evaluation and reflection. Evaluation usually takes place through developer screening, expert appraisal, pilots, and/or tryouts, each of which could use a variety of instruments (e.g., document analysis schemes, interview protocols, pre /posttests). Developer screening helps critique internal consistency and alignment with design goals through a formalized process of examining designs in light of initial intentions. Expert appraisal features external review to validate or improve specific aspects of the design. Pilots help understand how interventions will perform; they are typically conducted early, under semi-authentic conditions (e.g., in pullout classes, taught by

408

S. McKenney and S. Brand-Gruwel

the designer, or with volunteers). Tryouts are conducted in fully naturalistic settings; they can yield insights into various aspects of design (e.g., soundness, local viability, effectiveness). Reflection pertains to retrospective consideration of the evaluation data and experiences. Practitioners sometimes participate in expert appraisals and often participate in pilots and tryouts. This process is illustrated by Long and Hall (2015), who report multiple evaluations (related to three design cycles over a period of 6 years) in which digital storytelling was explored as a means to enhance preservice teachers’ reflective practice.

Implementation and Spread Throughout the three phases described above, attention is given to implementation and spread. Implementation entails adoption (deciding to engage with the intervention), enactment (the intervention takes place), and sustained maintenance (continuing the intervention in a sustainable way). Spread pertains to the diffusion and dissemination of key ideas and/or the intervention itself. Practitioners are typically key players in the processes of implementing and spreading interventions, as well as those underlying ideas that hold practical application. For example, Bakah et al. (2012) described stakeholder perspectives on the large-scale implementation and sustainability of redesigned technology curricula in two polytechnics in Ghana.

Multiple Roles As may be gleaned from the descriptions above, the tasks undertaken in each core design research process involve multiple roles. While additional subtle differences could easily be identified, we distinguish three different and crucial roles that design researchers play as they interact with practitioners throughout entire projects and within specific phases: consultant/facilitator, designer, and researcher. Below, we explain what is meant by each role and relationships to the design research phases and note key factors that contribute to role performance.

Consultant In line with the breadth of educational consultant work, fulfilling this role includes offering of professional development opportunities, networking opportunities, and consultation with practitioners (Matthews & Foster, 2005). An important part of professional development, consultants play a crucial role in supporting strategic planning (Krabbe-Sillasen & Valero, 2013). Good consultants work collaboratively with stakeholders on problem definition and framing, program development that explicitly involves the target community in planning, and joint research and evaluation (Nelson, Amio, Prilleltensky, & Nickels, 2000). Further, a crucial function of the consultant is sustaining contacts within professional learning networks (KrabbeSillasen & Valero, 2013) and facilitating access to additional expertise. And, as the term suggests, this role includes consultation. Whereas educational consultants in higher education typically work with individual instructors (Brinkley-Etzkorn, Schumann, White, & Smith, 2016), those in K-12 settings must be able to

16

Roles and Competencies of Educational Design Researchers: One. . .

409

accommodate both individual and team needs (Matthews & Foster, 2005). While some consultants may remain external and share expertise on an episodic basis, others work as process-oriented collaborators, often serving students directly (individually or through co-teaching) and sharing responsibility for them (Kirschenbaum, Armstrong, & Landrum, 1999). Thus, in recent decades, educational consultants have grown to take on the role of change agents and, at times, actively participate alongside their clients. In design research, this role is especially required during analysis, to help stakeholders expose their problems and knowledge thereof. But it is also present in design in the form of expertise sharing and structuring human processes. During evaluation, the consultant role centers on helping to understand what is happening and troubleshooting. During implementation, the consultant role includes modeling, coaching, and serving as program champion. This can include helping others to get/stay in touch with their reason for being involved, which is often tied to a sense of moral purpose.

Designer A designer is one who plans the appearance, form, or workings of something that does not yet exist. Educational designers plan and typically help construct innovations in the form of programs, processes, products, or policies. Good designers understand the processes, perspectives, and practices that enable their work. While the processes that facilitate educational design include analysis, design, development, implementation, and evaluation (sometimes referred to as ADDIE), different experts emphasize different facets. For example, Hoadley and Cox (2009) consider requirements, specifications, building, deployment, maintenance, and redesign to be key stages. In addition to an iterative and interactive (as opposed to linear and isolated) design process, Schunn (2008) notes the following processes have been shown important in engineering design: exploring problem representation, creating requirements and metrics, exploring alternatives, and exploring end-user perspectives. Throughout these processes, Burkhardt (2009) emphasizes that robust educational design is research-based, starting with review of research, of craft-based knowledge, and of earlier innovations and informing design and development through an iterative process that yields feedback from trials. Further, good designers are aware of the perspectives guiding their work. This includes values such as usability, usefulness, participation, or user-centeredness (Hoadley & Cox, 2009) that underpin their decisions. For example, Visscher-Voerman and Gustafson (2004) identified three paradigms that explained the decisions made by 24 expert designers in actual projects: (1) particularly high value on expert knowledge, including a systematic process (instrumental), (2) sharing responsibility with clients and placing high value on client need articulation (communicative), and (3) relying heavily on user ideas for the design as well as information underpinning it (pragmatic). Finally, a key element of designer work is developing their design repertoire. Experts have stressed the need for designers to develop or adopt guiding principles, the design patterns, and varied techniques (Hoadley & Cox, 2009). These are needed in relation to both more general design insights (e.g., knowledge of how people

410

S. McKenney and S. Brand-Gruwel

learn, media selection, subject matter research, and task analysis) and more specific insights (e.g., domain expertise, storyboarding, editing, scriptwriting) (MacLean & Scott, 2011; McKenney & Visscher-Voerman, 2013). In design research, this role is of course heavily present during the design and construction phase, steering the design process and shaping the designed products. However, it also plays a role in other phases, as foundational knowledge for design continues to develop. Because design researchers develop interventions to address practical challenges, they are served by understanding of the interactions of the design, how it is used, and the people who it aims to serve. Burkhardt (2009) refers to this as strategic design, and it includes identifying a specific opportunity for improvement, choosing or devising a model of change, identifying the resources that are needed to do the job well and the compromises that are acceptable, recognizing and questioning constraints, and advising stakeholders on the likely implications of their various decisions and offering alternatives where appropriate (Burkhardt, 2009).

Researcher The role of researcher pertains to conducting systematic investigation to develop new knowledge (facts, principles, theories, etc.). As such, their primary tasks are to design studies, collect and analyze data, and report the findings, in ways that are consistently ethical and legal guidelines (ESRC, 2001). Increasingly, researchers, practitioners, and policymakers are calling for strengthening attention to the researcher’s skills for disseminating and facilitating the use of new knowledge. While measuring societal impact remains challenging, some funding organizations and universities are beginning to include this facet in their assessments of research productivity (Levin, 2013; McKenney & Visscher-Voerman, 2013). This can be visible through unilateral approaches such as writing accessible publications for practitioners or more bilateral links between research and practice which leverage the interactive, social, and gradual nature of knowledge production by stressing the cooperation between researchers and practitioners during the co-creation of new knowledge (Levin, 2013; Vanderlinde & van Braak, 2010). While there is little debate about the importance of these basic researcher tasks, it is important to note notions of “good quality research” vary greatly from discipline to discipline (Vanderlinde & van Braak, 2010) and that even within disciplines, the epistemologies of scholars (and for graduate students, most notably the epistemologies of their advisors) differ tremendously and sometimes even conflict (Metz, 2001). Thus, another task of the researcher is to develop productive habits of mind. This includes becoming acquainted with the literature of a field and socializing into its disciplinary norms and identities (Golde, 2007). Further, researchers need to develop sensitivity to the fact that cultural assumptions and reflections of status can implicitly be built into theory, research questions, and methodological choices (Metz, 2001), as dealing with one’s own position presents a substantial epistemological and ethical consideration (Scott, Hinton-Smith, Härmä, & Broome, 2012).

16

Roles and Competencies of Educational Design Researchers: One. . .

411

In design research, the researcher role is most clearly present during the phases of empirical investigation: analysis and exploration and evaluation and reflection. But researcher expertise also serves design, e.g., by providing literature-based insights to ground the design and anticipate ways to increase its effectiveness, usability, and relevance. Because design research inherently involves multiple areas of focus (e.g., question to probe a problem, understand how a solution works, or assess the quality of an intervention), a broad understanding of qualitative and quantitative methods helps researchers to conceive of different kinds of questions that could be asked at different stages of inquiry and to align approaches accordingly (McKenney & Visscher-Voerman, 2013). This ability is crucial to the tighter integration of research and development, which can render educational research “more useful to practitioners and to policymakers, allowing the latter to make better-informed, less-speculative decisions that will improve practice more reliably” (Burkhardt & Schoenfeld, 2003, p. 3).

Crosscutting Competencies The descriptions above included key aspects of each role. Reflecting on these aspects across all three roles, several foundational competencies can be discerned, which are crucial to fulfilling each: orchestration, empathy, flexibility, and social competence. Asserting that these foundational and crosscutting competencies can help design researcher performance within and across each role, the remainder of this section elaborates how and why.

Orchestration Orchestration in the classroom pertains to the design, enactment, and management of diverse interactions and processes at multiple levels simultaneously: individual, in small groups, or for the whole class (Prieto, Holenko Dlab, Gutiérrez, Abdulwahed, & Balid, 2011). In design research, orchestration pertains to coordination of the many and diverse activities that are happening in parallel. This competency is needed for simultaneously attending to key aspects of each phase (e.g., research framing, data collection, solution design) as well as implementation and spread (which also include infrastructure woes and stakeholder ownership). Orchestration is required to fulfill each of the aforementioned roles. For the consultant, it is important to be able to oversee and support the overall change process (Matthews & Foster, 2005), which can also include being able to coordinate the mobilization of external resources as part of strategic planning (Krabbe-Sillasen & Valero, 2013). For the designer, project management, monitoring, and quality assurance skills have been identified as crucial in instructional designer competency frameworks (MacLean & Scott, 2011). This can also include working to retain as much space as possible for the creative talents in a design team and the systematic development that refines the products (Burkhardt, 2009). And for the researcher, management of parallel processes has been identified as a crucial skill for researchers by the United Kingdom’s Economic and Social Research Council (ESRC, 2001),

412

S. McKenney and S. Brand-Gruwel

which supports not only the production of new insights but also their mobilization for use in policy and practice (Levin, 2013).

Empathy Empathy concerns the sensitivity to and understanding of others, their situations, concerns, and feelings. In design literature, empathy is seen as an aspect of the design process that is influenced by the expertise of the designer, which can be enriched through the use of specific techniques (Kouprie & Visser, 2009). In design research, empathy is needed for exploring and attending to the needs, wishes, and concerns of stakeholders; creating designs that are usable, practical, and congruent with stakeholder concerns; helping researchers understand and interpret (especially qualitative) data; taking into account (un)shared goals; or becoming exposed to the incentives, motives, and reward structures in different settings. Empathy especially serves the roles of consultant, designer, and researcher. Understanding the perspectives of consultees is important for consultants (Brinkley-Etzkorn et al., 2016). As researchers call for consultants to recognize the wealth of knowledge inherent among those that live the day-to-day educational contexts (Nelson et al., 2000), an understanding of their needs, wishes, constraints, and rewards can be crucial to being able to leverage that expertise. Further, the consultant’s own level of enthusiasm has been cited an important factor for success (Kirschenbaum et al., 1999; Matthews & Foster, 2005). Designers also possess empathy and continuously seek to understand the (social) dynamics of the systems they wish to improve (Burkhardt, 2009) and the end users of specific designs (Schunn, 2008). For the researcher, a basic understanding of practitioner perspectives can help in research planning and execution such as anticipating the feasibility or resistance to various data collection approaches. It also helps researchers attune dissemination efforts to user needs (Vanderlinde & van Braak, 2010) and develop appreciation for different kinds of research along with any accompanying tacit assumptions (Metz, 2001). Flexibility Flexibility is the capacity of an individual to adjust to new or unexpected situations. Both cognitive flexibility, the ability to think about more than one task at a time or to switch quickly between tasks (Canas, Quesada, Antolí, & Fajardo, 2003; Spiro, 1988), and psychological flexibility, the ability to change or balance one’s standpoint, perspective, or convictions given multiple priorities (Kashdan & Rottenberg, 2010), are important for the design researcher. Flexibility is needed for balancing well-framed investigation with open-mindedness, staying focused on design goals and utilizing unplanned opportunities, and drawing conclusions and deriving new questions. Such flexibility also serves orchestration, e.g., optimizing the human and material resources available in ways that remain aligned with overall project goals. Flexibility is important for each of the aforementioned roles. It benefits the consultant by enabling perspective taking. This is important for understanding value (in)congruence between different stakeholders, as well as for engaging in the

16

Roles and Competencies of Educational Design Researchers: One. . .

413

self-reflection that consultants need to understand their role and functioning when engaging in transformative projects (Nelson et al., 2000). Flexibility benefits the designer because it aids in exploration of productive habits of mind (Tracey & Boling, 2014) as well as in “trying on different hats” to become attuned to various values and considerations that drive a specific design project, which can also include the designer’s own tricks and traps such as design fixation or groupthink (Hoadley & Cox, 2009). Flexibility benefits the research process by helping understand and leverage conceptual and methodological insights from other disciplines (Metz, 2001) as well as by enabling the adjustment of methods and schedules to opportunities in the field. Further, it helps the researcher as a developing human to see and understand aspects of themselves as well as their participants (Scott et al., 2012), to identify how implicit or explicit allegiances might be connected with other factors such as class, race, or gender (Metz, 2001), and – especially for those who are balancing research alongside complex and demanding home and work lives – to adopt new ways of interacting and challenging existing habits (Golde, 2007).

Social Competence From early stages of design research trajectories, social competence is important. While nuances in definitions vary, experts agree that social competence concerns the receiving, experiencing/processing, and sending of verbal and nonverbal communications with others (Feldman, Philippot, & Custrini, 1991; Halberstadt, Denham, & Dunsmore, 2001). Social competence is needed to develop trust, build relationships, invite people to feel safe, and speak frankly; during design, these skills are needed to negotiate design team tensions and to stimulate new thinking; during evaluation, these skills help engender cooperation, ease frustrations, and encourage participants to see things through and remain objective until results are in; for implementation and spread, these skills are needed to provide leadership and model positive attitudes. Social competence is important for all three roles. Consultants require effectiveness in areas of interactive communication (Kirschenbaum et al., 1999) including maintenance of stable contacts (Krabbe-Sillasen & Valero, 2013). Social competence helps them identify and merge the strengths of different partners as well as, and where appropriate, engage boundary spanners (Nelson et al., 2000). Internationally recognized frameworks of educational designer skills emphasize factors related to social competence, including leadership, communication, and client management (MacLean & Scott, 2011). Further, designers aiming for large-scale impact (such as curricula for widespread use) require social competence to network with the public and the media, as well as policy makers, funders, and fellow designers (Burkhardt, 2009). Further, the work of researchers has long been recognized as highly social, relying heavily on verbal and text-based discourse for researcherrespondent, researcher-researcher, and researcher-audience interactions. Social competence is required during most interactions with participants and is crucial for particularly meaningful encounters such as in-depth interviews, which rely heavily on rapport, humor, and humility (Scott et al., 2012; Vanderlinde & van Braak, 2010). It also supports knowledge mobilization (ESRC, 2001; Levin, 2013) and

414

S. McKenney and S. Brand-Gruwel

communications with fellow researchers, which are crucial not only for examining methods and findings but also for developing self-awareness and socializing into the profession (Golde, 2007; Scott et al., 2012).

Developing Design Researcher Capacity Design Researcher Learning Framework In the preceding sections, we have discussed the nature of design research and the importance of educating design researchers with varied sets of skills to interact with practitioners. Key phases of design research were articulated (analysis and exploration, design and construction, evaluation and reflection – each of which interacts with implementation and spread), as well as the main activities undertaken and the roles of practitioners in each. Thereafter, three different and crucial roles played by that design researchers were discussed. Descriptions of each role (consultant/facilitator, designer, and researcher) highlighted competencies needed for each. Finally, four crosscutting and foundational competencies were identified, and each was discussed: orchestration, empathy, flexibility, and social competence. Based on these discussions, Table 1 presents a framework for organizing the focus of design researcher learning. It articulates crucial areas in which design researchers learn through and for collaboration with practitioners. The columns emphasize how multiple roles and competencies come into play within specific phases, while the table as a whole illustrates the diverse capacity needed across entire projects. While individual development and needs would vary highly, the table could be helpful for shaping expectations and targeting learning supports to design researchers at various points in time.

Situated and Whole-Task Learning Rationale Taken together, the roles and crosscutting competencies design researchers must acquire (articulated in Table 1) speak to the complexity of this form of inquiry. In addition, every setting is different, the problems to be tackled are rarely welldefined, and there are many different ways to go about solving them. To develop the skills required to solve real-world problems, design researcher learning must be situated in the complex reality of everyday educational settings. This kind of a situated and whole-task approach to learning, which is rooted in social constructivism, is becoming increasingly common in the field of education (Van Merriënboer & Kester, 2008). In a whole-task approach, learning takes place by working on meaningful situated tasks that demand certain skills and knowledge to perform that task.

a

Attending to needs, wishes, concerns of stakeholders Critically investigate problem; uncover opportunities Developing trust, building relationships, inviting frankness

Empathy

Social competence

Flexibility

Literature review Field study Site visits and networking

Orchestration

Researcher

Designer

Consultant

Analysis and exploration Gets people to expose their (knowledge of) the problem(s) Gathers descriptions and explanations Frames and studies problem

Bold denotes especially heavy emphasis on this role in this phase

Crosscutting competencies (key uses in each phase)

Researcher learning about Roles (key work in each phase)

Table 1 Design researcher learning frameworka

Negotiation, stimulation

Creating designs that are usable, practical, and congruent with target group needs/wishes Remain focused on achieving goals; seek creative alternatives

Exploring solutions Mapping solutions Constructing solutions

Crafts design process as well as designed products Supports design with research

Design and construction Supports design with expertise, manages people processes

Engendering cooperation, mitigating frustration, encouraging objectivity

Deduce and induce; question why and what if

Screening Expert appraisal Pilots Tryouts Structured and organic reflection Understanding and interpreting data

Recommendations for revision/use Rigorously investigates solutions

Evaluation and reflection Troubleshoots when plans derail

Providing leadership, modeling positive attitudes

Implementation and spread Supports with advice/ expertise; champion, moral purpose New ideas for what could (not) work Observes to broaden understanding of context Adoption Enactment Sustained maintenance Dissemination and diffusion Understanding how designs fit (or not) in specific contexts Goal-oriented improvisation

16 Roles and Competencies of Educational Design Researchers: One. . . 415

416

S. McKenney and S. Brand-Gruwel

Definition A whole-task approach to education advocates using real-world problems and the integration of supportive contents, knowledge, skills, and attitudes, leading to learning situations that can be deemed as a coherent, interconnected, and meaningful whole. This is opposed to a fragmentized and compartmentalized learning situation, where learners can have difficulties combining pieces of information and integrating knowledge, skills, and attitudes, which result in low transfer of learning (Van Merriënboer & Kester, 2008). Different whole-task models aim at supporting the development of training programs for learners who need to develop and transfer professional competences or complex cognitive skills to an increasingly varied set of real-world contexts and settings. These models try to deal with complexity without losing sight of the relationships between elements (Van Merriënboer & Kester, 2008). Given the complexities inherent in conducting educational design research, whole-task models can provide useful insights in how to help researchers to develop their expertise. They accommodate the development of multiple competences as needed when taking on the different roles in the four phases of the design process while working on design studies rooted in real-life settings. Three models are described here: elaboration theory (Reigeluth, 1987, 1999), goal-based scenarios (Schank, 1993/1994), and four-component instructional design (Van Merriënboer, 1997). Reigeluth’s (1987, 1999) elaboration theory is a precursor of the whole-task approach and emphasizes starting with the simplest version of a learning task or domain and working toward more complex versions. It starts with an overview of the topic and zooms in on the related aspects of a topic. In essence, the theory focuses on sequencing instructional concepts and theoretical domains. Learning content (conceptual and theoretical) and related support aim toward the integration of knowledge, skills, and attitudes in which constructing mental models is central. For the design researcher, this might include mental models of the overall and phase-specific processes or the roles and how to enact them. In his theory on goal-based scenarios, Schank (1993/1994) emphasizes the need to practice skills using relevant content knowledge to help learners to achieve their goals. Learning by doing is a point of departure, and to support this learning, seven components are of importance: goal, mission, cover story, role, scenario operations, resources, and feedback. Goal-based scenarios stimulate the integration of knowledge, skills, and attitudes in meaningful settings and stress the importance of learner control over contents and strategies. This framework can be used to design various learning trajectories, including computer-based learning environments (Schank, Fano, Bell, & Jona, 1994). We have incorporated relevant aspects of these components into the guidelines for design researcher learning presented in the next section. Van Merriënboer (1997) developed the “four-component instructional design” (4CID) model. Learning tasks, supportive information, part tasks, and procedural information are the four components that should be designed in order to foster the learning of complex cognitive skills. Whole learning tasks are the backbone, and

16

Roles and Competencies of Educational Design Researchers: One. . .

417

sequencing learning tasks from simple to complex – while giving support but also fostering self-directed learning– should help develop learning and transfer. Van Merriënboer (1997) also stresses the importance of learning in authentic settings and draws on the importance situated learning (Lave & Wenger, 1991) which emphasizes that learning takes place in the same context in which it is applied. Fostering educational design researcher capacity mostly takes place while conducting a design study in practice and thereby is in a situated learning context where authenticity is guaranteed.

Relevance to the three Roles The importance of a whole-task approach and situated learning can further be examined with regard to each role. For example, Handley, Clark, Fincham, and Sturdy (2007) stress that consultants learn the practices and identities appropriate to joint projects through participation in various workplace communities. Translated to the learning of design researchers, this would include (a) the dominant workplace community associated with the consultant’s current place of work (typically a university for employment and a school for the research), (b) a wider network of practice across organizations which employ consultants with similar roles (in this case, other educational design researchers), and (c) peripheral communities which less directly influence the development of identity and practice (such as a research school). Experts on the learning of educational designers stress several considerations that point toward the value of whole-task and situated approaches. First, they stress the need for novice designers to be exposed to design models that encompass the whole design process (McKenney & Visscher-Voerman, 2013; Tracey & Boling, 2014). Second, the crucial role of firsthand experiences for designer learning is widely recognized (Hoadley & Cox, 2009), in part because designers frequently reason from previously encountered solutions (Tracey & Boling, 2014). Third, experts note that designer learning is predominantly informal and on the job (Yanchar & Hawkley, 2014), situated more in the work of designing than anywhere else. Finally, whole-task models help us attend to not only core design tasks but also to productive design habits. An example of this is the crucial habit of designer reflection (Hoadley & Cox, 2009; Yanchar & Hawkley, 2014) which, if well-timed and executed, can yield important and/or timely insights for live design work, as well as for the designer’s own professional learning. Also from the perspective of the researcher, whole-task and situated approaches are also important. Their complexity often prompts researchers’ “need to know” about new methods, thereby stimulating the growth of quantitative and qualitative skills (McKenney & Visscher-Voerman, 2013). Especially for researchers new to the field or to this genre of inquiry, whole-task and situated approaches support scaffolding during the transition to independent research – a notoriously important and difficult step for doctoral students (Gardner, 2008). The emphasis on whole, authentic tasks facilitates enculturation into the academy, for example, by participation in disciplinary cultures (Gardner, 2008). It also offers exposure to the complexity and ambiguity of real-world settings, which helps researchers develop understanding of

418

S. McKenney and S. Brand-Gruwel

epistemological variety and even tackle emotional challenges (Metz, 2001). These experiences can provide focal areas for structured reflections, which are extremely valuable for learning to share and debate difficulties, develop researcher identity, and empower researchers as professionals (Scott et al., 2012).

Guidelines for Design Researcher Learning Trajectories In the preceding sections, we have discussed the nature of design research and a framework depicting the phases in the process, the roles involved, and the competencies needed. Taking this framework and the theoretical ideas concerning whole-task approaches for (situated) learning, we offer guidelines to help learners and their mentors to foster the capacity development of design researchers. Specifically, we discuss seven guidelines and offer examples of how these guidelines can be used in practice. The guidelines can be used to shape not only individual learning trajectories but also group ones.

Guideline 1: Assess the Existing Design Researcher (Learner) Capacity To set up learning trajectories for learners or groups of learners, it is important to obtain insight into the capacities of the learners, in order to tailor trajectories accordingly. Working in a real-life context with a high complexity level makes learners easily susceptible to drowning in all the possible areas to address. Insight into the learners’ existing strengths and lacunas helps to prioritize the competencies to work on and identify strengths that could be leveraged in this process. The framework offered in Table 1 can help to inventory the learners’ competency levels. Based on a draft version of this framework, Jongstra, Pauw, and McKenney (2016, 2017) developed a self-report questionnaire to identify areas for development. In this questionnaire, learners respond to 61 statements (using a 5-point Likert scale) concerning (their own perceptions of) their crosscutting competencies in relation to the different phases of design research. For example, an item pertaining to flexibility during analysis and exploration is: “I approach problems using different perspectives.” The questionnaire and its results can be used to structure discussion of the learners’ capacities. As an example, the conclusion for Lisa, an experienced educational designer, is that she needs to work on her flexibility, because she finds it difficult to remain open minded (as a consultant) in the analysis and exploration phase. She finds her previous design project experience useful but also distracting as she must resist the tendency to color her perceptions of the current situation by her previous experiences, and she finds it tempting to consider ready-made solutions based on her previous work. The learning trajectory for this person should include learning situations in which this element can be practiced and evaluated. Guideline 2: Establish Clear Mission and Goal In light of the research aims and stages, as well as the assessment of existing capacity, it is of importance to identify and prioritize goals for design researcher

16

Roles and Competencies of Educational Design Researchers: One. . .

419

learning within a feasible time frame. As suggested in Schank’s goal-based scenarios (Schank, 1993/1994), setting goals and defining a mission are essential in learningby-doing settings. Design research settings are complex and rarely clearly structured. To foster learning, clear goals help select learning situations in which the learner can practice and reflect on specific competences. It may be obvious that the goals should be based on the assessment of needs and that the goals should be formulated using SMART (specific, measurable, assignable, realistic, and time-related) guidelines. For example, Jim just graduated from an academic master in the educational sciences and will work with a primary school on a project to design a program to foster children’s computational thinking and programming skills. As a beginning researcher, he only experienced the role of designer during his studies. His mission is to help the project to be successful but also to become a better educational design researcher and to obtain evidence of that. One of his goals is to be able to lead a group of teachers and designers to come up with designs that they all believe in and will later be able to evaluate based on objective criteria (phase, design and construction; role, designer; competence, social competence).

Guideline 3: Define Assessment Criteria The 4CID model by Van Merriënboer (1997) describes how performance criteria and standards can best be designed in order to make proper judgments of learning and formulate further learning needs. To start, a hierarchy is needed to define the constituent skills on which the competency assessment should take place. Given the constituent skills, the criteria and standards should then be explicated. Going a step further, a rubric can be designed to get more insight into if and how learners meet the standards. In a study by Kicken, Brand-Gruwel, Van Merriënboer, and Slot (2009a), performance standards and criteria were defined for a vocational educational program and used to shape an electronic development portfolio that helped learners assess current skills and plan subsequent learning tasks. For example, part of the researchers’ role in the analysis and exploration phase is doing a literature review. Literature review can be further broken down in constituent skills: 1) defining the research question, 2) searching for information, 3) selection and scanning of relevant and reliable sources, 4) processing information, and 5) presenting the information in a review (e.g., Brand-Gruwel, Wopereis, & Vermetten, 2005). Also, these constituent skills can be further divided, and criteria can be formulated. Helvoort et al. (2017) constructed a rubric for this skill, and a criterion and standard are, for instance: the learner used appropriate keyword when searching for sourcing and information. The standard for proper behavior is formulated as follows: the learner uses specific search terms that are relevant for the topic, uses synonyms and operators, and also takes languages into account. Guideline 4: Create Learning Opportunities Situated learning and learning by doing in the creation of appropriate learning opportunities are of importance. The 4CID model (Van Merriënboer, 1997) argues that the design of learning tasks from simple to complex and situated in an authentic setting with embedded support fits learning best, especially when aspects such as

420

S. McKenney and S. Brand-Gruwel

cognitive load are taken into account. Four possible task solutions could be relevant and are described here: example-based learning, completion problems, emphasis manipulation, and part task solutions. Using these solutions in a variety of authentic settings will facilitate the transfer of the learned competencies. Example-based learning finds support in disciplines such as Bandura’s social learning theory (Bandura, 1977). From this perspective, skills learning takes place by observing an expert executing the skill. Observational learning can be facilitated by giving learners modeling examples, showing a model performing the skill while thinking out loud, and providing important insight into the thought processes and decision-making that otherwise remains covert. Studies reveal that modeling examples are effective to foster learning of complex skills (Van Gog, Paas, & Van Merriënboer, 2008). When learning to conduct educational design research, observing experts and analyzing and reflecting on their behaviors (for instance, when leading design sessions) using the framework provided in this chapter give good opportunities for identifying aspects of performance and possibly discussing them afterward. Completion problems are problems in which the learner is provided with a given state and a partial solution. After studying the partial solution, the learner completes the remaining steps to solve the problem. This method stimulates active processing of the given solution steps because they give the essential information the learner needs before being able to continue. For example, in the role of consultant, a learner can get information and study the given state concerning the problem definition and also observe a mentor in the first meeting and analyze the process. Given the current state, the learner can prepare the next step in the process and lead the next session with the stakeholder and focus on, for instance, being emphatic and using social competence to create room in a discussion and encourage all stakeholders to share their own ideas. Emphasis manipulation means that, during learning in a complex process, the focus can be directed to learning specific competence (Frerejean, Van Strien, Kirschner, & Brand-Gruwel, 2016). The cognitive load can be too high if a learner should focus on all competences at the same time. Focusing on a specific competence also means that after performing, using the assessment criteria, the evaluation and reflection will lead to formulation of further learning needs. For example, when in the role of consultant, the learners’ focus may be on empathy while leading a session with the stakeholders. After the session, the learner and mentor(s) and possibly peers can evaluate if the learner gave all participants room to reflect on ideas and was not too directive. Also, other criteria should be evaluated. A clear overview of the competences and assessment criteria is a must when using emphasis manipulation in supporting learners to become educational design researchers. Part tasks can be designed when specific aspects with more a routine character are at stake (Van Merriënboer, 1997). For instance, when in a researcher’s role working on a literature review, one should know how to search in databases. This can be practiced in isolation. It is of importance to identify these aspects and design these part tasks and give the learners just in time information concerning the procedures that should be used.

16

Roles and Competencies of Educational Design Researchers: One. . .

421

Guideline 5: Create Awareness Learners and mentors need to be aware of what doing educational design research encompasses. They need to build mental models of the different phases of the process, the roles, and the competencies, analyzing them in the domain and representing them in mental models. The models concern the regularities, cognitive strategies, and problem-solving approaches when conducting educational design research. The framework presented in Table 1 can be starting point to think about approaches that help the learner to solve problems during the process. For instance, what is an approach when, in the role of designer in the design phase, the members of the design group focusing on developing students’ computational thinking skills have divergent ideas about what would be effective instructional measures for teaching computational thinking skills? A strategy could be to make an overview of the ideas and to conduct a literature review to gather evidence for the different ideas to underpin the various choices. Digital tools (e.g., mind mapping software) can help to generate and structure ideas. Creating the awareness concerning different approaches and strategies enables learners (and their mentors) to make well-grounded decisions about how to handle upcoming situations and reduce ad hoc decisions in unpredictable situations. Guideline 6: Stimulate Self-Directed Learning In situated learning, especially with adults, the importance of self-directed learning and defining one’s own learning needs has long been recognized (Knowles, 1975). Giving learners the opportunity to direct their own learning can have a positive effect on the learning results, because they can adapt the learning to their particular needs. Directing one’s own learning and creating tailored learning trajectories make learning more personally relevant, thereby fostering motivation. When learners experience responsibility for their own learning, it offers the opportunity to develop selfdirected learning skills and to prepare for lifelong learning as independent learners. Well-functioning design research teams have a natural tendency to engage in adaptive management, “an iterative process that involves stakeholders who learn through a cyclical process of setting objectives, planning, taking action, monitoring, and reflection on the outcomes, learning and taking action again” (Cundill, Cumming, Biggs, & Fabricius, 2012). This kind of learning is heavily focused on the task and seems worth articulating. At the same time, researchers may be so embroiled in the task facing them that attending to their own personal learning (e.g., roles and competencies) might feel like an unnecessary luxury. Good mentors as well as institutional routines (e.g., the common requirement of a PhD personal development plan) can stimulate the pursuit and monitoring of personal learning goals. Guideline 7: Give Support and Feedback During the learning process, support and feedback are essential to help learners focus on their own learning needs and to meet their learning goals. Design researcher support can take many forms, including advice (generic or tailored), tools (e.g., templates, checklists), and examples (procedural or conceptual). Feedback includes

422

S. McKenney and S. Brand-Gruwel

corrective or affirming comments about past behavior and works well with feedforward, i.e., corrective or affirming comments about future behavior. While basic process support for conducting educational design research is available (e.g., McKenney & Reeves, 2012), design researchers often rely on mentors, coaches, and peers for helping translate and adapt general ideas to their specific settings or for feedback on their personal learning and performance. One effective way of support is using reflective dialogues (Kicken, BrandGruwel, Van Merriënboer, & Slot, 2009b). By asking reflective questions such as “How do you think the meeting went with regard to learning goals you focused on?,” “Do you think you can simultaneously work on these three goals?,” or “How long do you expect it to take you to reach this goal?,” the mentor can help the learner to reflect on the performance, define points of improvements, and formulate realistic goals. Moreover, reflective dialogue gives the mentor and learner a better insight in the learner’s level of self-directed learning skills. To sum up, to support learners to become educational design researchers that can deal with a wide variety of complex situations during the process, multiple instructional measures can be taken. These measures are highlighted in the abovementioned guidelines. The framework offered in Table 1 can provide structure for using these guidelines. Namely, it provides starting points for investigating and monitoring design researcher learning.

Concluding Remarks We have argued for the development of design researchers with multiple skill sets. We have highlighted the importance of three roles and four kinds of competencies. And we have discussed research-based ways to support the development of those competencies. We have tackled this work from the perspective of conducting educational design research together with practitioners. We conclude with reflections on the role of context, individual design researcher expertise, and skills for modern researchers in general. As described previously, conducting educational design research constitutes a complex task. The complexity stems from the many different and connected aspects that bear consideration (often simultaneously) and require multiple skill sets. Additionally, the task is complex because it takes place in exciting, dynamic, and highly diverse ecologies of educational practice. While the competencies and roles described here are common across design research settings, each context brings its own considerations, including affordances and constraints. As a result, even when similar goals are articulated (e.g., increasing physics teacher capacity to make effective use of online modeling tools), their specific manifestations will most certainly vary due to contextual differences, for example, in stakeholder values, available resources, or leadership priorities. While specific settings have some influence on how design researchers (can) fulfill their roles and competencies, this process is also shaped by their own vision and judgment, two important aspects of expertise.

16

Roles and Competencies of Educational Design Researchers: One. . .

423

To some extent, design approaches appear to be a matter of personal preference or conviction (Visscher-Voerman & Gustafson, 2004). Yet professional orientations, and certainly one’s expertise, are powerfully shaped by one’s own experiences. Like most complex tasks, design research relies on both adaptive and routine expertise. Adaptive expertise is used to complete tasks which are novel (for those involved). For example, creating an innovative lesson series using online labs for inquiry learning to foster deep understanding of diffusion and osmosis would require a design researcher’s adaptive expertise if such work has not been undertaken previously. In contrast, routine expertise is used to create additional instances of tasks previously undertaken. This would be used if a designer researcher were to revise the aforementioned lesson series or create a second lesson series based on the same principles of inquiry learning but this time for Mendelian inheritance. This distinction seems important, given that scholarship has emphasized key differences in how each type of expertise develops (Lin, Schwartz, & Bransford, 2007; Bransford et al., 2010). Since both adaptive and routine expertise are required for design research, it seems important to understand and provide adequate opportunities for (supporting) the development of each. Finally, it seems important to point out that, while the skills mentioned here develop through and later also serve interaction with practitioners, they are increasingly crucial skills for all modern researchers. Writing in the Journal of Investigative Surgery, Toledo-Pereyra (2012) suggests the following ten qualities of a good researcher: interest, motivation, inquisitiveness, commitment, sacrifice, excelling, knowledge, recognition, scholarly approach, and integration. Through extensive international research, Lamblin and Etienne (2010) identified three sets of competencies required by researchers now and in the future: scientific competencies (scientific knowledge, ability to learn and adapt, ability to formulate a research issue, capacity for analysis and grasp of sophisticated technology tools, ability to work in an interdisciplinary environment, and ability to incorporate existing knowledge), project and team management skills (ability to work in a team, ability to develop a network, communication skills, ability to asses, language skills, business culture and management skills, project management skills, ability to manage and steer teams, awareness of the pertinence of the research and its impact on the environment), and personal aptitudes/interpersonal skills (creativity, open-minded approach, motivation/involvement, adaptability, ability to self-asses). Clearly, the foundational and crosscutting competencies described here (empathy, orchestration, flexibility, and social competence) align well with existing literature on researcher competencies. Thus, well-prepared researchers have much more than robust methodological skill sets. As Hostetler (2012, p. 16) indicates, The question of what counts as good education research. . . [is too often] conceived principally as a methodological question rather than an ethical one. Good education research is a matter not only of sound procedures but also of beneficial aims and results; our ultimate aim as researchers and educators is to serve people’s well-being.

424

S. McKenney and S. Brand-Gruwel

Design research has great potential to contribute to educational research in general (Anderson & Shattuck, 2012) and to the field of learning and instruction, specifically (Gravemeijer & Cobb, 2006). Because design research activities themselves simultaneously contribute to improving theoretical understanding and design practices of professionals, the researcher-practitioner learning through design research might best be characterized using Levin’s (2013) notion of knowledge mobilization – stressing the interactive, social, and gradual nature of the bilateral connections between research and practice. Yet this potential contribution can only be realized when investigator skills include those of the consultant, designer, and researcher. Currently, few (graduate) programs support researcher learning in the domains described above, and little explicit attention is given to the crosscutting and foundational competencies described. It may be that traditional research institutions have undervalued the contributions these roles have to make to both research and development in education (Burkhardt & Schoenfeld, 2003), but more modern ones have begun to emphasize, stimulate, and reward researcher attention to the co-creation, uptake, and use of knowledge. This chapter offers considerations for targeting such efforts and offers specific examples with regard to educational technology design research.

References Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. Bakah, M., Voogt, J., & Pieters, J. (2012). Advancing perspectives of sustainability and large-scale implementation of design teams in Ghana’s polytechnics: Issues and opportunities. International Journal of Educational Development, 32, 787–796. Bandura, A. (1977). Social learning theory. New York: General Learning Press. Barab, S., Dodge, T., Thomas, M., Jackson, C. & Tuzun, H. (2007). Our designs and the social agendas they carry. Journal of the Learning Sciences, 16(2), 263–305. Bereiter, C. (2002). Design research for sustained innovation. Cognitive Studies, Bulletin of the Japanese Cognitive Science. Society, 9(3), 321–327. Boschman, F., McKenney, S., & Voogt, J. (2014). Understanding decision making in teachers’ curriculum design approaches. Educational Technology Research and Development, 62, 393–416. Brand-Gruwel, S., Wopereis, I., & Vermetten, Y. (2005). Information problem solving by experts and novices: Analysis of a complex cognitive skill. Computers in Human Behaviour, 21, 487–508. Bransford, J., Mosberg, S., Copland, M., Honig, M., Nelson, H., Gawel, D., & Vye, N. (2010). Adaptive people and adaptive systems: Issues of learning and design. In A. Hargreaves, A. Lieberman, M. Fullan, & D. Hopkins (Eds.), Second international handbook of educational change (pp. 825–856). London: Springer. Brinkley-Etzkorn, K. E., Schumann, D., White, B., & Smith, T. (2016). Designing an evaluation of instructional consultation in a higher education context. To Improve the Academy, 35, 121–152. Burkhardt, H. (2009). On strategic design. Educational Designer, 1(3), 1–49. Burkhardt, H., & Schoenfeld, A. (2003). Improving educational research: Toward a more useful more influential and better-funded enterprise. Educational Researcher, 32(9), 3–14.

16

Roles and Competencies of Educational Design Researchers: One. . .

425

Canas, J., Quesada, J., Antolí, A., & Fajardo, I. (2003). Cognitive flexibility and adaptability to environmental changes in dynamic complex problem-solving tasks. Ergonomics, 46(5), 482–501. Cundill, G., Cumming, G. S., Biggs, D., & Fabricius, C. (2012). Soft systems thinking and social learning for adaptive management. Conservation Biology, 26(1), 13–20. https://doi.org/ 10.1111/j.1523-1739.2011.01755.x DBRC. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. Edelson, D., Gordin, D. N., & Pea, R. D. (1999). Addressing the challenges of qinuirybased learning through technology and curriculum design. Journal of the Learning Sciences, 8(3&4), 391–450. ESRC. (2001). Postgraduate training guidelines (3rd ed.). Swindon: ESRC (Economic and Social Research Council). Feldman, R. S., Philippot, P., & Custrini, R. J. (1991). Social competence and nonverbal behavior. Frerejean, J., Van Strien, J. L. H., Kirschner, P. A., & Brand-Gruwel, S. (2016). Completion strategy or emphasis manipulation? Task support for teaching information problem solving. Computers in Human Behavior, 62, 90–104. Gardner, S. K. (2008). “What's too much and what's too little?” the process of becoming an independent researcher in doctoral education. The Journal of Higher Education, 79(3), 326–350. Golde, C. M. (2007). Signature pedagogies in doctoral education: Are they adaptable for the preparation of education researchers? Educational Researcher, 36(6), 344–351. Gravemeijer, K., & Cobb, P. (2006). Outline of a method for design research in mathematics education. In J. V. D. Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 17–51). London: Routledge. Halberstadt, A. G., Denham, S. A., & Dunsmore, J. C. (2001). Affective social competence. Social Development, 10(1), 79–119. Handley, K., Clark, T., Fincham, R., & Sturdy, A. (2007). Researching situated learning: Participation, identity and practices in client-consultant relationships. Management Learning, 38(2), 173–191. Hoadley, C. (2004). Methodological alignment in design-based research. Educational Psychologist, 39(4), 203–212. Hoadley, C., & Cox, C. (2009). What is design knowledge and how do we teach it? In C. Digiano, S. Goodman, & M. Chorost (Eds.), Educating learning technology designers: Guiding and inspiring creators of innovative educational tools (pp. 19–35). New York: Taylor & Francis. Hostetler, K. (2012). What is "good" educational research? Educational Researcher, 34(6), 16–21. Jongstra, W., Pauw, I. & McKenney, S. (2016). Ontwerpgericht onderzoek in de master: Hoe faciliteren we dat? Presented at the Onderwijs Research Dagen [Educational Research Days], May 26–27: Rotterdam. Jongstra, W., Pauw, I., & McKenney, S. (2017). Competenties ontwikkelen voor ontwerpgericht onderzoek; Richtlijnen voor de HBO masteropleiding. Tijdschrift voor Lerarenopleiders, 38(4), 69–80. Kali, Y. (2006). Collaborative knowledge building using the design principles database. International Journal of Computer-Supported Collaborative Learning, 1(2), 187–201. https://doi.org/ 10.1007/s11412-006-8993-x Kashdan, T. B., & Rottenberg, J. (2010). Psychological flexibility as a fundamental aspect of health. Clinical Psychology Review, 30(7), 865–878. Kicken, W., Brand-Gruwel, S., Van Merriënboer, J. J. G., & Slot, W. (2009a). Design and evaluation of a development portfolio: How to improve Students' self-directed learning skills. Instructional Science, 37, 453–473. Kicken, W., Brand-Gruwel, S., Van Merriënboer, J. J. G., & Slot, W. (2009b). The effects of portfolio-based advice on the development of self-directed learning skills in secondary vocational education. Educational, Technology, Research & Development, 57, 439–460.

426

S. McKenney and S. Brand-Gruwel

Kirschenbaum, R. J., Armstrong, D. C., & Landrum, M. S. (1999). Resource consultation model in gifted education to support talent development in today’s inclusive schools. Gifted Child Quarterly, 43, 39–47. Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers. Englewood Cliffs, NJ: Prentice Hall/Cambridge. Kouprie, M., & Visser, F. S. (2009). A framework for empathy in design: Stepping into and out of the user's life. Journal of Engineering Design, 20(5), 437–448. Krabbe-Sillasen, M., & Valero, P. (2013). Municipal consultants’ participation in building networks to support science teachers’ work. Cultural Studies of Science Education, 8, 595–618. Lamblin, P. & Etienne, C. (2010). Skills and competencies needed in the research field: Objectives 2020. Neuilly-sur-Seine Cedex, France: Deloitte & Apec. Lave, J., & Wenger, E. (1991). Situated learning. Legitimate peripheral participation. Cambridge: University of Cambridge Press. Levin, B. (2013). To know is not enough: Research knowledge and its use. Review of Education, 1(1), 2–31. Lin, X., Schwartz, D. L., & Bransford, J. (2007). Intercultural adaptive expertise: Explicit and implicit lessons from Dr. Hatano. Human Development, 50(1), 65–72. Long, B. T., & Hall, T. (2015). R-NEST: Design-based research for technology-enhanced reflective practice in initial teacher education. Australasian Journal of Educational Technology, 31(5), 572–596. MacLean, P., & Scott, B. (2011). Competencies for learning design: A review of the literature and a proposed framework. British Journal of Educational Technology, 42(4), 557–572. Matthews, D. J., & Foster, J. (2005). A dynamic scaffolding model of teacher development: The gifted education consultant as catalyst for change. Gifted Child Quarterly, 49(3), 222–230. McKenney, S. (2017). Researcher-practitioner collaboration in educational design research: Processes, roles, values & expectations. In M. A. Evans, M. J. Packer, & K. Sawyer (Eds.), Reflections on the learning sciences (pp. 155–188). Cambridge: Cambridge University Press. McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. London: Routledge. McKenney, S., & Visscher-Voerman, I. (2013). Formal education of curriculum and instructional designers. Educational Designer, 2(6), 1–20. Metz, M. H. (2001). Intellectual border crossing in graduate education: A report from the field. Educational Researcher, 30(5), 1–7. Nelson, G., Amio, J., Prilleltensky, I., & Nickels, P. (2000). Partnerships for implementing school and community prevention programs. Journal of Educational and Psychological Consultation, 11(1), 121–145. Ormel, B., Pareja Roblin, N., McKenney, S., Voogt, J., & Pieters, J. (2012). Research-practice interactions as reported in recent design studies: Still promising, still hazy. Educational Technology Research & Development, 60(6), 967–986. Prieto, L. P., Holenko Dlab, M., Gutiérrez, I., Abdulwahed, M., & Balid, W. (2011). Orchestrating technology enhanced learning: A literature review and a conceptual framework. International Journal of Technology Enhanced Learning, 3(6), 583–598. Reigeluth, C. M. (1987). Lesson blueprints based on the elaboration theory of instruction. In C. M. Reigeluth (Ed.), Instructional theories in action: Lessons illustrating selected theories and models (pp. 245–288). Hillsdale, NJ: Lawrence Erlbaum Associates. Reigeluth, C. M. (1999). The elaboration theory: Guidance for scope and sequence decisions. In C. M. Reigeluth (Ed.), Instructional-design theories and models. A new paradigm of instruction (pp. 425–453). Mahwah, NJ: Lawrence Erlbaum Associates. Schank, R. C. (1993/1994). Goal-based scenarios: A radical look at education. Journal of the Learning Sciences., 3, 429–453. Schank, R. C., Fano, A., Bell, B., & Jona, M. (1994). The design of goal-based scenarios. Journal of the Learning Sciences, 3(4), 305–345. Schunn, C. D. (2008). Engineering educational design. Educational Designer, 1(1), 1–21.

16

Roles and Competencies of Educational Design Researchers: One. . .

427

Scott, S., Hinton-Smith, T., Härmä, V., & Broome, K. (2012). The reluctant researcher: Shyness in the field. Qualitative Research, 12(6), 715–734. Spiro, R. J. (1988). Cognitive Flexibility Theory: Advanced Knowledge Acquisition in Ill-Structured Domains. Technical Report No. 441. Toledo-Pereyra, L. H. (2012). Ten qualities of a good researcher. Journal of Investigative Surgery, 25, 201–202. Tracey, M. W., & Boling, E. (2014). Preparing instructional designers: Traditional and emerging perspectives. In M. Spector, M. Merril, J. Elen, & M. Bischop (Eds.), In handbook of research on educational communications and technology (pp. 653–660). New York: Springer. Van Gog, T., Paas, F., & Van Merriënboer, J. J. G. (2008). Effects of studying sequences of processoriented and product-oriented worked examples on troubleshooting transfer efficiency. Learning and Instruction, 18, 211–222. Van Helvoort, J., Brand-Gruwel, S., Huysmans, F., & Sjoer, E. (2017). Reliability and validity test of a Scoring Rubric for Information Literacy. Journal of Documentation, 73(2), 305–316. Van Merriënboer, J. J. G. (1997). Training complex cognitive skills. Englewood Cliffs, NJ: Educational Technology Publications. Van Merriënboer, J. J. G., & Kester, L. (2008). Whole task models in education. In J. M. Spector, M. D. Merrill, J. J. G. van Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (pp. 441–456). New York/Routledge: Taylor & Francis Group. Vanderlinde, R., & van Braak, J. (2010). The gap between educational research and practice: Views of teachers, school leaders, intermediaries and researchers. British Educational Research Journal, 36(2), 299–316. Visscher-Voerman, I., & Gustafson, K. (2004). Paradigms in the theory and practice of education and training design. Educational Technology Research and Development, 52(2), 69–89. Wagner, J. (1997). The unavoidable intervention of educational research: A framework for reconsidering researcher–practitioner cooperation. Educational Researcher, 26(7), 13–22. Yanchar, S., & Hawkley, M. (2014). “There’s got to be a better way to do this”: A qualitative investigation of informal learning among instructional designers. Educational Technology Research & Development, 62, 271–291. https://doi.org/10.1007/s11423-014-9336-7

Prof. Dr. Susan McKenney co-leads ELAN, the Department of Teacher Professional Development within the Faculty of Behavioral and Management Sciences at the University of Twente and is Visiting Professor in the Learning Sciences & Policy Group at the University of Pittsburgh. Her research focuses on understanding and facilitating the interplay between curriculum development and teacher professional development and often emphasizes the supportive role of technology in these processes. As such, she also studies processes of design that can be applied in the field of education and synergetic research-practice interactions. She is committed to exploring how educational research can serve the development of scientific understanding while also developing sustainable solutions to real problems in educational practice. Since design-based (implementation) research lends itself to these dual aims, her writing and teaching often provide ideas about how to conduct this exciting form of inquiry. In addition to authoring numerous articles, she co-edited the book, Educational Design Research and, together with Tom Reeves, wrote the book, Conducting Educational Design Research. She has served as guest editor of special issues in Instructional Science, European Journal of Education, Australasian Journal of Educational Technology, Pedagogische Studiën, and Technology, Pedagogy and Education. She is currently Associate Editor for the Journal of the Learning Sciences and has authored over 100 peer-reviewed publications. Prof. Dr. Saskia Brand-Gruwel is Dean of the faculty Psychology and Educational Sciences of the Open University of the Netherlands. Her research, positioned in the Welten Institute, Research Centre of Learning, Teaching and Technology, focuses on information literacy, self-regulated learning, and instructional design to foster these higher-order skills with a special interest on the

428

S. McKenney and S. Brand-Gruwel

use of technology in education. She studies the cognitive processes involved in the mentioned skills and the effects of instructional measures to support the skill acquisition in the interplay of more experimental research and design-based research. In addition to authoring numerous articles, she was guest editor for different special issues for the journals Learning and Instruction, Computers in Human Behavior, and Journal of Computer Assisted Learning. She guides Ph.D. and master’s students and, furthermore, has authored over 100 peer-reviewed publications.

Board Games as Part of Effective Game-Based Learning Strategies

17

Antonio Santos

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Teachers Creating Their Own Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Sociocultural Constructivist Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Method to Develop a Board Game for Game-Based Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part 1: Instructional Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . State Subject Matter: State Learning Objective – Build the Problem . . . . . . . . . . . . . . . . . . . . . . Part 2: Game Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design: Test the Prototype – Modify . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part 3: Evaluation/Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design Classroom Application: Apply – Modify . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . First Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Second Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

430 431 433 434 435 437 437 439 439 440 440 441 441 443 445 446

Abstract

In the educational field, practitioners view games as being effective tools, and research supports this belief. In general, accept that the field requires no further studies aiming to assess whether games are effective as educational tools but rather research that explains how we can integrate them into everyday teaching and learning processes. Although operational and cost-related issues that differentiate digital and nondigital games occur, some questions of interest with regard to both digital and nondigital game-based learning researchers and producers arise, such as how to better align instructional and game design. This chapter focuses on the use of board games as cost-effective instructional materials A. Santos (*) Department of Educational Sciences, Universidad de las Américas Puebla, Cholula, Mexico e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_142

429

430

A. Santos

integrated into a set of game-based learning strategies. As its main objective, it proposes a method (called LuDu) of aiding teachers in designing and building a complete game-based learning experience using a board game. Basically, this method proposes that, using an iterative approach, teachers start from their actual learning objectives, develop their own board game, and, finally, integrate it effectively into their daily teaching experiences. The author explains the theoretical underpinnings of this method, describes the process in terms of decisions and actions, and details a case study that illustrates test implementations of the method by two different groups of teachers in Mexico. Keywords

Educational technology · Game-based learning · Educational board games · Game design · Instructional design · Constructivist pedagogy

Introduction Teachers have been much interested in integrating the use of games into their educational practices because they have believed that games could serve as an effective tool. Supporting this belief, Schrier (2014) stated that evidence indicates that the processes of learning and engaging in game playing are related, but one can make the point that “we need to not only understand whether a game can teach, but [to also understand] the conditions under which it can (or cannot) help someone learn” (p. 2). Clark, Tanner-Smith, and Killingsworth (2014) did a meta-analysis study whose results agree with Schrier’s position. Indeed, they sustained that research on game-based learning should move from research investigating whether games can aid learning to research on how theoretically based design decisions affect learning. In fact, a decade ago, Richard van Eck (2006) was already stating that no more studies regarding the effectiveness of games as instructional resources were needed, but that what was needed was research that explained why games were so engaging and how they could better integrate them into everyday teaching and learning processes. Since teachers see games as being useful resources as far as they blend adequately with their educational practices (Arnab et al., 2013), understanding how to develop learning experiences that include the adequate use of games to support and enhance learning should constitute an important task for game-based learning researchers. In this way, instructional designers and teachers can have and apply sound strategies to adequately integrate games into their didactic practices, always with the main objective of enhancing students’ quality of learning. Although most of the previous considerations regarding the use of games in education could apply to both board games and videogames, some operational and cost-related aspects that differentiate them (and which the author will further explain in the next section) do, of course, occur. Particularly, this chapter focuses on the use of board games as being cost-effective instructional materials that educators have the ability to integrate into a set of game-based learning strategies. Thus, the proposal of a

17

Board Games as Part of Effective Game-Based Learning Strategies

431

method to aid teachers in designing and building a complete game-based learning experience using a board game constitutes the main objective of this work. Basically, this method proposes that, using an iterative approach, teachers start from their actual learning objectives, develop their own board game, and, finally, integrate it effectively into their daily teaching experiences. The author purposes to present to the teacher/ designer a set of decisions and actions that she/he needs to take in a nonlinear way in order to design a complete educational board game, build it using inexpensive materials, and evaluate it with his/her students. In this way, in this chapter, the author seeks to identify effective conditions under which a board game can promote learning. In the following sections, the author will explain the theoretical underpinnings of the proposal, describe the method in terms of decisions and actions, and detail a case study illustrating its test implementation with two different groups of teachers in Mexico.

Background Recently, a growing body of literature has analyzed research evidences of the effectiveness of the use of games for learning (e.g., Clark, 2007; de Freitas & Liarokapis, 2011; Prensky, 2012; Shaffer, 2006; Sitzmann, 2011; Tobias & Fletcher, 2007; Tobias et al., 2011; Wouters, van Nimwegen, van Oostendorp, & van der Spek, 2013). Although an important portion of this literature relates to videogames, one need not restrict the educational potential of such research to digital games alone; educators can certainly extend the application of the research findings to the use of other types of games (such as tabletop games), of which board games play an important part. Both videogames and board games share the potential to serve as mediums for students to learn and have fun while playing. Berland and Lee (2011) argued that “At their most base level, [both] games are systems of rules in which players operate on representations” (p. 65). Nevertheless, they also recognized that while in digital games, the hardware and software perform calculations and enforce the system of rules, in board games, players do these operations themselves as part of the whole gameplay. Wu, Chen, and Huang (2014) presented an interesting argument supporting board games in schools. They state that, although board games generally have a shorter game time and complexity as opposed to, for example, simulation online games, these same characteristics properly make them better suited for use in a classroom context. In fact, board games have been used as teaching materials since ancient times – mainly because, while playing them, players increase “their critical thinking, problem solving, analysis, reasoning, planning, and communication skills” (Hinebaugh, 2009, p. 10). Also, Mostowfi, Mamaghani, and Khorramar (2016) remarked that board games can foster collaborative and communication abilities by requiring intensive teamwork while solving conflicts during gameplay. In the same vein, Chiarello and Castellano (2016) highlighted the importance of the downtimes that commonly happen in board games – such as when players wait for each other’s moves – saying that they foster reflection, discussion, and a natural clarification of the topics.

432

A. Santos

Acknowledging the differences between both, some issues of interest to both digital and nondigital game-based learning researchers and producers arise. For instance, a significant question presents itself: how to create games with the right match between the students’ learning processes themselves and what the game affords, such as motivation, engagement, and fun. In other words, the challenge centers around how to adequately align both instructional design and game design. Kebritchi and Hirumi (2008) did a literature review to identify how educational videogame producers were using learning and instructional theories as the basis for their design decisions. They found that less than half of the authors reviewed stated the pedagogical theories on which they based their games. This result underscores the need for game studies to clearly identify how game design decisions have been integrated with the selected instructional strategies so that designers and researchers can better understand the relationship between game elements and learning. Van Staalduinen and de Freitas (2011) argued that “Pedagogy and game design currently seem to be two separated worlds” (p. 29). In the same vein, Becker and Parker (2014) accurately paraphrased the two (struggling) positions as either “game designers suck all the learning out of games” or “instructional designers suck all the fun out of games” (p. 183). The argument says that commercial game producers know about game design, but not about pedagogy, and that educators know about pedagogy, but not principles of game design. The concern presents this divorce between instructional design and game design as hindering the possibilities for game-based learning developers to operationally apply sound methodologies to obtain the desired learning outcomes (Egenfeldt-Nielsen, 2005, in van Staalduinen & de Freitas, 2011). In short, the risk exists that research may end with games “that neither instruct nor engage the learner” (Garris, Ahlers, & Driskell, 2002). The result of the meta-analysis that Clark et al. (2014) did to reevaluate the efficacy of digital games for learning (in studies published between 2000 and 2012) underscores this issue of the need for using well-developed design guidelines when producing instruction materials (such as digital or board games). They analyzed value-added game research – that is, studies where comparisons were made between the standard version of a game and its enhanced form (where they added a design strategy) to test its efficacy. They found that learning was improved with the enhanced versions of the games, thus attesting to the importance of design to the fostering of learning. Young et al. (2012) also did a literature review to inquire into the relationship between videogames and academic achievement in the K-12 curriculum. They were not able to find solid evidence regarding how best to use games in the educational structure of K-12 schooling. They explained their results pondering upon the fact that game play represents a rather complex, contextualized, nonlinear experience and that every time a student plays the same game again, it becomes a different experience. This conclusion highlights the need of using alternative research methods to understand what it means to engage cognitively and physically in both digital and nondigital games. Besides the experimental approach, researchers could use different research approaches – such as the interpretivist qualitative paradigm,

17

Board Games as Part of Effective Game-Based Learning Strategies

433

which assumes that people build meaning and reality socially (and which thus can prove to (very much) aid in the understanding of how students live the experience of game-based learning in the context of social life and how they interpret those experiences. Another result from the literature, also relevant for digital and nondigital games, was what Sitzmann (2011) concluded when doing a literature review on the instructional effectiveness of digital simulation games for improving adults’ work-related competencies. This author compared the effects on learners in game conditions vs. nongame conditions in affective, behavioral, cognitive, and skill-based training outcomes. He found higher gains when the game required trainees to learn actively as opposed to its using a more passive method, which suggests an instructional strategy potentially relevant for all types of games. Actually, this result accords with the experiential learning approach, which basically understands learning as being the process of constructing knowledge through comprehending and transforming one’s experience (Kolb, 1984) – a common framework when developing simulation games. This last genre of games includes both digital and nondigital versions and looks to situate a player in a context that models a real-life situation, where she/he can engage with the content in different types of problem-solving and decisionmaking experiences. However, some differences are apparent (By, 2012): the digital type of game simulates the event or situation using a core preprogrammed software algorithm that would constitute the basis for the mechanics of the game and which would react according to the different student’s actions during game play; thus, the rules are not visible to the players. With a game board, although players also experiment the simulated situation through following well-designed mechanics, the players themselves put the rules into action. As for simulation wargames, By (2012) stated that this difference affects how people play this type of game. Indeed, in the digital version, game players do not have equal access to rules and relevant information, while in a wargame played on a board with all sorts of accessories, such as maps and cards, the participants can “publicly examin[e] and discus[s] [the rules] in detail” (By, 2012, p. 177), so that all players can have a clear understanding of them.

Teachers Creating Their Own Games As the introduction of this chapter highlighted, the author aims to propose a method to aid teachers in their developing educational board games and successfully integrating them into their teaching. This may present a challenge, because teachers might not have the necessary knowledge and skills for game design or because they might not have enough motivation to develop new learning materials (considering their habitually rather high workload). The method that this chapter suggests addresses both issues by proposing that teachers develop their own games. It recommends the application of this tactic based on the following considerations:

434

A. Santos

1. Teachers already have the requisite expertise in both learning content and instructional design; and professional development can further teach them the basics of game design, so that they can create their own games. 2. Since knowing how to integrate content into game play constitutes one of the difficulties behind game-based learning (Kalloo, Mohan, & Kinshuk, 2015), teachers possessing both learning content mastery and expertise in instructional and game design would likely have more capability in solving this issue. 3. Teachers developing their own educational materials would likely have more motivation to integrate them into their everyday teaching practices. Of notable importance, it bears highlighting that, besides these three considerations described above, efficiency (time, cost, and effort vs. learning gains) also constitutes a relevant element worth contemplating when asking teachers to develop new learning materials. The author means “efficiency” in the sense of it not proving worthwhile to have teachers investing a lot of time and effort producing an educational game that would not have a significant impact on their students’ quality of learning. For example, one would not advise starting the production of an educational board game for learning objectives that are basically aiming for drill and practice (e.g., simple math operations) or to learn simple mental associations (e.g., countries and their capitals). Not that one could not possibly create a game board for these types of tasks, but in both of these examples, students could better learn such basic content through more cost-effective and efficient instructional strategies. Thus, teachers’ projects should aim for the achievement of more complex and challenging cognitive abilities, such as complex decision-making and problem-solving.

The Sociocultural Constructivist Perspective As the last section underlined, the method proposed in this chapter considers the production of instructional materials – a game board, in this case – that gives the learner/player the opportunity to engage in higher-order thinking processes. As a result, the author selected constructivism as the theoretical framework for the methodology. In fact, developers of game-based learning experiences very commonly use the constructivist approach as their theoretical framework (Wang & Burton, 2012) because it understands learning as being a social construction of knowledge and teaching as the development of environments where students can engage in complex social activities. In this way, one can conceptualize a game (whether digital or nondigital) designed for educational purposes as constituting a learning environment; thus, one can design such a game by applying many of the research results already published regarding the proper development of learning environments (e.g., Jonassen, 1999; Perkins, 1991; Wilson, 1996). Also, some educational game designers have thought of the sociocultural constructivist framework as representing the best theoretical framework for the development of a game because it encourages a situated approach to learning. In other words, under the sociocultural constructivist view, learning depends not only on the

17

Board Games as Part of Effective Game-Based Learning Strategies

435

identity of the pupil (in terms of his/her cultural background, past experiences, and knowledge) but also on the experiences the learner lives during the act of learning (in terms of context, activities, and tools). This represents a situated perspective of cognition because it basically accepts that our “ways of reasoning are socially determined” (Resnick, 1991, p. 8) – namely, that human cognition and the social situation in which cognition happens do not occur independently of each other. Sociocultural constructivists viewed learning as being the result of what Lave and Wenger (1991) call legitimate peripheral participation, that is, a social process that – step-by-step – allows novices to become experts. In fact, this constitutes the natural way individuals use to solve problems during everyday activities. People engage all the time in all types of activities to solve their problems; and, during the process, they build relevant knowledge regarding not just how to solve a certain type of problems but also how to select and use pertinent tools in an integral way. Thus, there exists the possibility of affirming that daily problem-solving activities have a situated, contextualized characteristic. As for these sociocultural constructivist notions to the process of game-based learning, one can conceptualize the act of playing/learning in a game as constituting essentially a social experience occurring in a context, which also includes activities done outside the game and the classroom space itself. Consequently, when individuals develop a game of any type – even as children playing with their friends – they are, in fact, building a sort of simulated social context (a learning environment) in which players can become engaged in situated cognition (as discussed earlier) through active problem-solving. For this reason, the method to develop educational board games described in this chapter basically proposes that one should conceptualize a game board as constituting a learning environment with a learning objective stated in terms of a problem for a student to solve and not as just some content for a student to learn. Therefore, when a player finally solves the central problem – following a clear set of rules – she/he would also engage mindfully themselves in achieving the general goal of the game. Moreover, during that process, the player would likewise engage themselves in learning all sorts of skills, the intended educative content, and the appropriate selection and use of different cognitive and physical tools. In conclusion, the method has, as its basic design strategy, the idea that educators should develop games in a way that encourages players to engage in complex problem-solving activities that require the players to think socially, using all types of tools at their disposal inside and outside the game.

Method to Develop a Board Game for Game-Based Learning The author named the method described in this chapter as “LuDu,” which means “to play” in Esperanto (Santos, 2017, also describes a previous version of this method), and created it to serve as a set of guidelines for helping game-based learning producers design low-cost board games. The strategies recommended by this method basically try to achieve a good blend between teaching/learning practices and game design. LuDu methodology proposes that the following facilitates blending:

436

A. Santos

(1) One employs a more holistic approach when creating a board game (i.e., considering not only issues regarding game design and instructional processes but also the contextual conditions of use). (2) Teachers themselves develop the games they intend to use in their classrooms. The author developed the LuDu method from several sociocultural constructivist pedagogical ideas and proposes four pedagogical strategies for producing an educational game: Strategy 1. The teacher states the learning objective in terms of an authentic problem for students to solve, not as just some content for students to learn. Strategy 2. In order to solve the problem, students need to learn and apply the content. They should have access to the content at all times. Content could be integrated either within or outside of the resources of the game itself. Strategy 3. Solving the problem must constitute the goal of the game. In order to play, students must perform activities that lead them to achieve the stated goal. Strategy 4. With regard to activities that students perform while playing, the game design should integrate such activities into the set of game mechanics (rules in action). In order to operationalize the abovementioned strategies, the author has organized the method into three parts: (1) instructional design, (2) game design, and (3) evaluation/integration. Each part of the method defines particular activities that the teacher should perform to create and apply his/her board game-based learning experience. Although the suggested activities can be performed following a linear order, the method attempts to have an iterative essence at all times; that is, it is applied as a process of continuous improvement. In other words – as professional game designers commonly do – teachers can design, test, and modify their decisions at any point in the process, as Fig. 1 shows.

1. Instructional Design

2. Game Design

3. Evaluation/Integration

State learning objective

Test prototype

Apply

State subject matter

Build problem

Fig. 1 The LuDu method

Design

Modify

Design classroom application

Modify

17

Board Games as Part of Effective Game-Based Learning Strategies

437

Part 1: Instructional Design State Subject Matter: State Learning Objective – Build the Problem First, this method asks teachers to establish the subject matter area in which they are experts and within which they would choose to work when developing the project. The subject matter could include history, geography, math, grammar, physics, or any other area that the teacher teaches. For the sake of clarifying how to apply the method, the author will include a (partly designed) example board game (one based within the subject domain of history) and will present it along with the accompanying description of the methodology. During this first part, the method asks teachers to state their learning objective (as written in their lesson plans), which they want their students to achieve while playing their board game. The learning objective for the history example could be “Students will demonstrate the ability to explain – in their own words – the social, political, and economic situation that led to the Mexican Revolution of 1910.” In accordance with the first pedagogical strategy (i.e., a teacher’s statement of the learning objective in terms of its offer of an authentic problem for students to solve and not just as some content for students to learn), the methodology next asks teachers to state their learning objective in terms of a problem for students to solve. This probably constitutes one of the most important activities of the LuDu method – because, by stating the problem, instructors are also starting to delineate some of the basic aspects of the dynamics and mechanics of their game. Thus, educators should create the problem carefully, taking into account these three relevant aspects: the learning objective, the authenticity of the problem, and the context within which that problem arises. First, the problem should evolve from the learning objective – because, once again, students will engage themselves in achieving the learning objective by solving the problem. Secondly, according to the constructivist perspective, the problem should have authenticity – that is, it should constitute a problem similar to the type of problems that members of a certain community of practice solve when practicing their professional activities in performing real-world tasks. Hence, in order to create an authentic problem, teachers should first clarify the system of activities of a particular community of practice – that is, those activities that certain professionals (e.g., mathematicians, historians, linguists, etc.) usually perform. In order to accomplish this, the method proposes an adaptation (a simplification) of the process based on the activity theory proposed by Jonassen and Rohrer-Murphy (1999) for analyzing a system of human activities. The first column in Table 1 provides a detailed list of what the methodology asks teachers to describe and helps stakeholders understand the system of activities of the community of practice in which instructors are interested. Further, the second column in Table 1 shows the system of activities of historians. Once teachers identify a basic version of the system of activities of the community of practice in which they are interested, they can then generate an authentic problem, considering that information and the stated learning objective. For the history example, the problem could be “The UN has decided to ask you, as a

438

A. Santos

Table 1 Technique to analyze the system of activities of a community of practice The methodology asks teachers to Describe the main purpose that members of the community of practice seek when performing their professional activities

Describe the general outcomes that result from performing the professional activities of the community of practice

List the tools that the community of practice employs to mediate its activities

List activities that members of the community of practice traditionally perform

As for the history example, historians Research, study, analyze, and interpret data Gather general information about events, historical periods, people, and places; then, interpret that information, and write a paper explaining their conclusions Manuscripts to disseminate information regarding human events, historical periods, people, and places Academic papers and published books Film and television documentaries Computers (tablets, laptops, and desk computers) Internet connections Recording devices for audio and images Research at libraries Query electronic databases and search within papers and other data files Research on the Internet Do field trips Interview people Reanalyze and revaluate, based on multiple perspectives, historical documents Act as consultants for the film and TV industries regarding historical events Write academic papers to spread ideas Write history textbooks for schools

proficient historian in the study of the Mexican Revolution, to join a group of other experts similarly selected to spend some time in the conflicted country of Centerland and to advise representatives of the conflict groups regarding which actions they could take to help prevent a social revolution.” Indeed, this way of creating the problem triggers the narrative of the game. Thirdly, in order to complete the problem, teachers should also imagine and describe in detail the context in which the problem is arising; in this way, the narrative of the game begins to unfold. For this example, one could describe the context of the imagined country of Centerland – because in the game, the problem is occurring there. Thus, one could conceive of the following scenario: “The country of Centerland is going through a very complicated social, political, and economic situation. Centerland is an island in the Pacific Ocean near the coast of Central America and has very close cultural relations with Mexico and all the other countries in the region. The country has a population of three million people and an area of 250 square kilometers. Some international experts believe that Centerland is about to enter an armed revolution that could turn (with the advent of revolution) very bloody.” In addition, one could sketch a map of the country identifying its states or provinces; and one could describe the situation for the time being. The idea is that, while creating this contextual information, teachers are

17

Board Games as Part of Effective Game-Based Learning Strategies

fo

In

nd

en

la ter

C ve

Sa

o nf

I

6

ob

Pr

fo

In

fo

In

fo

In

In

Jo

1

ob

Pr

sé

n Sa

4

fo

3

8

ob

Pr

5

7

ob

ob

Pr

ob

Pr

Pr

439

fo

In

ob

Pr

2

Fig. 2 A sketch of a possible board for the game “Save Centerland!”

already previsualizing a possible board design. For example, the design of the board can be based on the map of Centerland (see Fig. 2 for a sketch of what such a board might look like). At this point in the process, teachers can iterate this instructional design process by first asking their students whether this problem and its context are clear enough and whether they might be interested in playing a game based on them. Depending on the learners’ answers, one can make necessary changes – mostly to the problem and its context because two givens for the course are the subject matter and the learning objective.

Part 2: Game Design Design: Test the Prototype – Modify During this second part, teachers design their board game on the basis of all the pieces of information created so far and by identifying several game elements. The first column of Table 2 lists these game elements, while the second column continues the history example. Teachers are now able to build a board game with all its elements to crystallize all the pieces of information identified so far. For this, they can use all kinds of materials – such as cardboard, glue, scissors, markers, crayons, and Post-it notes. In this step, the educator seeks to finally have a prototype version of the board game, so that he/she could then test it with a small group of students in the classroom (see Fig. 2). Teachers are now ready to test the prototypes of their board games. In order to successfully complete this step, the methodology advises them to test it first with a small group of students to identify whether the ideas for the mechanics of their game are working properly. During this first iteration, teachers should have a greater interest in testing the board game itself than in testing their students’ learning of

440

A. Santos

Table 2 Relevant game elements The method asks teachers to State the goal of the game – that is, what players need to do to win the game. For instance, to conquer as many planets as possible, to reach a certain point on the board, to solve a puzzle or a mystery, etc. Also at this stage, one can imagine a catchy name for the game Identify the actions, activities, or tasks that their students plan to perform to achieve the goal of the game. For example, what do players do to start the game and to move and advance towards the goal? What do the rules permit, and what do they not permit? When or how do players reach the end of the game?

Identify the actions that the game does. For example, the game allows each player a limited time to move, increases difficulty when a player reaches a certain square in the game, gives bonus points each time a player reaches a certain square, etc. Identify what would impede players to reach the game’s goal. For example, other players block their movements or destroy their space vehicles, a player loses or starts again when getting double sixes rolling the dice or when losing all their lives in the game, etc.

For the history example Goal of the game: “As part of a group of experts assigned by the UN, prevent the country of Centerland from entering an armed social revolution” This game is collaborative because, to achieve its goal, players should play as a team The game is named “Save Centerland!” Players should start at the capital city (San José) of Centerland; each player should roll a pair of dice to advance through the board; in each province of the country, each player reads an “Information” and a “Problem” card; the game ends when solving all the problems regarding the situation in Centerland. At the end, depending on their actions, players could help prevent a social revolution. The would-be statements that coincide with the learning content would constitute the right answers As the players move along the map, the game gives them information and presents smaller problems for them to solve. Players should choose their answers and move accordingly to different possible places on the map If the players choose wrong answers along the course of gameplay, then, instead of preventing social revolution in Centerland, the social situation of the country remains the same or worsens

the subject matter. Then, teachers can repeat this Part 2 to whatever extent they may feel necessary; and after each iteration, according to the results, they should make the necessary modifications to their games.

Part 3: Evaluation/Integration Design Classroom Application: Apply – Modify Once teachers have iterated the testing of their board game prototype several times and feel comfortable with using the game in their regular classroom, they are trained in designing an instructional plan for their classroom application. Teachers should organize their instructional plans into four phases: (1) preparation (a phase to explain students both the learning objective and the goal of the game), (2) playing (students start to play the board game, familiarize themselves with its rules, and attempt to

17

Board Games as Part of Effective Game-Based Learning Strategies

441

reach its goal), (3) debriefing (an after-the-game session during which students can reflect upon the experience of playing the game and ask and answer questions regarding the learning content to clarify possible errors), and (4) evaluating (students undergo evaluation to assess whether they achieved the stated learning objective, learned the expected content, and been able to develop problem solving skills). In this latter phase, teachers can design a written test to assess whether their students achieved the learning objective, as defined in terms of the knowledge they constructed during their experience with the game and as defined by the skills they were able to develop. In order to assess students’ knowledge of the subject matter, teachers can ask them to write an essay. For example, for the abovementioned history case, students can write an essay to explain – in their own words – the social, political, and economic situation that led to the Mexican Revolution. As to the evaluation of learners’ skills, instructors can provide the opportunity for them to solve problems that are similar to the ones they worked out in the game. At this stage, teachers should principally assess the process the students followed to solve the problem rather than just evaluating the final proposed solution. Once teachers have designed an instructional plan, they can finally apply it to their regular classroom following its four phases. However, it is important for the educator to bear in mind the fact that many things might go wrong when applying the instructional plan the first time and that several iterations could prove to be necessary prior to the achievement – through usage of the board game – of a well-tested gamebased learning experience with the board game.

Case Study First Iteration The author tested the method to two occasions by applying it to different groups of teachers. In the first instance, he applied it to a group of 11 high school teachers from a local private school, during a 4-h workshop divided into 2 sessions. He video recorded the whole workshop for further analysis. In this case, teachers were not required to apply and test their prototypes with their students, because the researchers were more interested in first assessing the process of using the method to develop an educational board game. The workshop started with an open-ended questionnaire teachers were asked to complete about their possible previous adoption of any type of games in their teaching practices; in case they had, they were invited to describe their experiences in detail. The analysis of the questionnaire showed that all the teachers were already familiar with the use of games in their teaching practices. One teacher explained how she had used dominoes to teach fractions; several had created their own version of some popular table game to teach their content (such as Jeopardy, Marathon, crosswords, or hangman); another had developed political simulations to teach civics and ethics; and two explained how they had used educational computer games to teach grammar rules. Then, participants were given the method in the form of a written manual which contained

442

A. Santos

precise, step-by-step instructions for developing a board game according to the LuDu method. The manual also included a complete example of how to design a game board with the method. In order to give teachers the opportunity to practice it, the workshop facilitator first read out loud an example of the application of the method to explain all its parts and then asked them to form teams and create a corresponding board game for that example using the available materials. The objective seeked with this activity was that teachers had the opportunity of practicing the steps suggested by the method without worrying about identifying an instructional objective of their own. The analysis of the video that was recorded during this activity showed that while the facilitator read the example, teachers looked very passive and rather demotivated. However, when teams were formed and they started to build the corresponding games, things changed, motivation increased, and the trainees spent most of their time working hard. At the beginning of the second session of the workshop, three new teams were formed according to the teachers’ subject matter expertise; this time, the teams were asked to follow the method on their own to create, step-by-step, their board game (all the necessary building materials were also facilitated). The three teams were able to create rather interesting educational board games. One team developed a board game in the domain of physics, teaching the scientific method; another developed one in the domain of history, teaching the social context in which people built a famous Mexican cathedral; and the third group developed one in the domain of language, teaching the analysis of literary works (see Fig. 3). Finally, in a focus group activity, teachers were asked to comment on the experience of creating their own educational board game. The analysis of the video showed some interesting results. First, teachers affirmed that, given their later success, they were surprised that they had been able to create a complete board game following the method and affirmed that they had initially been uncertain about their ability to translate their academic expertise into a game and build it. However, they also said that, although they felt interested and motivated, they found the manual rather repetitive and confusing; and they expressed that the different steps were asking almost the same thing. They were motivated to create the board game following the example in the manual, but not so much to repeat every step for the development of their own game during the second session. In fact, they tended to anticipate a later step in the method all the time, imagining things about their game before the method actually asked them. Also, another teacher said that although he had found the activity

Fig. 3 Board games developed by the teachers

17

Board Games as Part of Effective Game-Based Learning Strategies

443

amusing, he had major concerns about finding the time to develop a board game; he honestly expressed that he thought that he would not create board games for his classes because he considered the process as being very time-consuming. Another participant stated that he did not understand how winning a game could relate to the purpose of learning something. In his view, achieving the learning objective and winning the game could represent rather different goals. This comment clearly shows how complex the issue of aligning instructional design and game design can be, as the end of the previous section remarked. In general, the test clarified that it had not been easy for the teachers to make a bridge between the instructional aspects of the task and how to translate them into a complete board game.

Second Iteration The author tested the method for a second time with a group of university teachers. In this occasion, he divided the workshop into two 3-h sessions because this time, teachers were asked to create their prototype games during the first meeting, test them with their students sometime during the week, and report the results during a second session. Thus, in this experience, the facilitator continuously stressed the iterative essence of the method, although they had time to test their products only once. The author offered the workshop to all the teachers at a local university; and 16 subscribed voluntarily, although only 11 teachers showed up on the day of the workshop. Taking into consideration the results of the previous test, he redesigned the method and its corresponding manual. He reduced the number of steps, deleting those that were considered repetitive. Also, he changed its instructional design in order to make it easily readable and understandable and to make it a self-learning instructional material. In addition, the author used a more humorous visual design so that teachers could feel more at ease following the steps. A decision was made to keep the example regarding how to design a board game using the method; however, teachers were not asked to actually build a board game for that particular example, as in the previous case, to avoid unnecessary repetitions when building their own game later on. Instead, the author appropriately inserted the example along the progression of methodological steps, so that teachers could use it as a reference at any time. At the beginning of the workshop, teachers answered an open-ended questionnaire. In this occasion, only four teachers stated that they had used games in their teaching practices. Three had used nondigital games, such as creating airplane paper models, using magnet letters to form words, and building diverse objects using cardboard. One teacher reported using the digital app Kahoot! for learning games at least once every semester. After answering the questionnaire, the author gave a general introduction regarding game-based learning to the whole group; subsequently, four teams of teachers were organized according to their academic expertise. They were given the new manual, asked to design and create their board games following the steps depicted in it by themselves, and instructed that they could ask for clarification of doubts at any time. The four teams were able to follow the method and design their own educational board games during the first 3-h session of the workshop. One team,

444

A. Santos

formed by a psychologist and a surgeon, created a game to teach the human brain anatomy and its different functions; another team, formed by business-oriented teachers, created a game to teach finances using the mechanics of the Monopoly game; another team, formed by a psychology teacher and three library specialists, developed a board game to teach students how to evaluate information resources located on the Internet when doing research; and a fourth team, formed by engineering teachers, created a board game where students had to build a structure that could hold a certain weight (see Fig. 4). At the end of this session, teachers were instructed to test their games with their students, if possible; and they were invited to present their results to the participants of the next session. After a week, the four teams gathered again and reported what had happened during the testing of the board games they had created. Three of the teams reported results (the other team had not had the time to test its prototype). The engineering team presented a Power Point with pictures of the actual testing session; they explained that, although they considered the experience as a whole to have been very successful, their students never used the board that had been created for their game. Thus, at the end, their design worked more as a simulation learning experience than as a board game. However, they affirmed that, after having learned from the testing experience, they had already designed a new board game which they thought would work better and which they were going to test again. This team (perhaps due to their engineering background) easily adopted the iterative nature suggested by the

Fig. 4 Board games developed by the teachers

17

Board Games as Part of Effective Game-Based Learning Strategies

445

method. The brain anatomy group also reported that they had tested their prototype; in particular, one teacher expressed that she was very happy with the whole experience and that she had discovered that teaching in general could be fun, although she knew that their game still needed several improvements. The business teachers reported that their prototype had worked pretty well and that their students looked motivated while playing their Monopoly-style board game. During the final focus group, all the teachers stated that they were satisfied with the experience and that they were very motivated to create and use board games in the future. However, analyzing the whole process that each team followed in developing their game, it was clear that it was difficult for all teams to pass from the instructional aspects to the game design aspects of the process. For example, the team that created a business board game following the mechanics of the Monopoly game first created a design on paper following the manual, which did not correspond with the final game they actually built. It was as if the production dynamics of the game pushed them towards a different direction and they simply forgot what they had previously stated in the manual as having been their instructional intent. Of course, as explained before, the LuDu methodology seeks the continuous evolution of the game being designed through its iterative essence, but the original learning objective should not be lost. In the case of designing and building educational materials, it is important that the students end by learning what was originally planned for them to learn, at least to a large extent. Undoubtedly, one would consider a certain leeway to be acceptable. Still, this case constitutes evidence of how the instructional design process and the game design process misalign very easily and should thus call for careful supervision.

Conclusion Within this manuscript, a method of integrating board games as part of effective gamebased learning strategies has been described. First, the author established the need for these types of materials and described the theoretical framings of the proposed method. Then, a two-part case study and its results were presented; and some parts of the method were evaluated. Although, this work detailed the successful implementation of many important aspects of the process of applying the method, the results showed that the method as a whole still needs much more testing. The next step will involve the implementation of an assessment plan that would test the whole base-learning experience – that is, both the creation of the game and its classroom application. Although this study allowed for gaining some understanding of how to properly align instructional and game design, appreciation of the whole picture requires further work. The author clearly observed that, during the testing of the method, teachers were able to state their instructional objectives; however, when they started imagining their game, they tended to forget their instructional intentions. In other words, they could state their instructional objectives and then imagine a board game; but they tended to end with a product rather separate from their initial objectives. Therefore, the quest to understand the process of properly bridging instruction and game design – to ensure students’ high quality of learning – still constitutes the central challenge. Lastly, a final

446

A. Santos

pertinent reflection pertains to the time required to develop a good game-based learning experience. In the case of teachers developing their own board games, they should properly get the extra time and support from their educational institutions so that the successful implementation of this potentially promising instructional strategy can assure a high quality of learning.

References Arnab, S., Brown, K., Clarke, S., Dunwell, I., Lim, T., Suttie, N., . . . de Freitas, S. (2013). The development approach of a pedagogically-driven serious game to support relationship and sex education (RSE) within a classroom setting. Computers & Education, 69, 15–30. Becker, K., & Parker, J. (2014). Methods of design: An overview of game design techniques. In K. Schrier (Ed.), Learning, education and games: Volume one: Curricular and design considerations (pp. 179–198). Pittsburgh, PA: ETC Press. Berland, M., & Lee, V. R. (2011). Collaborative strategic board games as a site for distributed computational thinking. International Journal of Game-Based Learning, 1(2), 65–81. https:// doi.org/10.4018/ijgbl.2011040105 By, T. (2012). Formalizing game-play. Simulation & Gaming, 43(2), 157–187. https://doi.org/ 10.1177/1046878110388239 Chiarello, F., & Castellano, M. G. (2016). Board games and board game design as learning tools for complex scientific concepts: Some experiences. International Journal of Game-Based Learning, 6(2), 1–14. https://doi.org/10.4018/IJGBL.2016040101 Clark, D., Tanner-Smith, E., & Killingsworth, S. (2014). Digital games, design and learning: A systematic review and meta-analysis (executive summary). Menlo Park, CA: SRI International. Clark, R. E. (2007). Learning from serious games? Arguments, evidence, and research suggestions. Educational Technology, 47(3), 56–59. de Freitas, S., & Liarokapis, F. (2011). Serious games: A new paradigm for education? In M. Ma, A. Oikonomou, & L. C. Lakhmi (Eds.), Serious games and edutainment applications (pp. 9–23). London, England: Springer. Egenfeldt-Nielsen, S. (2005). Beyond Edutainment: Exploring the Educational Potential of Computer Games. Copenhagen, ITUniversity of Copenhagen. Garris, R., Ahlers, R., & Driskell, J. (2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4), 441–467. https://doi.org/10.1177/ 1046878102238607 Hinebaugh, J. P. (2009). A board game education. Lanham, MD: Rowman & Littlefield Education. Jonassen, D., & Rohrer-Murphy, L. (1999). Activity theory as a framework for designing constructivist learning environments. Educational Technology Research and Development, 47(1), 61–79. Jonassen, D. H. (1999). Designing constructivist learning environments. In C. M. Reigeluth (Ed.), Instructional-design theories and models (2nd ed., pp. 215–239). Mahwah, NJ: Lawrence Erlbaum Associates. Kalloo, V., Mohan, P., & Kinshuk, D. (2015). A technique for mapping mathematics to game design. International Journal of Serious Games, 2(4), 73–92. Kebritchi, M., & Hirumi, A. (2008). Examining the pedagogical foundations of modern educational computer games. Computers & Education, 51(4), 1729–1743. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. New Jersey, NJ: Prentice-Hall. Lave, J., & Wenger, E. (1991). Situated learning. Legitimate peripheral participation. New York, NY: Cambridge University Press. Mostowfi, S. S., Mamaghani, N. K., & Khorramar, M. (2016). Designing playful learning by using [an] educational board game for children in the age range of 7–12: A case study: Recycling and

17

Board Games as Part of Effective Game-Based Learning Strategies

447

waste separation education board game. International Journal of Environmental & Science Education, 11(12), 5453–5476. Perkins, D. N. (1991). Technology meets constructivism: Do they make a marriage? Educational Technology, 31(5), 18–23. Prensky, M. (2012). From digital natives to digital wisdom: Hopeful essays for 21st century learning. Thousand Oaks, CA: Corwin. Resnick, L. B. (1991). Shared cognition: Thinking as social practice. In L. B. Resnick, J. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 1–20). Washington, D.C: American Psychological Association. Santos, A. (2017). Instructional strategies for game-based learning. In T. Kidd & L. R. Morris (Eds.), Handbook of research on instructional systems and educational technology (pp. 164–173). Hershey, PA: IGI-Global. https://doi.org/10.4018/978-1-5225-2399-4 Schrier, K. (2014). Introduction. In K. Schrier (Ed.), Learning, education and games: Volume one: Curricular and design considerations (pp. 1–4). Pittsburgh, PA: ETC Press. Shaffer, D. W. (2006). How computer games help children play. New York, NY: Palgrave Macmillan. Sitzmann, T. (2011). A meta-analytic examination of the instructional effectiveness of computerbased simulation games. Personnel Psychology, 64, 489–528. Tobias, S., & Fletcher, J. D. (2007). What research has to say about designing computer games for learning. Educational Technology, 47(5), 20–29. Tobias, S., Fletcher, J. D., Yun Dai, D., & Wind, A. P. (2011). Review of research on computer games. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 127–221). Charlotte, NC: Information Age Publishing. van Eck, R. (2006). Digital game-based learning: It’s not just the digital natives who are restless. Educause Review, 41(2), 16–30. Retrieved from http://er.educause.edu/articles/2006/1/digitalgamebased-learning-its-not-just-the-digital-natives-who-are-restless van Staalduinen, J. P., & de Freitas, S. (2011). A game-based learning framework: Linking game design and learning outcomes. In M. S. Khine (Ed.), Learning to play: Exploring the future of education with video games (pp. 29–54). New York, NY: Peter Lang. Wang, F., & Burton, J. (2012). Second life in education: A review of publications from its launch to 2011. British Journal of Educational Technology, 44(3), 357–371. Wilson, B. (Ed.). (1996). Constructivist learning environments. Englewood Cliffs, NJ: Educational Technology Publications. Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265. Wu, C., Chen, G., & Huang, C. (2014). Using digital board games for genuine communication in EFL classrooms. Educational Technology Research and Development, 62(2), 209–226. https:// doi.org/10.1007/s11423-013-9329-y Young, M. F., Slota, S., Cutter, A. B., Jalette, G., Mulin, G., Lai, B., . . . Yukhymenko, M. (2012). Our princess is in another castle: A review of trends in serious gaming for education. Review of Educational Research, 82(1), 61–89. https://doi.org/10.3102/0034654312436980

Antonio Santos, Ed.D. from Indiana University, has been a professor, researcher, and consultant in the field of educational technology in several Mexican universities, and has also been a producer of instructional materials for teachers’ training at different educational and industrial organizations. Santos has published several articles in the areas of constructivist learning environments, the use of technology in developing countries, and serious games and simulations, and has also presented papers in world conferences on themes like digital divide, distance education, contemporary pedagogies, and games and simulations. Currently Santos is a developer of teacher’s training projects and designer and producer of serious video games.

Section II Impact of Educational Policies and Research on Educational Practice

Impact of Educational Policies and Research on Educational Practice: A Section Introduction

18

Robert G. Doyle and Drew Polly

Abstract

This serves as a brief introduction to the section entitled “Impact of Educational Policies and Research on Educational Practice.” The chapters in this section include systematic reviews, meta-analyses, and syntheses of empirical research and theoretical frameworks related to design and policy. Keywords

Analyses · Design · Framework · Policy · Systematic review · Theoretic framework

Introduction Gwendolyn Morel and Heather Keahey share a meta-trend analysis of abstracts of publications in the SCOPUS database from 2009 to 2018, which focused on research published on Massive Online Open Courses (MOOCs), in ▶ Chap. 19, “Insight into MOOCs Research: A Meta-trend Analysis of Publications (2009–2018).” An analysis of 708 articles which met the authors’ criteria were analyzed. The data analysis indicated that the most frequently published topics included student engagement and performance, teaching and design, and community and social interaction. Based on their findings, authors provided recommendations for future research such as replicating the study using other databases or using different analytical methods to conduct the meta-trend analysis. R. G. Doyle Harvard University, Cambridge, MA, USA D. Polly (*) University of North Carolina at Charlotte, Charlotte, NC, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_129

451

452

R. G. Doyle and D. Polly

Yale Kali and colleagues share a ▶ Chap. 20, “Design-Centric Research-Practice Partnerships: Three Key Lenses for Building Productive Bridges Between Theory and Practice,” examining research-to-practice partnerships (RPPs) that attempt to bridge the historically large gap between theory and practice. The authors share a framework that includes the intersection of three meta-design principles and three pragmatic design principles followed by two cases that examine these principles in action. The authors highlight ways to frame RPPs through these empirically based design principles and give readers considerations for the design and implementation of future RPPs. Shirley Dawson and Vicki Napper share a ▶ Chap. 21, “The Confluence Effect of Policy, Mental Models, and Ethics on Individual Behavior,” where they consider the intersection of policy, mental models, and how those models are applied to describe and examine behavior with regard to learning. Their detailed review of the literature leads to recommendations for careful examination of past policies and considering the implications of future policies that may influence training, learning opportunities, and behavior. Brent Philipsen and colleagues share a ▶ Chap. 22, “Teacher Professional Development for Online Teaching: An Update of Insights Stemming from Contemporary Research,” that includes a systematic meta-aggregative approach to analyze qualitative data from articles published between 2016 and 2020 on the topic of teacher professional development for online teaching. The analysis narrowed the potential number of articles from 567 to 5 articles that met the authors’ qualifications for inclusion. Their synthesis led to six empirically based recommendations for the field including: (1) design and develop a supportive Teacher Professional Development (TPD) program and environment for online teaching; (2) acknowledge the existing context regarding online teaching; (3) address teacher change associated with the transition to online teaching; (4) determine the overall goals and relevance of TPD for online teaching; (5) acknowledge teacher professional development strategies associated with the transition to online teaching; and (6) disseminate knowledge, skills, and attitudes about online teaching and evaluate the teacher professional development program.

Dr. Robert Doyle is a retired Associate Dean and Academic Advisor at Harvard University. He has received numerous AECT awards including the recent 2022 Presidential Service Award, the 2021 Distinguished Service Award, the 2021 International Division Distinguished Service Award, the 2018 J. Michael Spector Appreciation Award, and others. Doyle has presented and served as a featured presenter or keynote speaker on the topics of flipped classrooms, the college admissions process in the United States, massive open online courses (MOOCs), assistive technology, and higher education technology design for learning spaces at conferences in the United States, Austria, Germany, Hungary, Turkey, Singapore, France, Colombia, Estonia, and Cyprus. He has also published numerous journal articles on similar topics. Drew Polly is a Professor in the Elementary Education program at the University of North Carolina at Charlotte. His research interests focus on supporting teachers’ and teacher candidates’ use of learner-centered pedagogies in their classrooms, including the use of digital technologies.

Insight into MOOCs Research: A Meta-trend Analysis of Publications (2009–2018)

19

Gwendolyn M. Morel and Heather L. Keahey

Contents Translation of MOOCs Research Trends into Practice: A Meta-Trend Analysis of Publications (2009-2018) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . History of MOOCs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MOOCS Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliometrics and Text Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Transformation and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Publication Time Trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prolific Countries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prolific Journals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Taxonomies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Time Trends of Article Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Publication Clusters and Countries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prolific Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Translating Research Trends into Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Open Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning Theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Informal Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Business of MOOCs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

454 455 456 458 459 459 459 460 460 461 461 463 464 465 466 467 469 470 470 471 472

G. M. Morel (*) Digital Learning, Texas Higher Education Coordinating Board, Austin, TX, USA e-mail: [email protected]; [email protected] H. L. Keahey Liberty University, School of Education, Lynchburg, VA, USA e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_72

453

454

G. M. Morel and H. L. Keahey

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Further Research Suggestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

473 474 474 475

Abstract

Despite the proliferation of research focused on Massive Open Online Courses (MOOCs), the debate continues about whether MOOCs are a viable educational method and what the future holds. Like many emerging technologies, the excitement for what MOOCs could do to affect the educational landscape was promising to early adopters. However, after several years of experimentation, MOOCs are still finding their place in the ranks of educational disruptors. This chapter is organized by a brief history of MOOCs, bibliometrics, and trends in MOOCs research published in the Scopus database for the period of 2009–2018, followed by a discussion of how the research offers insight into practice. Keywords

MOOC · Massive open online course · Bibliometrics · Text mining

Translation of MOOCs Research Trends into Practice: A MetaTrend Analysis of Publications (2009-2018) Massive open online courses (MOOCs), a phenomenon in online learning, provide an avenue for the dissemination of academic course content by giving students access to online resources including an expert in an area of study and a connection to other learners (McAuley, Stewart, Siemens, & Cormier, 2010). Published research focused on MOOCs has increased considerably in recent years. However, deliberation continues about whether MOOCs can offer students a worthwhile educational experience. Previous research focused on MOOCs is partial in scope, leaving a shortage of shared, comprehensive answers to critical questions. Some examples include: How has the published research changed over the past years? Who is contributing to the conversation? What are the observed trends associated with research and peer-reviewed publications? And perhaps most importantly, how does the research translate into practice for both educators and learners? Trends in research help to provide a look at the implications for teaching and learning as well as aiding in identifying future areas of study. Using bibliometric measures and text mining, a longitudinal view of MOOCs research is provided for the period of 2009 to 2018 in an attempt to provide a comprehensive review of previous research so that practitioners may have a source for a systematic application in their own practice.

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

455

Our research questions were: 1. What are the bibliometrics of published research in MOOCs? 2. What taxonomies can be derived from MOOCs research as indicated by peerreviewed article abstracts? 3. What are the current and emerging trends in research on MOOCs?

History of MOOCs Although 2008 marks the year in which the first Massive Open Online Course (MOOC), a term first coined by Dave Cormier, was offered (Downes, 2008), it would be three years later when the concept gained increased attention. In 2011, Stanford professors, Sebastian Thrun and Peter Norvig, offered free courses to anyone who wanted to take part (Haber, 2014). Introduction to Artificial Intelligence attracted over 120,000 students which seemed to indicate a viable market for these types of courses early on (Hill, 2014). Following was the emergence of “spin-off companies” such as Udacity and Coursera (Anderson & McGreal, 2012, p. 385) and higher education consortiums such as EdX and the Open Learning Initiative (Breslow, DeBoer, Stump, Ho, & Seaton, 2013) all serving as platforms dedicated to MOOC delivery. The attention MOOCs were attracting prompted the New York Times to announce 2012 as “The Year of the MOOC” (Pappano, 2012). However, as (Haber, 2014) points out: Faith in the combined virtues of universal education and technical progress meant that from the nineteenth century onward, any new breakthrough in communications technology was almost immediately put to work toward the goal of educating the masses, with MOOCs being the latest manifestation of this historic impulse. (p. 20)

Prior to MOOCs, distance learning was conducted in the form of correspondence courses, dating back to the eighteenth century when Caleb Phillips advertised in the Boston Gazette shorthand lessons in which he would mail weekly lessons to students by mail (Holmberg, 1995). The development of reliable postal services allowed for formalized correspondence schools to emerge in the United States and Great Britain (Harting & Erthal, 2005; Keegan, 1990; MacKenzie & Christensen, 1971). In the twenty-first century, with the expansion of the Internet, increased accessibility to online and mobile technologies allowed for a shift from distance learning toward e-learning. Open learning, informal learning, personal learning networks, and most recently, learning platforms for the masses have benefited from this shift. MOOCs began as open source online courses with no prerequisites, fees, or accreditation that integrated “the connectivity of social networking, the facilitation of an acknowledged expert in a field of study, and a collection of freely accessible online resources” (McAuley et al., 2010, p. 4) boasting the potential to draw

456

G. M. Morel and H. L. Keahey

enrollment sizes into the thousands. A review of scholarly literature has shown various classifications of MOOCs with regard to the concept of “openness” (Ebben & Murphy, 2014, p. 336). For example, xMOOCs, sometimes referred to as “AI-Stanford” as a tribute to the 2011 course, are viewed as having an individualist, more traditional university course approach. The cMOOCs are the courses with a connectivist, social approach to learning. However, not all researchers make or prioritize these distinctions in their research (Liyanagunawardena, 2013), while others argue for the differentiation because of the underlying pedagogical underpinnings for each (Rodriguez, 2012, 2013). Regardless of the distinction in type of MOOC, they have been utilized in a variety of settings. In addition to the spin-off companies and consortiums offering MOOCs from established higher education institutions, corporate branches such as the HP Foundation created HP LIFE, an online set of “micro courses” designed for entrepreneurs globally (Hewlett Packard, 2015). The purpose was to serve unemployed and underemployed people around the world by providing practical educational resources and IT skills training. Courses were also designed to provide instructor support materials should K-12 teachers choose to bring the courses into their face-to-face classrooms as supplemental material (Hewlett Packard, 2015).

MOOCS Research The first incremental increase in published research on MOOCs appeared in 2011 when researchers were interested in how the novel format would “possibly enable the construction of a redesigned educational landscape that better fits this Knowledge Age” (deWaard et al., 2015, p. 95). For example, terms related to MOOCs research in the earlier years included language such as disruption, chaos, maze, myth, and mania in which researchers were interested in what the emergence of MOOCs would do to substantially change formal education (Daniel, 2012; deWaard et al., 2015; Mangan, 2012; Pence, 2012; Skiba, 2012). Throughout the duration of MOOCs research, advocates and skeptics resided on both sides of the debate where some predicted a revolution and others an apocalypse. Progressively, research on MOOCs, while not leaving behind the broad spectrum of potential, began focusing on particular elements of MOOC design. For example, design elements related to asynchronous communication, self-directed learning, problem solving strategies, time on task, video lectures, and peer assessment were reviewed (Breslow et al., 2013; Johnston, 2015; Lin & Cranton, 2015). User experience reports also began emerging (Bali, 2014; Haber, 2014; Liyanagunawardena, 2013) perhaps in response to the fragmented body of research on the topic where simply experiencing a MOOC for oneself was the most direct path to understanding them. Course design, quality, and learner expectations grew as strong focus areas for research beginning in 2015 (Lowenthal & Hodges, 2015; Veletsianos, Collier, & Schneider, 2015). By 2018, many of the research areas in MOOCs had been established; however, the addition of

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

457

MOOCs case studies from a global perspective was increasingly notable in the body of knowledge (Chen, Lee, & Hsiao, 2018; Jafari, Vajarah, Arefi, & Rezaeizadeh, 2018; Magaña-Valladares et al., 2018; Soyemi, Ojo, & Abolarin, 2018). Skeptics and advocates debate the merits of MOOCs in areas such as the quality of education, access to education, state funding, economic viability, assessment, and fraud (Gul, Mahaja, Shafiq, Shafi, & Shah, 2018; Hew & Cheung, 2014; McAuley et al., 2010). Research also reviews specific elements of course management such as enrollment, achievement, and student experience (Gaševic, Kovanovic, Joksimovic, & Siemens, 2014). In addition to being a disruptor to the current higher education system, those interested in leveraging the MOOC concept offer it as a model for transforming corporate training and employee development (Anderson & McGreal, 2012; Dodson, Kitburi, & Berge, 2015; Ebben & Murphy, 2014; Radford, Coningham, & Horn, 2015). Content analysis has recently emerged as a branch of research in response to an interest in understanding MOOCs and those who study MOOCs. Liyanagunawardena, Adams, and Williams (2013) claim to be the first to systematically review MOOC literature. They focused on 45 peer-reviewed papers during the period of 2008–2012 in an effort to establish areas of interest by publication type, publication year, and contributor. Researchers identified published papers “as relevant if their primary focus was to explore the concept of a MOOC or the implications for higher education, report on experiments with MOOCs, or compare MOOCs with other educational approaches” (p. 205). Of the papers collected, 80% were classified as case studies (21 articles) and educational theory (15 articles) which explained the focus of research from the learner perspective. Authors recommended creator/facilitator perspective, technological aspects, cultural tensions, and ethical aspects as suggestions for future research (Liyanagunawardena et al., 2013). Ebben and Murphy (2014) reviewed scholarly writing between the years of 2009–2013 to investigate themes chronologically. Researchers identified 25 peer-reviewed journal articles from nine major databases that revealed two major phases of research interests. Research imperatives were Connectivist MOOCs, Engagement, and Creativity for the period of 2009 to 2011, while imperatives for the period of 2012 to 2013 were xMOOCs, Learning Analytics, Assessment, and Critical Discourses about MOOCs (Ebben & Murphy, 2014). Authors suggest discipline-based and interdisciplinary research “to assess more fully the ramifications of massive open online education and to shape its future directions” (p. 343). Similarly, Sangrà and González-Sanmamed (2015) reviewed articles for the years 2013 and 2014 to also clarify publication year, type, and categories of interest. Veletsianos and Shepherdson (2015) conducted research using bibliometrics to investigate interdisciplinary MOOC research between 2013 and 2015. They were interested in addressing “complex scientific problems” while bringing in “novel perspectives into a field other than their own” (p. 1). Researchers identified 102 relevant papers using the following criteria: “(1) empirical, (2) published in a peerreviewed journal, in conference proceedings, or in Educause Review, (3) published

458

G. M. Morel and H. L. Keahey

or was available online as in press between January 2013 and January 2015, and (4) written in English” (Veletsianos & Shepherdson, Who studies MOOCs? Interdisciplinary in MOOC research and its changes over time, 2015, p. 5). In a comparison to the disciplines represented in Liyanagunawardena et al. (2013), Veletsianos and Shepherdson (2015) found increased interdisciplinary representation by recognizing an increase in computer science although education was the dominating discipline represented in both studies. Authors suggested further interdisciplinary focus in collaborative research efforts. Content analysis research studies continue through 2018 with focus areas such as social media use and course design (Costello, Brown, Mhichíl, & Zhang, 2018), vocational training (Paton, Fluck, & Scanlan, 2018), and social mobility (van de Oudeweetering & Agirdag, 2018).

Bibliometrics and Text Mining Managing and analyzing large amounts of data can be difficult for researchers. However, content analysis of collections of work can help scholars with a holistic understanding of a topic area and determine trends in research. A bibliometric analysis is a process to assess published research by determining specific indicators (Thelwall, 2008). The analysis includes quantitative statistics summarizing the publication information. Customary bibliometric analysis includes growth of papers, ranking of geographical distribution of authors, most prolific journals with other statistics associated with reference to authorship, frequency distribution of subject descriptors, and reference percentages per paper (Keshaval & Gowda, 2008). A bibliometric analysis can provide valuable information about trends within a specific field of study. Content analysis can supplement and augment bibliometric measures. Computerized exploration is one way to cope with mass amounts of information to discover relevance and interest (Feldman & Dagan, 1995). Such analysis can be a burden for researchers for a variety of reasons including time and effort. Text mining techniques can be employed as both an efficient and effective method for content analysis and categorizations (Hung & Zhang, 2012). Text mining denotes the practice of obtaining meaningful, nontrivial patterns from text-based files. The process involves structuring the input text by parsing and running it through a program to recognize and generates data based on various aspects of the text such as term frequency. Through text mining programs, high-quality information is generated for researchers by employing algorithms involving pattern recognition and statistical pattern learning (Nayak, Prasad, & Senapiti, 2015). Text mining has been used in prior research in learning technologies in the area of e-learning (Hung, 2012) and mobile learning trends (Hung & Zhang, 2012). This study extends the use of text mining techniques to analyze the growing body of peer-reviewed published research on the subject of MOOCs.

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

459

Method Data Collection Considering the growing number of published articles in print, online, and open access, it is nearly impossible to access and catalog journal articles without specific search criteria. This study focused on articles published in one of the most popular and widely accessed databases, Scopus (Elsevier, 2019). Scopus is a highly respected database known as the largest abstract and citation database of peerreviewed publications (Elsevier, 2019). It also contains export features that make a comprehensive bibliometric analysis feasible. The initial search was conducted using the keyword MOOC relative to topics and titles of articles within the database. The publication dates were restricted to years prior to 2019 in order to analyze complete years of research. Additional search restrictions include entries categorized as social science, journal articles, and written in English. A total of 736 articles were initially retrieved and exported to an excel spreadsheet for organizational purposes. From the initial list of articles, 28 were removed from the list because they did not fit in the scope of this study. Specifically, 12 did not contain an abstract, seven were duplicate entries, eight were not related to the topic, and the last one was an editorial. A total of 708 articles contained in 231 unique journals met the selection criteria and were used in this study.

Cleaning Data Peripheral abstract information such as copyright information and introductory statements such as abstract or summary were removed from the text in the abstract fields. Also, a new column in the article data spreadsheet was created to house the country of the first author. Though affiliation and correspondence address were generally provided, in order to run a bibliometric analysis of the contributing countries, a column with the singular entry of the country was needed. The final step in the cleaning process was to assign each article a unique identification number (ID). The information for each article was housed in unique rows in the spreadsheet. The articles were sorted alphabetically by the first author’s name and a unique ID was given to each one.

Data Transformation and Analysis Bibliometrics Within the spreadsheet, the data was organized and sorted based on the organizational parameters to examine a specific bibliometric measure. A variety of bibliometric measures were used in describing the characteristics of MOOC research in an attempt to provide a broad picture of overall publication trends of MOOC research. These measures included numerical growth of papers annually, a ranked list of the most prolific journals and authors, data discerning countries of origin for the articles, and top cited articles.

460

G. M. Morel and H. L. Keahey

Text Mining and Clustering Analysis The abstracts of the articles were used as data sources for the creation of the categorical taxonomy. The reasoning for using the abstracts is twofold. First, an abstract provides a concise yet comprehensive overview of the full article. Second, abstracts are much shorter in length than a full article, thus diminishing the influence of meaningless data which occurs within any publication. For the text mining process, Rapid Miner Studio 7.6 (Rapid Miner, 2017), an open source predictive analytics platform was used. Two main processes were used to create the categories, text mining and text clustering. The process for text mining included several steps. First, the abstract data was tokenized creating a collection of words contained in the source documents. The cases for all the words were transformed to lowercase. The data was then sent through a filter elimination for stop words (e.g., the, and, it). The data was then tokenized so that no term shorter than two characters in length was in the output. The stem operator was employed to strip the suffixes from words so that the same root with different tenses would be counted as the same word (e.g., “organize,” “organizes,” “organized”). The output for this process was word vectors using the term frequency-inverse documents frequency (tf-idf) statistic which denoted the importance of a word to a document by calculating the frequency of the word and offsetting that quantity by the frequency in which the word appears in all abstracts. Once this process was complete, the word vectors were run through a clustering algorithm within Rapid Miner. A k-means algorithm was chosen because it is a simple, straightforward, widely used process in data mining applications (Arthur & Vassilvitskii, 2007; Berkhin, 2006). This procedure produced a flat, nonhierarchical, cluster type categorization. A k-value of 17 representing the number of clusters was set by the researchers based off the comparison results of the group sum of squares and the number of clusters. Once the 17 clusters were calculated, they were interpreted and labeled by two specialists in the field of learning technologies to confirm inter-rater reliability.

Results Publication Time Trends A summary of the number of retrieved journal articles per year (prior to 2019) extracted from the Scopus database is given in Fig. 1. The first four years (2009–2012) of MOOC research publishing contain a total of six articles. The following year in 2013, the number of articles published grew to 24, the equivalent to four times the previous years combined. In 2014, the following year, the total number of publications increased to 78, a substantial growth rate of 225% over the prior year. The next year, 2015, the publications increased to 131, or a 68% growth rate over 2014. For the three years, 2015, 2016, and 2017, the publications remained relatively stable with a yearly fluctuation percentage of less

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

461

Published MOOC Research by Year 250

Number of Articles

213 200 150

131

123

2015

2016

133

78

100 50

24 1

0

2

3

0 2009

2010

2011

2012

2013

2014

2017

2018

Year Fig. 1 MOOC publication trends by year from 2009 to 2018

than 9%. The final and most recent year in the analysis, 2018, showed a notable increase of published articles from 133 to 213, or a 60% increase over the previous year, 2017.

Prolific Countries The bibliometric analysis identified the top nations, based on author affiliation, which produced the most publications concerning the topic of MOOCS during the specified time period (Fig. 2). The nation of origin was determined by the affiliation location of the primary author. Authors from 58 different countries contributed to the body of research. The countries with a contribution of over one percent from the corpus of articles are included in the graph. The top five most prolific nations were the USA (27.7%), United Kingdom (9.9%), Spain (9.2%), Canada (8.2%), and Australia (5.9%). The remaining countries shown in the graph contributed at least ten articles to the corpus of research on MOOCs.

Prolific Journals The journals that published the most articles on massive open online courses are listed in Table 1. The journals with the highest number of articles in descending order were International Review of Research in Open and Distributed Learning, International

462

G. M. Morel and H. L. Keahey

Prolific Countries Contributing MOOC Research 30.00% 27.68%

PERCENTAGE

25.00% 20.00% 15.00% 10.00%

9.89% 9.18%

8.19% 5.93%

5.00%

3.95%

2.54% 2.26%2.12% 1.98%1.84%

1.41%1.41%1.27%1.27%1.27% 1.13%

0.00%

COUNTRY

Fig. 2 The top 17 of 58 countries publishing the most MOOC journal articles Table 1 Journals publishing the most MOOC articles

Source title (h5-index)a International Review of Research in Open and Distributed Learning (46) International Journal of Emerging Technologies in Learning (15) Computers & Education (91) Distance Education (30) British Journal of Educational Technology (57) Educational Media International (18) Turkish Online Journal of Distance Education (30) Comunicar (38) Journal of Computer Assisted Learning (34) Open Learning (46) RUSC Universities and Knowledge Society Journal (21) a

Number of articles 87

% of articles 12.29%

28

3.95%

26

3.67%

15 14

2.12% 1.98%

14 13

1.98% 1.84%

12 12

1.69% 1.69%

11

1.55%

11

1.55%

Theme Research, theory, and practice in open and distributed learning Trends and research results in technology enhanced learning Education and digital technology Open and distance learning Digital educational and training technologies Educational Media Distance education and open learning applications Communication and education Information and communication technology supporting learning Open, flexible, and distance learning Information and communication technology in higher education

From Google Scholar

Journal of Emerging Technologies in Learning, Computers & Education, Distance Education, British Journal of Educational Technology, Educational Medial International, Turkish Online Journal of Distance education, Comunicar, Journal of Computer Assisted Learning, Open Learning, and RUSC Universities and

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

463

Knowledge Society. Each of these 11 journals published at least 11 articles about MOOC research and come from varied origins of publication. The h5-index value is the largest number, h, representing h published articles which have been cited at least h times within the past five years (Google, n.d.). It should be noted that the last journal on the list, RUSC Universities and Knowledge Society, is known presently as International Journal of Educational Technology in Higher Education. The 11 journals represent 34.3% of the literature contained in the selected corpus of MOOC literature. Each of the remaining journals contributed one to ten articles to the body of literature. Over one-third of the article abstracts used in this study came from the 11 most prolific journals.

Taxonomies The employed k-means algorithm classified the articles into categories based on document similarity within the abstracts. Of the 708 articles analyzed, abstracts resulted in 17 single-level cluster divisions shown in Table 2. The clusters were examined and labeled independently by two specialists after examining the output of the data mining computer program. The seventeen clusters were labeled by the computer generation from CL0 to CL16. This labeling scheme is consistent throughout the document. The results of the clustering analysis along with the given names are shown in Table 2. The top two populated clusters within the selected MOOC research publications were student engagement and performance (138 articles 19.5%) and teaching and Table 2 Clusters labels and contribution Cluster CL0 CL1 CL2 CL3 CL4 CL5 CL6 CL7 CL8 CL9 CL10 CL11 CL12 CL13 CL14 CL15 CL16

Label Course formats Teaching and design Library Science Student engagement and performance Course administration Lifelong learning Learner perception and self-regulation Engineering and education Social causes Open education Teaching applications Markets and motivations Teacher considerations Assessments Professional development Research Community and social interaction

# of articles 32 92 12 138 23 31 30 23 12 45 42 33 29 26 24 48 68

% contribution 4.5 13.0 1.7 19.5 3.2 4.4 4.2 3.2 1.7 6.4 5.9 4.7 4.1 3.7 3.3 6.8 9.6

464

G. M. Morel and H. L. Keahey

design (92 articles, 13.0%). The third most populated cluster was community and social interaction (68 articles, 9.6%), with the fourth being Research (48 articles, 6.8%), and the fifth was Open Education (45 articles, 6.4%). Together, these five clusters represent 53.5% of the selected MOOC articles meaning, over one half of the articles were contained in these five clusters.

Time Trends of Article Clusters The results of the time trend analysis of the publication clusters are shown in Fig. 3. The trend indicates topics of increasing or decreasing research interest during the ten-year time period. Six clusters have one publication before 2013. These are CL5 (Lifelong Learning – 2009), CL12 (Teacher Considerations – 2011), CL 15 (Research – 2011), CL0 (Course Formats – 2012), and CL 9 (Open Education – 2012). Most clusters show a general increase in quantity of journal publications throughout the time period. All clusters continued to be represented in 2018 except for one, CL2, Library Science and CL8, Social Causes had a notable decline with only a

MOOC Research Clusters 250

Number of Articles

200

150

100

50

0

CL0 Course Formats CL1 Teaching and Design CL2 Library Science CL3 Student Engagement and CL16 Performance CL15 CL4 Course Administration CL14 CL5 Lifelong Learning CL13 CL6 Learner Perception and SelfCL12 Regulation CL11 CL7 Engineering and Education CL10 CL8 Social Causes CL16 CL16 CL9 CL15 CL9 Open Education CL16 CL14 CL15 CL13 CL8CL7 CL10 Teaching Application CL12 CL6 CL15 CL13CL14 CL11 CL11 Markets and Motivations CL12 CL14 CL10 CL5 CL11 CL12 Teacher Considerations CL12 CL16 CL10CL9 CL11 CL9 CL4 CL8 CL13 Assessments CL10 CL8CL7 CL7 CL15 CL6 CL6 CL14 Professional Development CL9 CL14 CL5 CL13 CL5 CL8 CL4 CL15 Research CL12 CL3 CL4 CL11 CL6CL7 CL 16 Community and Social CL10CL9 CL3 CL4CL5 CL7 CL6 CL3 CL5CL4 Interaction CL2 CL3 CL15 CL2 CL3 CL14 CL2 CL1 CL5CL9 CL2 CL10 CL3 CL1 CL1 CL1 CL5 CL16 CL15 CL1 CL2 CL1 CL0 CL0 CL0 CL0 CL0 CL9 CL0 CL0 CL12 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018

Year CL0

CL1

CL2

CL3

CL4

Fig. 3 Trend of MOOC publication clusters by year

CL5

CL6

CL7

CL8

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

465

single article published in 2018. Seven clusters experienced over a 100% increase from 2017 to 2018. In ascending order, these clusters are: • • • • • • •

Cluster 7, Engineering and Education, 4–8 articles, 100% increase Cluster 3, Student Engagement and Performance, 23–52 articles, 116.7% increase Cluster 5, Lifelong Learning, 5–11 articles, 120% increase Cluster 13, Assessments, 5–12 articles, 140% increase Cluster 16, Community and Social Interaction, 9–22 articles, 144% increase Cluster 10, Teaching Applications, 8–20 articles, 150% increase Cluster 12, Teacher Considerations, 4–13 articles, 225% increase

Two clusters, CL 5 and CL 12, appeared before 2013 and then demonstrated an increase in publications during 2017 and 2018. The articles in cluster 5 have a general focus on the extended learning MOOCs can provide, and the content of cluster 12 articles bring attention to the teachers of MOOCs.

Publication Clusters and Countries A summary of publications by the 11 most prolific countries is found in Table 3. The table contains all clusters and the leading countries by percent contribution for that country. For example, in the first row, of the articles published from Australia, 5% were in cluster 0, 26% appear in cluster 1, etc. The countries in the table represent a combined 535 articles or 76% of all articles. While all countries contributed to many different clusters, some of the countries had a substantial portion, 25% or more, of their respective articles contributing to a single cluster. Taiwan had one-half of its total articles published under Cluster 3, Student Engagement and Performance. Taiwan had another cluster which contained a substantial percentage of its articles. Cluster 6, Learner Perceptions and Self-regulation, contained 25% of Taiwan’s articles. This cluster also contained 39% of Germany’s articles. Cluster 10, Teaching and Applications, is home to 41% of articles originating in Canada. Australia and India contribute 26% and 28% of their respective articles to Cluster 1, Teaching and Design. For this discussion, dominance indicates a country whose publication total was more than the other countries’ publication percentage by a factor of 2 or more. Figure 4 shows the percent of articles contributed by each country in each cluster. The country that dominated the most clusters was the USA which was expected since this country contributed the most articles overall. Authors from the USA contributed dominantly to six clusters: CL1 (Teaching and Design) with 34%, CL2 (Library Science) with 75%, CL3(Student Engagement and Performance) with 33%, CL4 (Course Administration) with 48%, CL8 (Social Causes) with 92%, and CL16 (Social Interaction) with 27%. The other country whose published articles dominated a cluster was China. Cluster 10, Teaching Applications, boasts 57% of its articles from China.

466

G. M. Morel and H. L. Keahey

Table 3 MOOC publication clusters by country Australia Canada China Germany India Italy Netherlands Spain Taiwan UK USA Australia Canada China Germany India Italy Netherlands Spain Taiwan UK USA

CL0 5% 7% 12% 0% 7% 0% 6% 2% 0% 0% 5% CL9 7% 4% 0% 0 14% 6% 10% 8% 6% 7% 3%

CL1 26% 18% 7% 8% 29% 0% 6% 17% 12% 4% 16% CL10 0% 4% 41% 0 0% 6% 8% 3% 0% 0% 1%

CL2 2% 0% 0% 0% 0% 0% 0% 0% 0% 3% 5% CL11 7% 0% 0% 8% 14% 0% 44% 3% 0% 10% 7%

CL3 16% 18% 12% 38% 7% 20% 22% 17% 50% 17% 24% CL12 10% 4% 3% 0% 0% 7% 0% 3% 0% 9% 3%

CL4 CL5 0.% 10% 11% 7% 2% 0% 0% 23% 7% 0% 0% 27% 0% 6% 3% 5% 0% 0% 0% 3% 6% 1% CL13 2% 4% 2% 0% 0% 7% 6% 8% 0% 3% 3%

CL6 0% 4% 9% 0% 0% 0% 0% 3% 25% 10% 3% CL14 0% 0% 3% 0% 0% 7% 6% 3% 0% 9% 4%

CL7 5% 0% 0% 0% 0% 0% 0% 5% 0% 7% 2% CL15 5% 11% 5% 15% 14% 7% 0% 11% 0% 9% 4%

CL8 0% 0% 0% 0% 0% 0% 0% 0% 0% 1% 6% CL16 5% 11% 3% 8% 7% 13% 17% 11% 6% 9% 11%

Note: CL0, Course Formats; CL1, Teaching and Design; CL2 Library Science, CL3 Student Engagement and Performance; CL4, Course Administration; CL5, Lifelong Learning; CL6 Learner Perception and Self-Regulation; CL7, Engineering and Education; CL8, Social Causes; CL9, Open Education; CL10, Teaching and Applications; CL11, Markets and Motivations; CL12, Teacher Considerations; CL13, Assessments; CL14, Professional Development; CL15 Research; CL16 Community and Social Interaction

Prolific Authors Of the 708 articles reviewed, 615 unique first-authors contributed to the corpus of work, with a majority contributing to a single article. One author contributed eight articles as first author, Sunnie Lee Watson, the most of any other author on the list. Four authors contributed four articles each, 24 contributed three articles each, and the remaining 116 contributed two articles each. Authors contributing a single article to the corpus were 544, or 89% of total authors, which suggests research in MOOCs is not a primary research agenda item, at least directly, for most of the authors contributing to the corpus. In a comparison of most contributing first authors to most cited articles, authors and article citations did not correspond. Using Google Scholar for citation figures (as of May 31, 2019), the top contributing article for the 2009–2018 timeframe was Liyanagunawardena’s, 2013 publication, MOOCs:

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

467

120.0% 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% CL0 CL1 CL2 CL3 CL4 CL5 CL6 CL7 CL8 CL9 CL10 CL11 CL12 CL13 CL14 CL15 CL16 Austrialia

Canada

China

Germany

India

Italy

Netherlands

Spain

Taiwan

United Kingdom

USA

OTHER

Fig. 4 Percentage of contribution by country for each cluster

A Systematic Study of the Published Literature 2008–2012, with 1042 cites. The articles cited at least 300 times is listed in Table 4. The second most cited article was Jordan’s 2014 article, Initial Trends in Enrolment and Completion of Massive Open Online Courses, with 671 cites. The third most cited article was Kop’s 2011 article, A Pedagogy of Abundance or A Pedagogy to Support Human Beings? Participant Support on Massive Open Online Courses with 536 cites, followed closely by the 2009 origin article contributed by Antonio Fini, The Technological Dimension of a Massive Open Online Course: The Case of the CCK08 Course Tools, with 536 cites. Margaryan’s 2015 article, Instructional quality of Massive Open Online Courses (MOOCs) had 489 cites and the only other article to break the 300 mark for the number of times the article had been cited was deWaard’s 2014 article, Using mLearning and MOOCs to Understand Chaos, Emergence, and Complexity in Education, with 320 cites. In addition to naturally referencing the first MOOC research article in the field of study, the topical areas of study for these highly cited articles give insight into what the research community has been focused on: content analysis in MOOCs research, trends in enrollment and completion, participation, instructional quality, and complex educational environments.

Discussion Perhaps unsurprising or even expected, the first article published in the Scopus database in 2009 about MOOCs was an analysis of the impact of open learning on life-long learning using the first true MOOC (CCK08) as a framework (Fini, 2009). The years between 2011 and 2013 marked the years in which new areas of interest began appearing in the research. Based on articles present in the Scopus database, research interest in MOOCs has increased dramatically since 2013. In that time

468

G. M. Morel and H. L. Keahey

Table 4 Top cited articles Citation 1042

Year 2013

Country United Kingdom

671

2014

United Kingdom

547

2011

Canada

536

2009

Italy

489

2015

United Kingdom

320

2011

Canada

Title and authors MOOCs: A Systematic Study of the Published Literature 2008–2012 Authors: Liyanagunawardena T.R., Adams A.A., Williams S.A. Initial Trends in Enrolment and Completion of Massive Open Online Courses Author: Jordan, K. A Pedagogy of Abundance or a Pedagogy to Support Human Beings? Participant Support on Massive Open Online Courses Authors: Kop R., Fournier H., Mak J.S.F. The Technological Dimension of a Massive Open Online Course: The Case of the CCK08 Course Tools Author: Fini, A. Instructional Quality of Massive Open Online Courses (MOOCs) Authors: Margaryan A., Bianco M., Littlejohn A. Using mLearning and MOOCs to Understand Chaos, Emergence, and Complexity in Education Authors: deWaard I., Abajian S., Gallagher M.S., Hogue R., Keskin N., Koutropoulos A., Rodriguez O.C.

frame, representation from an international perspective has expanded from a single source to a global phenomenon contributing to a full body of research. However, the number of journals frequently publishing MOOCs-related research remains relatively small compared to the number of journals that have shown an interest in the subject matter. Countries of origin show a similar trend in that although a total of 58 countries have contributed to the body of knowledge, only 11 represent 75% of the total contributed research. The continued expansion of published research through 2017 and then practically doubling in 2018 support research in MOOCs as a valuable contribution to the body of knowledge in educational technology and online learning environments. Questions and aims in early research seem to suggest an interest in MOOCs as an emerging technology that could help inform design and technology use in online learning. Fini (2009) was specifically interested in the “learners’ views about the multi-tool environments adopted in the course and to give some suggestions for setting up multi-tool course environments” (p. 2). Likewise, both articles from 2011 use case studies to examine design principles contributing to learning environments. It should be noted both articles published in 2011 were from Canada and were among the list of most cited articles. The top-cited article for the entire corpus was published in 2013, also at the time MOOCs research increased in number and scope. This and the top cited article in 2014 both focus on a review of the MOOCs literature to date, whereas one aims to systematically review the published research (and claims to be the first to do so) and the other specifies an interest in enrollment and completion rates using data found in the public domain. The body of research began

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

469

to take shape in 2015 when all topical areas derived from the clustering analysis were represented. The publications to follow further provide the nuanced differences necessary in determining which topical areas expand and diminish. Research publications generally stabilize in number before reaching a striking growth in all areas in 2018 with student engagement and performance, teacher application, and community and social interaction receiving more attention than in years past.

Translating Research Trends into Practice Although MOOCs have begun to establish a place in educational technology research, time will continue to reveal how the research will evolve and what the implications will be on the educational landscape. It could be hypothesized that the growth in research is approaching the peak of inflated expectations in Gartner Group’s Hype Cycle (2016), a model for how a technology will evolve over time. According to Gartner (2016), in the peak of inflated expectations, “early publicity produces a number of success stories often accompanied by scores of failures” (para. 8). In this place, some will take action and others will not. The significant increase in publications from 2017 to 2018 could be further evidence that MOOCs research is stabilizing in the peak of inflated expectations stage of the Hype Cycle or that research efforts may continue to climb. For a technology to proliferate, it will have to pass the trough of disillusionment and enter the slope of enlightenment before mainstream adoption takes off in the plateau of productivity. Figure 5 shows how visibility changes as the technology matures and an estimate for where MOOCs could be in the cycle. This is a pivotal time for MOOCs because of the unknown

Fig. 5 Estimated position of MOOCs in the Hype Cycle (Source: Jeremykemp at English Wikipedia)

470

G. M. Morel and H. L. Keahey

nature of the decline following the peak of inflated expectations. However, based on the research thus far, MOOCs may still be finding the most comfortable fit in a broad educational landscape. At this point in the trajectory, four interrelated areas are offered for consideration by the practitioner: Open Education Resources, Learning Theories, Informal Learning, and the Business of MOOCs.

Open Education MOOCs contribute to the academic offerings under the umbrella of open education. The recommended definition of Open Educational Resources (OER) is “the open provision of educational resources, enabled by information and communication technologies, for consultation, use and adaptation by a community of users for non-commercial purposes” (UNESCO, 2002, p. 24). This definition of OER implies that MOOCs are closely related but may be differentiated by a social approach to learning (Ebben & Murphy, 2014; Liyanagunawardena, 2013; Liyanagunawardena et al., 2013). In 2011, the Department of Education and the Department of Defense announced The Learning Registry, a joint effort intended to support the creating and sharing of learning resources in an online community environment (U.S. Department of Education, 2011). The effort was in response to a recognition of the difficulty educators had in sharing information and resources (Learning Registry.org, n.d.). The decision of universities to open up online courses for students world-wide, in the form of MOOCs, can be viewed as another step forward in OER (Godwin-Jones, 2012). This opinion supports the notion that MOOCs are the natural next step with online learning combining technology with reaching the masses (Bali, 2014). A general educational movement toward more accessible online learning resources that operate within a shared community is a shift from the individual nature of the shared resources of the past. Research tied to open education and MOOCs first appeared in 2012 and remained an area with steady contribution through 2018 (see Cluster 9, Open Education). With the progress of the open education movement, practitioners will have access to more content that has been quality-reviewed as well as opportunities to share their own work with their learning communities. MOOCs serve as another platform in which open education resources are enriching the educational landscape and making them a viable source for learning.

Learning Theories An underlying learning theory supporting MOOCs, especially cMOOCs, is Connectivism. George Siemens first introduced Connectivism in 2005 as a response to the limitations to other learning theories such as behaviorism, cognitivism, and constructivism. The root of Siemens’ (2005) argument is “learning is a process that occurs within nebulous environments of shifting core elements – not entirely under the control of the individual” (p. 5). The important skill for the learner to possess is

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

471

the ability to navigate the importance of the information and to make the appropriate connections for continual learning (Siemens, 2005). A second theory associated with MOOCs is rhizomatic learning. In 2014, Dave Cormier offered a MOOC on the subject to “provide a space for considering rhizomatic learning” (Mackness & Bell, 2015, p. 31). The rhizome is a metaphor for teaching and learning first introduced by Deleuze and Guattari (1987) but has found a reemergence with MOOC technology through Cormier and others (Bozkurt & Keefer, 2018). Cormier (2008) used the metaphor to describe a view of knowledge where it “can only be negotiated, and the contextual, collaborative learning experience shared by constructivist and connectivist pedagogies is a social as well as a personal knowledge-creation process with mutable goals and constantly negotiated premises” (para. 3). Both theories support a shift in thinking about teaching, learning, and knowledge. The flexibility inherent to both theories promotes learning as a continual process that is less related to stated objectives and observable outcomes and more about community, negotiation, and fluidity. MOOC research has also inspired discussion about other theories relevant in the MOOC landscape. Examples include Rogers’ diffusion of innovation (DoI) theory (Annabi & Muller, 2016), e-learning concepts ecosystem (Aparicio, Bacao, & Oliveira, 2016), community of practice as they relate to large online courses (Baxter & Haycock, 2014), communication theories and Hofstede’s cultural dimensions (Bayeck & Choi, 2018), critical posthumanism (Bayne, 2015), and even a political theory of autonomist Marxism (Curinga, 2016). As with much of educational technology research, practitioners may benefit from the learning theories explored in MOOC research when defining learning goals for students, designing learning environments, and assessing the value of content and assessment measurements. However, perhaps even more enlightening was the broad spectrum of theories indirectly related to the dialogue of MOOCs research and the broadening formal and informal educational landscape.

Informal Learning A major criticism of MOOCs is the low completion rates of enrolled students. Jordan (2015) reported completion rates for 221 courses ranged “from 0.7% to 52.1%, with a median value of 12.6%” (p. 341). Jordan also found an inverse correlation between length of course and completion rates. For example, 5-week courses had higher completion rates than those scheduled for 10 weeks or more. One could interpret these findings as proof of the dismal effect that MOOCs are having as a traditional educational offering. However, this interpretation begins with a belief that completion of a course is an indicator of success or learning. Another interpretation of the figures could stem from a connectivism perspective. Rather than completion being an indicator of learning, a connectivist may posit that the learner gained the desired information, made the connections necessary, and navigated appropriately, even if it meant navigating away from the MOOC. DeBoer, Ho, Stump, and Breslow (2014) support a reconceptualization of educational variables because “conventional

472

G. M. Morel and H. L. Keahey

measures of achievement seem to be disconnected from what many users intend to achieve” (p. 23). Another perspective could be that learners use MOOC opportunities to try something they would not normally invest time or money in because commitment is optional. The opportunity for a safe place to fail or discover new things without a long-term commitment in a formal learning environment is rare and therefore could make MOOC learning appealing. Being that follow-up with learners is difficult if not impossible, measuring learning impact beyond completion is partial at best. Similar to MOOCs, Personal Learning Environments (PLE) are a form of learning (although informal) that has emerged as a result of greater technological advances and a need from learners to learn on demand. They also serve as a venue for learning outside of institutional or formal learning environments where learners are leveraging the Internet to expand the support of peers and peer networks to facilitate learning (Martindale & Dowdy, 2010). Like MOOCs, PLEs have the ability to bring together large communities of like-minded individuals interested in the same content areas. At the heart of PLEs, however, is the learner-centered, selfdirected learning path. According to Martindale and Dowdy (2010), the clearest argument for the PLE is the ability that learners have to connect to their peers and to create and share resources. Learners may be using MOOCs as a form of PLE rather than as a traditional environment based on course enrollment and completion. Practitioners may benefit from a recognition that “learning is no longer an internal, individualistic activity” (Siemens, 2005, Conclusion, para. 2) but one where access and choice provides learners with a learning environment where they may guide their own continuous learning pathways beyond any one teacher’s classroom. Additionally, creative approaches to teaching and learning could be to incorporate both formal and informal learning methods and practices in the classroom to help students leverage both types of learning.

The Business of MOOCs A discussion focused on MOOCs would not be complete without addressing cost and accessibility. Although MOOCs may have emerged into the educational landscape as a disruptor to the system, time suggests that MOOCs may not offer as much potential in terms of a business model. From the viewpoint of technology, the requirements for offering a course to 100 students may be negligible when increasing the offering to 10,000 students (Yoram, 2014). However, what began as a completely open environment may be moving away from truly open or free and toward varying degrees of each. One example is the emergence of the Small Private Online Courses (SPOCs), or the sharing of knowledge that values learning in a social manner without the necessity of a massive stage (Freitas & Paredes, 2018; Guo, 2017) Recognizing the cost and resources required for staff in any course development endeavor, universities may go the route of variable offerings with variable rates to recoup some of the cost of business (Yoram, 2014). Other sources of income may

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

473

include governmental funding, payment by MOOC participants for completion certificates or other institutional services, in-course advertising, or the sale of participant data to third-party entities (Dellarocas & Van Alstyne, 2013). Less direct may be pathways MOOCs create in recruiting potential students to enroll in traditional degree programs once they have experienced the university offering or earned transferable credit. However, leading universities, typically also highly selective, have been those with the resources and funding to offer MOOCs. For some faculty and administrators, remaining a completely free offering without a sustainable way to address the cost associated with MOOC production and delivery creates issues with practices that are at odds with traditional beliefs about higher education and the purpose it serves (Abeles, 2014). In the digital age of learning, web accessibility has become a key component in quality online learning. The appeal of MOOCs to offer learning opportunities to broader populations outside of previously traditional environments includes older learners and those with disabilities. However, accessibility compliance requires work and expertise and may not be a priority success criterion for some MOOC providers (Sanchez-Gordon & Luján-Mora, 2013). The other side of the accessibility debate is related to discussions about the reality of the digital divide, another component in whether MOOCs are truly open or not. Researchers are finding that rather than closing the digital divide MOOCs may be another source for perpetuating it (Lee, Hong, & Hwang, 2018; Rohs & Ganz, 2015).

Conclusion Since the first MOOC was offered in 2008, the MOOC landscape has changed considerably. Research on MOOCs has provided practitioners multiple perspectives and lenses to view education. The philosophical debates and reimagining of access to education, credit for education, and the ways of and reasons for learning can be partially credited to the emergence of MOOCs in the educational landscape. From the most practical standpoint, the ability to reach large populations of research participants due to the large enrollments found in MOOC courses makes them an attractive setting for educational researchers. Perhaps, the disruption often referred to when discussing MOOCs may be more about pushing on long standing beliefs in a variety of areas related to education and less about taking over current higher education business models. Being that research efforts were represented by worldwide contribution, researchers, educators, and administrators globally may have more in common. The automated approach to analysis using text mining techniques on article abstracts offers an impartial examination of the body of research. The resultant clustering analysis showed that the topical offerings of MOOCs were varied in both topic and sustained interest. Although the Hype Cycle is a helpful tool in imagining the trajectory that MOOCs evolution is on, it is not meant to imply a strict adherence to the pathway. As those who have come before us know, simply basing future projections from recent experiences is a limiting way to imagine the potential

474

G. M. Morel and H. L. Keahey

for this educational technology or any other (Papert, 1980). The four focus areas presented for consideration were intended to highlight key areas in current MOOC research that may be the most relevant for practitioners in the field. The ways in which MOOCs can be used in a variety of settings for a variety of audiences are currently being explored at all levels of education as well as in private sectors for employee training. With readily available access to information, learners can self-direct their learning pathways, make connections with other like-minded peers, and contribute and receive content for their own use. MOOCs are an avenue to participate in this movement and serve as a readily available resource for anyone willing and able. Explorations with integrating the formal and informal aspects of teaching and learning will be an ongoing challenge for traditional learning environments as MOOCs continue to find their place along the technology evolution cycle.

Limitations The social science journals in the Scopus database maintain stringent review standards. These articles generally are regarded as having a high impact factor. Given the high standard of the journals, they may not reflect the most recent trends because the publication time in top journals can take two years. Also, since the popularity of MOOCs is a moderately recent phenomenon, much of the research could be included in conference proceedings and educational journals that are not part of this effort. It should also be recognized that although the term MOOC seems to yield an international representation of publications, this term may not fully encompass all research about open, online, large enrollment offerings if those offering the courses or researching them do not use the term Massive Open Online Course. While the k-means is a respected cluster generating algorithm, the choice for the number of clusters, the k value, is a researcher-dependent decision. For this research, the k value for the k-means statistic was derived based on calculated comparisons. There is not one single formula to obtain the perfect number and a deeper analysis could yield a different and potentially more suitable k-value.

Further Research Suggestions Future research should focus on similar studies that broaden to other databases to determine similarities and differences in findings. A future attempt at this study will incorporate expanded search terms for the originating list of articles. Although the authors have attempted to define trends in research questions from early MOOCs publications, an elaboration of the research questions over time will be a more valuable contribution to the current body of knowledge. With more time, future studies could help illuminate movement as it aligns with Gartner Group’s Hype Curve. Other research analysis approaches could be utilized, such as a different categorical algorithm, perhaps a hierarchical one, which could produce clusters and a corresponding hierarchy to the research trends.

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

475

References Abeles, T. P. (2014). The university - the shifting past. On The Horizon, 22(2), 101–110. Anderson, T., & McGreal, R. (2012). Disruptive pedagogies and technologies in universities. Education, Technology and Society, 15(4), 380–389. Retrieved from http://www.ifets.info/ journals/15_4/32.pdf Annabi, C. A., & Muller, M. (2016). Learning from the adoption of MOOCs in two international branch campuses in the UAE. Journal of Studies in International Education, 20(3), 260–281. Aparicio, M., Bacao, F., & Oliveira, T. (2016). An e-learning theoretical framework. Educational Technology & Society, 19(1), 292–307. Arthur, D., & Vassilvitskii, S. (2007). k-means++: The advantages of careful seeding. In Eighteenth annual ACM-SIAM symposium on Discrete algorithms (pp. 1027–1035). Society for Industrial and Applied Mathematics. Bali, M. (2014). MOOC Pedagogy: Gleaning good practice from existing MOOCs. Journal of Online Learning and Teaching, 19(1), 44–56. Baxter, J. A., & Haycock, J. (2014). Roles and student identities in online large course forums: Implications for practice. International Review of Research in Open and Distance Learning, 15(1), 20–40. Bayeck, R. Y., & Choi, J. (2018). The influence of national culture on educational videos: The Case of MOOCs. International Review of Research in Open and Distance Learning, 19(1), 186–201. Bayne, S. (2015). Teacherbot: Interventions in automated teaching. Teaching in Higher Education, 20(4), 455–467. Berkhin, P. (2006). A survey of clustering data mining techniques. In Grouping multidimensional data (pp. 25–71). Berlin, Germany/Heidelberg, Germany: Springer. https://doi.org/10.1007/3540-28349-8_2 Bozkurt, A., & Keefer, J. (2018). Participatory learning culture and community formation in connectivist MOOCs. Interactive Learning Environments, 26(6), 776–788. Breslow, L. P., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013). Studying and learning in the worldwide classroom research into edX’s first MOOC. Research & Practice in Assessment, 8(1), 13–25. Chen, C.-C., Lee, C.-H., & Hsiao, K.-L. (2018). Comparing the determinants of non-MOOC and MOOC continuance intention in Taiwan: Effects of interactivity and openness. Library Hi Tech, 36(4), 705–719. Cormier, D. (2008). Rhizomatic education: Community as curriculum. Journal of Online Education, 4(5). http://nsuworks.nova.edu/cgi/viewcontent.cgi?article¼1045&context¼innovate Costello, E., Brown, M., Mhichíl, M. N., & Zhang, J. (2018). Big course small talk: twitter and MOOCs – A systematic review of research designs 2011–2017. International Journal of Educational Technology in Higher Education, 15(1), 1. Curinga, M. (2016). The MOOC and the multitude. Educational Theory, 66(3), 369–387. Daniel, J. (2012). Making sense of MOOCs: Musings in a maze of myth, paradox and possibility. Journal of Interactive Media in Education, 3, 18. https://doi.org/10.5334/2012-18 DeBoer, J., Ho, A., Stump, G., & Breslow, L. (2014). Changing “course”: Reconceptualizing educational variables for Massive Open Online Courses. Educational Researcher, 43(2), 74–84. Deleuze, G., & Guattari, R. (1987). A thousand plateaus. University of Minnesota Press. Dellarocas, C., & Van Alstyne, M. (2013). Money models for MOOCs. Communications of the ACM, 56, 25–28. deWaard, I., Abajain, S., Gallagher, M. S., Hogue, R., Keskin, N., Koutropoulos, A., & Rodriguez, O. C. (2015). Using mlearning and MOOCs to understand chaos, emergence, and complexity in education. International Review of Research in Open & Distance Learning, 12(7), 94–115. Dodson, M. N., Kitburi, K., & Berge, Z. L. (2015). Possibilities for MOOCs in corporate training and development. Performance Improvement, 54, 14–21. https://doi.org/10.1002/pfi.21532

476

G. M. Morel and H. L. Keahey

Downes, S. (2008). Places to go: Connectivism & connective knowledge. Innovate: Journal of Online Education, 5(1). Retrieved from http://nsuworks.nova.edu/cgi/viewcontent.cgi? article¼1037&context¼innovate Ebben, M., & Murphy, J. S. (2014). Unpacking MOOC scholarly discourse: A review of nascent MOOC scholarship. Learning, Media and Technology, 39(3), 328. https://doi.org/10.1080/ 17439884.2013.878352 Elsevier. (2019). Retrieved from Scopus: https://www.elsevier.com/solutions/scopus Feldman, R., & Dagan, I. (1995). Knowledge discovery in textual databases (KDT). In Proceedings of the first international conference on knowledge discovery and data mining (KDD-95) (pp. 112–117). Fini, A. (2009). The technological dimension of a massive open online course: The case of the CCK08 course tools. The International Review of Research in Open and Distributed Learning, 10(15). https://doi.org/10.19173/irrodl.v10i5.643 Freitas, A., & Paredes, J. (2018). Understanding the faculty perspectives influencing their innovative practices in MOOCs/SPOCs: A case study. International Journal of Educational Technology in Higher Education, 15(1), 1. Gartner. (2016). Gartner Hype Cycle. Retrieved April 10, 2016, from Gartner: http://www.gartner. com/technology/research/methodologies/hype-cycle.jsp Gaševic, D., Kovanovic, V., Joksimovic, S., & Siemens, G. (2014). Where is research on massive open online courses headed? A data analysis of the MOOC research initiative. International Review of Research in Open and Distance Learning, 15, 134–176. https://doi.org/10.19173/ irrodl.v15i5.1954 Godwin-Jones, R. (2012). Emerging technologies: Challenging hegemonies in online learning. Language, Learning & Technology, 16(2), 4–13. Google. (n.d.). Google Scholar. Retrieved May 24, 2019, from Google Scholar Metrics: https:// scholar.google.com/intl/en/scholar/metrics.html#metrics Gul, S., Mahaja, I., Shafiq, H., Shafi, M., & Shah, T. (2018). Massive open online courses: Hype and hope. Journal of Library and Information Technology, 38(1), 63–66. Guo, P. (2017). MOOC and SPOC, which one is better? Eurasia Journal of Mathematics Science and Technology Education, 13(8), 5961–5967. Haber, J. (2014). MOOCs. Cambridge, MA: The MIT Press. Harting, K., & Erthal, M. (2005). History of distance education. Information Technology, Learning, and Performance Journal, 23(1), 35–44. Hew, K. F., & Cheung, W. S. (2014). Review: Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 43(1), 5–16. https://doi.org/10.1016/j.edurev.2014.05.001 Hewlett Packard. (2015). HP 2015 sustainability report. Palo Alto, CA: Hewlett Packard. Hill, P. (2014, August 13). MOOC mania: Stanford AI course creates media sensation two years ago. Retrieved from e-Literate: http://mfeldstein.com/mooc-mania-stanford-ai-course-createsmedia-sensation-two-years-ago/ Holmberg. (1995). Theory and practice of distance education. London, England: Routledge. Hung, J. (2012). Trends of e-learning research from 2000 to 2008: Use of text mining and bibliometrics. British Journal of Educational Technology, 43(1), 5–16. https://doi.org/10. 1111/j.1467-8535.2010.01144.x Hung, J. L., & Zhang, K. (2012). Examining mobile learning trends 2003–2008: A categorical meta-trend analysis using text mining techniques. Journal of Computing in Higher Education, 24(1), 1–17. https://doi.org/10.1007/s12528-011-9044-9 Jafari, E., Vajarah, K. F., Arefi, M., & Rezaeizadeh, M. (2018). MOOC-based curriculum model validation in higher education in Iran. Turkish Online Journal of Distance Education, 19(3), 112–127. Johnston, T. C. (2015). Lessons from MOOCs: Video lectures and peer assessment. Academy of Educational Leadership Journal, 19(2), 91–97.

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

477

Jordan, K. (2015). Massive open online course completion rates revisited: Assessment, length, and attrition. The International Review of Research in Open and Distance Learning, 16(3), 341–358. Keegan. (1990). Foundations of distance education. New York, NY: Routledge. Keshaval, G. A., & Gowda, M. P. (2008). ACM transaction on information systems (1989–2006): A bibliometric study. Information Studies, 14(4), 223–234. Learning Registry.org. (n.d.). Retrieved from learningregistry.org: http://www.learningregistry.org Lee, J., Hong, A., & Hwang, J. (2018). A review of massive open online courses: MOOC’s approach to bridge the digital divide. In 22nd Biennial conference of the international telecommunications society. Seoul, South Korea. Lin, L., & Cranton, P. (2015). Informal and self-directed learning in the age of Massive Open Online Courses (MOOCs). In O. Mejiuni, P. Cranton, & O. Taiwo (Eds.), Measuring and analyzing informal learning in the digital age (pp. 91–104). Hershey, PA: IGI Global. Liyanagunawardena, T. (2013). MOOC experience: A participant’s reflection. SIGCAS Computers and Society, 44(1), 9–14. Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008–2012. International Review of Research in Open and Distance Learning, 14, 202–227. Lowenthal, P., & Hodges, C. (2015). In search of quality: Using quality matters to analyze the quality of massive, open, online courses (MOOCs). International Review of Research in Open and Distance Learning, 16(5), 83–101. MacKenzie, O., & Christensen, E. L. (1971). The changing world of correspondence study: International readings. University Park, PA: Pennsylvania State University Press. Mackness, J., & Bell, B. (2015). Rhizo 14: A rhizomatic learning cOOC in sunlight and shade. Open Praxis, 7(1), 25–38. Magaña-Valladares, L., Rosas-Magallanes, C., Montoya-Rodríguez, A., Calvillo-Jacobo, G., Alpuche-Arande, C. M., & García-Saisó, S. (2018). A MOOC as an immediate strategy to train health personnel in the cholera outbreak in Mexico. BMC Medical Education, 18(1), 1–7. Mangan, K. (2012, October 1). MOOC mania [Special report: Online learning]. Retrieved from The Chronicle of Higher Education: http://chronicle.com/article/Massive-Excitement-About/ 134678/ Martindale, T., & Dowdy, M. (2010). Personal learning environments. In G. Veletsianos (Ed.), Emerging technologies in distance education (pp. 177–193). Edmonton, AB: Athabasca University Press. McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for digital practice. SSHRS Knowledge Synthesis Grant on the Digital Economy. Retrieved from http:// www.edukwest.com/wp-content/uploads/2011/07/MOOC_Final.pdf Nayak, T., Prasad, S., & Senapiti, M. (2015). A survey on web text information retrieval in text mining. Research Journal of Applied Sciences, Engineering and Technology, 10(10), 1164–1174. https://doi.org/10.19026/rjaset.10.1884 Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. Brighton, UK: Harvester. Pappano, L. (2012, November 2). The year of the MOOC. The New York Times. Retrieved from http://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-are-multi plying-at-a-rapid-pace.html?_r¼0 Paton, R., Fluck, A. E., & Scanlan, J. (2018). Engagement and retention in VET MOOCs and online courses: A systematic review of literature from 2013 to 2017. Computers & Education, 125, 191–201. Pence, H. E. (2012). When will college truly leave the building: If MOOCs are the answer, what is the question? Journal of Educational Technology Systems, 41(1), 25–33. Radford, A. W., Coningham, B., & Horn, L. (2015). MOOCs: Not just for college students-How organizations can use MOOCs for professional development. Employment Relations Today, 41(4), 1–15. https://doi.org/10.1002/ert.21469 Rapid Miner. (2017). Rapid Miner. Retrieved from https://rapidminer.com/products/studio/

478

G. M. Morel and H. L. Keahey

Rodriguez, O. (2012). MOOCs and the AI-Stanford like courses: Two successful and distinct course formats for Massive Open Online Courses. European Journal of Open, Distance and E-learning (1). Retrieved from http://www.eurodl.org/?article¼516 Rodriguez, O. (2013). The concept of openness behind c and x-MOOCs. Open Praxis, 5(1), 67–73. Rohs, M., & Ganz, M. (2015). MOOCs and the claim of education for all: A disillusion by empirical data. International Review of Research in Open & Distance Learning, 16(6), 1–18. Sanchez-Gordon, S., & Luján-Mora, S. (2013). Web accessibility of MOOCs for elderly students. In International conference on information technology based higher education and training. Antalya, Turkey: IEEE. Sangrà, A., & González-Sanmamed, M. (2015). Meta-analysis of the research about MOOCs during 2013–2014. Educación XX1, 18(2), 1–28. Siemens, G. (2005). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2(1). Retrieved from http://www.itdl.org/ Journal/Jan_05/article01.htm Skiba, D. J. (2012). Disruption in higher education: Massively open online courses (MOOCs). Nursing Education Perspectives, 33(6). http://libproxy.library.unt.edu:2143/ps/i.do?id¼GALE %7CA313344873&sid¼summon&v¼2.1&u¼txshracd2679&it¼r&p¼HRCA&sw¼w& asid¼dec1c9928f6dca2ca0e700a1437549e7 Soyemi, O., Ojo, A., & Abolarin, M. (2018). Digital literacy skills and MOOC participation among lecturers in a private University in Nigeria. Library Philosophy and Practice. Thelwall, M. (2008). Bibliometrics to webometrics. Journal of Information Science, 34(4), 605–621. U.S. Department of Education. (2011, November 8). Departments of Education and Defense to Launch “Learning Registry” Tools and Community [Press Release]. Retrieved from http:// www.ed.gov/news/press-releases/departments-education-and-defense-launch-learning-registrytools-and-community UNESCO. (2002). Forum on the impact of open courseware for higher education in developing countries: Final Report. Paris, France: UNESCO. Retrieved from http://unesdoc.unesco.org/ images/0012/001285/128515e.pdf van de Oudeweetering, K., & Agirdag, O. (2018). MOOCS as accelerators of social mobility? A systematic review. Educational Technology & Society, 21(1), 1–11. Veletsianos, G., Collier, A., & Schneider, E. (2015). Digging deeper into learners’ experiences in MOOCs: Participation in social networks outside of MOOCs, notetaking and contexts surrounding content consumption. British Journal of Educational Technology, 46(3), 570–587. Veletsianos, G., & Shepherdson, P. (2015). Who studies MOOCs? Interdisciplinary in MOOC research and its changes over time. International Review of Research in Open & Distance Learning, 16(3), 1–17. Yoram, M. K. (2014). A race to the bottom: MOOCs and higher education business models. Open Learning, 29(1), 5–14.

Gwendolyn M. Morel is a Director in the Division of Digital Learning at the Texas Higher Education Coordinating Board. Previously, she served as the Director of the Office of Distance and Extended Learning at Texas State University. She took an early interest in educational technology while working for Webex Communications in the late 1990s, when the concept of being able to “gather together” anytime, anywhere through virtual meetings was first starting to emerge. Since then, Gwen has developed a curriculum and delivered training to professionals around the world in diverse environments ranging from aviation, to software development, and higher education. Gwen serves as the Assistant Editor for the Development section of the Educational Technology Research and Development journal. Gwen received her Bachelor of Science in Organizational Behavior, a master’s degree in Learning Technologies, and a Ph.D. in Learning Technologies from the University of North Texas.

19

Insight into MOOCs Research: A Meta-trend Analysis of. . .

479

Heather L. Keahey received her Ph.D. from the University of North Texas in Learning Technologies. She is currently a part of the Adjunct Faculty at Liberty University in the School of Education teaching doctoral students online. She is on the Mathematics faculty at Wharton County Junior College. She spent several years teaching Mathematics at Miller Career and Technology Center in Katy, Texas, serving as Department Chair. While with Katy ISD, she received recognition for the integration and implementation of technology in the classroom including the Digital Star Teacher of the Year for the district. She worked on the development team and taught the first online course offered by the district. She started her career as a systems engineer with Compaq Computers. Heather holds a Master of Science in Mathematics from the University of West Florida, a Bachelor of Science in Computer Science with a Minor in Mechanical Engineering, and a Master of Science in Industrial Engineering with a Minor in Computer Science from Texas A&M University.

Design-Centric Research-Practice Partnerships: Three Key Lenses for Building Productive Bridges Between Theory and Practice

20

Yael Kali, Bat-Sheva Eylon, Susan McKenney, and Adi Kidron

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Research-Practice Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . RPPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . About This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Key Lenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Existing Insights About DC-RPPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Scholarship of Teaching and Practitioner Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Change Laboratory Formative Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multilevel Boundary Crossing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design Principles for Productive DC-RPPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MDP1: Define and Tailor Key Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MDP2: Facilitate Engagement Through Dynamic Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MDP3: Cultivate Productive Habits-of-Mind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Employing DC-RPP Design Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Case 1: Fostering Interdisciplinarity in Middle Schools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Case 2: Innovating Physics Teaching and Learning Workshop . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

482 482 483 483 484 484 486 486 487 488 488 491 494 495 496 501 506 507 508

Y. Kali (*) · A. Kidron University of Haifa, Haifa, Israel e-mail: [email protected]; [email protected] B.-S. Eylon The Science Teaching Department, The Weizmann Institute of Science, Rehovot, Israel e-mail: [email protected] S. McKenney ELAN, Department of Teacher Professional Development, Faculty of Behavioral, Management and Social Sciences, University of Twente, Enschede, The Netherlands e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_122

481

482

Y. Kali et al.

Abstract

The last decade has witnessed a strong increase in research that moves toward mutually beneficial collaboration between researchers and practitioners. This chapter focuses on such collaborations that aim to design resources for use in schools while also advancing theoretical understanding of the dynamics within such partnership. We refer to such endeavors as design-centric research-practice partnerships (DC-RPPs). To guide the development of productive DC-RPPs, we synthesize insights from three theoretical lenses: (1) scholarship of teaching and practitioner research, (2) change laboratory formative interventions, and (3) multilevel boundary crossing. These lenses, together with a framework that characterizes DC-RPPs based on the practical constructs of (1) processes, (2) roles, and (3) habits-of-mind, are used in a 3  3 theory-practice matrix to elicit and articulate nine design principles that can support productive DC-RPPs. We describe two cases that illustrate how the design principles come to life in authentic DC-RPPs (one with 3 middle schools, focusing on interdisciplinary learning, and the other with 22 high schools, focusing on physics) and conclude with a discussion of emerging work that could support DC-RPPs and recommendations for future research. Keywords

Research-practice partnerships (RPPs) · Design-based research (DBR) · Design principles · Scholarship of teaching · Change laboratories · Boundary crossing

Introduction Research-Practice Interactions Educational research is conducted to inform scientific understanding but also for the purpose of understanding and improving educational practice. With this dual goal in mind, investigations have been undertaken to explore how educational professionals access, value, and use research (Broekkamp & van Hout-Wolters, 2007; de Vries & Pieters, 2007; Vanderlinde & van Braak, 2010). Similarly, studies have been conducted to describe modes through which knowledge is generated and shared in the field of education (Bauer & Fischer, 2007; Lavis, Robertson, Woodside, McLeod, & Abelson, 2003; Nutley, Walter, & Davies, 2007). While most educational research today can still be characterized by a hierarchical relationship in which practitioners are the object of study and researchers translate their findings for them, the last decade has witnessed a strong increase in research that moves beyond data extraction agreements (Wagner, 1997) toward mutually beneficial collaboration between researchers and practitioners (van Braak & Vanderlinde, 2012). Policymakers, funders, and scholars are pioneering ways for researchers and practitioners to work together, and research-practice partnerships (RPPs) are particularly promising in this regard (Coburn & Penuel, 2016).

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

483

RPPs RPPs are long-term collaborations between practitioners and researchers that are organized to investigate problems of practice and solutions for improving schools and school districts (Coburn, Penuel, & Geil, 2013). The partnerships are not established solely for short-term projects but rather cultivate long-term relationships that can provide the infrastructure for improvement (Penuel, 2015). Sustainable educational partnerships feature mutual benefit and dependencies (Dede, Rockman, & Knox, 2007), as well as continued use of social resources (contact among people), conceptual resources (ideas and processes), and physical resources (materials, tools). By focusing on real-time local challenges, RPPs can lead to research that is informative, timely, and relevant to local stakeholders (Henrick, Munoz, & Cobb, 2016). The RPP interactions themselves can benefit practitioners by increasing awareness of important advances in relevant scholarship or by creating opportunities to develop and apply new knowledge (Coburn & Penuel, 2016). Benefits to researchers are also present, since serving practice is a fundamental goal and collaboration enables fulfilling this goal especially because (1) studies situated in practice bring with them the ecological validity that renders the derived knowledge more useful and (2) regular interactions with practitioners help keep researchers aware of, and sensitized to, the realities and concerns of those working in classrooms (McKenney & Pareja-Roblin, 2018). In addition, RPPs can lead to innovations in both practice and research, including transformative learning for both design researchers and practitioners (Kali, 2016). Educational RPPs can involve various types of practitioners (e.g., teachers, administrators, parents), work at small or large scales (e.g., one school, a citywide school district), and tackle myriad kinds of problems (e.g., teacher turnover, learning outcomes, bullying). They may aim to (1) generate research findings that can inform practice or policy, (2) increase schools’ capacities to engage in a sustained, disciplined effort at improvement, or (3) design resources for use in schools. Such aims are sought while also advancing theoretical understanding (Coburn et al., 2013). The remainder of this chapter focuses on RPPs that seek to address all these aims, referred to here as design-centric research-practice partnerships (DC-RPPs). In particular, given the focus of this handbook, it focuses on researcher and practitioner learning during DC-RPP engagement, with emphasis on examples involving learning technologies.

About This Chapter The overarching goal of this chapter is to abstract design knowledge from existing theoretical lenses, relevant to DC-RPPs, and to synthesize this knowledge in a way that can help increase current understanding on research-practice synergy. Thus, by offering considerations to guide DC-RPPs, this chapter aims to contribute to the development of productive bridges between theory and practice. Following this introduction, the “Key Lenses” section starts with a literature review on DC-RPPs,

484

Y. Kali et al.

much of which stems from design-based (implementation) research. In so doing, we examine the processes, roles, and habits-of-mind that enable DC-RPPs. The remainder of the “Key Lenses” section synthesizes insights from three lenses that can enrich and guide (1) learning to inform DC-RPPs, (2) learning through DC-RPPs, and (3) learning from DC-RPPs. These refer, respectively, to three theoretical lenses: (1) scholarship of teaching and practitioner research, (2) change laboratory, and (3) boundary crossing. In the following section – “Design Principles for Productive DC-RPPs” – we use these lenses to elicit design principles that can support the processes, roles, and habitsof-mind of productive DC-RPPs. Then, the “Employing DC-RPP Design Principles” section describes two cases (one with 3 middle schools, focusing on interdisciplinary learning, and the other with 22 high schools, focusing on physics) that illustrate how the design principles come to life in authentic DC-RPPs. The chapter concludes with a discussion of the dynamic nature of DC-RPPs, as portrayed in different trajectories taken by participants in the two cases, and recommends directions for future research.

Key Lenses Existing Insights About DC-RPPs Much of what we know about DC-RPPs stems from a family of approaches that connect basic and applied educational research in, on, or through design. These include design experiments (e.g., Collins, 1992), design-based research (e.g., DBRC, 2003), design-based implementation research (e.g., Penuel, Fishman, Cheng, & Sabelli, 2011), and educational design research (e.g., McKenney & Reeves, 2012). It is important to note that these terms are not interchangeable and that design-centric research in the field of education does not always involve the kinds of partnerships described above. Indeed, some experts have gone to lengths to describe key differences (Reinking & Bradley, 2008; Penuel et al., 2011). Here, we focus on insights relating to long-term research-practice partnerships that share the dual aim of deriving new knowledge through the design of solutions to problems in educational practice, as well as their implementation. We relate these insights to three categories of challenges in DC-RPPs described by McKenney (2016) as knowledge of design processes, ability to envision and take on new roles, and developing the habits-of-mind that can serve the enterprise, as follows. First, while educators and researchers are typically familiar with the concept of design, many lack experience or formal training in the process of designing and implementing educational innovation. The overall DC-RPP process entails three core phases: analysis, design, and evaluation. During analysis, discussions are held to shape a better understanding of the educational problem to be addressed, the target context, and the stakeholder needs. Often, this phase also includes an open-ended exploration of where and how similar problems have been tackled elsewhere. Throughout design, potential solutions to the problem are generated, explored, considered, and then mapped and constructed. During this process, the core ideas

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

485

underpinning the design – including their theoretical and practical grounding – are articulated. This enables underlying assumptions and the design framework to be shared and critiqued by the various stakeholders. In evaluation, depending on the phase of development, elements of a partial or full design are tested for soundness, feasibility, local viability, broader institutionalization, immediate effectiveness, or long-term impact. Each of these phases is undertaken not in isolation but rather with explicit consideration of concerns relevant to the implementation of the designed solution. Because each of these phases can be shaped differently depending on the aims and context, DC-RPPs are recommended to Define and tailor key processes. We refer to this recommendation as the first of three meta design principles (MDPs) that we suggest for guiding productive DC-RPPs (MDP1). Second, as may be gleaned from the descriptions above, the tasks undertaken in each core process require DC-RPP participants to take on more than the traditional roles and often share responsibilities of consultant/facilitator, designer, and researcher. The consultant/facilitator role encompasses sharing expertise to help problem-solving, strategy, and planning (mostly consultant), as well as supporting others in the team to achieve an outcome by providing structure, guidance, or supervision (facilitator). In DC-RPPs, this latter role is especially required during analysis (helping people to expose their problems and knowledge thereof). But it is also present in design (sharing expertise, managing resources), evaluation (helping to understand what is happening/troubleshooting), and implementation, especially if the facilitator serves as “program champion” who helps others become and remain in touch with their reason for being involved (often tied to sense of moral purpose). The designer role includes developing and realizing a plan for the appearance, form, or workings of something that does not yet exist – most likely programs, processes, products, or policies. Naturally, this role is heavily present during the design phase (influencing the design process as well as the designed products), but it is also important in other phases, as foundational knowledge for design continues to develop. The role of the researcher is fulfilled by anyone conducting systematic investigation to develop new knowledge (facts, principles, theories, etc.). This role is most clearly present during the phases of empirical investigation, analysis and evaluation, but researcher expertise also serves design, e.g., by providing researchbased insights. Though preferences may exist, each of these roles can be taken on by either researchers or practitioners, and they may shift over time. Clear understanding and agreements about roles are crucial to fulfilling them well. As a result, DC-RPPs are recommended to Facilitate engagement through dynamic roles. We refer to this recommendation as MDP2 for productive DC-RPPs. Third, since DC-RPPs intend to develop the capacity for sustaining change, the inculcation of habits-of-mind that serve the process is required, most notably trust, empathy, and flexibility. Creating capacity for change requires the ability to work across contexts which relies on trust, which is served by norms of interaction, and shared commitments (Donovan, Snow, & Daro, 2014). In productive educational partnerships, trust is developed by engagement that is deep, direct, and frequent (Penuel, Bell, Bevan, Buffington, & Falk, 2016). Empathy is needed for exploring and attending to the needs, wishes, and concerns of stakeholders; creating designs

486

Y. Kali et al.

that are usable, practical, and congruent with stakeholder concerns; helping understand and interpret data; and taking into account (un)shared goals or the incentives, motives, and reward structures in different settings. Finally, flexibility is needed for (1) balancing well-framed investigation with open-mindedness, (2) staying focused on design goals while utilizing unplanned opportunities (Kali, 2016), and (3) drawing conclusions and deriving new questions. Flexibility also serves the orchestration required to optimize the use of the human and material resources available in ways that remain aligned with overall project goals. Therefore, DC-RPP participants are advised to Cultivate productive habits-of-mind (MDP3). The three MDPs introduced above are used in this chapter to elicit pragmatic design principles (PDPs) from each of the three theoretical lenses. We therefore begin by introducing these lenses.

The Scholarship of Teaching and Practitioner Research The scholarship of teaching (Hutchings & Shulman, 1999; Shulman, 2011; Trigwell, Martin, Benjamin, & Prosser, 2000) and practitioner research (Cochran-Smith & Lytle, 2001, 2009) are two movements that represent a rich field of research characterizing the nature of teaching as a profession and ways that scholarship develops within communities of teachers. Although scholarship of teaching usually refers to higher education, whereas practitioner research typically refers to K-12 settings, there are many similarities in the ways these movements view professional teaching. This includes three main principles: (1) commitment to systematically exploring own teaching, as reflected in students’ learning, or what Cochran-Smith and Lytle (2001) refer to as “generating local knowledge of practice”; (2) sharing this knowledge within a community of teachers and thus making it “community property” (Shulman, 1998, 2011) that can be critiqued, negotiated, and improved; and (3) a community effort to “go meta” (Shulman, 1998, 2011) and develop conceptual frameworks for understanding practice (Cochran-Smith & Lytle, 2001). Education researchers, instructional designers, content specialists, and learning scientists often take part, as partners, within such communities (see, e.g., conferences organized by the International Society for the Scholarship of Teaching and Learning, ISSOTL, 2017). In DC-RPPs, the systematic explorations and collective learning take place as an integral part of the collaborative design process. The understandings derived from these explorations regarding local conditions for implementation and spread, as well as regarding the stakeholders (learners, teachers, colleagues), are prerequisites for, and can inform, design. We thus refer to the scholarship of teaching and practitioner research as a lens that can enrich our conceptualization of DC-RPPs in situations in which learning can inform design.

Change Laboratory Formative Interventions Change laboratory is a method of intervention and a theoretical framework that has been developed by researchers at the Center for Activity Theory and Developmental

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

487

Work Research (Engeström, 2007; Engeström, Virkkunen, Helle, Pihlaja, & Poikela, 1996; Sannino, Engeström, & Lemos, 2016; Virkkunen, 2013). It is based on collaboration between researchers and practitioners who work together to identify and design solutions to disturbances and bottlenecks in the practitioners’ prevailing work practices. Inspired by Vygotsky’s work, the change laboratory method includes a set of instruments and conceptual tools, which the researcher-facilitator uses to facilitate the group’s collective analysis of their work activity system and sometimes for its (re) conceptualization. The intervention is referred to as “formative” due to the developmental nature of the change process in which the group gradually designs tentative solutions and experiments with them. These eventually become generative solutions that can lead to practical systemic transformation within the community, as well as to the development of theory. Change laboratory researchers refer to this whole process (collective design, participatory analyses, and implementation phases) as “expansive learning.” For this reason, we view change laboratory formative interventions as lenses that can help conceptualize learning through design in DC-RPPs.

Multilevel Boundary Crossing Teacher-researcher partnerships are sometimes conceptualized in relation to the boundary crossing literature (as in Penuel, Allen, Coburn, & Farrell, 2015). The concept of boundary crossing refers to ways people from two or more different communities of practice learn to productively work with each other. In DC-RPPs, these are the researchers’ and the practitioners’ communities of practice. The boundaries that participants need to cross are therefore sociocultural differences between the groups. Following a comprehensive review of the literature, Akkerman and Bakker (2011) defined four learning mechanisms (or boundary crossing processes) that take place at boundaries between different communities of practice: identification, coordination, reflection, and transformation. Although these processes may progress in different order (Akkerman & Bruining, 2016), reflection and eventually transformation usually stem from earlier identification and/or coordination processes. The most dramatic process of transformation in boundary crossing denotes a process in which people change their existing practices or develop new practices at the boundary between the different communities of practice. In a DC-RPP, this may occur when a researcher, as a result of the partnership with practitioners, develops new ways to make sense of processes that take place in the learning environment being explored or when a teacher, as a result of the partnership with researchers, begins to reflect in a more systematic manner on how teaching affects student learning. The co-designed learning environment in DC-RPPs fulfills an important bridging function, referred to as “boundary object” in the boundary crossing literature (coined by Star, 1989 and Star & Greisemer, 1989). The boundary crossing can take place at different levels: the institutional, interpersonal, and intrapersonal (Akkerman & Bruining, 2016). Taken together, the boundary crossing literature serves as an important lens for conceptualizing learning from design partners in DC-RPPs.

488

Y. Kali et al.

Design Principles for Productive DC-RPPs Design principles integrate descriptive, explanatory, and predictive understanding to guide the development of interventions (McKenney & Reeves, 2012). They serve as research-based guidelines for instructional design and can be articulated at different grain sizes (Kali, 2006, 2008). In this chapter, we distinguish principles relevant to DC-RPPs in two layers of abstraction. The more abstract layer consists of MDPs, such as the three described above, related to practical aspects of DC-RPPs: processes, roles, and habits-of-mind. The MDPs, which serve as the rows in the theorypractice matrix below (Table 1), are made more concrete with pragmatic design principles (PDPs), articulated for each of the three lenses presented (scholarship of teaching, change laboratories, and boundary crossing) in the columns of the matrix. In this way, this chapter presents PDPs to guide learning (1) as input for, (2) through, or (3) from DC-RPPs, respectively.

MDP1: Define and Tailor Key Processes PDP1: Scholarship Processes The scholarship in teaching and practitioner research movements describe the processes through which the profession of teaching advances. These include processes in which teachers are involved in systematically collecting evidence on their own practice and exploring relationships between these practices and their students’ learning. It also involves sharing the evidence and negotiating insights with peers. Cochran-Smith & Lytle (2001) describes the way teachers generate such “local knowledge of practice” within professional inquiry communities as follows: Through talk and writing, they make their tacit knowledge more visible, call into question assumptions about common practices, and generate data that make possible the consideration of alternatives. Part of the culture of inquiry communities is that rich descriptive talk and writing help make visible and accessible the day-to-day events, norms, and practices of teaching and learning and the ways different teachers, students, administrators, and families understand them. In this way, participants conjointly uncover relationships between concrete cases and more general issues and constructs. (pp. 53–54)

The process in which in situ explorations of an individual teacher become more generalized and applicable for teaching in other contexts (or what Shulman calls “going meta”) involves a commitment to making these explorations and their outcomes “community property,” so that they can become “subject to peer review and evaluation and accessible for exchange and use by members of one’s disciplinary community” (Shulman, 2011, p. 4). This also enables developing principles that cut across contexts, and “building, interrogating, elaborating, and critiquing conceptual frameworks that link action and problem-posing to the immediate context as well as to larger social, cultural, and political issues” (Cochran-Smith & Lytle, 2001). Teacher inquiry can serve DC-RPPs by fostering close connections to insights based on daily practice. Any design process involves making a myriad of design

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

489

Table 1 Design principles for productive DC-RPPs (theory-practice matrix) Meta design principles (MDPs) MDP1: Define and tailor key processes

MDP2: Facilitate engagement through specific and dynamic roles

MDP3: Cultivate productive habits-of-mind

Pragmatic design principles (PDPs) Change laboratory Scholarship lens lens PDP1: Scholarship PDP2: Change processes laboratory processes Systematically Reflect on the investigate, within a collective’s object of professional activity to identify community of historically formed teaching, questions contradictions; develop that relate own practice and experiment with and student learning generative and while committing to concrete solutions (for sharing findings, contradictions) to negotiating, and reach “theoretically refining cross-cutting mastered principles developments” PDP4: Scholarship PDP5: Change roles laboratory roles Develop the social, Consider role division organizational, and in which practitioners digital infrastructures serve as leaders of that will enable all change, designers, and DC-RPP participants experimenters, while to assume roles of researchers serve as researchers and peer provokers and reviewers who supporters of process constantly seek to advance practice, as well as the profession of teaching PDP7: Scholarship PDP8: Change habits-of-mind laboratory habits-ofView teaching as mind profession, in which Be prepared to question accepted knowledge that is developed in a wide set practices, and analyze problematic situations, of inquiries serves as a which may require a “community property” somewhat to lead social change. “revolutionary Trust may play a mindset” crucial role for such sharing

Boundary crossing lens PDP3: Boundary crossing processes Engage in activities that help individuals, groups, and institutions identify and respect the various expertise within the partnership, coordinate distributed work, and reflect at own practice

PDP6: Boundary crossing roles Attend to the role of brokers as linking between groups that differ socioculturally (e.g., researchers and practitioners)

PDP9: Boundary crossing habits-ofmind Be open and flexible to take up others’ perspectives and to transform on an interpersonal, intrapersonal, and/or institutional level

decisions. One of the benefits of DC-RPPs is that the close relationship between researchers and practitioners enables multiple cycles of in situ testing of tentative design conjectures. Building on “local knowledge of practice” within this process increases the ecological validity of the design decisions. To fulfill this potential, this PDP advises participants within DC-RPPs to adopt processes that are typical to the practitioner research and scholarship of teaching

490

Y. Kali et al.

movements, that is, to (1) develop means for systematically collecting evidence on student learning with the designed learning materials, (2) to create opportunities for DC-RPP participants to collaboratively develop and negotiate local knowledge of practice, as well as to (3) engage in sharing, critiquing, and synthesizing design knowledge with larger communities of scholars in the field, using public infrastructures such as the design principle database (Kali, 2006, 2008; Kali & Linn, 2008).

PDP2: Change Laboratory Processes The key processes in change laboratory formative interventions refer to the way collectives (e.g., a group of practitioners who seek to evoke a change within their school) initiate and engender change within their community. The change process usually begins by the group’s deep reflection on their own practices. Of special interest in this reflection is identifying the object of the group’s activity (their vision for what they want to achieve as a group). Such reflection enables participants to identify historically formed contradictions or conflict of motives. After identifying the contradictions, the change laboratory approach guides the group, while they embark on a collective design effort to understand and face these identified gaps. “The collective design effort is itself the core of an expansive learning process, involving reconceptualization and practical transformation of the object of the learner’s activity” (Sannino et al., p. 2). The artifact designed by the group serves for practical experimentation of the ideas developed (e.g., trying out a prototype in school). This iterative experimentation and development enables the group to develop a solution to the identified contradiction. In expansive learning, the solution is inspired by, but deviates from, the theoretical notions that the interventionist brings to the table. Ideally, the solutions developed should be generative, meaning that they would promote additional actions that serve the community in reaching the aspired vision. Exposing and addressing conflicting motives within DC-RPPs is crucial to their productivity. Many DC-RPPs stumble upon impediments in the co-design process, which stem from various contradictions or conflicts of motives that have not been surfaced and discussed within the partnership. A conflict of interest could be, for instance, a desire of school leaders to adopt a certain innovative educational approach but with a lack of capacity or even will of the teaching staff to embrace such a change. Contradictions may also stem from different agendas that researchers and practitioners bring to the partnership. To prevent such impediments, and to foster awareness of conflicting motives, this PDP (based on the change laboratory literature) recommends DC-RPPs to engage in processes specifically designed to (1) surface the contradictions among participants or those that the innovation might provoke, (2) develop a shared vision that resolves the conflicts, (3) develop a genuinely authentic goal for the co-design process and product, (4) experiment with prototypes and evolving versions of the design, and (5) reach a generative design product that will inspire additional activities that serve the shared vision.

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

491

PDP3: Boundary Crossing Processes In their review of the literature, Akkerman and Bakker (2011) found that mutual identification occurs when “intersecting practices are (re)defined in light of one another. In this process, people are concerned with (re)defining the way in which the intersecting practices are different from one another and how they can legitimately coexist” (p. 142). They also found that the process of coordination of the different practices takes place when “means and procedures are sought, allowing diverse practices to cooperate efficiently in distributed work, even in the absence of consensus” (p. 143). Their analysis showed that boundary crossing can lead to a reflection process, “which is about mutually defining the different perspectives that each intersecting practice can bring, and openness to take up others’ perspectives to look at one’s own practice.” Finally, usually as a result of the former processes, a transformation process occurs, leading to “profound changes in practices, potentially even the creation of a new, in-between practice, sometimes called a boundary practice” (p. 146). The sociocultural differences between researchers and practitioners are often underestimated, as is the need for explicit attention to bridging them. Similar to other partnerships between multi-expertise partnerships, DC-RPPs might become quite challenging (Akkerman et al., 2013). Researchers and practitioners bring with them very different sets of practices, beliefs, and values. These sociocultural differences can result in misunderstandings regarding all aspects of the collaborative work. To avoid such misunderstandings, and to foster productive communication among various participants of a DC-RPP, this PDP (based on the boundary crossing literature) recommends to develop activities designed to promote identification, coordination, and reflection processes. These activities should be tailored to the specific (changing) needs of the partnership and attend to the various levels in which boundary crossing processes may occur (i.e., intrapersonal, interpersonal, and institutional). Engaging in such activities may lead to transformation processes at all levels. For instance, at the institutional level, as a result of such a collaboration, a school or department in a college or university may adopt or develop new practices and procedures.

MDP2: Facilitate Engagement Through Dynamic Roles PDP4: Scholarship Roles A main construct in Shulman’s vision regarding the scholarship of teaching includes teachers’ active participation in a scholarly community of teaching. He maintained that: A scholarship of teaching will entail a public account of some or all of the full act of teaching – vision, design, enactment, outcomes, and analysis – in a manner susceptible to critical review by the teacher’s professional peers and amenable to productive employment in future work by members of that same community. (Shulman, 1998, p. 6)

492

Y. Kali et al.

A similar notion regarding the need for a community to encourage professionalism can be found in Cochran-Smith’s work: From an inquiry stance, teacher leadership and group membership look very different from how they might look when teachers are ‘trained’ in workshops or staff development projects. Taking an inquiry stance on leadership means that teachers challenge the purposes and underlying assumptions of educational change efforts rather than simply helping to specify or carry out the most effective methods for predetermined ends. (Cochran-Smith & Lytle, 2001, p. 54)

Note that in both accounts, the roles of teachers extend the more traditional notions of teaching and include roles that researchers typically play within their communities, including research and peer review. Many DC-RPPs are planned (and funded) for several years; after which practitioners are expected to take full ownership of maintaining and continuing to develop the innovation. One of the challenges of DC-RPPs is to develop infrastructures that will enable sustainability of the practices associated with implementation of the designed innovation over time despite various dynamics within a school (e.g., teacher turnover, new policies, etc.). This is especially true for technology-enhanced innovations that might be more susceptible to change. To sustain the innovation, this PDP (based on the scholarship of teaching literature) suggests developing social, organizational, and digital infrastructures that will enable and encourage all DC-RPP participants (researchers and practitioners from various institutions) to take part in a scholarly community, assuming roles of researchers and peer reviewers who constantly seek to advance not only local practice but also broader insights that extend what is currently known about professional teaching. To do so, DC-RPP leaders should develop, together with participants, the mechanisms that will allow participants to play these various scholarship roles in the inquiry community processes. These mechanisms, ultimately, should be embedded within the regular activities of the practitioners and enable them to gain credit for their efforts.

PDP5: Change Laboratory Roles One of the prominent aspects of change laboratory formative interventions is that the practitioners are those who take a leadership role in the change process. As part of this role, practitioners are those who identify the tensions in prevailing practices, design tentative solutions, and explore them. The research interventionist’s role is to provoke and support this processes, as described in PDP4. In fact, (Engeström, 2011) views the agency of the practitioners in formative interventions as a foundational point of departure, which distinguishes this tradition of research from design-based research. According to his analysis, design-based research is “associated with notions of perfection, completeness, and finality” (p. 600) that are in conflict with such practitioner agency. But there are many different perspectives on design-based research in this regard. Whereas Engestrom argues that design-based researchers

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

493

typically take the role of leaders and are those who “make the grand design,” while practitioners assume the role of implementers who contribute to its modification, McKenney and Reeves (2012) emphasize the importance of collaborative agenda setting in this genre of inquiry. But regardless of one’s perspective on design-based research, we believe that the change laboratory formative interventions lens can inspire a range of alternative roles and divisions of labor in DC-RPPs. These may be used to enrich current conceptualizations of design research (e.g., McKenney, 2016), and especially design-based implementation research (Penuel et al., 2011), which already highlight practitioner agency and ownership as key elements and encourage a more distributed role taking between researchers and practitioners. Such enrichment is required because fostering participants’ agency and ownership of change processes that combine external innovation with local visions of practitioners is a challenging endeavor. Yet it is also a vital component in sustainability (Bergen & Van Veen, 2004; Cviko, McKenney, & Voogt, 2015; Ketelhut & Schifter, 2011). The change laboratory formative interventions can become a useful lens in delineating the roles of participants and researchers-interventionists in DC-RPPs. Specifically, this PDP urges DC-RPP participants to consider a division of roles in which practitioners will lead the process of defining the problem and designing and experimenting with solutions, while researchers will serve as provokers and supporters of the process. Such a division of labor can broaden the range possibilities for distributing roles among participants and suggests an alternative to the more common role division, in which researchers lead and design and practitioners implement and contribute to modification.

PDP6: Boundary Crossing Roles The multilevel boundary crossing literature examines the dynamic nature of roles within partnerships that include different communities of practice. Of special interest is the role of brokers. Akkerman and Bakker’s literature review and Akkerman and Bruning’s further conceptualization (2011) revealed “the potentially significant role of individual people in (re)establishing continuity, especially in situations in which there is not yet a formalized structure for collaboration between different practices” (Akkerman & Bruining, 2016, p. 11). In those cases, there are often one or a few people doing the crossing, who have the authority and status to join people together for mutual benefit. In many DC-RPPs, individual researchers (typically graduate students) and individual practitioners (typically enthusiastic teachers, deeply engaged in the partnership) serve as brokers in partnerships, which may include other established researchers, as well as additional teachers and school leaders. By experiencing some of the practices of the other profession, these individuals may develop common language and understanding, which enable them to advance the co-design process and knowledge building within the DC-RPP. Akkerman and Bruning (2016) note that due to the crucial role that brokers play in partnerships that involve sociocultural differences, special attention should be paid to this role. First, “when these brokers appear to have most responsibility for establishing continuity across

494

Y. Kali et al.

different practices, they are likely to have a challenging political position and can probably gain from organizational recognition and support” (p. 41). Second, For sustainability reasons. . . partnerships might gain from circulating broker positions over time among various actors rather than relying too strongly on a few actors doing the crossing. This or alternative approaches is needed to make sure that others can also be involved and consequently learn from mutual activity. (p. 41)

One of the biggest challenges of DC-RPPs is to develop the capacity for sustaining change. Careful attention to the role of brokers can help address this challenge. Therefore, this PDP recommends DC-RPPs to identify individuals who may inherently become brokers within the partnership as a first step and to encourage these brokers to share or even pass this role to others as the partnership develops.

MDP3: Cultivate Productive Habits-of-Mind PDP7: Scholarship Habits-of-Mind In the preface of their book, Cochran-Smith and Lytle (2009) describe inquiry as a stance as follows: . . . a worldview and habit of mind – a way of knowing and being in the world of educational practice that carries across educational contexts and various points in one’s professional career and that links individuals to larger groups and social movements intended to challenge the inequities perpetuated by the educational status quo. (p. xiii)

This habit-of-mind requires not only cultivation of scholarship skills but also trustbuilding processes that allow teachers to surface, discuss, and critique their day-today events, norms, and practices, among peer practitioners as well as researchers. This PDP stresses the importance for DC-RPPs to design and develop work environments in which such trust is cultivated.

PDP8: Change Laboratory Habits-of-Mind The expansive learning involved in change laboratory formative interventions “requires breaking away from the given frame of action and taking the initiative to transform it. The new concepts and practices generated in an expansive learning process carry future-oriented visions loaded with initiative and commitment by the learners” (p. 5). In DC-RPPs, such a “revolutionary” habit-of-mind is especially needed when the design endeavor largely deviates from prevailing school practices. Evoking such a habit-of-mind among practitioners requires a delicate exploration of the needs, desires, and expectations of participants. Thus, this PDP advises DC-RPP leaders to carefully question accepted practices and analyze problematic situations, as a basis for cultivating the “revolutionary” stance required for leading profound changes within an educational system. The processes and roles described in PDP2 and PDP5 enable such exploration.

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

495

PDP9: Boundary Crossing Habits-of-Mind Based on their review of the literature, Akkerman and Bakker (2011) claim that “all learning involves boundaries. . . The boundary of the domain or community is constitutive of what counts as expertise or as central participation. When we consider learning in terms of identity development, a key question is the distinction between what is part of me versus what is not (yet) part of me” (Akkerman & Bakker, 2011, p. 132). The boundary crossing literature views learning as the development of identity, in which “others” with different experiences, knowledge, beliefs, and values play a crucial role. But in order to be able to learn from interactions with others, one is required to cultivate an “otherness” habit-of-mind, that is, to be open to identify and value the perspectives that others bring to a partnership and to be flexible to take up other’s perspectives, in a way that may change own (professional) identity. To enable productive collaborative work between researchers and practitioners, such openness is required. Therefore, this PDP advises DC-RPP participants to cultivate an atmosphere that will foster such a flexible state of mind. Here too, the processes and roles described in PDP3 and PDP6 can foster such an environment.

Employing DC-RPP Design Principles To illustrate the value of the set of PDPs in the theory-practice matrix described above, this section presents two DC-RPP cases and illustrates how the various PDPs were employed in them. The first case describes a 2-year program that aimed to foster interdisciplinarity in three middle schools, and the second case describes an 18-month program aimed at improving and innovating physics teaching and learning by leading teachers from 22 high schools. The rationale for choosing the two cases is multifaceted. First, we view them as representing productive DC-RPPs, as they both embed strong and deep ties between theory and practice, seek innovation in both theory and practice, and seek extended applicability (in Sannino et al.’s terminology – they are generative). Second, as described below, the value of the outcomes stemming from both programs has been formally acknowledged by circles wider than the DC-RPP by both researchers and practitioners. Third, the processes within both DC-RPPs were documented and studied. Fourth, authors of the current chapter were involved in the partnerships, enabling to base the characterization in both cases on information beyond the published record. Fifth, they represent two different profiles of the use of the set of PDPs in the theory-practice matrix. Specifically, while both cases employed all the nine PDPs, the profile of case 1 emphasizes the formative intervention in change laboratory lens (PDPs #2, 5, and 8), while the profile of case 2 emphasizes the scholarship of teaching and practitioner research lens (PDPs #1, 4, and 7). Our characterization of the two cases is a retrospective enterprise, based on the PDPs described above. It should be noted that in real time, both cases were perceived as teacher professional development programs that view design as an important method for teacher learning but not as DC-RPPs. This conceptualization was

496

Y. Kali et al.

developed retrospectively as well. We introduce each case, first, generally, and then in relation to the PDPs. Since our analysis of the cases showed that there was a strong relationship between processes, roles, and habits-of-mind for each of the lenses (we further discuss this finding in the “Discussion” section), we present the employment of PDPs in sets related to the theoretical lenses they represent (columns in the theory-practice matrix).

Case 1: Fostering Interdisciplinarity in Middle Schools General Description of the Program This case describes a DC-RPP that was formed with the vision of fostering interdisciplinarity within schools. Three schools, which were interested in developing a partnership around this vision, participated. The goal of the partnership was to adopt and adapt a technology-enhanced instructional model, originally designed to promote interdisciplinary learning in higher education (Kidron & Kali, 2015), for the use of middle school students (Kidron & Kali, 2018). The model, entitled Boundary Breaking for Interdisciplinary Learning (BBIL), refers to three perspectives – curricular, pedagogical, and organizational. Each of these perspectives builds on theoretical grounding and is represented as a design principle: breaking boundaries between disciplines, breaking boundaries between learners, and breaking boundaries between organizational hierarchies, respectively. An implementation of the model in higher education, via a course named “learning in a networked society (LINKS),” has been shown to promote interdisciplinary understanding of undergraduate students (Kidron & Kali, 2015). The LINKS course, which involved six disciplinary knowledge domains – learning sciences, science-communication, health sciences, cognition, communication, and information sciences – served as a reference for the DC-RPP in adaptation of the BBIL model (Table 2 describes selected features for each of the BBIL model’s design principles and illustrates how they were implemented in the LINKS course). The partnership included researchers, members of an educational nongovernmental organization (NGO), ministry of education supervisors, and school principals and teachers from the three schools. Overall, a total of about 40 participants were involved. The mechanisms that supported the collaborative work in the partnership included the following: (1) kickoff meetings in each of the schools, with a small forum of representatives to discuss the goals and suggest an “operation model” (Table 3); (2) co-design workshops (30 h in each school, in each year of the study), facilitated by one of the researchers, which enabled teachers to get familiar with the BBIL model and design their preliminary technology-enhanced learning environments and enabled researchers to learn about the characteristics, needs, affordances, and constraints in the schools; (3) reflection in practice meetings that were held within each team (often with one of the researchers) during enactment in each of the schools; (4) whole DC-RPP retreats that were conducted once a year, in which all participants met in order to share insights and lesson learned in each of the schools, as well as provide peer feedback, and reexamine the insights in light of the

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

497

Table 2 The BBIL model’s principles, features, and implementation in the LINKS course. (Adapted from Kidron & Kali, 2015) BBIL design principle Breaking boundaries between disciplines Curricular perspective building on theoretical notions of interdisciplinarity and knowledge integration

Breaking boundaries between learners Pedagogical perspective building on theoretical notions of learning communities

Breaking boundaries between organizational hierarchies Organizational perspective building on the notion of cognitive apprenticeship

Selected features and example implementation in the LINKS course Cross-cutting interdisciplinary theme A theme that serves as a backbone through which knowledge from the different disciplines is integrated. The cross-cutting theme chosen for the LINKS course was “learning in a networked society” Integrative cross-domain artifact Following a series of lessons in more than one disciplinary domain, students create an original artifact that integrates their ideas in these domains. The LINKS course integrative artifact was a short essay Integrative lenses A set of predefined generic questions for each disciplinary domain designed to foster interdisciplinary connection making. In LINKS, the integrative lenses included questions such as how learning is conceptualized in the various disciplinary domains Streamlining learning between community members Sequenced activities in which artefacts developed and shared in the community are later on used by other community members. In the LINKS course, community artifacts developed using the integrative lenses were used by individuals for creating their integrative artifact Learning community norm prompts Prompts designed to promote productive community learning norms (e.g., respect other ways of thinking). In the LINKS course, norms were directly discussed (online) in the community and published in the course website Personal mentoring between levels of hierarchy Technology-enhanced communication channels that enable personal mentoring of novices by advanced community members. The LINKS undergraduate students were mentored by graduates in a parallel course Modeling artifacts between levels of hierarchy Artifacts developed by advanced community members to make visible their ways of thinking are shared with novices. In the LINKS course, artifacts developed by the graduate student community were shared following the study of each domain with the undergraduate students

theory; and (5) a DC-RPP website, in which the processes and products of the collaborative work were constantly documented by all participants. During the first year of the partnership, each of the practitioner teams adapted the BBIL instructional model (Table 2) for their specific school contexts. Specifically, they collaboratively designed, enacted, and evaluated their own technologyenhanced interdisciplinary learning environments. Toward the end of the year, all learning environments comprised of technology-enhanced activities designed for

498

Y. Kali et al.

Table 3 Learning environments developed by the three practitioner teams Cross-cutting theme Disciplinary domains and topics

Operation model

Identity of the interdisciplinary moderator

School 1 Connections

School 2 Revolutions

School 3 Futurism

Biology (animal communication) Chemistry (chemical bonds) Social studies (social structures) Art (relations and feelings) Two eighth-grade classes taught for a total of 32 h spanning 16 weeks Disciplinary experts (development team)

Biology (the discovery of microorganisms) History (the French revolution) Geography (continent discovery) Art (the invention of the camera) Four eighth-grade classes taught for a total of 36 h spanning 4 weeks The homeroom teachers (some were also disciplinary experts)

Biology (genetic engineering) Physics (alternative energy) Geography (pollution)

Two eighth-grade classes taught for a total of 40 h spanning 6 weeks Disciplinary experts + homeroom teachers

students to study disciplinary contents as well as explore interdisciplinary connections between them (as demonstrated in the menu of the “Connections” learning environment, designed by School 1, in Fig. 1). Based on the design principles of the BBIL model, they all included an integrative assignment (the integrative crossdomain artifact in Table 2), which they chose to design as a culminating activity. The learning environments differed in their themes, content domains, operation models, and distribution of roles between disciplinary and interdisciplinary teaching, as demonstrated in Table 3. In the second year, additional teachers joined each practitioner team for a second iteration, in which the learning environments were either revised or completely redesigned based mainly on practitioner reflections but also on student reflections regarding the enactment in the first iteration (student reflections were collected using a survey designed by the teachers, as well as via some interviews conducted by the researcher).

Strong Affiliation with the Change Laboratory Lens Employing change laboratory roles (PDP5). Since the goal of the DC-RPP in this case was to adopt the notion of interdisciplinarity, and since the schools were interested in adopting the BBIL model and exploring new ways to implement it in each of the schools, the roles of the researcher and practitioners naturally took a change laboratory character. That is, the practitioners were those who served as leaders of change in their schools, as well as designers of the technology-enhanced learning environments and experimenters of the tentative solutions that they designed. The researchers, who initially suggested the BBIL model as an inspiration for practitioners to invent their own solutions, were naturally positioned as provokers and supporters. These roles enabled the processes described in the next section.

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

499

Fig. 1 Homepage of the learning environment designed by School 1 practitioner team

Employing change laboratory processes (PDP2). As suggested in PDP2, one of the initial stages was analysis of each of the practitioner teams’ prevailing school practices and identification of contradictions with the interdisciplinarity vision (as portrayed in the BBIL model) that the schools sought to adopt. Since the original model was designed for the context of higher education, many dilemmas emerged regarding various aspects of implementation. Kidron and Kali (2018) document 22 dilemmas that were raised, some of which represent “productive deviations” (Sannino et al., 2016) from the original designed solutions. For example, a major design consideration was raised in the DC-RPP around ways to put together the various disciplinary perspectives as part of one coherent learning sequence. In the context of the higher education course, it was quite simple to teach each of the perspectives as a whole, in sessions of about 2 weeks each. Students were provided with a set of “integrative lens” questions (see Table 2) after each of these sessions, which they discussed in an online forum to promote their knowledge integration between perspectives. However, organizational constraints made it quite difficult for the three schools to implement such a learning sequence. An alternative solution that came up involved splitting all the contents according to several “integrative lenses” questions (see Table 2), rather than according to the disciplinary domains. This enabled each of the domain-expert teachers to continue to meet every week with all students (as was the case prior to the intervention). The dilemma that was raised was whether such a solution might prevent students from developing deep understanding in each disciplinary domain and engage them too early in making connections between domains.

500

Y. Kali et al.

As suggested by PDP2, the three practitioner teams experimented with such solutions and shared the practical knowledge gained in each of the three practitioner teams over a period of 2 years. This enabled the teams not only to improve the technology-enhanced learning environments they designed but also to improve the design principles of the BBIL model with additional design considerations and a range of new solutions that extend its applicability to broader contexts. The resultant “enhanced principled instructional model” (Kidron & Kali, 2018) can be viewed as a type of generative solution (as PDP2 advises) developed in the DC-RPPS, which can inspire additional activities that serve the vision of promoting interdisciplinarity in schools. To enable the design knowledge developed in the DC-RPP to reach a wider audience, members of the DC-RPP (the NGO representatives and the facilitating researcher) are currently writing a book intended for teachers, principals, and educational policymakers. Employing change laboratory habit-of-mind (PDP8). The roles and processes described above required practitioners to adopt a habit-of-mind of questioning accepted processes (regarding the interdisciplinarity vision) analyzing problematic situations with regard to this vision and designing and experimenting with the solutions they designed (as advised in PDP8). In two of the schools, based on the knowledge and experience they gained in the program, teachers continued to develop additional interdisciplinary projects even after the intervention has ended. One of the teachers who moved to a different school initiated a totally new interdisciplinary program in his new school, demonstrating his cultivation of this “revolutionary” habit-of-mind. That said, it is important to note that this habit-of-mind, and the leadership roles and processes associated with it, was very demanding of the teachers. Perhaps for this reason, the reflection on explanatory models aspect of PDP8 was only moderately employed, as described in the section below describing the way in which the scholarship of teaching PDPs was employed.

Affiliation with Other Lenses Employing boundary crossing PDPs. The various mechanisms that supported the DC-RPP (e.g., the kickoff meetings, annual retreats, collaborative maintenance of the partnership’s website) enabled all participants to engage in boundary crossing processes (PDP3). These processes were especially prominent in the co-design workshop and reflection-in-action meetings. In these intensive meetings, as documented in interviews with teachers and in a journal kept by the researcher (Kidron & Kali, 2018), the teachers and the facilitating researcher had many opportunities to get acquainted with and develop a more knowledgeable appreciation of the expertise of each other. Such identification processes, as well as coordination of practices within the team, enabled the teachers and the researcher to meaningfully reflect on their own practices and sometimes to transform their own practices. Such transformation occurred at all levels described by Akkerman and Bruining (2016). Specifically, through the interpersonal joint work, the three teams developed new ways to foster interdisciplinary teaching and learning in each of the schools. At the institutional level, this required transforming organizational structures for enactment of these programs (for instance, pooling all teaching hours to enable an intensive

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

501

period devoted for the project). At the intrapersonal level, some teachers, who were especially open to take up the role of designers and embrace it as part of their professional identity (PDP9), began to serve as facilitators of their peers in the second year of the partnership. Their role in this year, which can be considered “brokering” (PDP6), represents a significant intrapersonal transformation. An additional boundary crossing process was on the side of the facilitating researcher (author 4 of this chapter). The work with the schools was originally external to his PhD, in which he intended to study interdisciplinarity only within higher education. His decision to incorporate an analysis of the DC-RPP work as a major part of his dissertation reflects a significant intrapersonal transformation process. Employing scholarship of teaching PDPs. As mentioned above, the balance between teachers’ role as leaders of change and designers (PDP5) and their role as researchers who explore their students’ learning in a community of scholars (PDP4) in this case was somewhat biased toward the former. Vast energy was required from teachers for engendering the change in their schools. As a result, only little time was devoted in the DC-RPP for systematically investigating students’ learning (PDP5) with the technology-enhanced learning environments the teachers designed and developed. The reflection in practice meetings, which was the mechanism of the DC-RPP in which such investigation could have taken place, was used mainly for addressing ongoing needs. For instance, the teams used these meetings for synchronizing activities within the team, discussing emerging implementation issues, refining the learning materials or activities for the students based on insights from the enactment, or solving unexpected problems (e.g., infrastructure fails or unplanned school activities which interrupted the planned timeline). Teachers also used these meetings to reflect on their experiences in the classes and to share emergent student insights, which was crucial for the team to support the students’ interdisciplinary understanding (Kidron & Kali, 2018). However, we view these latter activities as a partial employment of scholarship of teaching and practitioner research processes (PDP 1), roles (PDP4), and habits-of-mind (PDP7). That said, the next stage planned for the DC-RPP – in those schools in which interdisciplinarity has become part of the schools’ activity – is intended to focus on deepening the process of investigating students’ development of interdisciplinary learning. We discuss the implications of this DC-RPP case together with case 2, following its description below.

Case 2: Innovating Physics Teaching and Learning Workshop General Description of the Workshop The workshop was part of a 3-year professional development program for 22 Jewish and Arab leading Israeli high school physics teachers from all over the country who met for a full day each week throughout the year. The program aimed at developing teacher leadership for promoting physics education in Israel. Indeed, following the program, most of the graduates became involved in leadership roles such as facilitators of regional professional learning communities of physics teaching and curriculum developers in additional DC-RPPs.

502

Y. Kali et al.

The DC-RPP workshop (330 h) included activities that were carried out as part of the program’s weekly meetings, while the participants continued to collaborate between meetings. The partnership included a team of 8 physics education researchers and expert practitioners from the staff of a science teaching department in a research university and all the 22 leading teachers. Within the wider goal of teacher leadership in physics education, the specific aim of the workshop was to develop teachers’ capabilities to carry out systematic research-based design of learning-centered activities. Learning about physics education research findings and ways to use them in practice were part of this aim. Participants of the DC-RPP worked in teams of 5–6 to design learning modules of between 6 and 10 h each. A science education researcher or expert practitioner served as mentor in each of the teams. Participants chose the module topics from a list of topics related to the existing syllabus (e.g., “the first and second laws of Newton”). All topics were characterized in the literature on physics education research (PER) as challenging for both teaching and learning and thus requiring genuine changes in the way they are taught. The modules developed by the teams included activities and resources (e.g., simple lab equipment), as well as suggestions of teaching sequences. During the workshop, teachers systematically and continuously collected data on their practice and on their students’ learning (an evidence-based approach). The DC-RPP teams shared with the plenum their experiences, insights, and challenges, raised questions regarding the relationship between practice and learning and about alternative ways to bring about change, and received feedback. This process led to the development of cross-cutting ideas. The DC-RPP workshop consisted of several stages (Table 4), each of which ended with a mini-conference attended by additional colleagues from the physics and physics education community (e.g., leading teachers, physics education researchers, interested physicists). Upon completing the workshop, some of the teams published a paper in the Israeli Journal of Physics Teachers, summarizing the module’s content, the pedagogy, the findings from the various investigations, and the reflections on the whole design process. The learning materials of each team were also published through a teachers’ website and are being used in a variety of forms until today. Findings were also presented in international meetings of researchers and teachers and published in peer-reviewed journals. The article by Eylon and Bagno (2006) became a practical input for teachers as it was included in a resource book published by the physics teacher education coalition (PTEC) (Meltzer & Shaffer, 2011). Teachers’ responses to questionnaires given immediately after the workshop and 6 years later suggest that the program has had lasting beneficial impacts on the participants’ attitudes toward teaching and on their classroom practice. In particular, most of the teachers singled out the design of the module as an activity that was most meaningful, useful, or important to them. Additionally, teachers adapted and adopted the approach exemplified in the workshop in their work as facilitators of regional teacher communities. The researchers benefited as well. The lessons learned from this intensive collaboration have been applied in a variety of research and development projects for professional development of physics teachers.

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

503

Strong Affiliation with the Scholarship Lens Employing scholarship processes (PDP1). A major mechanism that promoted a genuine change in the participants’ views and practices throughout the workshop was the goal-driven, evidence-based iterative design process that the DC-RPP participants carried out. This process was supported by resources within the workshop. Participants systematically investigated questions related to their own practice (physics teaching) and student learning. Additionally, a successive refinement of goals took place throughout the workshop. For example, as described in Table 4, in Table 4 Stages in the DC-RPP, features, and implementation. (Adapted from Eylon & Bagno, 2006) Stages in the DC-RPP Stage I: Defining teaching and learning goals (about 120 h) 1. Initial definition of goals 2. Review of the literature 3. Diagnosis 4. Revision of goals Mini-conference I Aim: Collaborative negotiation of the required innovation based on PER literature and practitioner (researchers and teachers) experience Stage II: Designing the module (about 120 h) 5. Initial design Mini-conference II 6. Development of module Aim: Initial design and development of activities based on collaborative learning of PER-based innovative teaching strategies and feedback from experts

Stage III: Performing smallscale research study (about 90 h) 7. Design and implementation 8. Consolidation and reporting Mini-conference III Aim: Identifying gaps between what is taught and what students learn, as well as reflecting on (practical and theoretical) lessons learned in the DC-RPP

Selected features and examples of implementation in the workshop Experience as learners Teachers carry out activities related to specific modules. This experience challenges teachers’ physics and physics teaching knowledge (matched to proximal zone of development) Iterative revision of goals Within each design team, teachers carry out content analysis and characterize relationships among concepts and principles relevant to the planned module in a concept map Diagnosis Teachers design a diagnostic questionnaire to probe students’ understanding. They administer, analyze, and summarize the findings and then discuss with the RPP and other implications for the design of the module (e.g., deciding to narrow its scope) Collaborative learning of innovative teaching strategies Each design team reviews a PER-based instructional strategy (e.g., the predict-observe-explain strategy), presents to the plenum, and leads discussion about challenges and advantages Consultation with experts Toward the mini-conference, participants approach expert teachers. Physics educators and physicists for feedback and assistance in explaining complex ideas The “story of the module” Toward the mini-conference, teachers develop “story of the module” posters including goals and rationale for strategies, alternative sequencing, and entry conditions. They also describe the process leading to the initial design Assessing student learning Within each design team, teachers formulate research questions and tools, implement the modules in their classes, and administer and analyze the assessment (e.g., using questionnaires and interviews). They start individually, and then they share, reflect, and collaboratively interpret findings and implications for design

504

Y. Kali et al.

stage I of the design process, the teachers developed a diagnostic questionnaire to examine student understanding of central ideas related to the initial goals of the module, administered it to their students, analyzed the answers, and reflected collaboratively with their peers in the team and also with the other teams. Based on the findings, members of the DC-RPP realized, for example, that some assumptions about students’ incoming knowledge were unfounded, and therefore the team revised the goals for the modules they designed in stage II. In some of the teams, these findings were quite shocking, as participants initially viewed the topic of the module (“the relationship between Newton’s first and second laws”) as too obvious and assumed that they had the required experience to teach it. Through the investigation, participants realized challenges that are often not attended to. The collaborative discussions within the professional community of teaching led to cross-cutting insights. For instance, many teachers grappled with basic questions concerning the design of “good” test questions for probing student thinking on a particular topic and tried to unpack what understanding means in that topic. In their reflections on the process, the teachers claimed that as the DC-RPP advanced, they attempted to reveal students’ thinking about physical phenomena rather than focusing on their technical skill to use equations. The teachers reported that the physics education research literature on students’ understanding was very useful in this process. In stage III, after teachers taught the first version of the module to their students, some teams were disappointed to find gaps between what they taught and what their students have learned (as analyzed in the DC-RPP using the diagnostic tool developed). In most of teams, this resulted in negotiating criteria for narrowing the scope of the module and reconsidering the goals. This challenging selection process required “going meta” while negotiating criteria for deciding what to include in the design of the module. For example, one cross-cutting criterion that came up was making the learning relevant to students. Facilitate engagement through specific and dynamic roles (PDP4). The feedback from colleagues within and between the design teams, as well as with additional colleagues that teachers met in stages I, II, and III, was a significant input to the DC-RPP. This process enabled all the participants to assume roles of researchers and peer reviewers. In the preparation process of the mini-conferences, members of the workshop discussed the structure and formats of the meeting. These discussions, supported by the experienced mentors, started a process of engagement in, and developing the mechanisms for sharing, critiquing and synthesizing design knowledge (as recommended in PDP4). Toward these mini-conferences, the groups worked intensively on consolidating the various inputs (e.g., realizations from their experience as learners in the workshop, findings of the investigations, their own practical knowledge) and explicating them toward making their ideas visible and public. In interviews, teachers described the challenge of preparation toward these meetings (Eylon & Bagno, 2006). They also explained that their own analysis of their practice and the requirement to explicate and explain their considerations brought about a “quantum jump” in their expertise as teachers. This included contents of their module, understanding of the aspects they should attend to in designing instruction, and the relationship between their teaching and their students’

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

505

learning. The accompanying research on the workshop examined various documents prepared by the DC-RPP, such as transcripts of the meetings and the conferences, the initial version of the module, and interviews with the teachers. The analyses and triangulation of data documenting the activities designed and developed in the DC-RPP, and the roles that participants took to develop them, indicated a considerable increase in the coherence of these activities (e.g., between stated goals and findings regarding student learning, as well as between stated goals and the actual content of the module)(Eylon and Bagno, 2006). Cultivate productive habits-of-mind (PDP7). It should be noted that the participating teachers were experienced and successful teachers with sound knowledge of physics. They were required to teach a packed curriculum and therefore were somewhat reluctant, at early stages of the partnership, to take up changes in practice that require considerable time and effort, even when they acknowledged the need for change. The building of trust was an essential component in teachers’ willingness to openly discuss their ideas. Initially, teachers attributed teaching and learning difficulties to their own inadequacies. The processes and roles described above (PDP1 and PDP4), in which teachers realized that their ideas and findings are respected and that other members of the DC-RPP are grappling with similar issues, were essential in their willingness to share and open up. Some of the central activity strands in the workshop were designed to form important habits-of-mind for “learner-centered” practice. An example is the “evidence based approach” in which teachers take an inquiry stance (Cochran-Smith & Lytle, 2009) and continuously follow their practice and student learning to make decisions on the basis of data they collect. These aspects played an important role in later activities of the researchers such as the binational UK-Israel collaboration on the development of an evidence-based continuing professional program for science teachers (Harrison, Hofstein, Eylon, & Simon, 2008).

Affiliation with Other Lenses Employing change laboratory and boundary crossing PDPs. As mentioned above, the workshop started as a professional development program in which the change laboratory and boundary crossing roles (PDP5 and PDP6) were not symmetric. The physics education researchers led the process. Retrospectively, it seems that they underestimated the sociocultural difference between themselves (as researchers) and the practitioners and were not sensitive enough to this gap. Through the deep involvement with the DC-RPP activities, the practitioners developed ownership on the design process and material development. There was a change in the division of roles within the partnership and teachers, who gradually began to serve as leaders of change, designers, and experimenters, while the researchers began to take more supportive roles (PDP5). In a similar manner, there was more sensitivity to the role of brokers (PDP6). For example, the teachers suggested adding expert curriculum developers to the teams. These experts enabled better linkage between the researchers and the teachers in the design process. Interestingly, the changes in the role division were strongly associated with

506

Y. Kali et al.

employment of PDP8 and PDP9 (change laboratory and boundary crossing habitsof-mind). While at early stages practitioners were reluctant to consider alternative modes of teaching, as time went on, they began to value the perspectives that researchers brought to the partnership and were more willing to try out new directions. In projects that were taken up by some members of the partnership in later years, the employment of these PDPs was enhanced and became an important a priori consideration in designing additional DC-RPPs.

Discussion The analysis of the two cases, as mentioned above, shows that the main processes, roles, and habits-of-mind in each of the cases are strongly related to a specific theoretical lens in the theory-practice matrix (one of the columns in Table 1). This affiliation can be explained by the fact that these practical aspects of a DC-RPP are related to, and may affect, each other. In case 1, for instance, the processes of engendering change within the schools, and exploring the contradictions that need to be addressed, required teachers to embrace that “revolutionary” habit-of-mind and assume the role of leaders of change and designers of new learning environments that would allow the change to take place. In case 2, the processes of investigating questions that relate own practice (physics teaching) and student learning were naturally related to teachers’ assuming roles of researchers and peer reviewers in the community. As all participants (researchers and teachers) developed trust as a habit-of-mind within the community, processes of sharing, critiquing, and synthesizing design knowledge became natural. That said, other PDPs from the theory-practice matrix, though not as saliently employed, were also part of the cases’ profiles, indicating that PDPs in the various lenses may strengthen each other. In case 1, for instance, the boundary crossing processes, roles, and habits-of-mind were crucial for participants to collaboratively engender a meaningful change within the schools. That is, in order for practitioners to embark on the complex task of designing their technology-enhanced learning environments intended to promote interdisciplinarity, they were required to develop an appreciation of design as a practice that can support teaching and that they would want to adopt. On the other hand, the researchers’ appreciation and understanding of the complexity of the teaching ecology within the schools enabled them to better support the design process of the teachers. PDPs that strengthen each other between lenses were also found in case 2. There was a strong relationship between changes over time in the habits-of-mind pragmatic principles PDP8 and PDP9. From being reluctant to consider alternative views and changes in practice, teachers became interested to consider alternatives and experiment with new strategies. A similar relationship between the lenses was observed in employing pragmatic principles PDP5 and PDP6 by the researchers. Overtime, they became more sensitive to the

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

507

sociocultural gaps between the researchers and practitioners in the partnership. This resulted in reconsideration of the division of roles and led to active seeking of brokers. The finding that PDPs from various theoretical lenses can strengthen each other supports earlier research on design principles. Kali, Levin-Peled, and Dori (2009), for instance, based on their research on design principles for promoting collaboration in higher education, recommend identifying clusters of design guidelines that strengthen each other. The set of PDPs in the research-practice matrix can be viewed as such a cluster for guiding productive DC-RPPs. Interestingly, PDPs that were only partially employed, such as the scholarship PDPs (#1, 4, and 7) in case 1, and change laboratory formative interventions PDPs (#5 and 8) in case 2 became much more salient in further stages in both cases. In case 1, this was illustrated by the intention to go deeper into investigating students’ interdisciplinary understanding in further steps of the program. In case 2, this was illustrated in the design of more opportunities for practitioners to serve as leaders of change in later stages of the project. These developments illustrate the dynamic use of PDPs over time in both cases. In fact, we view the dynamics in case 1 as moving from focusing on change laboratory PDPs to a dual focus that also emphasizes scholarship of teaching PDPs. In case 2, the dynamics represent the other direction, that is, moving from a focus on scholarship of teaching PDPs to the dual focus in which change laboratory formative intervention PDPs are also emphasized. We believe that these trajectories portray the complex nature of conducting designbased research in general (e.g., Akkerman et al., 2013) and in DC-RPPs in particular. This complexity was addressed in different trajectories in the two cases.

Conclusion Communicating the complex nature of design research endeavors, especially when they are conducted in DC-RPPs, is a challenging task, due to the multiple aspects involved in (a) design, (b) research, and (c) partnerships. The theory-practice matrix of PDPs introduced in this chapter addresses this challenge. By enabling exploration of specific connections between theory and practice in DC-RPPs, it has provided a productive way for characterizing, comparing, and contrasting between two DC-RPP cases. Although the cases varied in many aspects (e.g., use of technology, disciplinary contents involved, the scope of design, the focus on student learning), the set of PDPs enabled exploration of the unique connections between theory and practice they represent. We believe that the abstraction and articulation of the PDPs in the theory-practice matrix will enable not only to accumulate and synthesize lessons learned from the emerging research trajectory of DC-RPPs but also to guide such future endeavors and thus build important bridges between research and practice in education. Finally, we view the current theory-practice matrix of PDPs as a growing endeavor and call additional DC-RPP researchers to expand it with additional relevant lenses.

508

Y. Kali et al.

References Akkerman, S. F., & Bakker, A. (2011). Boundary crossing and boundary objects. Review of Educational Research, 81(2), 132–169. Akkerman, S. F., Bronkhorst, L. H., & Zitter, I. (2013). The complexity of educational design research. Quality and Quantity, 47(1), 421–439. Akkerman, S. F., & Bruining, T. (2016). Multi-level boundary crossing in a professional development school partnership. Journal of the Learning Sciences, 25(2), 240–284. Bauer, K., & Fischer, F. (2007). The educational research-practice interface revisited: A scripting perspective. Educational Research and Evaluation, 13(3), 221–236. Bergen, T., & Van Veen, K. (2004). Het leren van leraren in een context van onderwijsvernieuwingen: Waarom is het zo moeilijk? [Teacher learning in the context of educational innovations: Why is it so difficult?]. VELON: Tijdschrift voor Lerarenopleiders, 25(4), 29–39. Broekkamp, H., & Van Hout-Wolters, B. (2007). The gap between educational research and practice: A literature review, symposium and questionnaire. Educational Research and Evaluation, 13, 203–220. Coburn, C. E., & Penuel, W. R. (2016). Research–practice partnerships in education outcomes, dynamics, and open questions. Educational Researcher, 45(1), 48–54. https://doi.org/10.3102/ 0013189X16631750 Coburn, C. E., Penuel, W. R., & Geil, K. (2013). Research-practice partnerships at the district level: A new strategy for leveraging research for educational improvement. New York, NY: William T. Grant Foundation. Cochran-smith, M., & Lytle, S. L. (2001). Beyond certainty: Taking an inquiry stance on practice. In A. Lieberman & L. Miller (Eds.), Teachers caught in the action professional development that matters (pp. 45–58). New York, NY: Teachers College Press. Cochran-Smith, M., & Lytle, S. L. (2009). Inquiry as stance: Practitioner research for the next generation. New York, NY: Teachers College Press. Collins, A. (1992). Toward a design science of education. In E. Lagemann & L. Shulman (Eds.), Issues in education research: Problems and possibilities (pp. 15–22). San Francisco, CA: Jossey-Bass. Cviko, A., McKenney, S., & Voogt, J. (2015). Teachers as co-designers of technology-rich learning activities for emergent literacy. Technology, Pedagogy and Education, 24(4), 443–459. DBRC. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. de Vries, B., & Pieters, J. (2007). Bridging the gap between research and practice: Exploring the role of knowledge communities in educational design. European Educational Research Journal, 6(4), 382–392. Dede, C., Rockman, S., & Knox, A. (2007). Lessons learned from studying how innovations can achieve scale. Threshold, 5(1), 4–10. Donovan, S., Snow, C., & Daro, P. (2014). The SERP approach to problem-solving research, development, and implementation. In B. J. Fishman, W. R. Penuel, A. R. Allen, & B. H. Cheng (Eds.), Design-based implementation research: Theories, methods, and exemplars. National society for the study of education yearbook (pp. 400–425). New York, NY: Teachers College Record. Engeström, Y. (2007). Putting Vygotsky to work: The change laboratory as an application of double stimulation. In The Cambridge companion to Vygotsky (pp. 363–382). Cambridge, MA/New York, NY: Cambridge University Press. Engeström, Y. (2011). From design experiments to formative interventions. Theory & Psychology, 21(5), 598–628. Engeström, Y., Virkkunen, J., Helle, M., Pihlaja, J., & Poikela, R. (1996). The change laboratory as a tool for transforming work. Lifelong Learning in Europe, 1(2), 10–17. Eylon, B., & Bagno, E. (2006). Research-design model for professional development of teachers: Designing lessons with physics education research. Physical Review Special Topics – Physics Education Research, 2, 020106-1–020106-14. https://doi.org/10.1103/PhysRevSTPER.2.020106

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

509

Harrison, C., Hofstein, A., Eylon, B., & Simon, S. (2008). Evidence-based professional development of teachers in two countries. International Journal of Research in Science Education, 30(5), 577–591. Henrick, E., Munoz, M. A., & Cobb, P. (2016). A better research-practice partnership. Phi Delta Kappan, 98(3), 23–27. Hutchings, P., & Shulman, L. S. (1999). The scholarship of teaching: New elaborations, new developments. Change, 31(5), 10–15. ISSOTL. (2017). International society of the scholarship of teaching and learning. Retrieved from http://www.issotl.com Kali, Y. (2006). Collaborative knowledge building using the design principles database. International Journal of Computer Support for Collaborative Learning, 1(2), 187–201. Kali, Y. (2008). The design principles database as means for promoting design-based research. In A. E. Kelly, R. A. Lesh, & J. Y. Baek (Eds.), Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching (pp. 423–438). Mahwah, NJ: Lawrence Erlbaum Associates. Kali, Y. (2016). Transformative learning in design research: The story behind the scenes. In C. K. Looi, J. L. Polman, U. Cress, & P. Reimann (Eds.), Transforming learning, empowering learners. The international conference of the learning sciences (ICLS) 2016 (Vol. 1, pp. 4–5). Singapore: International Society of the Learning Sciences. Kali, Y., Levin-Peled, R., & Dori, Y. J. (2009). The role of design-principles in designing courses that promote collaborative learning in higher-education. Computers in Human Behavior, 3(1), 55–65. Kali, Y., & Linn, M. C. (2008). Technology-enhanced support strategies for inquiry learning. Handbook of research on educational communications and technology, 145–161. Ketelhut, D. J., & Schifter, C. C. (2011). Teachers and game-based learning: Improving understanding of how to increase efficacy of adoption. Computers & Education, 56, 539–546. Kidron, A., & Kali, Y. (2015). Boundary breaking for interdisciplinary learning. Research in Learning Technology, 23, 1–17. Kidron, A., & Kali, Y. (2018). Extending the applicability of design-based research through research-practice partnerships. Educational Design Research, 1(2). Lavis, J. N., Robertson, D., Woodside, J. M., McLeod, C. B., & Abelson, J. (2003). How can research organizations more effectively transfer research knowledge to decision makers? Milbank Quarterly, 81(2), 221–248. McKenney, S. (2016). Researcher-practitioner collaboration in educational design research: Processes, roles, values & expectations. In M. A. Evans, M. J. Packer, & R. K. Sawyer (Eds.), Reflections on the learning sciences (pp. 155–188). New York, NY: Cambridge University Press. McKenney, S., & Pareja-Roblin, N. (2018). Connecting research and practice: Teacher inquiry and design-based research. In J. Voogt, G. Knezek, R. Christensen, & K. Lai (Eds.), International handbook of information technology in primary and secondary education (2nd ed.). London, England: Springer. McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. London, England: Routledge. Meltzer, D. E., & Shaffer, P. S. (2011). Teacher education in physics: Research, curriculum and practice. College Park, MD: American Physics Society. Nutley, S., Walter, I., & Davies, H. (2007). Using evidence. How research can inform public services. Bristol, UK: The Policy Press. Penuel, W. R. (2015). Infrastructuring as a practice for promoting transformation and equity in design-based implementation research. Keynote address at the ISDDE annual conference. September 22–25, Boulder. Penuel, W. R., Allen, A.-R., Coburn, C. E., & Farrell, C. (2015). Conceptualizing research–practice partnerships as joint work at boundaries. Journal of Education for Students Placed at Risk, 20(1–2), 182–197.

510

Y. Kali et al.

Penuel, W. R., Bell, P., Bevan, B., Buffington, P., & Falk, J. (2016). Enhancing use of learning sciences research in planning for and supporting educational change: Leveraging and building social networks. Journal of Educational Change, 17(2), 251–278. Penuel, W. R., Fishman, B., Cheng, B., & Sabelli, N. (2011). Organizing research and development and the intersection of learning, implementation and design. Educational Researcher, 40(7), 331–337. Reinking, D., & Bradley, B. (2008). Formative and design experiments: Approaches to language and literacy research. New York, NY: Teachers College Press. Sannino, A., Engeström, Y., & Lemos, M. (2016). Formative interventions for expansive learning and transformative agency. Journal of the Learning Sciences, 8406, 10508406.2016.1204547. Shulman, L. S. (1998). Course anatomy: The dissection and analysis of knowledge through teaching. In P. Hutchings (Ed.), The course portfolio: How faculty can improve their teaching to advance practice and improve student learning. Washington, DC: American Association of Higher Education. Shulman, L. S. (2011). Feature essays: The scholarship of teaching and learning: A personal account and reflection. International Journal of the Scholarship of Teaching and Learning, 5(1) (Article 30). Star, S. L. (1989). The structure of ill-structured solutions: Boundary objects and heterogeneous distributed problem solving. In M. Huhns & L. Gasser (Eds.), Readings in distributed artificial intelligence. Menlo Park, CA: Morgan Kaufman. Star, S. L., & Greisemer, J. R. (1989). Institutional ecology, “translations” and boundary objects: Amateurs and professionals in Berkeley’s museum of vertebrate zoology, 1907-39. Social Studies of Science, 19(3), 387–420. Trigwell, K., Martin, E., Benjamin, J., & Prosser, M. (2000). Scholarship of teaching: A model. Higher Education Research and Development, 19(2), 155–168. van Braak, J., & Vanderlinde, R. (2012). Het profiel van onderwijsonderzoekers en hun opvattingen over samenwerking met de onderwijspraktijk. Pedagogische Studiën, 89(6), 364–376. Vanderlinde, R., & van Braak, J. (2010). The gap between educational research and practice: Views of teachers, school leaders, intermediaries and researchers. British Educational Research Journal, 36(2), 299–316. Virkkunen, J. (2013). The change laboratory: A tool for collaborative development of work and education. Heidelberg, Germany: Springer Science & Business Media. Wagner, J. (1997). The unavoidable intervention of educational research: A framework for reconsidering researcher-practitioner cooperation. Educational Researcher, 26(7), 13–22.

Yael Kali is Professor of Technology Enhanced Learning at the University of Haifa, and a Fellow of the International Society of the Learning Sciences (ISLS) since 2021. She is the Founding Director of two Israeli centers of research excellence – Learning In a NetworKed Society (LINKS, 2012–2018) and Taking Citizen Science to School (TCSS, 2017–2023). Using design-based research and design-based implementation research, Kali explores technology-enhanced learning and teaching at various contexts and age levels, from junior high school to higher education, and as part of teacher professional development programs. Her work focuses on the role of design principles for supporting learning and collaborative design, especially within networks of research-practice partnerships, and in teachers-as-designers contexts. She has served as an Associate Editor for the journal Instructional Science from 2012 to 2020, and was a visiting scholar at the Centre for Research on Computer Supported Learning & Cognition, University of Sydney (2010–2011), and at the School of Education, University of Colorado Boulder (2018). Bat-Sheva Eylon is a Professor (Emeritus) in the Science Teaching Department at the Weizmann Institute of Science. She headed the department from 2007 to 2015. She is a Fellow of the American Association for the Advancement of Science (AAAS) since 2006, a Fellow of the International Society of the Learning Sciences (ISLS) since 2021, and a recipient of the 2015 Israel EMET prize

20

Design-Centric Research-Practice Partnerships: Three Key Lenses for. . .

511

in Social Science. Her main research areas are the learning and teaching of physics in high school and integrated science in middle school, and the continuing professional development of teachers. Eylon’s approach reflects her strong belief in research-practice-partnerships (RPP), in research on designs that advance RPP collaborations and in the importance of teachers’ leadership. With peers, many of whom her graduate students, she designed and led a variety of large-scale research-based educational frameworks such as the national networks of Professional Learning Communities for science and for physics teachers, the Rothschild-Weizmann MSc “Program for Excellence in Science Teaching” for acting teachers, and the PeTeL digital environment. This environment empowers teachers’ collaborations and functioning as producers, consumers, and designers of Personalized Teaching and Learning. The Knowledge-Integration perspective, developed with Linn and collaborators, has played an important role in all these endeavors. Prof. Dr. Susan McKenney co-leads ELAN, the Department of Teacher Professional Development within the Faculty of Behavioral and Management Sciences at the University of Twente, and is a Visiting Professor in the Learning Sciences & Policy Group at the University of Pittsburgh. Her research focuses on understanding and facilitating the interplay between curriculum development and teacher professional development and often emphasizes the supportive role of technology in these processes. As such, she also studies processes of design that can be applied in the field of education and synergetic research-practice interactions. She is committed to exploring how educational research can serve the development of scientific understanding while also developing sustainable solutions to real problems in educational practice. Since design-based (implementation) research lends itself to these dual aims, her writing and teaching often provide ideas about how to conduct this exciting form of inquiry. In addition to authoring numerous articles, she co-edited the book Educational Design Research and, together with Tom Reeves, wrote the book Conducting Educational Design Research. She has served as guest editor of special issues in Instructional Science, European Journal of Education, Australasian Journal of Educational Technology, Pedagogische Studiën, and Technology, Pedagogy and Education. She is currently Associate Editor for the Journal of the Learning Sciences and has authored over 100 peer-reviewed publications. Adi Kidron is head of the educational technologies unit (EduCore) at the Department of Science Teaching at the Weizmann Institute of Science, and is responsible for leading a team of experts in the development of innovative and effective educational technologies that support educational research. He holds a Ph.D. in Education Technologies from the University of Haifa, as well as a post-doctoral position within the WISE team at the University of California, Berkeley. His research focuses on interdisciplinary learning within technology-enhanced learning communities. He has also gained over 25 years of experience as an instructional designer in various ed-tech companies.

The Confluence Effect of Policy, Mental Models, and Ethics on Individual Behavior

21

Shirley A. Dawson and Vicki S. Napper

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mental Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Models of for Defining Ethical Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rest’s Decision-Making Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jones’ Moral Intensity Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cottone and Claus Theoretical Model Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Craft Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Nexus of Policy, Mental Models, and Ethical Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

514 515 516 517 518 518 519 520 521 522 522

Abstract

The study of how people learn and behave has changed significantly. Current educational practices are built on the foundations of a variety of behavioral theories and stage development theories that have evolved into practices based on cognitive learning theory. The individual is now the application focus of educational planning. However, development of policy and codes of ethical conduct are not based on how people construct their behavior but rather still reflect group or organizational thinking to control behavior. This chapter reviews types of policy development, development of mental models, and how those models may define behavior. It also presents possible research areas to help understand the interaction between individual mental models and development of policy and codes of ethics.

S. A. Dawson · V. S. Napper (*) College of Education, Department of Teacher Education, Weber State University, Ogden, UT, USA e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_127

513

514

S. A. Dawson and V. S. Napper

Keywords

Policy · Code of ethics · Ethical behavior · Mental model

Introduction Understanding how people learn evolved in the twenty-first century to focus on understanding how individuals perceive and process life experiences and how those experiences affect their behaviors. Prior to the advent of theories of individualized cognition, it was a common practice to develop policy or codes of ethical conduct based on observable behaviors. However, what is observed and what is understood are often different. This difference between justice focused laws and individual decision-making factors may have led to a mismatch of behavior-based policy or codes of ethics and how people actually choose to react toward those directives. The foci of the following chapter are to present another lens for viewing policies and codes of ethics, to discuss why an awareness of factors people may use to construct personal decision-making is critical, and explore the relationship between policy and mental models. The study of how people learn and, as a result, behave has changed significantly from the models treasured by practioners of the twentieth century. Current educational practices are built on the foundations of a variety of behavioral theories and stage development theories (i.e., Piaget, Kohlberg, Gagne, Knowles). Extensive research on cognitive learning theories in the later part of the twentieth century has evolved into current practice and research based on the idea of individualized learning; the individual is now the focus of educational planning and policies. Identifying factors that best match individual needs and motivations are the foundation of current practice. The focus of this chapter is to show the relationship between three areas: policy, mental models, and decision-making models on ethical behavior. Policy, mental models, and ethics represent important constructs that may influence each other. Policy focuses on identification of and regulating of moral actions; whereas, mental models represent the underlying unique paradigms of ethical values and behaviors developed by individuals. The authors suggest that justice-focused constructs of policy and codes are influenced by individual or organizational mental models on ethical behavior. This chapter considers the following factors: the nature and role of policy, the development of mental models, decision-making models, and the relationship between policy and individual meaning in ethical settings. First explored are the constructs of policy and ethics and their influence on professional behavior in educational setting. Next, individual mental models and research on decision-making processes are reviewed to discover how these constructs may interact with the development of policy.

21

The Confluence Effect of Policy, Mental Models, and Ethics on. . .

515

Policy An informed analysis of policies is dependent on an understanding of previous and current policy analyses and the relationship between policy and practice (Grossman & Thompson, 2004). Policy is an oft used but rarely defined term in ethical discussions. Policy guides prudent principles and procedures to manage or direct current or future decisions (Business Dictionary, n.d.; Merriam-Webster, n.d.). More than just a plan, policy is a governance structure to ensure goals are achieved and to enforce adherence to specified principles and acceptable actions (Mansfield & Thackik, 2016). Clearly written policies can ensure fidelity of program or law implementation (Barlett, Johnson, Lopez, Sugarman, & Wilson, 2005). Policy is not the same as law, but both are governing sources for individual actions and decisions. There are primarily three law types in the United States. Statutes emanate from legislative bodies and are generally more permanent than regulations because they require executive approval to become law. Regulations (or rules), written by the administrative branch, may depend on statutory law for existence or permanence, may provide further clarification, or be independent of other law. Lastly, judicial or case law is created when the judicial branch interprets or refines existing regulatory or statutory law. Governments, professional organizations, boards, or other bodies write policies to give further directions, to supplement laws, or to supersede law if one does not exist. Policy is a less precise term (Sindelair et al., 2010; Zirkel, 2014) and more easily revised as legislative, executive, or judicial action is not required. The vagueness in meaning and permanence can result in variation in policy interpretation, creation, implementation, intentions, and analysis (Guba, 1984). Despite its vagueness and because of its broadness, the power of policy is not overlooked and ranges across a wide diversity of professions (Rose, Kremen, Thrupp, Gemmill-Herren, Graub, & Azzu, 2014; Rogers, Fenton, & Hutchings, 2015). The request from professional oversight committees for more regulation may stem from the belief that sustained change requires policy change (Hardman, 2008; Garcia, Zeglin, Matray, Froehlich, Marable, & McGuire-Kuletz, 2016; Rose et al., 2014). Polices can provide safeguards to ensure continued practice (Flynn & Nolan, 2008), provide an effective lever to implement new practices (Barlett et al., 2005; Harmsen, 2016), or create coherency for practice (Hirsch, Rorrer, Sindelar, Dawson, Heretick, & Jia, 2009). The use of power must be carefully considered when defining, framing, and creating policies to solve problems (Koduah, Agyepong, & van Dijk, 2016). The unequal distribution of power in education could be attributed to unequal educational policies and practices (Darling-Hammond & Sykes, 2003: Liasidou, 2016; Mansfield & Thackik, 2016; Sutcher, Darling-Hammond, & CarverThomas, 2016; Torgerson, Beare, & Spagna, 2016). It is unrealistic and simplistic to expect that the complex problems such as teacher shortages (Boe, Cook, & Sunderland, 2008; Hirsch et al., 2009; Sutcher et al., 2016),

516

S. A. Dawson and V. S. Napper

maternal health, (Koduah et al., 2016), behavior disabilities (Liasidou, 2016), or ethics can be resolved solely through policies. Faulty understanding (Harmsen, 2016) or incorrect analysis (McDonnell & Grubb, 1991) of the problem may create policies that result in unintended or unanticipated consequences. However well-intentioned, policies can and often do conflict with ethics, morality, commitments, or purposes (ILRI, 1995; Richerme, 2016). Inappropriate policies have been an important cause of poor function for many years (ILRI, 1995) and could contribute to ongoing challenges and current ethics discourse. The reported number of reported unethical practices in public education leads to the realization that policies are not sufficient to solve the problem. For example, a report, GAO 14-42, prepared by the U.S. Government Accountability Office for the U.S. Department of Education (2014) stated “nearly 9.6 percent of K-12 students are victims of sexual abuse by school personnel – such as teachers, principals, coaches, and school bus drivers – sometime during their school career (p. 6).” Rather than seeking a policy to solve problems, it is time to look at other avenues to begin to solve a crisis of unethical behaviors.

Mental Models Ethical values driven by external codes of ethics or policies and organized as a mental model in the psyche of the individual based on external values would not include necessary information to process complex, personal situations. According to Johnson-Laird (2005), Charles Sanders Peirce, described as the nineteenth century grandfather of mental models, developed a psychological representation in 1896 of real, hypothetical, or imaginary situations called predicate calculus. Peirce used a diagramic system of icons to explain human reasoning. This concept later became known as a mental model. Johnson-Laird (2005) stated, “mental models represent entities and persons, events and processes, and the operation of complex systems” (p. 187). He also developed a definition of mental models through the principle of iconicity (a visual structure that represents what is known), the principle of possibilities (representation of possibilities of what is known), and the principle of strategic variations (ways to explore what is known). An underling premise of this definition is that the human mind creates models of what is understood and may change that model based on what is learned or experienced in the future. A personal mental model is a philosophical construct that organizes meaning to help clarify or guide the individual’s understanding of complex thoughts to make sense of what is known. Mental models are usually explained or shown through actions or words and may change overtime through the experiences of life. The creation of a mental model precedes sense-making at the individual level because it helps organize what has been experienced in a way that is useful to the individual. This sense-making is a necessarily part of metacognition (Brock, Vert, Kligyte, Waples, Sevier, & Mumford, 2008).

21

The Confluence Effect of Policy, Mental Models, and Ethics on. . .

517

Mental models exist at the level of organizations because organizations are composed of a myriad of individual leaders who are tasked to some how manage the daily functions of their institution. A way of understanding the role of an organizational mental model is to view how the organization functions: Mental models are images, representations, or schemes of how we perceive and understand the world around us. Like all models, mental models are abstractions of reality. The model is less complex than the real world. No matter how well constructed, all models are wrong in some context or time. As the economy evolves, the mental models that were once successful become outdated. Therefore, refreshing the mental model of organization leaders is a chief requirement of a strategic management competency. (Creative Advantage, 2015 Definition of Ethics, para. 1)

In this context, the codes of ethical behavior may inform organizational policy or vice versa as an expression of formalized mental models that underlie planning and legislatures of all types. Codified behavior and organizational ethics should be updated as the environment of work changes at national, state, or local levels.

Models of for Defining Ethical Behavior There exist many models of ethical behavior as defined by codes and policies. Ethics is a system of philosophy that recommends or defends concepts of right or wrong conduct. An example of ethical behaviors within an organizational policy for human work performance can be found in the five core job characteristic model (JCM) as developed by Hackman and Oldham (1975). This model implies a broad extension of the concepts of policy and ethics to guide human behaviors in that it sets institutional standards for actions to be adhered to by employees. According to Taylor (2015), the model identified: (1) skill variety (the range of tasks and needed skills), (2) task identity (tasks with identifiable objectives), and (3) task significance (the impact of the job on other people’s lives), (4) autonomy (authority to act independently) and (5) feedback (information on how the worker is doing in the job and how to improve). (p. 7–8)

The job characteristic model (Heery & Noon, 2017) defines how an individual works: “. . .the JCM suggests that positive outcomes will occur for the individual and the organization [through] high motivation, high-quality performance, high job satisfaction, low absenteeism, and low labour turnover” (para. 1). This model could be considered an expression of ethical values in that it could generate policy guidelines to set institutional standards for actions that are to be adhered to by employees. Specifically, the identification of employee autonomous conduct would have boundaries set by policy. The significance of the task may also generate policy to encourage beneficence and fairness toward other employees or customers. A framework model of educator actions and beliefs (Hutchings, 2014: Rogers et al., 2015) has served as a foundation for the National Association of State

518

S. A. Dawson and V. S. Napper

Directors of Teacher Education and Certification (NASDTEC) Model Code of Ethics for Educators since 2016. The regulatory framework includes laws and policies and guides decision-making by enforcing compliance via external means such as employment, certification, or legal sanctions. The disposition framework encompasses personal values, beliefs, and attitudes that guide decision-making through internal compulsion. The ethical framework is comprised of professional standards that guide decision-making and enforce compliance through professional sanctions. These three frameworks must work together to create equilibrium or ethical misconduct will result (Hutchings, 2014: Rogers et al., 2015). Dawson, Hofland, Lynes, and Squire (2015) completed a comprehensive study to categorize and compile educator ethic laws from all 50 states. The results established the operationalization of ethics as dependent on three constructs: lawful adherence, individual responsibility, and professional standards. These three structures affect ethical actions and become the foundation for formalized professional standards. As also noted by Rogers et al. (2015), ambiguity in one area could create confusion in another construct, heightening the opportunity for unethical decisions; whereas, clarity of understanding in all areas is necessary to promote understanding and expression of ethical action.

Rest’s Decision-Making Model Ethical decision-making processes are important for creating policies arising from ethically valid situations. Rest’s decision-making model argues that the process of decision-making logically occurs in a predictable order (Rest, 1994). This model consists of four stages for individual ethical decision-making and behavior: (1) recognize the moral issue, (2) make a moral judgment, (3) resolve to place moral concerns ahead of other concerns, and (4) act on the moral concerns. Rest indicated the foundation of moral awareness is the ability to interpret a situation as being moral and precedes action. Moral judgment allows the decision maker to decide which course of action is morally correct. Moral intent promotes the desire to prioritize moral values over other values. An additional stage in the development of moral behavior is applying moral intention into a situation (Craft, 2012). Policies can be developed to guide future ethical behavior in a predictable manner to help eliminate the type of ethical situations that initially arose and created the dilemma.

Jones’ Moral Intensity Model Another prominent model for identifying decision-making preceding ethical behaviors is the Jones’ model of moral intensity (Jones, 1991). This model has six components: 1. Magnitude of consequences: The sum of the harm or benefits of the moral act to those involved.

21

The Confluence Effect of Policy, Mental Models, and Ethics on. . .

519

2. Social consensus: The degree of social agreement that a proposed act is good or bad. 3. Probability of effect: The probability that the act will actually take place and will harm or benefit those involved. 4. Temporal immediacy: length of time between now and the intended act. 5. Proximity: The feeling of immediacy to those involved. 6. Concentration of effect: Strength of consequences for those involved. Jones (1991) stated, “Moral intensity is a construct that captures the extent of issue-related moral imperative in a situation. It is multidimensional, and its component parts are characteristics of the moral issue. . .” (p. 372). Because of the multidimensional nature of this model, it would be more difficult to develop a simple policy to govern the six associated behavioral factors. However, an understanding of the six moral intensity factors may ultimately lead to a more complex understanding of the inherent features of ethical decision-making and how people react in complex situations.

Cottone and Claus Theoretical Model Review In a review of decision-making approaches, Cottone and Claus (2000) identified multiple categories. In the category of theoretical models, Cottone and Claus listed several models in use at the time of the review. Hare’s Philosophical Basis of Psychiatric Ethics (1991) was founded on the idea of the rights and duties of the psychiatrist and the greatest good for the greatest number of patients. The Rest model (described above) drew heavily on Kolberg’s theory of moral development as well as research findings. In a 1991 text on psychiatry and law, Gutheil, Bursztajn, Brodsky, and Alexander argued for “decision analysis” as a formal decision-making tool. It “is a step-by-step procedure enabling us to break down a decision into its components, to lay them out in an orderly fashion, and to trace the sequence of events that might follow from choosing one course of action or another” (Cottone & Claus, 2000, p. 276). Berne’s transactional analysis therapy approach was also listed for analysis of values and ethical decision-making. The value of emotional responses of the counselor within the social context of the therapeutic relationship was the foundational theory of Hill, Glaser, and Harden in 1995. This theory emerged from theories of feminism and power. Betan proposed the idea that interpretation of ethical situations brought an understanding of the knowledge and process of knowing. The practice-based models were the largest category in the Cottone and Claus (2000) review. Nine studies were identified as using steps or stages. Many of the practice-based models approached decision-making as sequential steps, whereas others promoted the use of scenarios or dilemmas to promote deeper thinking about the underlying values. Cottone and Claus proposed that practice-based ethical decision-making models do not make decisions for the user of the model but rather provide processes that facilitate the final decision-making process.

520

S. A. Dawson and V. S. Napper

In the empirical findings category, Cottone and Claus (2000) cited two studies. The first study by cited concluded that “interpersonal relations influenced the ethicaldecision-making of graduate counseling student participants when they were asked to reconsider a decision” (p. 277). The second empirical study cited that concluded “decision-making was affected negatively by pressure [of others] but that participants relied little on legal guidelines in making responses to ethical conflict dilemmas” (p. 278). This supported their hypothesis that graduate student decision-making is based on social system theory which is the interrelationships that exist between individuals, groups, and institutions. An important construct that emerged from the Cottone and Claus review was the prominence of individual understanding and actions in the process of decisionmaking. Decision-making processes begin at the level of the individual and reflect individual moral choices but also emerge through their interrelationships with groups and institutions.

Craft Review In a review of ethical decision-making literature, Craft (2012) used the keywords: ethical decision-making, awareness, behavior, judgment, intent, Rest’s model, moral intensity, and literature review. Three dominant mental construct categories emerged from factors identified in the review of 357 studies: individual constructs (16 factors identified), organizational constructs (14 factors identified), and moral intensity constructs (5 factors identified). Within the construct of individual factors, personality was the most studied. Awareness and judgment arising from personality were the most cited dependent variable in those studies, followed by behavior and intent. Gender and individual philosophy or value orientation were also heavily studied with the most studied dependent variables being judgment and intent. Within the construct of organizational factors, rewards and sanctions were the most studied with the primary dependent variables being judgment and intent. The second most studied factor was ethical culture again with the primary dependent variables being judgment and intent. Overall, the dominant mental constructs found were individual factors followed by organizational factors. The least studied construct in Craft’s review was the moral intensity factors identified in the Jones’ model of moral intensity with no studies on how behavior affects the moral intensity of subjects. The lack of studies on moral intensity is interesting because Robin, Reidenbac, and Forrest (1996) argued that perceived importance of an ethical issue influenced behavioral intention and therefore overt behavior. A model that supports the importance of behavioral intent is Rest’s four component model for ethical (1986). It addressed this importance by its inclusion of behavioral components requiring judgment and intended action. In the Craft review, the least studied dependent variable in all construct areas was behavior with only 37 of 357 identified variables. The research movement away from individual behavior is a disconnect to what is happening at an individual level

21

The Confluence Effect of Policy, Mental Models, and Ethics on. . .

521

or an academic fascination with constructivist theory as evidenced by the most researched individual factor being intent. The foundation of action is internal and arises from intended actions. The codes of ethics studies reviewed by Craft indicated codes positively impact ethical decision-making, but two studies (O’Leary & Stewart, 2007; Rottig, Koufteros, & Umphress, 2011) found codes of ethics were insufficient to influence ethical behavior. In the notes on reported findings, the wording of the codes or existence of the codes did not appear to be sufficient to influence the internal model of the individual when they faced with a dilemma. However, organizational culture as well as policies and procedures were identified as having a complex impact on individual awareness of ethics. These findings support the idea that the mental model of ethics held collectively by the organization may influence individual ethical behavior but the individual mental model may not necessarily be consistent or in step with the organizational model.

The Nexus of Policy, Mental Models, and Ethical Behavior The existence of policy to prescribe and guide ethical behavior is found in multiple sources of law and codes. Theory and research attest to the existence of personal mental models that influence ethical decision-making. The alignment or match among external ethical polices, internal decision-making, and personal mental models is not clear; whereas, the connection between the three have been alluded (Dawson et al., 2015; Rogers et al., 2015). There has not been a concerted effort to examine the influence of mental models and decision-making on policy or the possible convergence of all three constructs in the decision-making process. Instead, emerging questions about the factors influencing behaviors and outcomes of individual actions solicit exploration and investigation. These questions include: 1. What are the characteristics of an environment that can lead toward development of a personal mental model of ethics? 2. Do mental models of ethical decision-making over ride policy? 3. Does a uniform policy on ethics create an environment where people align their mental models to conform? 4. How do moral intensity factors affect individual behavior and mental model development? 5. Do taught mental models match the various codes of ethical behavior expounded by state boards of education for students and employees in their organizational system? Are public education environments inherently ethical because of the mental models of teachers being enacted in their classrooms or are ethical values imposed from a different source? Do teachers express their mental model of ethical behavior in their educational environments and if so, how? 6. Do the ethical standards espoused by employers create the foundation for employee mental models of ethical behaviors or are mental models primarily

522

S. A. Dawson and V. S. Napper

based on personal experience? Do personal mental models change based on personal experience gained while working, or during life activities, or both?

Conclusion Although many mental constructs have been identified, there is little understanding of the policy and decision-making factors that either create or influence the development of those mental constructs. An argument can be made that creation of policies or codes of ethics can influence how the individual behaves, but it is not clear why individuals choose to break from established policy or codes. Prior work on the factors influencing behaviors and outcomes of individual actions may be useful in not only training decision-making skills for individuals in professional settings but also to help guide development of policy. Understanding of mental models could help to bridge the disconnect between a person’s knowledge of what is ethically required and a person’s chosen actions.

References Barlett, L., Johnson, L., Lopez, D., Sugarman, E., & Wilson, M. (2005). Teacher induction in the midwest: Illinois, Wisconsin, and Ohio. Santa Cruz: New Teacher Center at University of California, Santa Cruz. Boe, E. E., Cook, L. H., & Sunderland, R. (2008). Teacher turnover: Examining exit attrition, teaching area transfer, and school migration. Exceptional Children, 75(1), 2–31. Brock, M. E., Vert, A., Kligyte, V., Waples, E. P., Sevier, S. T., & Mumford, S. D. (2008). Mental models; An alternative evaluation of a sensemaking approach to ethics instruction. Science & Engineering Ethics, 14(3), 449–472. https://doi.org/10.1007/s11948-008-9076-3. Cottone, R. R., & Claus, R. E. (2000). Ethical decision-making models: A review of the literature. Journal of Counseling & Development, 78(3), 275–283. Craft, J. L. (2012). A review of the empirical ethical decision-making literature: 2004–2011. Journal of Business Ethics, 117, 221–259. https://doi.org/10.1007/s10551-012-1518-9. Creative Advantage. (2015). Resource definition of a mental model of ethics. Retrieved from http:// www.createadvantage.com/glossary/mental-model. Darling-Hammond, L., & Sykes, G. (2003). Wanted: A national teacher supply policy for education: The right way to meet the “highly qualified teacher” challenge. Education Policy Analysis Archives, 11(33). Retrieved from http://epaa.asu.edu/epaa/v11n33/. https://doi.org/10.14507/ etaa.v11n33.2003 Dawson, S., Hofland, B., Lynes, & Squire. (2015). Aligning state ethic laws with CEC ethical principles and practice standards. Paper presentation at the Council for Exceptional Children conference, 8–11 Apr 2015, San Diego. Flynn, G. V., & Nolan, B. (2008). The rise and fall of a successful mentor program: What lessons can be learned? The Clearing House, 81(4), 173–179. Garcia, J., Zeglin, R. J., Matray, S., Froehlich, R., Marable, R., & McGuire-Kuletz, M. (2016). An analysis of the use and policies regarding social media use as a work tool in public rehabilitation. Rehabilitation Research, Policy, and Education, 30, 161–175. Grossman, P., & Thompson, C. (2004). District policy and beginning teachers: A lens on teacher learning. Educational Evaluation & Policy Analysis, 26(4), 281–301. Guba, E. G. (1984). The effect of definitions of policy on the nature and outcomes of policy analysis. Educational Leadership, 42, 63–70.

21

The Confluence Effect of Policy, Mental Models, and Ethics on. . .

523

Hackman, J. R., & Oldham, G. R. (1975). Development of the job diagnostic survey. Journal of Applied Psychology, 60(2), 159–170. Hardman, M. L. (2008). Special education in a 21st century world: A personal view. Paper presented at the Teacher Education and Special Education in Changing Times: Personnel Preparation and Classroom Interventions, Dallas. Harmsen, R. (2016). The impact of applying different metrics in target definitions: Lessons for policy design. Energy Efficiency, 9(4), 951–964. https://doi.org/10.1007/s12053-016-9453-8. 1570646X. Hirsch, E., Rorrer, A., Sindelar, P. T., Dawson, S. A., Heretick, J., & Jia, C. L. (2009). State policies to improve the mentoring of beginning special education teachers. NCIPP Doc. No. PA-1Es. Retrieved May, 2015 from University of Florida, National Center to Inform Policy and Practice in Special Education Professional Development Web site http://www.ncipp.org/reports/pa_1es.pdf. Hutchings, T. (2014). Working towards a model code educator ethics. PPT National Association of State Directors of Teacher Education and Certification. Retrieved from http://www.nasdtec.net/ search/all.asp?bst=code+of+ethics. ILRI (International Livestock Research Institute). (1995). Livestock policy analysis (ILRI Training Manual 2, p. 264). Nairobi: ILRI. ISBN 92-9146-003-6. Job Characteristics Model. (2017). In E. Heery & M. Noon (Eds.), A dictionary of human resource management (2nd rev. ed.). Retrived from http://www.oxfordreference.com/view/10.1093/oi/ authority.20110803100021207. Johnson-Laird, P. N. (2005). Mental models and thought. In Holyoak, K. J. & Morrison, R. G. (Eds.), The Cambridge handbook of thinking and reasoning (pp. 185–208). Cambridge: MA. Cambridge University Press. Jones, T. M. (1991). Ethical decision-making by individuals in organizations: An issue-contingent model. The Academy of Management Review, 16(2), 366–395. Retrieved from http://www.jstor. org/sable/258867. Koduah, A., Agyepong, I. A., & van Dijk, H. (2016). ‘The one with the purse makes policy’: Power, problem definition, framing and maternal health policies and programmes evolution in national level institutionalised policy making processes in Ghana. Social Science & Medicine, 167, 79–87. https://doi.org/10.1016/j.socscimed.2016.08.051. Liasidou, A. (2016). Discourse, power interplays and ‘disordered identities’: An intersectional framework for analysis and policy development. Emotional and Behavioural Difficulties, 21(2), 228–240. https://doi.org/10.1080/13632752.2015.1095009. Mansfield, K. C., & Thackik, S. L. (2016). A critical policy analysis of Texas’ “Closing the gaps 2015”. Education Policy Analysis Archives, 24(3), 33pp. EISSN-1068 2341. McDonnell, L., & Grubb, W. N. (1991). Education and training for work: The policy instruments and the institutions. Santa Monica: The Rand Corporation. O’Leary, C., & Stewart, J. (2007). Governance factors affecting internal auditors’ ethical decisionmaking: An exploratory study. Managerial Auditing Journal, 22(8), 787–808. https://doi.org/ 10.1108/02686900710819643. Policy. (n.d.). In Business Dictionary’s online dictionary. Retrieved from http://www.business dictionary.com/definition/policy.html. Policy. (n.d.). In Merriam-Webster’s online dictionary. Retrieved from https://www.merriam-web ster.com/dictionary/policy. Rest, J. R. (1986). Moral development: Advances in research and theory. New York: Praeger. Rest, J. R. (1994). Background: Theory and research. In J. R. Rest & D. Narváez (Eds.), Moral development in the professions: Psychology and applied ethics (pp. 1–26). Mahwah: Lawrence Erlbaum Associates. Richerme, L. K. (2016). Uncommon commonalities: Cosmopolitan ethics as a framework for music education policy analysis. Arts Education Policy Review, 117(2), 87–95. Robin, D. P., Reidenbac, R. E., & Forrest, P. J. (1996). The perceived importance of an ethical issues as an influence on ethical decision-making of ad managers. Journal of Business Research, 35, 17–28. https://doi.org/10.1016/0148-2963(94)00080-8.

524

S. A. Dawson and V. S. Napper

Rogers, P. S., Fenton, A. M., & Hutchings, T. R. (March 5, 2015). Building the case for a model code of ethics for educators to inform educator preparation. Retrieved from http://c.ymcdn.com/ sites/www.nasdtec.net/resource/collection/7C8FAAA3-65CF-4B6E-B0B4-801DDA91A35F/Build ing_the_Case_for_a_Model_Code_of_Ethics.pdf. Rose, T., Kremen, C., Thrupp, A., Gemmill-Herren, B., Graub, B., & Azzu, N. (2014). Policy analysis paper: Mainstreaming of biodiversity and ecosystem services with a focus on pollination. Rome: United Nations Food and Agriculure Organization. http://www.fao.org/3/ai4242e.pdf. Rottig, D., Koufteros, X., & Umphress, E. (2011). Formal infrastructure and ethical decision making: An empirical investigation and implications for supply management. Decision Sciences Journal, 42(1), 163–204. https://doi.org/10.1111/j.1540-5915.2010.00305.x. Sindelair, P. T., Brownell, M. T., Billingsley, B., Cook, R. J., Jones-Bromenshenkel, M., Huisinga, S., & Mullins, F. (2010). Special education teacher education research: Current status and future directions. Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children, 33(1), 8–24. https://doi.org/ 10.1177/0888406409358593. Sutcher, L., Darling-Hammond, L., & Carver-Thomas, D. (2016). A coming crisis in teaching? Teacher supply, demand, and shortages in the U.S. Palo Alto: Learning Policy Institute. Taylor, G. (2015). Hackman and Oldham’s job characteristics model. Teaching Business & Economics, 19(2), 7–9. Torgerson, C., Beare, P., & Spagna, M. (2016). Quality of educator preparation: How the California State University collaborates to prepare education professionals and refute the claims of policy makers. Journal of Education & Social Policy, 3(1), 36–45. United States Government Accountability Office, Report to Ranking Member, Committee on Education and the Workforce, House of Representatives. (2014). Child welfare: Federal agencies can better support state efforts to prevent and respond to sexual abuse by school personnel. (Publication No. GOA-14-42). Retrieved from https://www.gao.gov/products/GAO-14-42. Zirkel, P. (2014). The law in the special education literature: A brief legal critique. Behavioral Disorders, 39(2), 102–107. Retrieved from http://www.jstor.org/stable/43153587.

Shirley Ann Dawson is an Assistant Professor in the Teacher Education Department at Weber State University. Previously she taught public school for 22 years as an elementary teacher, special education resource teacher for students with disabilities, cluster teacher for students with severe autism, and state gifted and talented coordinator for junior high students. She prepared teacher candidates at University of Utah, Westminster College, Argosy University, and Salt Lake Community College before joining faculty at Weber State University. She always wanted to become a teacher and has spent her career in the greatest profession, teaching. She holds Level 3 Teaching Certification in Elementary Education 1-8, Special Education K-12+, and teaching endorsements in gifted and talented, and mild to moderate disabilities. She earned a Ph.D. from University of Utah in Special Education. Areas of specialization include special education law, and currently she serves as the Teaching Assistant Pathway to Teaching Director, Departmental Honors Chair, Utah State Council for Exceptional Children Action Network Coordinator, and the Student Council for Exceptional Children Advisor. Her research interests include educator ethics, post school training, mentoring, and teacher preparation. She has published and presented in areas of ethic law, special education law, teacher preparation, and teacher mentoring. Vicki S. Napper is a retired Professor of Education from Weber State University, Ogden, UT. Previously she was an Instructional Designer for the Center for Distance Learning at the University of Alaska, Anchorage; Research Analyst for Southwest Research Institute at Hill Air Force Base, UT; Graduate Assistant in the Special Education Department at Utah State University, Logan, UT. She graduated from Utah State University, Logan, UT, with a Ph.D. in Education and emphasis in Instructional Design. Her research emphasis is on identifying factors that contribute

21

The Confluence Effect of Policy, Mental Models, and Ethics on. . .

525

toward physical injury or potential harm of adults and children using computer technology. Another emphasis is development of ethical conduct. She is currently serving on the Association for Educational Communications and Technology (AECT) Professional Ethics Committee and was Chair from 2006 until 2015 and a Member of the AECT Board of Directors from 2012 to 2015. During her employment at Weber State University she also served on committees for creation of Code of Ethics for pre-service teachers. She created and edited the Ethically Speaking column for Tech Trends from 2000 to 2001. She has published multiple articles and made numerous presentation about developing ethical values as well as topics related to the health and safety of persons of all ages who use technology.

Teacher Professional Development for Online Teaching: An Update of Insights Stemming from Contemporary Research

22

Brent Philipsen, Jo Tondeur, Yves Blieck, and Silke Vanslambrouck

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Theoretical Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Why Invest in Teacher Professional Development? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Online and Blended Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Teacher Professional Development for Online and Blended Teaching . . . . . . . . . . . . . . . . . . . . . Purpose of This Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Synthesized Finding 1: Design and Develop a Supportive TPD Program and Environment for Online Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Synthesized Finding 2: Acknowledge the Existing Context Regarding Online Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Synthesized Finding 3: Address Teacher Change Associated with the Transition to Online Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

528 529 529 531 531 532 533 534 536 538 538 540 541

B. Philipsen (*) Educational Sciences, Vrije Universiteit Brussel, Brussels, Belgium e-mail: [email protected] J. Tondeur Vrije Universiteit Brussel, Brussels, Belgium University of Wollongong, Wollongong, Australia e-mail: [email protected] Y. Blieck Vrije Universiteit Brussel, Brussels, Belgium Open University Hasselt, Hasselt, Belgium e-mail: [email protected] S. Vanslambrouck Vrije Universiteit Brussel, Brussels, Belgium e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_167

527

528

B. Philipsen et al.

Synthesized Finding 4: Determine the Overall Goals and Relevance of TPD for Online Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Synthesized Finding 5: Acknowledge Teacher Professional Development Strategies Associated with the Transition to Online Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Synthesized Finding 6: Disseminate Knowledge, Skills, and Attitudes About Online Teaching and Evaluate the Teacher Professional Development Program . . . . . . . . . . . . . . . . . . An Updated Comprehensive Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Limitations and Suggestions for Further Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

542 544 547 547 548 550 550 550

Abstract

Teacher professional development is one of the most examined elements in educational research. In the past decade, there has been a high increase in courses being offered in an online or blended format. However, with the COVID-19 lockdowns the number of teachers who are facing the question on how to teach in an online or blended environment is higher than ever. In this respect, the current chapter aims to shed light on the important components of teacher professional development for online and, by extent, blended teaching and learning. Building on the results of an earlier study that examined these features, this chapter presents an update of those findings and adds recent insights and action recommendations for practice. By adopting a meta-aggregative approach, findings are synthesized and their importance is uncovered and substantiated. Keywords

COVID-19 pandemic · Online and blended teaching · Teacher professional development · Teacher training · Qualitative research

Introduction In the past few years, there was a significant increase in the use of online and blended courses (IGI Global, 2018). In this light, many professional development programs were developed, and much research was done to uncover their peculiarities which elucidated important insights (e.g., Philipsen, Tondeur, McKenney, & Zhu, 2019a; Philipsen, Tondeur, Pynoo, Vanslambrouck, & Zhu, 2019c). However, even in “normal” times (i.e., without a COVID-19 situation), it is not easy to switch to online teaching. Thus, when this switch is being made obligated due to COVID-19, one can already imagine the amount of questions rising. Furthermore, nobody expected the worldwide impact of the COVID-19 situation on education (Education International, 2020). For the future generations, the COVID-19 situation (currently: 2019–2022) refers to a pandemic outbreak of a highly contagious coronavirus that can affect the respiratory and cardiovascular organs (Education International, 2020; World Health Organization, 2020). Due to being highly transmittable many governments worldwide declared a general lockdown forcing schools to close, resulting in

22

Teacher Professional Development for Online Teaching: An Update. . .

529

an unseen mandatory increase in courses being offered online (UNESCO, 2020). It is clear that the current COVID-19 crisis forces teachers to transition abruptly, from a situation in which face-to-face instruction is the norm, towards online education. In a first response to the COVID-19 crisis, we saw that the instructional survival in the majority of cases took place via emergency remote teaching (ERT) or online teaching (OT) (Hodges, Moore, Lockee, Trust, & Bond, 2020). Though similar, ERT and OT are strikingly different. Emergency remote teaching (ERT) can be seen as shifting the instructional delivery to an alternate delivery mode (Hodges et al., 2020). This shift is temporary and due to specific crisis circumstances. OT is different than ERT because in OT all of the instructional components were deliberately designed to be offered online (Hodges et al., 2020). This chapter targets mainly instructional components adhering to OT but nevertheless can be useful for ERT situations. In this respect, it is now more than ever important to understand the important components of teacher professional development (TPD) for online teaching. From the perspective of teachers, empirical research indicates that the process to develop qualitative online courses is time-intensive (Salmon, 2011; Stavredes, 2011; Blieck et al., 2019). Teachers might also overlook what they actually value in (online) teaching and focus on instructional surviving. Therefore, teachers need to take time, take a step back and examine what good teaching is and how good education is formed (Philipsen, 2019). This allows teachers to depart from what they value in order to construct online courses, rather than letting online courses limit what they value. Teachers thus need to have an awareness of how the process of teacher professional development (TPD) for online (and blended) learning/teaching takes form (Philipsen, 2019) and moreover which components deem to be important in TPD for OBL (Philipsen, Tondeur, Pareja Roblin, Vanslambrouck, & Zhu, 2019b).

Theoretical Background Why Invest in Teacher Professional Development? While teacher professional development is approached in various ways (Consuegra & Engels, 2016; Clarke & Hollingsworth, 2002; Desimone & Garet, 2015; Evans, 2014), this chapter adheres to the notion as proposed by Evans (2014): “[teacher] professional development is the process whereby people’s professionalism may be considered to be enhanced, with a degree of permanence that exceeds transitoriness” (p. 188). Extensive research on TPD contributed substantially to our understanding of it and its processes (Evans, 2014). TPD is an important element in striving towards successful learners’ learning and in achieving educational quality. This dual importance becomes apparent from the fact that TPD should always embrace a theory of instruction and a theory of change (van Veen, Zwart, Meirink, & Verloop, 2010). Philipsen (2019) states that “The former is concerned with how the TPD will affect the students’ learning while the latter is concerned with how the TPD will affect a teacher’s teaching practice” (p. 860).

530

B. Philipsen et al.

With respect to students’ learning, one has to keep in mind that teachers can never fully control the learners’ learning. Students most often hold the ultimate decision whether they want to learn something or not. In this view, teachers can “only” do their best in creating environments that stimulate students’ learning. While acknowledging that this is rather a narrow view on learning, one cannot neglect the students’ motivation or willingness to learn. This is because motivation provokes students in making choices, engaging and persisting in education (Dörnye & Ushioda, 2011). Motivation is partly shaped by a mental process called self-efficacy (Dörnye & Ushioda, 2011), which is related to the success of students in online or blended environments (Vanslambrouck, Zhu, Lombaerts, Philipsen, & Tondeur, 2018). While self-efficacy is defined by Bandura (1986) as the belief of students how well they can perform their tasks in a specific context, online learning challenges students’ self-efficacy and, thus, their motivation by stimulating them to learn autonomously. These autonomous online and blended environments require students and teachers alike to self-regulate their motivation and learning (Zimmerman, 2015). Moreover, the study of Vanslambrouck, Zhu, Pynoo, Thomas, Lombaerts, and Tondeur (2019) shows that students in online and blended environments often use various motivational strategies. Students rely on teachers and important others such as peers and family to motivate them to learn and persist (Vanslambrouck et al., 2019). While students normally focus on their goals, such as the exams as a motivation to learn (Vanslambrouck et al., 2019), they currently experience, often together with teachers, insecurity due to changing exam formats. Contextual problems like these could cause motivational problems (Engelschalk, Steuer, & Dresel, 2016), which stresses the importance of paying attention to and supporting the motivation of students. From this follows that it is strikingly difficult to substantiate the assumption that TPD efforts eventually lead to enhanced student learning. Any educational researcher knows that student learning is a complex process influenced by a lot of variables. Nevertheless: A teacher can never know enough about how a student learns, what impedes the student’s learning, and how the teacher’s instruction can increase the student’s learning. Professional development is the only means for teachers to gain such knowledge. Whether students are high, low, or average achievers, they will learn more if their teachers regularly engage in high-quality professional development. (Mizell, 2010, pp. 18)

In following Mizell (2010), the authors of the present chapter would like to elucidate that professional development can be conceptualized in both formal and informal ways. Many teachers professionalize – often unconsciously – by informally talking with their fellow colleagues (Philipsen, 2019). With respect to teacher’s teaching practice, TPD is aimed to install changes in these practices that are required to achieve or maintain high-standing quality education. In this way, TPD is important for the institutional quality improvement process. Like TPD, institutional quality improvement is also ultimately focused on improving students’ learning (Harvey & Green, 2006). Institutions are challenged when they (are coerced to) transition towards online teaching (Jara & Mellar, 2009; Moskal, Dziuban, & Hartman, 2013). Indeed,

22

Teacher Professional Development for Online Teaching: An Update. . .

531

successful adoption, implementation, and improvement of Online and Blended Learning (OBL) requires that the needs of the different stakeholders in institutions are taken into account (Moskal et al., 2013). This is important to ensure that the institution puts reliable and robust infrastructure and sufficient resources in place to support the teachers, which includes TPD, and the students during OBL (Moskal et al., 2013).

Online and Blended Teaching To define online and blended teaching, this chapter separately examines online teaching and blended teaching. Online teaching can be seen as a new and digital way of orchestrating learning activities (Salmon, 2011). In comparison, it seems less easy to define blended teaching, which is also described as mixed, combined, or hybrid teaching (Alammary, Sheard, & Carbone, 2014; Catalano, 2014). After examining various definitions of blended teaching, this chapter chooses to adhere to one proposed by Boelens, Van Laer, De Wever, and Eelen (2015), who describe blended learning – and which in this chapter is extended to blended teaching – as “learning that happens in an instructional context which is characterised by a deliberate combination of online and classroom-based interventions to instigate and support learning” (p. 2). However, it should be noted that during the COVID-19 crisis, most classes shifted to ERT. In what follows, this chapter will focus on online learning and teaching. When the educational measures brought into effect with the COVID-19 crisis, e.g., ERT, are loosened, some teachers will gradually move from ERT to blended learning, online learning or back to completely face-to-face learning and teaching.

Teacher Professional Development for Online and Blended Teaching Having defined and argued the concepts of “teacher professional development” and “online and blended teaching” earlier, another question should not be addressed, that is, whether or not TPD for online teaching should be distinguished from other TPD approaches (Philipsen, 2019). Previous contemporary research shows that many of the important features of TPD align with features of TPD for online and blended teaching (Philipsen et al., 2019a, b, c; Philipsen, 2019). However, there seem to be some subtle yet important differences at play. Tschida, Hodge, and Smith (2016) argue that changing from face-to-face teaching to online teaching entails a change in role from respectively a lecturer to a facilitator. Moreover, Wang, Chen, and Levy (2010) state that moving towards online teaching often entails a psychological change, a change that is far too often neglected in contemporary research. Philipsen et al. (2019a, b, c) and Philipsen (2019) showed that there are indeed differences in TPD for online teaching and other TPD approaches. Yet, they also indicate that these differences are quite subtle but nevertheless should be acknowledged. To illustrate, TPD for online teaching seems to pay more attention to the identity and the role of the teacher. This is due to the fact that it is not necessarily the content of what is being taught that changes, but more the mode of how that content

532

B. Philipsen et al.

is being taught (Philipsen, 2019). Baran and Correia (2014) and Pareja Roblin et al. (2018) argue that in online teaching, teachers’ acceptance, motivation, and participation relates to various levels of teacher support. Support at the teaching level is, for example, further specified into inter alia technology support and pedagogical support: “[. . .], support related to the transformation of faculty members’ content for the online environment is another critical factor in promoting successful online teaching practices” (Baran & Correia, 2014, p. 98). In the process of TPD for online teaching, there is a clear key role for the teachers in creating online courses (Baran, Correia, & Thompson, 2011; Comas-Quinn, 2011; Philipsen, Tondeur & Zhu, 2015). Changes made by these teachers should be acknowledged and appreciated. This applies to general approaches to TPD (Evans, 2014) and to TPD that is related to online or blended learning (Philipsen, 2019). However, as conflicting as it may seem, professional development for teaching online should not be technology-centered (Baran & Correia, 2014). This aligns with what was stated earlier in this chapter that professional development should depart from what is valued in education and teaching and then applied to teaching online. Thus, it seems that professional development for online and blended teaching could be described as value-driven to instigate the best results. In this respect, transformative learning experiences should be encouraged (Baran & Correia, 2014). Having defined both online and blended learning earlier, this chapter now looks into recent data pertaining to whether teachers feel ready to teach online. This chapter draws from the “Readiness for online learning” survey (2020) led by Prof. Jo Tondeur from the Vrije Universiteit Brussel (Belgium). The survey originated from a collaboration between the Vrije Universiteit Brussel (Belgium), the University of Wollongong (Australia), and the University of South-Eastern Norway and the University of Oslo (Norway). To date, more than 1000 respondents were questioned over 40 different countries, and the study is still ongoing. Thus, the results presented are just preliminary. The idea behind the survey was to examine the challenges perceived by teachers who are now “forced” to teach online due to the COVID-19 outbreak. Most of the respondents are employed in the higher and secondary education levels. The first results show that 73.5% of the respondents had to switch/shift to online teaching. However, 59.8% of the respondents did not have any experience with online learning prior to the COVID-19 outbreak (Tondeur, 2020). Thus, 59.8% of the respondents who participated in the survey had to teach online without any prior experience. This is a strikingly high number. And a number that more than ever highlights the importance of understanding the crucial components of TPD for OBL.

Purpose of This Study As a result of the current COVID-19 lockdown, the main reason to transition to online learning is to ensure that learning and educational opportunities are available for learners who cannot/are not allowed to attend face-to-face education. Yet, the results from the “Readiness for online learning” survey (Tondeur, 2020) indicate that a significant number of teachers are neither experienced with nor prepared for online

22

Teacher Professional Development for Online Teaching: An Update. . .

533

Fig. 1 A framework that presents the important components of TPD for OBL. (Reprinted from “Improving teacher professional development for online and blended learning: A systematic metaaggregative review” by B. Philipsen et al., 2019, Education Tech Research Dev 67, 1145–1174. Copyright 2019 by Education Technology Research and Development)

teaching. This highlights a strong need for teacher professional development in online teaching. Building on the important components of TPD for OBL as proposed by Philipsen et al. (2019a, b, c), this chapter integrates more recent studies and examines whether the components should be updated. The review study done by Philipsen et al. (2019a, b, c) integrated studies from 2004 to 2015. This review study will examine the more up-to-date papers starting from 2016 to 2020. These state-ofthe-art papers could entail valuable information which could lead to omitting or adding important components to the framework as proposed by Philipsen et al. (2019a, b, c). As a clarification, in Fig. 1, this chapter presents the framework as originally presented by Philipsen et al. (2019a, b, c). This chapter will make use of a systematic meta-aggregative approach (Joanna Briggs Institute, 2014). A meta-aggregative approach “offers the possibility of providing stronger qualitative evidence and elucidating patterns across the literature than findings from single studies allow [and] entails that the identified categories are aggregated into synthesized findings and formulates action recommendations” (Philipsen et al., 2019a, b, c, p. 1148). Synthesized data portray findings and results that have already occurred (Hannes & Lockwood, 2011). Action recommendations are targeting future actions and are therefore formulated in a different way to inform and guide practice and academic use (Hannes & Lockwood, 2011; Tondeur, van Braak, Ertmer, & Ottenbreit-Leftwich, 2017).

Method For this study, the authors used a systematic meta-aggregative review to analyze the qualitative data (Hannes & Lockwood, 2011; Joanna Briggs Institute, 2014; Lockwood, Munn, & Porrit, 2015). Meta-aggregation is still fairly new in

534

B. Philipsen et al.

educational research to approach qualitative evidence synthesis. However, recent studies have shown that its remarkable procedure leads to concrete and clear action recommendations (e.g., Hannes, Raes, Vangenechten, Heyvaert, & Dochy, 2013; Philipsen et al., 2019a, b, c; Tondeur et al., 2017). The main idea behind metaaggregation is to construct synthesized findings which in turn lead to action recommendations (Hannes, 2010; Hannes & Lockwood, 2011). Important is that each synthesized finding is supported by credible or unequivocal evidence. Only when adequate and unequivocal or credible evidence is found – e.g., textual information in the papers reviewed – data will be deemed appropriate. It shows to a certain extent a similarity with a thematic analysis (Miles & Huberman, 1994; Patton, 2015), yet the difference lies in the creation of action recommendations (Philipsen et al., 2019a, b, c). In line with the previous review study: This study focused solely on qualitative data. This included quotations from trainers, e-learning managers, and teachers, and also the researchers’ conclusions if the authors of this study considered that their—the researchers of the studies reviewed—conclusions were supported by unequivocal or credible empirical evidence. (Philipsen et al., 2019a, b, c, p. 1149)

This leads to defining what is viewed as unequivocal or credible evidence. On the one hand, unequivocal evidence refers to evidence that is directly observable in the reviewed study and that is clearly supported by textual data, to illustrate: I immediately felt a match with Meryl. I do not know if this was because I was sitting next to her, but we started talking right away. You can notice that she is a very open person and that attracts me. What she does in her professional life and what I do, we have some similarities and you do not have that with everyone. (Philipsen et al., 2019a, b, c, p. 52)

The example given shows a teacher that clearly reports a certain experience – in this case, the feeling of connectedness – within a TPD program for OBL. Quotes and data like the example given are clear in their meaning, and thus leave little room for interpretation (Hannes, 2010). On the other hand, credible evidence consists mostly of the researchers’ understanding, but is considered to be plausible and justifiable regarding the specific context (Hannes, 2010). In this review study, the authors only included peer-reviewed articles, excluding books or book chapters. Without questioning the value of books as a good source of information, they do not always include a clear and peer-reviewed method section (Philipsen et al., 2019a, b, c).

Data Collection The strategy entailed the examination of leading high-impact scientific databases by use of key search-items. The current study is based upon a previous study done by Philipsen et al. (2019a, b, c). Three out of five authors of the current – updated –

22

Teacher Professional Development for Online Teaching: An Update. . .

535

chapter were also involved in the study done by Philipsen et al. (2019a, b, c). This implies that for some parts (e.g., search strategy and overall methodology), the authors were already highly accustomed to the procedures inherent to this meta-aggregative approach. However, two new authors were added. They were both not involved in the previous study (Philipsen et al., 2019a, b, c), which implies that they can bring unbiased, new, and fresh insights and ideas into this new chapter. In line with previous research done by (Philipsen et al., 2019a, b, c), the authors of the current chapter kept the same key search-terms, namely: “teacher professional development” or “teacher training” combined with “online learning/teaching,” “blended learning/teaching,” and “e-learning/teaching.” The studies were sought in four main multidisciplinary databases (i.e., EBSCOHost, SCOPUS, Web of Science, and ERIC), and subsequently, references from appropriate studies were checked. Whereas the initial paper (Philipsen et al., 2019a, b, c) focused on studies published from 2004 to 2015, the current study will continue with studies published from the year 2016 to 2020. In this way, it can be examined if the initial results need finetuning (Patton, 2015). Furthermore, it is important to note that the current chapter did not select any studies based on their reported effectiveness or success, the educational level of the teachers, or the geographical location. The different combinations of key-terms, across the four databases, led to 567 possible studies, published between 2016 and 2020. The next step in the data collection was the examination of the studies’ titles and abstracts. This chapter adheres to same inclusion criteria as in the previous study by Philipsen et al. (2019a, b, c): A study was considered relevant when it related to the professional development or training of teachers for online and blended learning or teaching, and when it was considered useful, the full text was sought. Therefore, the first inclusion criterion was that the title and abstract must relate to TPD for OBL and that a full text must be available (Philipsen et al., 2019a, b, c, p. 1150).

Based on this criterion, 49 studies were deemed suitable for further examination. As in the previous review study, it was also noted that this big difference (567 versus 49) is partially due to the interchangeable use of “online TPD” and “TPD for online teaching.” For this study, it is not important how the TPD program was offered (online, blended or face-to-face) in the studies reviewed, as long as the focus of the TPD program was learning to teach (partially or fully) online. Following Philipsen et al. (2019a, b, c), the 49 remaining studies underwent a critical appraisal. This critical appraisal consists of two steps. The first step encompasses the assessment of the studies’ methodology. For this study, which makes use of systematic meta-aggregation, it is important that the studies reviewed have a qualitative or mixed-method research design. Eighteen of the 49 studies were deemed appropriate for further examination. The second part of the critical appraisal entailed the “fit for research.” This is a detailed reading of the 18 studies in order to assess their compatibility with this paper’s aim.

536

B. Philipsen et al.

Table 1 Number of studies identified for each criterion Search strategy 1. Scanning the databases using search terms 2. Examining the titles, abstracts, and full-text possibility 3. Critical appraisal of the articles deemed important

Inclusion criteria 1. Written in English and published between 2016 and 2020 2. Targeting TPD for OBL and availability of a full-text 3.1 Qualitative or mixed method research 3.2 “Fit for research”

Number of articles included 567 49 18 5

Five studies were deemed appropriate. This chapter thus reviewed five additional studies to the original 15 studies reviewed by Philipsen et al. (2019a, b, c). Table 1 shows the number of studies identified for each criterion.

Data Analysis The current chapter used an inductive analysis that aims to identify specific themes emerging from the data (Patton, 2015). The unit of analysis was the textual data in the results and conclusions sections of five studies, which showed to have a qualitative or mixed methods research design. The textual data was only deemed suitable for analysis if it showed unequivocal or credible evidence. This means that the results were either completely transparent in their meaning or deemed credible based on the information at hand. Figure 2 shows the methodological steps in the process of a systematic meta-aggregation. The data analysis in a meta-aggregative review involves generally three distinct steps (Hannes & Lockwood, 2011; Philipsen et al., 2019a, b, c). First, a specific open coding process of the textual data (Miles & Huberman, 1994); second, an aggregation into larger categories; and third and final the creation of synthesized findings and action recommendations. The five studies deemed appropriate for review were imported into NVIVO12, a program that allows to code textual data. The first step in the data analysis was the specific open coding process in combination with the axial open coding process (Miles & Huberman, 1994). The specific open coding process entails the coding process of short parts of textual data. The axial coding process sought out themes in the codes attributed to the textual data. The process was somehow different than in the initial study (i.e., Philipsen et al., 2019a, b, c) because now there was already a framework to guide the data analysis. Thus, in the coding process, this study examined if the data could be allocated to some of the existing categories identified by Philipsen et al. (2019a, b, c) (e.g., peer support) or whether it concerned a new category. If new categories were identified, they were further examined regardless of whether those could be allocated to existing synthesized findings, which form the base for action recommendations. Table 2 presents the five studies reviewed for this research. Both programs focusing on pre- and in-service teachers were deemed suitable.

22

Teacher Professional Development for Online Teaching: An Update. . .

537

Fig. 2 Clarification of methodological steps. (Reprinted from “Improving teacher professional development for online and blended learning: A systematic meta-aggregative review” by B. Philipsen et al., 2019, Education Tech Research Dev 67, 1145–1174. Copyright 2019 by Education Technology Research and Development) Table 2 Selected studies deemed appropriate for review Author(s) Luo, Murray, & Crompton Adnan

Year 2017

Number of participants 48 37

Philipsen, Tondeur, Pynoo, Vanslambrouck, & Zhu Borup & Evmenova Philipsen, Tondeur, McKenny, & Zhu

2019

5

2019

18

2019

20

TPD program Pre-service teachers designing an online Course e-Tutor program that prepares teachers for online – language – teaching Blended TPD program focusing on learning how to teach online

Professional development course designed improve effective online teaching Online professional development program (digital didactics) focusing on teaching online

538

B. Philipsen et al.

Results Further, in this chapter, the authors explain how the results from the current study are complemented with the results from the previous review study (Philipsen et al., 2019a, b, c). For each synthesized finding (SF), the main categories that form the base of that SF will be presented. It will be indicated how many of the five reviewed papers, from the current study, A (X/5), support the identified categories. Next to that, the results of the earlier review study (i.e., Philipsen et al., 2019a, b, c), in which 15 papers between 2004 and 2015 were reviewed, will be integrated as B (X/15). Then we will add the insights from this study with the study from Philipsen et al. (2019a, b, c) as c (X/20). To illustrate, the category “peer support” was identified as an important component of TPD for OBL in all of the five studies reviewed in the current study A (5/5). In Philipsen et al.’s study (2019a, b, c), “peer support” was indicated 13 out of 15 B (13/15) times as an important component of TPD for OBL; hence, in total 18 out of 20 C (18/20) studies support the category of peer support as an important component of TPD for OBL. Thus, A (X/5) refers to the results of the extra research done for this paper, while B (X/15) refers to Philipsen et al. (2019a, b, c). Figure 3 aims to clarify this distinction.

Synthesized Finding 1: Design and Develop a Supportive TPD Program and Environment for Online Teaching The TPD for online teaching should be designed and subsequently developed in a way that it supports teachers throughout the whole process. The initial main categories (Philipsen et al., 2019a, b, c) underpinning this SF were “support and feedback,” “intended process,” and “duration.” Based on the current review study, no adjustments should be made pertaining to these categories. Support and feedback are best given by experts in educational technology A(3/5) B(11/15) C(14/20). Sufficient feedback moments should be planned in advance and need to be communicated to the participants by preference at the start of the TPD trajectory. Next to that, five out of five studies (A(5/5) B(2/15) C(7/20)) reviewed mentioned that providing examples of a possible end product could be beneficial for the participating teachers: “It was helpful to see specifics.” Participant eight added that the multiple student and instructor examples were “very helpful so you got to see

Fig. 3 Clarification of reference

22

Teacher Professional Development for Online Teaching: An Update. . .

539

other ways to do it. Something I would not thought of probably” (Borup & Evmenova, 2019, p. 9). A second main category underpinning the first SF is the intended process A(5/5) B(4/15) C(9/20) of the TPD program. This category encompasses the general outline of the TPD program. Teacher trainers should think about what happens after the training program because teachers often indicate that they would like to continue to make use of the TPD’s opportunities (e.g., the forum, the examples of their peers, the training materials) (Philipsen et al., 2019a, b, c). Next to that, consideration should be given to the systematic and/or cyclic (different iterations) process of the program. A systematic approach could entail that the TPD program strongly focuses on participant learning (i.e., the teachers who follow the TPD) (Nihuka & Voogt, 2012). The last main category underpinning SF 1 is the duration of the TPD program. Overall, this category is formed by two smaller sub-categories, namely, teachers indicating a lack of time A(2/5) B(11/15) C(13/20) and spreading out the training days over a feasible period A(2/5) B(3/15) C(5/20). The TPD program should be organized in a way that it has a good balance between the amount of work that is expected from the participants (i.e., the teachers) and their willingness to invest that time. Hence, keeping in mind that many teachers often indicate having a lack of time, trainers could consider developing a longer trajectory that allows them to spread out the training days. After reviewing the five additional studies, the five extra studies mentioned in Table 2, it can be argued that the first SF does not need to be adjusted, hence this chapter can also retain the first action recommendation: “To better address teachers’ needs, a TPD for OBL should include a supportive environment with regular and just-in-time support and feedback, a well-considered TPD process, and a feasible duration” (Philipsen et al., 2019a, b, c, p. 1156). Figure 4 shows how the different main and subcategories form the first synthesized finding. The quotes used are from both this chapter and the initial review study from 2019 (Philipsen et al., 2019a, b, c).

Fig. 4 Construction of SF 1

540

B. Philipsen et al.

Synthesized Finding 2: Acknowledge the Existing Context Regarding Online Teaching The initial review study (Philipsen et al., 2019a, b, c) identified four main categories underpinning SF 2. Those were: (a) institutional and personal planning, (b) institutional characteristics, (c) already existing programs for TPD related to online (and blended) teaching, and (d) the financial component. Based on the current review study, it is argued that SF 2 can remain largely unchanged in itself. However, the current study would like to make some changes in the categories that form the base for SF 2. Several main categories (and the components they encapsulate) were mentioned remarkably less than other categories and components, which led to revising the categories underpinning SF 2. The first main category at the base of SF 2 is the institutional and personal planning A(3/5) B(5/15) C(8/20). In essence, it is highly favorable that both the institution and the teachers have a clear view of the best possible moments to participate in professional development initiatives. This allows for a better planning of both the institutions’ and teachers’ professional calendars and workloads. The second main category that leads to SF 2 targets the institutional characteristics. The category encompasses three smaller subcategories: the institutional context and culture, the institutional approach and support towards TPD for OBL, and leadership. Surprisingly, none of the five reviewed studies showed sufficiently supported evidence for this category or simply did not mention it. This was somewhat unexpected given the fact that in the initial review study (Philipsen et al., 2019a, b, c), 33% of the reviewed papers clearly mentioned these categories (with the exception of leadership). Acknowledging the institutional characteristics and culture A(0/5) B(5/15) C(5/20) is needed if one aims to contextualize the TPD program, which in turn, together with institutional support A(0/5) B(5/15) C(5/20), can lead to teachers feeling more connected to the TPD program (Philipsen et al., 2019a, b, c) which could positively affect teachers experience with the program. Closely related to institutional support is leadership A(0/5) B(1/15) C(1/20). “Leadership can influence teachers’ dispositions and judgements about online learning (Cowan, 2013), [. . .] a principal plays a vital role in ‘supporting the cultural change’ in the transition to OBL” (Philipsen et al., 2019a, b, c, p. 1157). The third and fourth main categories underpinning SF 2 are “the consideration of existing TPD programs” and the “financial component.” Similar to the other main categories underpinning SF 2, almost none of the reviewed additional papers mentioned the importance of considering existing TPD programs A(1/5) B(1/20) C (2/20) or financial factors A(0/5) B(2/15) C(2/20). Without questioning the importance of the former two categories, it does raise the question of whether they should be explicitly included as main categories. It was noticed earlier that leadership was mentioned in just one out of the 20 papers reviewed. Following this, leadership is now presented as a smaller sub-category and is therefore not explicitly visible in the new comprehensive framework (see Fig. 10). Thus, in light of these results, this chapter will not present “the consideration of existing TPD programs” and the “financial component” as main categories. The consideration of existing TPD

22

Teacher Professional Development for Online Teaching: An Update. . .

541

Fig. 5 Construction of SF 2

programs will now be placed under institutional and personal planning. Checking whether relevant programs for online teaching exist can be seen as part of institutional planning. The financial component will now be placed under institutional characteristics and context as a smaller subcategory. The financial resources available for professional development can definitely be seen as part of contextual and institutional elements. The action recommendation stemming from SF 2 is: To enhance the overall acceptance, a TPD for OBL should acknowledge the existing context of the teachers by taking institutional and personal planning and institutional characteristics into account. Figure 5 shows how synthesized finding 2 is constructed. The quotes are mainly from the initial review study (Philipsen et al., 2019a, b, c).

Synthesized Finding 3: Address Teacher Change Associated with the Transition to Online Teaching At the base of SF 3, there were initially two main components identified, namely, “rethinking roles” and “professional identity and educational beliefs.” These main categories remain unchanged. Yet an extra subcategory is added to the main category of professional identity and educational beliefs, namely, “willingness to change” and “feelings.”

542

B. Philipsen et al.

The first category at the base of SF 3 is “rethinking roles.” This category is directed at both teachers and their students. Teachers need to rethink their own role in an online environment A(3/5) B(5/15) C(8/20) because “teaching with online technologies places instructors in a novel, unusual context involving different roles, responsibilities, and competencies” (Adnan, 2018, p. 105). Next to that, teachers also need to think about the role of their students in the online environment, A(1/5) B(5/15) C(6/20). When students can see each other’s assignments, for example, they can act as peerevaluator. However, prior to assigning students that role, teachers need to think about how they see that role and what it encompasses. In the process of rethinking both roles, teachers often compare online teaching with their former face-to-face classroom experiences, A(4/5) B(3/15) C(7/20). Doing so could inform the teachers about what they value in teaching, which could help them in creating online courses. This leads to the second main category underpinning SF 3: teachers’ professional identities and their educational beliefs, A(2/5) B(4/15) C(6/20). In this respect, Philipsen et al. (2019a, b, c) report that “Wang et al. (2010) argue that the transition to [online teaching] often entails a psychological or mental change related to one’s professional identity and educational beliefs” (p. 1158). Next to that, it also depends on teachers’ willingness to change A(1/5) (1/15) C(2/20) which is a smaller subcategory of “professional identities and educational beliefs.” How teachers see themselves in their profession can reveal a great deal what they value in education and thus inform teacher trainers about the creation of TPD programs for online teaching (Philipsen et al., 2019a, b, c). Next to that, a new subcategory is added to the main category of “professional identities and educational beliefs.” This is based on the insights stemming from the additional five reviewed papers and is conveyed as the subcategory “feelings” A(4/5) B(0/15) C(4/20). Understanding teachers’ feelings during a TPD program for online teaching is important for teacher trainers and program designers because it provides them with valuable information on how the process of professional development is experienced: This means that coming to understand the feelings of teachers or practitioners during a TPD program for online teaching can give one a better idea of how they see themselves as online teachers. This can lead, in turn, to increased knowledge regarding how to design TPD programs for online teaching. (Philipsen et al., 2019a, b, c, p. 55)

The third SF leads to the following action recommendation: to address teacher change associated with the transition to online teaching, a TPD for online teaching should consider the effects of this transition on teachers’ professional identity and educational beliefs and provide them with the opportunity to re-examine their own professional role and the ones attributed to their students. Figure 6 shows how SF 3 is constructed.

Synthesized Finding 4: Determine the Overall Goals and Relevance of TPD for Online Teaching SF 4 contemplates the overall goals and relevance of the TPD for online teaching. Teacher trainers and program designers need to have a clear idea about the objectives

22

Teacher Professional Development for Online Teaching: An Update. . .

543

Fig. 6 Construction of SF 3

and related procedures of the program. Next to that, there should also be attention towards the relevance of the program. The first main category underpinning SF 4 is “goals and procedures.” If one can work towards clearly specified goals A(2/5) B(5/15) C(7/20), then it might become easier for participating teachers to focus on their learning process. However, it is important to keep in mind that throughout the TPD program, personal objectives can change due to the fact that certain experiences or insights alter teachers’ needs and by implication their personal objectives. Strongly related to the goals of the program are the procedures A(2/5) B(5/15) C(7/20) used to achieve them. This might seem similar to the intended process of SF 1, but “the difference is that these procedures are linked to reaching prior stated goals. They represent the steps needed to achieve the objectives of [the program]. The intended process from SF1 is more directed towards the overall outline of [the program]” (Philipsen et al., 2019a, b, c, p. 1160). In the initial review study (Philipsen et al., 2019a, b, c), a distinction was made between three different knowledge domains that should be incorporated as content into the program (i.e., technological, content, and pedagogical domains). In the review study presented here, the authors decided to not include these different knowledge domains as separate components of a TPD for online teaching. Based on the five additional reviewed studies and new insights, the authors of the current study argue that the TPD’s content probably emerges more organically when the personal and program-related goals and objectives are outlined. The second main category at the base of SF 4 is “relevance,” which consists of three smaller subcategories. The first subcategory is that teachers need to see a clear link to their practice, A(5/5) B(11/15) C(16/20). There is preferably a translation to teachers’ everyday practice so that teachers can be given the chance to see how the

544

B. Philipsen et al.

Fig. 7 Construction of SF 4

TPD program can benefit them. This is evidently closely related to teachers seeing a merit for their students A(1/5) B(13/15) C(14/20), the second subcategory. The TPD for online teaching should convince teachers that their students can benefit from changing to an online environment as well: “Using online technologies was expected to ‘increase access to learning opportunities and flexibility for students’ (R27). Instructors believed that students were familiar with technology, and technology in learning environments would stimulate engagement” (Adnan, 2018, p. 101). The last and final subcategory is the fact that the TPD should address a specific need from the participating teachers, A(3/5) B(4/15) C(7/20). Teachers need to have a concrete and specific reason – a need – to professionalize themselves. If not, changes are often less sustained on a long-term basis. The action recommendation stemming from SF 4 is: to tailor for teachers’ needs and increase relevance, a TPD for online teaching should set clear objectives and procedures. Figure 7 shows how SF 4 is constructed.

Synthesized Finding 5: Acknowledge Teacher Professional Development Strategies Associated with the Transition to Online Teaching Teacher professional development strategies can be seen as, deliberately incorporated, specific teacher actions and dispositions used to ameliorate the TPD process.

22

Teacher Professional Development for Online Teaching: An Update. . .

545

After reviewing five additional papers, this study’s authors argue that it is no longer suitable to keep the same organization of the initial main categories. The authors have identified three new main categories, and simply adding them to the comprehensive framework (see Fig. 1) would lead, just as in the textual body of this chapter, to an overload of information. Hence, the authors re-examined all the existing and new main categories underpinning SF 5 and suggest a new way to re-organize these categories. The strategies can be organized into three groups. Behavioral strategies, cognitive strategies, which also include meta-cognitive strategies and finally, social strategies. The classification of the identified categories into these groups is mainly done to provide a clearer overview. The behavioral strategies involve teachers being active A(4/5) B(4/15) C(8/20) and experiential A(5/5) B(11/15) C(16/20) in their professional development process for online teaching. In essence, this means that teachers should be given the chance to actively experience online teaching. To illustrate, in the study of Philipsen et al. (2019a, b, c) participants were given the chance to digitalize their course content. In doing so, they create a useful product as end A(3/5) B(6/15) C(9/20). The cognitive strategies entail the main categories “awareness,” “reflection,” and “confidence and motivation.” Creating awareness is a new main category A(3/5) B(0/15) C(3/20) underpinning SF 5. Awareness relates to teachers realizing that they can improve their current teaching situation. Basically they have to be aware of a different approach to teaching their courses: “Most expected to gain new perspectives about this ‘new phase in education’, [. . .] ‘To keep up with the times, I should be aware and capable of exploiting new methods, techniques and tools’ (R32)” (Adnan, 2018, p.100). Next, being reflective A(4/5) B(8/15) C(12/20) can help teachers to take a step back and re-examine their current position, and it often goes together with opportunities for self-assessment A(1/5) B(4/20) C(5/20): “Looking back at your work every week forces you to take time and evaluate what you have done. When I put my finished assignment aside for some days and then take a second look, I notice my mistakes more easily” (Philipsen et al., 2019a, b, c, p. 245). The last main category within the group of cognitive strategies is being confident and motivated. Teachers need to have confidence A(1/5) B(7/15) C(8/20) in their teaching ability in an online environment if they are to be successful. Closely related to confidence is a teacher’s motivation A(4/5) B(4/15) C(8/20) to teach online. The following quote shows how motivation – together with peer support – affects teachers’ choices and behavior (Philipsen et al., 2019a, b, c, p. 247): The peer coaching that the participants experienced motivated some participants to continue with the programme. One participant indicated: ‘This feedback that I got from [X] helped me realise again why I enrolled myself in this course. I thought about quitting the programme but I should finish what I started’.

As a final group, there are social strategies. Social strategies are all about how peers affect one’s actions and thoughts during and after a TPD program for online teaching. First of all, there is a new category of positive recognition and appreciation A(3/5) B(0/15) C(3/20). If teachers are to sustain changes made (i.e., teaching

546

B. Philipsen et al.

online), they need to receive a sort of validation for those changes. They need to experience that what they are doing differently is being positively appreciated and recognized. This can come from themselves (e.g., noticing that online teaching holds ample benefits) or from colleagues and managers (e.g., a fellow teacher telling that the changes made are really well done) (Philipsen et al., 2019a, b, c). Closely related to positively recognizing and appreciating the changes made is creating a sense of mutual endeavor A(3/5) B(0/15) C(3/20), which is also presented as a new category: “In this study, the participants’ feelings of connectivity were identified as a sense of a mutual endeavour and a recognition that one learns from one another” (Philipsen, 2019, p. 52). Finally, peer support and feedback A(5/5) B(13/15) C(18/20) is one of the most mentioned strategies. Hence, this indicates the importance of peers in a professional development trajectory. Figure 8 shows the construction of synthesized finding 5. This synthesized finding leads to the following action recommendation: To better support teachers in their professional development process, a TPD for OT should encapsulate specific behavioral, cognitive, and social strategies.

Fig. 8 Construction of SF 5

22

Teacher Professional Development for Online Teaching: An Update. . .

547

Fig. 9 Construction of SF 6

Synthesized Finding 6: Disseminate Knowledge, Skills, and Attitudes About Online Teaching and Evaluate the Teacher Professional Development Program The last SF suggests that TPD programs for online teaching should be evaluated A (0/5) B(4/15) C(4/20) and that the knowledge, skills, and attitudes should be cascaded A(0/5) B(3/15) C(3/20) towards teachers’ colleagues. Although both categories are deemed logical and important, they were not mentioned with sufficient unequivocal evidence in the five additional studies reviewed. For now, both categories are still included in this review as important components of TPD for OT. However, in the future, the authors of the current study will further examine the components’ position within the comprehensive framework. Figure 9 shows how SF 6 was constructed in the previous review. SF 6 leads to the following action recommendation: “To extend possibilities for knowledge sharing and tailor further TPD initiatives to existing contexts and needs, a TPD for OBL should encourage the dissemination of knowledge, skills and attitudes about OBL, and perform a continuous evaluation of TPD processes” (Philipsen et al., 2019a, b, c, p. 1163).

An Updated Comprehensive Framework In the remainder of this chapter, an updated version of the comprehensive framework will be presented (Fig. 10). This new version that encapsulates the important components of a TPD program for online teaching and, by an extent, for blended teaching. In blended teaching, teachers need to examine which content they offer online and which content or activities are better offered face-to-face. However, this distinction is not in the scope of this study. Thus, while the components also apply to blended teaching, one needs to be aware that with blended teaching additional elements are at play. “The framework does not represent a linear process or a TPD

548

B. Philipsen et al.

Fig. 10 Updated framework. (Based on Philipsen et al., 2019, Education Tech Research Dev 67, 1145–1174. Copyright original picture: 2019 by Education Technology Research and Development)

[trajectory], but rather provides an overview of important components of TPD for [online and blended teaching]” (Philipsen et al., 2019a, b, c, p. 1165).

Discussion The current study sought out to present an update of important components inherent to TPD for online teaching and, by an extent, blended teaching and emergency remote teaching. We wish to acknowledge the pertaining differences between online teaching, blended teaching, and emergency remote teaching, yet due to the specific focus of this chapter we will remain our focus on blended and online teaching. As discussed earlier, blended teaching requires the additional effort of deciding which content will be delivered online or in a face-to-face setting. However, many of the identified components – if not all – can also be applied to blended environments. In the initial review paper, Philipsen et al. (2019a, b, c) synthesized the results and subsequently constructed action recommendations. These synthesized findings and action recommendations are largely the same as those formulated in the current study. However, this study also brought up some new insights pertaining to the important components of online and blended teaching and was conducted in a unique situation of a global pandemic. While typing this chapter, the COVID-19 pandemic became a global threat for all of us and one that expanded with a velocity that originally many people severely underestimated (Van Audenhove, Mariën, & Philipsen, 2020). Due to the pandemic, many teachers and students were obligated to orchestrate their teaching and learning activities in an online environment (WHO, 2020). A change that not only brought solutions but also painfully showed that teachers and students alike often lacked the skills, knowledge, attitudes, networks, and motivation needed for successful high-quality online education. Researchers and policy-makers should take into account that is demanding to go online additionally

22

Teacher Professional Development for Online Teaching: An Update. . .

549

means that specific groups of teachers and students face digital exclusion (Van Audenhove et al., 2020). As discussed earlier, motivation is needed for students and teachers in making choices, engaging, and persisting in educational processes (Dörnye & Ushioda, 2011). Motivation is partly shaped by the mental process of self-efficacy (Dörnye & Ushioda, 2011), which relates to successfully participating in online or blended learning environments (Vanslambrouck et al., 2018). This strongly suggests that the role of “motivation” cannot be underestimated in the process of TPD (e.g., Borup & Evmenova, 2019). Almost half of all the reviewed studies highlight “motivation” as an important component of online and blended learning or teaching. Yet, one has to realize that being obliged to teach online – due to COVID-19 – can affect severely peoples’ motivation and self-regulation; while one cannot expect adults to all be selfregulatory (King & Lawler, 2003; Knowles, Holton, & Swanson, 2015) even though online and blended learning environments often do require high levels of motivation and self-regulation. (Vanslambrouck, Zhu, Pynoo, Thomas, & Lombaerts, 2019). Thus, contextual or global crisis situations – like COVID-19 – could cause motivational problems (Engelschalk et al., 2016), which stresses the importance of paying attention to, and supporting the motivation of teachers and students if one is aiming for high-quality online education. This insight aligns strongly with the social strategies mentioned in synthesized finding 5. When one focuses on the aimed effects of TPD, one might notice that TPD usually aims to provoke changes in teaching practices in order to achieve or maintain high-standing quality education. In this way, TPD – for online and blended teaching – is important for the institutional quality improvement process. All of the aforementioned components, synthesized findings, and action recommendations are useable in processes or initiatives to enhance teachers’ skills, knowledge, attitudes, or networks (Philipsen et al., 2019a, b, c). Furthermore, it is often questioned what the end result of TPD should be (Evans, 2014). The authors of the current study would like to stress that this often depends on where exactly research is conducted. Evans (2014) highlights that in an American approach, TPD is more often related to enhanced student learning, whereas in Western-European contexts, teacher change is more frequently seen as a justifiable end in itself (Evans, 2014; Philipsen, 2019). Because student learning is such a complex process, affected by many variables among which TPD, the authors of the present study tend to adhere more to the Western-European viewpoint and thus do not make any claims as to how the components, synthesized findings and action recommendations might affect student learning. Nevertheless, TPD is eventually, just like institutional quality improvement, and ultimately focused on improving student learning (Harvey & Green, 1993, 2006). Thus, institutions are challenged in their transition toward online teaching (Jara & Mellar, 2009; Moskal et al., 2013). A successful design, development, implementation, and improvement of online teaching requires that the needs of different stakeholders are taken into account (Moskal et al., 2013). This is important to further ensure that the institution puts sufficient resources in place to support the teachers, which includes TPD for the online teacher (Moskal et al., 2013).

550

B. Philipsen et al.

Limitations and Suggestions for Further Research The current review is based on a previous study by the authors (Philipsen et al., 2019a, b, c), which examined 15 peer-reviewed studies using a meta-aggregative approach (Hannes et al., 2013). The current chapter added five studies to the previously reviewed corpus of research. Hence, in total, this led to 20 reviewed studies. Some might view this as a rather limited amount of studies for a systematic review, but contemporary research shows that this is a sufficient number (Gast, Schildkamp, & van der Veen, 2017; Hwang, Bartlett, Greben, & Hand, 2017; Kurilovas, Dvareckiené, & Jevsikova, 2016; Spolaôr & Vavassori Benitti, 2017) while using a metaaggregative approach (e.g., Tondeur et al., 2017). “The meta-aggregative approach can incorporate any number of studies, and there is no ‘ideal’ number nor cut-off” (Lockwood et al., 2015, p. 183). Furthermore, the authors did not use reported effectiveness of the studies reviewed as a selection criterion. “This would be an interesting topic for further research to see if the same components could be derived” (Philipsen et al., 2019a, b, c, p. 1168). We, as authors, are aware of the fact that there is still a huge wealth of literature on online teaching in the K-12 environment since the early 2000s (e.g., Rice & Dawley, 2007). We invite other researchers to keep looking into the similarities and differences as presented here in this chapter.

Conclusion The current chapter aimed at shedding light on the important components of teacher professional development for online and, by extent, blended learning. Building on the results of the earlier study that examined these features (Philipsen et al., 2019a, b, c), this new study presented an update of these findings and added recent insights and action recommendations for practice. In total, 20 studies were taken into account, and the six synthesized findings are: (1) design and develop a supportive TPD program and environment for online teaching; (2) acknowledge the existing context regarding online teaching; (3) address teacher change associated with the transition to online teaching; (4) determine the overall goals and relevance of TPD for online teaching; (5) acknowledge teacher professional development strategies associated with the transition to online teaching; and (6) disseminate knowledge, skills, and attitudes about online teaching and evaluate the teacher professional development program. These results can inform policy and practice in their decision-making processes pertaining to teacher professional development for online and blended learning or teaching.

References Adnan, M. (2018). Professional development in the transition to online teaching: The voice of entrant online instructors. ReCALL, 30(1), 88–111. https://doi.org/10.1017/S0958344017000106 Alammary, A., Sheard, J., & Carbone, A. (2014). Blended learning in higher education: Three different design approaches. Australasian Journal of Educational Technology, 30(4), 440–454.

22

Teacher Professional Development for Online Teaching: An Update. . .

551

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice- Hall, Inc. Baran, E., & Correia, A. (2014). Professional development framework for online teaching. TechTrends, 58, 95–101. https://doi.org/10.1007/s11528-014-0791 Baran, E., Correia, A., & Thompson, A. (2011). Transforming online teaching practice: Critical analysis of the literature on the roles and competencies of online teachers. Distance Education, 32(3), 421–439. https://doi.org/10.1080/01587919.2011.610293 Blieck, Y., Kauwenberghs, K., Zhu, C., Struyven, K., Pynoo, B., & Depryck, K. (2019). Investigating the relationship between success factors and student participation in online and blended learning in adult education. Journal of Computer Assisted Learning. https://doi.org/10.1111/ jcal.12351 Boelens, R., Van Laer, S., De Wever, B., Eelen, J. (2015). Blended Learning in Adult Education: Towards a Definition of Blended Learning. Adult Learners Online. Borup, J., & Evmenova, A. S. (2019). The effectiveness of professional development in overcoming obstacles to effective online instruction in a college of education. Online Learning, 23(2), 1–20. https://doi.org/10.24059/olj.v23i2.1468 Catalano, H. (2014). The opportunity of blended-learning training programs in adult education – Ascertaining study. Procedia – Social and Behavioral Sciences, 142, 762–768. https://doi.org/ 10.1016/j.sbspro.2014.07.612 Clarke, D., & Hollingsworth, H. (2002). Elaborating a model of teacher professional growth. Teaching and Teacher Education, 18, 947–967. Comas-Quinn, A. (2011). Learning to teach online or learning to become an online teacher: An exploration of teachers’ experiences in a blended learning course. ReCALL, 23(3), 218–232. https://doi.org/10.1017/S0958344011000152 Consuegra, E., & Engels, N. (2016). Effects of professional development on teachers’ gendered feedback patterns, students’ misbehaviour and students’ sense of equity: Results from a one year-quasi-experimental study. British Educational Research Journal, 1–24. https://doi.org/10. 1002/bjer.3238 Cowan, P. (2013). The 4I model for scaffolding the professional development of experienced teachers in the use of virtual learning environments for classroom teaching. Contemporary Issues in Technology and Teacher Education, 13(1), 82–98. Desimone, L., & Garet, M. (2015). Best practices in teachers’ professional development in the United States. Psychology, Society and Education, 7(3), 252–263. Dörnye, Z., & Ushioda, E. (2011). Teaching and researching motivation (2nd ed.). Harlow, UK: Pearson education limited. Education International. (2020, April 15). Education International COVID-19 tracker. Retrieved from Edcuation International: https://www.ei-ie.org/en/detail/16669/education-internationalcovid-19-tracker Engelschalk, T., Steuer, G., & Dresel, M. (2016). Effectiveness of motivational regulation: Dependence on specific motivational problems. Learning and Individual Differences, 52, 72–78. https://doi.org/10.1016/J.lindif.2016.10.011.l Evans, L. (2014). Leadership for professional development: Enhancing our understanding of how teachers develop. Cambridge Journal of Education, 44(2), 179–198. Gast, I., Schildkamp, K., & van der Veen, J. T. (2017). Team-based professional development interventions in higher education: A systematic review. Review of Educational Research, 87(4), 736–767. https://doi.org/10.3102/0034654317704306 Hannes, K. (2010). Het qualitative assessment and review instrument (QARI) ter ondersteuning van synthesen van kwalitatief onderzoek. KWALON, 15(3), 35–44. Hannes, K., & Lockwood, C. (2011). Pragmatism as the philosophical foundation for the Joanna Briggs meta-aggregative approach to qualitative evidence synthesis. Journal of Advanced Nursing, 67(7), 1632–1642. Hannes, K., Raes, E., Vangenechten, K., Heyvaert, M., & Dochy, F. (2013). Experiences from employees with team learning in a vocational learning or work setting: A systematic review of qualitative evidence. Educational Research Review, 10, 116–132. https://doi.org/10.1177/ 0022487108327554

552

B. Philipsen et al.

Harvey, L., & Green, D. (2006). Defining quality. Assessment & Evaluation in Higher Education, 18(1), 9–34. 159. https://doi.org/10.1080/0260293930180102 Hodges, C. B., Moore, S., Lockee, B. B., Trust, T., & Bond, M. A. (2020). The difference between emergency remote teaching and online learning. Educause. https://er.educause.edu/articles/ 2020/3/the-difference-between-emergency-remote-teaching-and-online-learning Hwang, Y.-S., Bartlett, B., Greben, M., & Hand, K. (2017). A systematic review of mindfulness interventions for in-service teachers: A tool to enhance teacher wellbeing and performance. Teaching and Teacher Education, 64, 26–42. https://doi.org/10.1016/j.tate.2017.01.015 IGI Global. (2018). Online course management. In I. S. Reference. Hershey, PA, USA: IGI Global. Jara, M., & Mellar, H. (2009). Factors affecting quality enhancement procedures for e-learning courses. Quality Assurance in Education, 17(3), 220–232. https://doi.org/10.1108/ 09684880910970632 Joanna Briggs Institute. (2014). Joanna Briggs institute reviewers’ manual (2014th ed.). Adelaide, SA: The Joanna Briggs Institute. King, K., & Lawler, P. (2003). Trends and issues in the professional development of teachers of adults. New Directions for Adult and Continuing Education, 98, 5–14. Knowles, M. S., Holton, E. F., & Swanson, R. A. (2015). The adult learner. The definitive classic in adult education and human resource development (8th ed.). New York, NY: Routledge. Kurilovas, E., Dvareckiené, V., & Jevsikova, T. (2016). Augmented reality-based learning systems: Personalisation framework. In J. Novotná & A. Jancarik (Eds.), Proceedings of the 15th European Conference on E-learning (pp. 391–398). Prague, Czech Republic: Charles University, Academic Conferences and Publishing International Limited. Lockwood, C., Munn, Z., & Porrit, K. (2015). Qualitative research synthesis: Methodological guidance for systematic reviewers utilizing meta-aggregation. International Journal of Evidence Based Healthcare, 13(3), 179–187. https://doi.org/10.1097/XEB.0000000000000062 Miles, M., & Huberman, A. M. (1994). An expanded sourcebook: Qualitative data analysis. Thousand Oaks, CA: Sage. Mizell, H. (2010). Why professional development matters. Oxford: Learning Forward. Retrieved from https://learningforward.org/wp-content/uploads/2017/08/professional-developmentmatters.pdf Moskal, P. D., Dziuban, C. D., & Hartman, J. (2013). Blended learning: A dangerous idea? Internet and Higher Education, 18, 15–23. https://doi.org/10.1016/j.iheduc.2012.12.001 Nihuka, K., & Voogt, J. (2012). Collaborative e-learning course design: Impacts on instructors in the Open University of Tanzania. Australasian Journal of Educational Technology, 28(2), 232–248. Pareja Roblin, N., Tondeur, J., Voogt, J., Bruggeman, B., Mathieu, G., & van Braak, J. (2018). Practical considerations informing teachers’ technology integration decisions: The case of tablet PCs. Technology, Pedagogy and Education, 27(2), 165–181. https://doi.org/10.1080/1475939x. 2017.1414714 Patton, M. Q. (2015). Qualitative research & evaluation methods. Thousand Oaks, CA: Sage. Philipsen, B. (2019). A professional development process model for online and blended learning: Introducing digital capital. Contemporary Issues in Technology and Teacher Education, 19(4), 850–867. Philipsen, B., Tondeur, J., McKenney, S., & Zhu, C. (2019a). Supporting teacher reflection during online professional development: A logic modelling approach. Technology, Pedagogy and Education, 28(2), 237–253. https://doi.org/10.1080/1475939X.2019.16020777 Philipsen, B., Tondeur, J., Pareja Roblin, N., Vanslambrouck, S., & Zhu, C. (2019b). Improving teacher professional development for online and blended learning: A systematic metaaggregative review. Educational Technology Research and Development, 67(5), 1145–1174. https://doi.org/10.1007/s11423-019-09645-8 Philipsen, B., Tondeur, J., Pynoo, B., Vanslambrouck, S., & Zhu, C. (2019c). Examining lived experiences in a professional development program for online teaching: A hermeneutic phenomenological approach. Australasian Journal of Educational Technology, 35(5), 46–59. https://doi.org/10.14742/ajet.4469

22

Teacher Professional Development for Online Teaching: An Update. . .

553

Rice, K., & Dawley, L. (2007). Going virtual! The status of professional development for K-12 online teachers. Boise State University. Salmon, G. (2011). E-moderating: The key to teaching and learning online. New York, NY: Routledge. Spolaôr, N., & Vavassori Benitti, F. (2017). Robotics applications grounded in learning theories on tertiary education: A systematic review. Computers & Education, 112, 97–107. https://doi.org/ 10.1016/j.compedu.2017.05.001 Stavredes, T. (2011). Effective online teaching: Foundations and strategies for student success. San Francisco, CA: Jossey-Bass. Tondeur, J. (2020, June 18). Preliminary results of the readiness for online learning survey. Retrieved from https://today.vub.be/nl/artikel/vub-onderzoek-toont-omvang-uitdagingverplicht-online-onderwijs-door-corona-23de-leerkrachten Tondeur, J., van Braak, J., Ertmer, P., & Ottenbreit-Leftwich, A. (2017). Understanding the relationship between teachers’ pedagogical beliefs and technology use in education: A systematic review of qualitative evidence. Educational Technology Research and Development, 65(3), 555–575. Tschida, C., Hodge, E., & Schmidt, S. (2016). Learning to teach online: Negotiating issues ofplatform, pedagogy and professional development. In V. Wang (Ed.), Handbook ofresearch on learning outcomes and opportunities in the digital age (pp. 664–684).Hershey, PA: Information Science Reference. UNESCO. (2020, April 15). COVID-19 education response. Retrieved from https://en.unesco.org/ covid19/educationresponse Van Audenhove, L., Mariën, I., & Philipsen, B. (2020). Digital life after Covid-19: More than just access. (2020, June 25). Media Leanring News. Retrieved from https://news.media-andlearning.eu/type/featured-articles/digital-life-after-covid-19-more-than-just-access/ van Veen, K., Zwart, R. C., Meirink, J. A., & Verloop, N. (2010). Professionele ontwikkeling van leraren: een reviewstudie naar effectieve kenmerken van professionaliseringsinterventies van leraren. ICLON. Vanslambrouck, S., Zhu, C., Lombaerts, K., Philipsen, B., & Tondeur, J. (2018). Students’ motivation and subjective task value of participating in online and blended learning environments. Internet and Higher Education, 36, 33–40. Vanslambrouck, S., Zhu, C., Pynoo, B., Thomas, V., Lombaerts, K., & Tondeur, J. (2019). An in-depth analysis of adult students in blended environments: Do they regulate their learning in an ‘old school’ way? Computers & Education, 128, 75–87. https://doi.org/10.1016/j.compedu. 2018.09.008-z Wang, Y., Chen, N.-S., & Levy, M. (2010). Teacher training in a synchronous cyber face-to-face classroom: Characterizing and supporting the online teachers’ learning process. Computer Assisted Language Learning, 23(4), 277–293. https://doi.org/10.1080/09588221.2010.493523 WHO. (2020, April 15). Coronavirus disease (COVID-19) pandemic. Retrieved from: https://www. who.int/emergencies/diseases/novel-coronavirus-2019 Zimmerman, B. J. (2015). Self-regulated learning: Theories, measures, and outcomes. International Encyclopedia of the Social & Behavioral Sciences, 21, 541–546. (2nd ed.).

Dr. Brent Philipsen received his Ph.D. in 2019 on a dissertation targeting teacher professional development for online and blended learning. He is now working as scientific researcher with iMEC-SMIT at the Vrije Universiteit Brussel. His interests are teacher professional development and digital capital. Prof. Dr. Jo Tondeur is currently working as Assistant Professor at the Vrije Universiteit Brussel (MILO). His research interests are in the field of instructional design and educational innovation. Most of his work focuses on ICT integration in teaching and learning processes.

554

B. Philipsen et al.

Dr. Yves Blieck received his Ph.D. in 2018 with a dissertation that targets quality management and effective institutional continuous quality improvement of online and blended learning. He is an experienced researcher and lecturer with a demonstrated history of working in the secondary, higher, and adult education industry. Dr. Silke Vanslambrouck received her Ph.D. in 2018 with a dissertation that targets adult learners’ self-regulation and motivational values in online and blended learning settings. She now works as a scientific researcher at the Vrije Universiteit Brussel.

Section III Technologies for Learning, Instruction, and Performance

Technologies for Learning, Instruction, and Performance: A Section Introduction

23

Dirk Ifenthaler

Abstract

Technologies for learning, instruction, and performances quickly found their way in the educational arena and thus represent an important asset for every learning environment. This section presents sound theoretical contributions and rigorous empirical findings emphasizing the importance of technologies for learning, instruction, and performance. It is the aim of this section to contribute in a major way to the exceedingly important conversation about educational technology and to provide different perspectives on implications for theory, methodology, and pedagogical practice. Keywords

Technology · Learning · Instruction · Performance · Educational technology · Instructional technology · Digital media

The term technology originated from Greek [tekhnologia] meaning systematic use of an art, craft, or method, hence, an expression of a skill. However, with the nearendless human inventions over the last centuries, technology is mostly referred to as a tool requiring specific skills to operate the technology. With further advances and innovations, technology is no longer only a physical tool; rather it is also referring to algorithms, analytics procedures, software code, or protocols, such as the Internet. Hence, a common understanding of technology includes a double meaning, (a) the

D. Ifenthaler (*) Learning, Design and Technology, University of Mannheim, Mannheim, Germany Curtin University, Perth, WA, Australia e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_130

557

558

D. Ifenthaler

systematic application of knowledge for a specific purpose and (b) a tool or digital code, i.e., hardware and software (Spector, 2013b). Since the invention of the first calculating machine in the early seventeenth century, digital technologies have evolved as powerful applications for all areas of the twenty-first century society and quickly found their way in the educational arena (Ifenthaler, 2010). Like many other technologies, technologies for learning, instruction, and performance change at an accelerated pace (Huang, Spector, & Yang, 2019; Spector, 2013a). These rapid changes have created both opportunities and areas of serious concern for learning, instruction, and performance. As changes in theory and technology happen on an accelerated pace, new terminologies are introduced – often without or limited reference to existing conceptual frameworks, thorough theories, or empirical evidence. As a result, there is an observable growing parallel profusion of different terminologies (Boaden & Lockett, 1991). Accordingly, valid theory building and rigorous empirical testing need to complement the ongoing inventions of technologies (Lowyck, 2014). The technological possibilities for designing effective learning environments are doubtlessly great, but the pedagogically significant question as to how learning, instruction, and performance (assessment) can be supported effectively by technologies is sometimes left out of the picture (Ifenthaler, 2010). Often consolidated as educational technology or instructional technology (Elen & Clarebout, 2012), the discipline needs to build further synergies between instructional design (Branch, 2009) and learning design (Ifenthaler, Gibson, & Dobozy, 2018a) and learning theories for better understanding the potentials and limitations of technologies for learning, instruction, and performance (Lowyck, 2014). For example, as technology-based assessment has been advancing rapidly and its growth is set to accelerate with the availability of automated data collection and analysis (Webb & Ifenthaler, 2018), the ethical dimension needs to be revisited from theory and investigated with advanced empirical testing. Revised theoretical frameworks need to define who has access to which data, where and how long the data will be stored, and which algorithms shall be applied for which purpose (Ifenthaler, Greiff, & Gibson, 2018b). Still, the overarching challenge is to understand the benefits and limitations of educational technologies and make effective use of technologies for supporting learning, instruction, and performance in different contexts. The aspiration of research and practice focusing on technologies for learning, instruction, and performance is to keep in step with the technological innovations and provide valid theories and evidence-based decisions for its pedagogical application. This section presents sound theoretical contributions and rigorous empirical findings emphasizing the importance of technologies for learning, instruction, and performance. It is the aim of this section to contribute in a major way to the exceedingly important conversation about educational technology and to provide different perspectives on implications for theory, methodology, and educational practice.

23

Technologies for Learning, Instruction, and Performance: A Section. . .

559

References Boaden, R., & Lockett, G. (1991). Information technology, information systems and information management: Definition and development. European Journal of Information Systems, 1(1), 23–32. https://doi.org/10.1057/ejis.1991.4 Branch, R. M. (2009). Instructional design: The ADDIE approach. New York, NY: Springer. Elen, J., & Clarebout, G. (2012). Learning technology. In N. M. Seel (Ed.), Encyclopedia of the sciences of learning (Vol. 12, pp. 1980–1981). New York, NY: Springer. Huang, R., Spector, J. M., & Yang, J. (2019). Emerging issues in educational technology. In R. Huang, J. M. Spector, & J. Yang (Eds.), Educational technology. Lecture notes in educational technology (pp. 231–241). Singapore, Singapore: Springer. Ifenthaler, D. (2010). Learning and instruction in the digital age. In J. M. Spector, D. Ifenthaler, P. Isaías, Kinshuk, & D. G. Sampson (Eds.), Learning and instruction in the digital age: Making a difference through cognitive approaches, technology-facilitated collaboration and assessment, and personalized communications (pp. 3–10). New York, NY: Springer. Ifenthaler, D., Gibson, D. C., & Dobozy, E. (2018a). Informing learning design through analytics: Applying network graph analysis. Australasian Journal of Educational Technology, 34(2), 117–132. https://doi.org/10.14742/ajet.3767 Ifenthaler, D., Greiff, S., & Gibson, D. C. (2018b). Making use of data for assessments: Harnessing analytics and data science. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), International handbook of IT in primary and secondary education (2nd ed., pp. 649–663). New York, NY: Springer. Lowyck, J. (2014). Bridging learning theories and technology-enhanced environments: A critical appraisal of its history. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed., pp. 3–20). New York, NY: Springer. Spector, J. M. (2013a). Emerging educational technologies and research directions. Educational Technolgy & Society, 16(2), 21–30. Spector, J. M. (2013b). Trends and research issues in educational technology. The Malaysian Online Journal of Educational Technology, 1(3), 1–9. Webb, M., & Ifenthaler, D. (2018). Assessment as, for and of 21st century learning using information technology: An overview. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), International handbook of IT in primary and secondary education (2nd ed., pp. 1–20). Cham, Switzerland: Springer.

Dirk Ifenthaler is Professor and Chair of Learning, Design and Technology at University of Mannheim, Germany and UNESCO Deputy Chair of Data Science in Higher Education Learning and Teaching at Curtin University, Australia. Dirk’s research focuses on the intersection of cognitive psychology, educational technology, data analytics, and organizational learning. He is the Editor-in-Chief of the Technology, Knowledge and Learning, Editor-in-Chief of Educational Technology & Society, and Senior Editor of Journal of Applied Research in Higher Education.

The Cognitive Theory of Multimedia Learning: The Impact of Social Cues

24

Sara West Bechtold

Contents Introduction: The Cognitive Theory of Multimedia Learning and Personalization . . . . . . . . . . . Reviewing the Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

562 563 571 572

Abstract

Multimedia, and in particular the careful manipulation of spoken text, can be personalized to enhance the satisfaction of lesson content and achievement of learners in multimodal learning environments. The literature in the field of educational psychology supports the theory that learners have a higher level of achievement and a more positive perception of learning content that uses a careful and well-planned approach to dual coding theory. Building upon the cognitive theory of multimedia’s personalization principle, which asserts that students learn better from spoken words and pictures than from words alone, the CTML, and its supporting principles, promotes a design structure for listening comprehension. Personalization has been simplified to give examples of seven crucial social cues I am an Anthropology Professor at Pima Community College, in Tucson Arizona. I am also completing my doctorate of education, specializing in instructional technology and distance education, at Nova Southeastern University in Fort Lauderdale Florida. My major research interest at present is the work of Dr. Richard E. Mayer. It is my desire to further test and apply the personalization principle of Mayer’s Cognitive Theory of Multimedia Learning (CTML), in order to increase learner knowledge retention. Additionally, the CTML could greatly improve instructional design for multimodal learning. It is my belief that learning alone can become a social event, particularly if the CTML theoretical framework is applied when designing instructional messages. I am also a professional musician, wife, and mother to my lovely daughter. It is my hope that multimodal instruction can transform education in a productive and moral, humanistic manner. S. W. Bechtold Pima Community College, Tucson, AZ, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_60

561

562

S. W. Bechtold

that improve learning. Through the application of the CTML’s personalization as critical framework for providing meaningful learning in multimodal learning environments, arguably, learners can experience a social event when learning alone. The learner can experience a social event with a computer that occurs as a result of social presence of the narrator, generated through well-designed instructional modules. When social cues such as politeness are embedded into the narration, a social event can occur between the online agent, which can positively impact the satisfaction of lesson content and achievement of the learner. The literature reviewed suggests that the personalization of narrated instructional messages through social cues from narrated lesson content could become a standard methodology for the design of multimodal lessons. Keywords

Cognitive theory of multimedia learning (CTML) · Social cues · Personalization · Social presence

Introduction: The Cognitive Theory of Multimedia Learning and Personalization There has been a call for research (Clark & Mayer, 2016) that investigates the impact that social cues can have on learner achievement and attitude toward lesson content within an interactive multimodal learning environment. The CTML (Mayer, 2009, 2016) is slowly becoming a popular underlying framework for instructional design in the fields of eLearning, instructional design, and educational psychology. Social cues, or aspects of the voices of the spoken text, can have a positive or negative impact on learner behavior toward lesson content and achievement, depending on the specific combination of gender and ethnicity of the learner and the narrator. The proposed areas of this study are educational psychology, instructional design, personalization, learner perception of lesson content, social presence, cognitive load, working memory capacity, and dual coding theory. The topic to be addressed in this review is that research in the fields of educational psychology and instructional design has indicated a need for instructional designers to personalize narrated instructional messages in multimodal learning in order to improve learners’ attitudes or behavior toward lesson content and learner achievement. There is evidence from research studies that indicate a need for instructional message design with audio and pictures to be carefully designed for learners, in order to promote their achievement and behavior toward lesson content. Instructional message design should be more personalized for individual learners, in order to further engage them and promote a positive increase in student achievement and attitude. The cognitive theory of multimedia learning (CTML) is based upon the concept that “Multimedia instructional messages should be designed in ways that are

24

The Cognitive Theory of Multimedia Learning: The Impact of Social Cues

563

consistent with a scientific research base of empirical evidence” (Mayer, 2009, p. 65). In 2009, the CTML had 12 design principles. The CTML is currently a theory with 18 principles, which are designated as the principles of multimedia design (Clark & Mayer, 2016, pp. 393–395). The personalization principle states that “People learn more deeply from multimedia lessons when learners experience heightened social presence, as when a conversational script with polite wording or learning agents are used” (Clark & Mayer, 2016 p. 465).

Reviewing the Literature The purpose of this review is to investigate the impact that social cues, presented through the application of social cues, can have on learner achievement and attitude toward lesson content within an interactive multimodal learning environment, using the CTML (Mayer, 2009, 2016) as the underlying framework for instructional design. The proposed areas of this study are educational psychology, instructional design, personalization, gender studies, ethnic studies, learner perception of lesson content, social presence, cognitive load, working memory capacity, and dual coding theory. The literature suggests that the narrator’s social cues, which can be examined through a gender and ethnicity experiment, have an impact on learning. However, student achievement and learner attitudes or behavior toward lesson content have not been studied using the CTML with an emphasis on social cues as the treatment of personalization, thus there is a need for further study. The modality principle (Clark & Mayer, 2016, pp. 113–130) has been studied and provided research evidence the most out of the existing principles of the CTML. The modality principle is combined when applying the personalization principle. “When it’s feasible to use audio, there is considerable evidence that presenting words in audio rather than on screen text can result in significant learning gains” (p. 113). The reversed modality effect posits that “Students learn better from text and pictures if the text is presented as written rather than spoken text” (p. 99). Conversely, the modality effect (Schnotz, 2014, p. 98) supports the indication that “students learn better from text and pictures if the text is presented as spoken rather than as written text” because of the resulting avoidance of split visual attention. The split attention effect occurs when learner attention is split between different sensory modes or media (Cierniak, Scheiter, & Gerjets, 2009). Sweller (2005) hypothesizes that working memory is often overloaded (Clark, 2001). The modality effect allows instructional designers to do a better job of designing and presenting graphics and words to help the learner more deeply understand academic content (Ambrose & Lovett, 2014). Ginns, Martin, and Marsh (2013) reviewed research on the impact of personalization or “conversational style” (p. 445) on how people learn. In the results, psychology and instructional design frameworks are refined using meta-analytic methods. Within the literature of the CTML, (2009, 2016) there is a call for further research which should analyze whether there are preferences with regard to the visual or the auditory modality in multimedia learning. This call for research could be answered

564

S. W. Bechtold

in part by examining the impact personalization has on the learner’s behavior toward computer-based learning material and if achievement is improved through using the CTML model designing narrated instructional messages. A review of the literature has revealed a gap in examining how narrative voices, and their social cues embedded within them, might have a positive impact on learner behavior and perception of lesson content in a multimodal learning environment. Research in the fields of educational psychology and instructional design has indicated a need for instructional designers to personalize narrated instructional messages in multimodal learning in order to improve learners’ attitudes or behavior toward lesson content and learner achievement. There is evidence from research studies that indicate a need for instructional message design with audio and pictures to be carefully designed for learners, in order to promote their achievement and behavior toward lesson content. Instructional message design should be more personalized for individual learners, in order to further engage them and promote a positive increase in student achievement and attitude. The (CTML) is based upon the concept that “Multimedia instructional messages should be designed in ways that are consistent with a scientific research base of empirical evidence” (Mayer, 2009, p. 65). The CTML is underpinned by standards which are designated as the principles of multimedia design (Clark & Mayer, 2016, pp. 393–395). The personalization principle states that “People learn more deeply from multimedia lessons when learners experience heightened social presence, as when a conversational script with polite wording or learning agents are used” (Clark & Mayer, 2016, p. 465). For those learners who have the ability to hear, many are not listening. The literature suggests that the narrator’s social cues are a method which can improve learning. Generative cognitive processing and learner engagement have been proven to improve with specially designed verbal and visual messages that have a humanized approach (Mayer, 2009). Social agency theory underpins the CTML and is interwoven into the perspective that learning alone can actually become a social event through best practices in instructional design and the conceptual frameworks that support them (Wei, Chen, & Kinshuk, 2012). While personalization could include various accents and dialects within human languages, social cues such as inflection on keywords in the text that a student must recall later have been proven to make a significant improvement in a learner’s generative processing and retention of information. However, student achievement and learner attitudes or behavior toward lesson content have not been studied using the CTML with an emphasis on social cues as the treatment of personalization, thus there is a need for this study. Earlier research into the modality principle has been critically reflected through the CATL, the cognitive-affective theory of learning with media (Moreno & Mayer, 1999). The literature suggests that the modality effect could be employed to improve the instructional designers so they can do a better job of designing the presentation of graphics and words to help the learner more deeply understand academic content (Ambrose & Lovett, 2014).

24

The Cognitive Theory of Multimedia Learning: The Impact of Social Cues

565

The CTML literature asks for further research which should analyze whether there are preferences with regard to the visual or the auditory modality in multimedia learning. This review aims to further this call for research by examining the impact personalization has on the learner’s behavior toward computer-based learning material and if achievement is improved through using the CTML model designing narrated instructional messages. A review of the literature has revealed a gap in examining how narrative voices with different genders or the ethnicity of the voice might have a positive impact on learner behavior and perception of lesson content in a multimodal learning environment. Narrated instructional messages are being designed with little or no consideration of the need for the personalization of the facilitator’s spoken narrative or lecture. The cognitive theory of multimedia learning (Mayer, 2009) has indicated through experiments and resulting empirical evidence that multimedia instructional messages should be more personalized and learner centered. Mental models are constructed in the mind of the learner through listening comprehension and reading comprehension. The cognitive theory of multimedia learning (Mayer, 2009) is embedded in the search for how people can learn better through the manipulation of auditory and visual information. Paivio’s (1986) dual coding theory takes into consideration the dual channels or the ear (auditory) and the eye (visual register). The channels have a limited capacity to cognize and send information (Schnotz, 2014, p. 83). The CTML provides empirical evidence for predicting under which conditions combinations of text and pictures will be beneficial for learning. This study aims to further this call for research by examining the learner perception of gender and ethnicity in narrated instructional messages. To specify the impact of personalization in this study, narrated instructional messages with the seven major social cues learners may hear, but not fully engage in listening actively, and the gender of the learner and the narrator should be examined. Embedded social cues are associated with specific social cues, which can further humanize the interactive multimodal environment and help improve achievement and attitude. The evidence supports that learners can develop a higher level of positive perception of the learning context and achieve higher cognitive processing of information when pictures and audio are combined. This study focuses on the auditory and not the pictorial channels of processing (Paivio, 1986), except to define what still images are most appropriate, when combined with narrated instructional messages. Embedded social cues within the speakers’ voice act as nonverbal social cues, which humanize the interactive multimodal environment and help create a social learning event. The CTML posits that (2009) personalization of vocal or spoken messages directed at the learner produces evidence that learning alone can become a social event (p. 242). Learners are more likely to view the narrator or instructor as a familiar communicator, and therefore the learner will make more of an effort to decode the instructional messages. Online learning has the potential to foster a realistic social event between the learner and the multimedia narrator, when the narration for the instructional model is designed in a personalized manner. The existing evidence suggests that the personalization principle is central to effective

566

S. W. Bechtold

instructional design. There is a need for the personalization principle to be applied through a gender and ethnic study. The main focus is to determine how varied gender and ethnicity in voices can impact learner behavior, and achievement, to discover whether there are preferences with regard to the auditory modality being the main feature instead of the visual modality in multimedia learning. Schnotz (2014) calls for future research to “analyze whether there are preferences with regard to the visual or the auditory modality in multimedia learning” (p. 97). Personalized narration requires further study into elements of the voice, both verbal (spoken contextual information) and nonverbal (tone, social cues, or inflections on specific words within the context). Mayer, Sobko, and Mautone (2003) noted that the deficiency or lack of evidence in the literature on personalization of the voice in multimodal instruction results from few researchers focusing on this topic. “Additional work is needed to pinpoint which aspects of voice are most important in promoting deep learning” (p. 424). Few studies have been done that measure learner attitude and achievement when taking multimodal instructional lessons with mixed genders (Ozogul, Johnson, Atkinson, & Reisslein, 2013). Learners, instructors, instructional designers, and researchers of instructional design are the intended audience. Additionally, this study may benefit the educational psychology field, as this is a study of how learners learn and provide scientific information about how to improve the relationship between the learner and instructor through the modification of the voice. Educational psychological research forms the theoretical framework of this study, and so it should also apply to educational technologists, psychologists, educators, and courseware developers. Articulate storyline will be used as the application platform within the LMS for the proposed interactive multimodal lesson the subjects will participate in. The information is to be used primarily by educational technologists and instructors. The personalization principle is designed to prime deeper learning through embedding social cues. These social cues are defined by Clark and Mayer (2016, p. 182) as seven crucial criteria. Polite wording rather than direct wording, conversational style rather than formal, vocal quality, tone, pitch, and pauses (silence) are the six examples of central social cues that can improve learning. Politeness has been shown to have the greatest impact on learning of any of the other critical social cues (Mayer, Johnson, Shaw, & Sandhu, 2006). A well-designed instructional multimedia message should utilize both narration and images. It should be noted that the image principle was once deemed an “unprinciple” (Mayer, 2009), as Mayer stated that an added screen image of a facilitator did not improve the learner’s retention of knowledge. Mayer’s call for further research can be answered in part by looking at different populations of subjects and new design approaches, such as embedded social cues in narrated messages, to fill gaps in the literature and further promote the CTML’s personalization principle in multimodal instructional design and delivery. In multimedia learning (2009), the personalization principle was considered as an agent for designing multimedia instruction, which transforms the process of knowledge acquisition for a learner in computer-based instruction into an interactive social experience (Mayer, 2009). The theoretical framework of Mayer addresses the problem of the disconnect between learner and instructor in asynchronous and

24

The Cognitive Theory of Multimedia Learning: The Impact of Social Cues

567

synchronous multimodal learning environments. A recent fourth edition of e-Learning and the Science of Instruction (Clark & Mayer, 2016) builds upon the aforementioned research. The 12 principles that comprise the CTML in 2009 have now grown to 19 (Mayer, 2014, pp. 8–9). The existing research was in 2009 missing verification regarding whether or not social cues stimulate social responses in students, and therefore direct events of social reaction were recommended to “be included in future research” (Mayer, 2009, p. 254). With the new research emerging regarding personalization, there is now an interest in the research community for multimodal personal agents. It can be argued that the personal agent need not be an avatar, but a voice that is designed for the learner. The major findings of research conducted thus far show a general agreement that people learn better with words and pictures (Mayer, 2009, 2016; Schnotz, 2014). This shift in education has transformed society and presents risks and challenges as well as new outlooks on educational psychology and instructional design. Building upon the work of Mayer, this model has a basic underlying question, “How do varied multimedia environments impact the way people learn?”. Research indicates a need for better design practices to improve the effectiveness and interactivity of narrated instructional messages (Clark & Mayer, 2016). One of the major discussions in the field of educational technology currently is that instructional designers can produce improved results via designing the presentation of graphics and spoken words to help the learner more deeply understand academic content (Ambrose & Lovett, 2014). The impact that personalized narrated instructional messages have on learners in interactive multimodal learning environments is key. Online learning has the potential to foster a realistic social event between the learner and the multimediabased instructor through social cues. When the narration for the instructional model is designed in a personalized manner. The existing evidence suggests that the cognitive theory of multimedia learning is central to effective instructional design. The social agency theory (Mayer, 2005a, 2005b, 2009) posits that personalized multimedia messages include social cues that activate the feeling of social presence. The learner feels more connected to the sender (e.g., computer) and puts more effort into understanding the learning content. This leads to deeper cognitive processing and better learning outcomes (Reichelt, Kammerer, Niegemann, & Zander, 2014). “The social agency theory argues that people interpret computers as social partners” (Louwerse, Graesser, Lu, & Mitchell, 2005). Social presence is the degree to which we as individuals perceive another as a real person and any interaction between the two of us as a relationship. Social presence theory suggests that different media convey different degrees of perceived substance to an interaction. The degree of the connection is based on the amount of nonverbal information available to the receiver through any particular channel (auditory or visual). Mayer et al. (2003) conducted two experiments with learners who experienced a “narrated animation” (p. 419) on the formation of lightening. A voice effect was discovered through this study, where the student achieved a better performance on the test and felt more connected to the narrator if the voice was human rather than machine synthesized and had a standard rather than “foreign” accent. The theoretical framework of social agency theory was

568

S. W. Bechtold

applied to look at social cues and how they can “prime the social conversation and schema in learners” (p. 419). Social agency theory underpins the CTML and is interwoven into the perspective that learning alone can actually become a social event through best practices in instructional design and the conceptual frameworks that support them (Wei et al., 2012). While personalization could include various accents and dialects within human languages, social cues such as inflection on keywords in the text that a student must recall later have been proven to make a significant improvement in a learner’s generative processing and retention of information. Studies supporting the framework of CTML have been continually published for over a decade. The CTML was in 2009 a complex theory with 12 principles, “principles of multimedia design” divided into three groups (Mayer, 2009, p. 266). These groups are termed principles for reducing extraneous processing (principles 1 through 5), principles for managing essential processing (principles 6 through 8), and principles for fostering generative processing or knowledge retention (principles 9 through 12). In 2016, the CTML has grown to incorporate 18 principles, which are designated as the principles of multimedia design (Clark & Mayer, 2016, pp. 393–395). “People learn more deeply from multimedia lessons when learners experience heightened social presence, as when a conversational script with polite wording or learning agents are used” (Clark & Mayer, 2016, p. 465). In a 1999 article about the role of modality and contiguity, which supports the cognitive nature of multimedia learning, Moreno and Mayer discussed one of the first experiments conducted in the field of instructional design research. The experiment results produced evidence that “Both experiments revealed a modality effect in which students learned better when verbal input was presented as speech rather than visually as text” (p. 358). Additional research is needed in order to establish the role of unique differences in multimedia learning. One supporting study concludes with the proposition: “When designing multimedia messages, designers should consider the role of social cues such as the speaker’s voice” (Mayer et al., 2003, p. 424). Mayer et al. (2003) conducted two experiments with learners who experienced a “narrated animation” (p. 419) on the formation of lightning. A vocal effect was discovered through this study. The student achieved a better performance on the test and felt more connected to the narrator if the voice was human rather than machine synthesized and had a standard rather than “foreign” accent. The theoretical framework of social agency theory was applied to look at social cues and how they can “prime the social conversation and schema in learners” (p. 419). In recent years researchers have stated the theory that the CTML is the most effective theoretical framework for multimodal learning (Fenesi, Heisz, Savage, Shore, & Kim, 2014). Fenesi et al. (2014) highlight that “Future work should address how multimedia instruction affects long-term learning since assessments of understanding occur days or weeks after multimedia exposure in realistic educational settings” (p. 260). The impact suggested in the literature that personalized narrated instructional messages can have on learners is measurable. Online learning has the potential to foster a realistic social event between the learner and the multimedia-

24

The Cognitive Theory of Multimedia Learning: The Impact of Social Cues

569

based instructor. When the narration for the instructional model is designed in a personalized manner, a social event may occur. A study in 2010 on the personalization principle at a Turkish University was done in Turkish, which adds insight to the possibility of having no personalization by using the third-person singular, with passive voice when necessary. This is not possible in English, which shows how the personalization of narrative design changes culturally depending on the learning environment. “Rigorous research in how self-referencing influences multimedia learning in varied linguistic and cultural contexts will help clarify the extent to which language style matters on a universal ground” (Kartal, 2010, p. 621). Kirschner, Sweller, and Clark (2006) wrote an article contending that minimal guidance during instruction does not work. Mayer’s work in the article is aligned with one of two sides of an argument. Mayer has called for direct instructional guidance. This method of guidance for learners delivers to the learner specific, clear information that also supports an underpinning that is well matched with the cognitive structure of the human brain. The other side of the argument is the minimally guided approach (p. 75), which is thought by leading cognitivist educational researchers such as Sweller, Kirschner, Clark, Moreno, and Kolb to be ineffective. The deficiency or lack of evidence in the literature results from few researchers focusing on this topic. Moreover, those who have employed the CTML are using populations that are not diverse subject groups. “Additional work is needed to pinpoint which aspects of voice are most important in promoting deep learning” (Mayer et al., 2003, p. 424). The keystone work of Sweller’s cognitive load theory (1994) was built into the CTML and has been greatly beneficial to the accuracy and reception of the CTML. Reichelt et al. (2014) took a subject group similar to Mayer’s and expanded it in a replicated study, yet it again used college students as subjects. Therefore, even the most current research still does not cover a diverse enough subject base, and perhaps this is the real issue with the CTML in current research. K-12 students, and also mature learners from diverse populations, should be studied in order to get closer to answering the question of whether vocal personalization can improve learner knowledge retention. Gender attribution in computer-mediated communication is another area of research that has both utilized and somewhat challenged the CTML. A recent study examined gender attribution for online support providers with male, female, or ambiguous usernames, who provided highly person-centered (HPC) or low person-centered (LPC) messages. (Spottswood, Walther, Holmstrom, & Ellison, 2013). The data resulted in showing that a lack of nonverbal cues in computermediated communication suggested “users make inferences and projections about message senders in order to construct impressions of others, which guide their responses to others’ messages” (p. 313). Gender issues in vocal personalization should be examined further over diverse populations of subjects, and this study did not have a diverse enough population. Stiller and Jedlicka’s (2010) study was based upon personalization in multimedia learning design, but it discussed text and pictures as opposed to the voice as the

570

S. W. Bechtold

dominant instructional tool to enhance learning and retention. The study was conducted using 65 tenth grade learners as subjects, from various German grammar schools who “received computerized instructions about the structure of the human eye, containing static pictures and on screen text” (p. 133). Stiller and Jedlicka found that the social agency theory could be used to investigate the effects and benefits of personalized multimedia design, but interaction and its effects might be better studied through cognitive theory. The social agency theory relates to the concept that a single learner might experience a relationship with the narrator, and a higher level of knowledge retention could then occur. Nass and Brave (2005) examined the human-computer relationship through voice interfaces. The focus of the study was the different aspects of voices in computer interfaces and gave recommendations for improvement in design methodology. Nass’ vision was that with careful and deep interface design “Users will not simply talk at and listen to computers. Instead, people and computers will cooperatively speak with one another” (Nass & Brave, 2005). Further research is required into the developments of better-designed interfaces that might allow for such a relationship to take place in computer-based instruction. The phenomena of the “multiple source effect” (Lee & Nass, 2004) relates to human and computer-synthesized speech in computer-based instruction. There was a call for further research to be conducted in this area, giving particular attention to the potential for a theoretical understanding of how users respond to multiple synthetic voices, which creates the multiple source effect. “Theorizing about doubly-disembodied speech, then, will continue to be important for our understanding of new media and advanced simulation technologies” (Lee & Nass, 2004, p. 203). The concept of learning as a social event can be applied to the human-computer relationship if the relationship is founded on direct instructional messages that are carefully personalized. Lee and Nass (2004) posited that the choice of employing a synthesized voice or natural human recorded voice does not make any significant difference in multimodal learning. What really made the impact on learning (Nass & Brave, 2005) were the social cues embedded into the narrated voice of the instructional message. While personalization could include various accents and dialects within human languages, social cues such as inflection on keywords in the text that a student must recall later have been found to make a significant improvement in a learner’s generative processing and retention of information. It can be concluded that the relationship developed between an electronic interface and a student, when narrated speech is carefully designed, can offer an equivalent educational experience for the learner in computer-based learning in comparison with that of a face-to-face lecture (Mayer et al., 2003; Thomas, 2013). Aarntzen (1993) was one of the first to discuss audio in courseware with regard to instructional design. In the article, the redundancy effect (p. 357) is postulated as an essential component of instructional message design for learning. The redundancy effect theory was first introduced by Paivio (1979) and relates to his dual coding theory (1991). The redundancy theory states that when the number of alternative memory codes increases, overall knowledge retention in learner’s increases. Coding an instructional message in more than one style enhances knowledge retention

24

The Cognitive Theory of Multimedia Learning: The Impact of Social Cues

571

(Paivio, 1991). What can be taken away is that both visual and auditory stimuli work better together with stand-alone verbal or visual stimuli. Symbols communicated through images produce different signals than verbal or spoken information. Thus, there should be a simultaneous presentation of visual and auditory channels of processing, where either the image or the spoken message is the predominant stimulus. A recent study (Park, 2015) titled “The Effects of Social Cue Principles on Cognitive Load, Situational Interest, Motivation, and Achievement in Pedagogical Agent Multimedia Learning” has given more insight into the role of social cues in instructional design. “The personalized narration was found to improve learners’ motivation in terms of relevance and confidence whether presented by a pedagogical agent or in on-screen text although no significant differences were found in the recall test and the comprehension test” (p. 211). Evidence from this study shows that spoken narrative of a human voice presented by a “pedagogical agent” was successful in the reduction of the learner’s perceived cognitive load when matched to computer-based text narration and “no narration conditions.” Arguably, a pedagogical agent can be a voice and nothing more. The departure into avatars and simulations with video are perhaps missing the main point, which is that it is the spoken narrative that guides and motivates the learner as the guide. Synthesizing the results of the research gathered thus far offers support for the CTML as the leading theoretical basis to improve the learner’s knowledge retention. It is also clear that there is much more research required in the fields of educational psychology and instructional design, employing the CTML with new and diverse populations of subjects. The focus on computer and human interaction also requires further study, and the work of Nass should be furthered in research environments that support the addition of social cues to human or synthesized narrated speech. Educators can be excellent communicators in the classroom, and now there is a need for them to be equivalently adept at online or computer-based instructional methodologies. Employing the CTML theory in the design of instruction may be applied to develop humanization of instructional messages and can be applied to face-to-face learning situations as well. Instructional designers in the computerbased learning environment can artificially construct a social presence for the learner, using the CTML as the main directive for best practices when designing instructional narrated messages. Instructional designers in the computer-based learning environment can artificially construct a social presence for the learner, using the CTML as the main directive for best practices when designing instructional narrated messages.

Summary To summarize, synthesizing the results of the research presented in this review offers overwhelming support of the CTML as the leading theoretical basis to improve a learner’s social response and engagement. It is also clear that there is much more research required in the fields of educational psychology and instructional design,

572

S. W. Bechtold

employing the CTML with new and diverse populations of subjects. The focus on computer and human interaction also requires further study, and the work of the late Clifford Nass should be furthered in research environments that support the addition of social cues to human or synthesized narrated speech. Educators must be excellent communicators in the classroom, and now there is a need for them to be equivalently adept at online or computer-based instructional methodologies. With the aid of innovative theories like CTML, the humanization of instructional messages can and will improve the learning and retention of the student and can be applied to face-to-face learning situations as well. There is much more research required in the fields of educational psychology and instructional design, employing the CTML with new and diverse populations of subjects. Instructional designers in the computer-based learning environment can artificially construct a social presence for the learner, using the CTML, and particularly the personalization principle as the main directive for best practices when designing instructional narrated messages. Instructional designers in the computer-based learning environment can artificially construct a social presence for the learner, using the CTML’s social cues as the agent of personalization. The CTML as a conceptual framework hold tremendous potential to become a main directive for best practices when designing instructional narrated messages to improve learning satisfaction and achievement. Acknowledgments I wish to acknowledge my mentors Dr. Charles Schlosser, Dr. Richard E. Mayer, the late Dr. Clifford Nass, my daughter Natalie, and Kevin, without whom this effort would not have been possible.

References Aarntzen, D. (1993). Audio in courseware: Design knowledge issues. Educational and Training Technology International, 30(4), 354–356. doi:10.1080/0954730930300406. Ambrose, S., & Lovett, M. (2014). Prior knowledge is more than content: Skills and beliefs also impact learning. In V. A. Benassi, C. E. Overson, & C. M. Hakala (Eds.), Applying science of learning in education: Infusing psychological science into the curriculum. Retrieved from the Society for the Teaching of Psychology web site: http://teachpsych.org/ebooks/asle2014/index.php Baddeley, A. (1992). Working memory. Science, 255, 556–559. Bishop, M. J. (2000). The systematic use of sound in multimedia instruction to enhance learning (Order No. 9980921). Available from ProQuest Dissertations & Theses Full Text. (304626733). Bloom, B. S. (1994). Reflections on the development and use of the taxonomy. In K. J. Rehage, L. W. Anderson, & L. A. Sosniak (Eds.), Bloom’s taxonomy: A forty-year retrospective, Yearbook of the national society for the study of education. Chicago: National Society for the Study of Education. Campbell, D. T., Stanley, J. C., & Gage, N. L. (1963). Experimental and quasi-experimental designs for research (no. 04; Q175, C3.). Boston: Houghton Mifflin. Cantoni, V., Cellario, M., & Porta, M. (2004). Perspectives and challenges in e-learning: Towards natural interaction paradigms. Journal of Visual Languages and Computing, 15(5), 333–345. Clark, R., Mayer, R., Clark, R. C., & Mayer, R. E. (2016). E-learning and the science of instruction: Proven guidelines for consumers. Hoboken, NJ: John Wiley & Sons.

24

The Cognitive Theory of Multimedia Learning: The Impact of Social Cues

573

Creswell, J. W. (2015). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (5th ed.). Upper Saddle River, NJ: Pearson. Fenesi, B., Heisz, J. J., Savage, P. I., Shore, D. I., & Kim, J. A. (2014). Combining best-practice and experimental approaches: Redundancy, images, and misperceptions in multimedia learning. The Journal of Experimental Education, 82(2), 253. Garrison, D. R., & Akyol, Z. (2013). The community of inquiry theoretical framework. In M. G. Moore (Ed.), Handbook of distance education (3rd ed., pp. 104–119). New York: Routledge. Ginns, P., Martin, A. J., & Marsh, H. W. (2013). Designing instructional text in a conversational style: A meta-analysis. Educational Psychology Review, 25(4), 445–472. Gong, L., & Nass, C. (2000, September). Speech interfaces from an evolutionary perspective. Communications of the ACM, 43(9), 36. Gong, L., & Nass, C. (2007). When a talking-face computer agent is half-human and halfhumanoid: Human identity and consistency preference. Human Communication Research, 33 (2), 163–193. Gunawardena, C. N. (1995). Social presence theory and implications for interaction and collaborative learning in computer conferences. International Journal of Educational Telecommunications, 1(2), 147–166. Huck, S. W. (2012). Reading statistics and research (6th ed.). Boston: Pearson Education. Kartal, G. (2010). Does language matter in multimedia learning? Personalization principle revisited. Journal of Educational Psychology, 102(3), 615. Kreijns, K., Van Acker, F., Vermeulan, M., & Van Buuren, H. (2014). Community of inquiry: Social presence revisited. E-Learning and Digital Media, 11(1), 5–18. https://doi.org/10.2304/ elea.2014.11.1.5. Lee, K. M., & Nass, C. (2004). The multiple source effect and synthesized speech. Human Communication Research, 30, 182–207. https://doi.org/10.1111/j.1468-2958.2004.tb00730.x. Lowenthal, P. R. (2009). The evolution and influence of social presence theory on online learning. In T. T. Kidd (Ed.), Online education and adult learning: New frontiers for teaching practices (pp. 124–139). Hershey, PA: IGI Global. Mayer, R. E. (2005a). The Cambridge handbook of multimedia learning. Cambridge, UK: Cambridge University Press. Mayer, R. E. (2005b). Cognitive theory of multimedia learning. In R. Mayer (Ed.), Cambridge handbook of multimedia learning (pp. 31–48). New York: Cambridge University Press. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press. Mayer, R. E. (2014). The Cambridge handbook of multimedia learning. Cambridge, UK: Cambridge University Press. Mayer, R. E., & R. Moreno (1998). A cognitive theory of multimedia learning: Implications for design principles. Retrieved from http://www.unm.edu/~moreno/PDFS/chi.pdf Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38, 43–52. Mayer, R. E., Fennell, S., Farmer, L., & Campbell, J. (2004). A personalization effect in multimedia learning: Students learn better when words are in conversational style rather than formal style. Journal of Educational Psychology, 96(2), 389–395. Mayer, R. E., Sobko, K., & Mautone, P. D. (2003). Social cues in multimedia learning: Role of speaker’s voice. Journal of Educational Psychology, 95(2), 419–425. https://doi.org/10.1037/ 0022-0663.95.2.419. Mayer, R. E. (2014). The Cambridge handbook of multimedia learning (2nd ed.). New York: Cambridge University Press. Molenda, M., & Pershing, J. (2008). Improving performance. In A. Januszewski & M. Molenda (Eds.), Educational technology: A definition with commentary (pp. 49–80). New York: Routledge. Moreno, R. (2005). Instructional technology: Promise and pitfalls. In L. PytlikZillig, M. Bodvarsson, & R. Bruning (Eds.), Technology-based education: Bringing researchers and practitioners together (pp. 1–19). Greenwich, CT: Information Age Publishing.

574

S. W. Bechtold

Moreno, R., & Mayer, R. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91, 358–368. Moreno, R., & Mayer, R. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309–326. Morgan, S. E., Reichert, T., & Harrison, T. R. (2002). From numbers to words: Reporting statistical results for the social sciences. Boston: Allyn and Bacon. Nass, C., & Brave, S. (2005). Wired for speech: How voice activates and advances the humancomputer relationship. Cambridge, MA: MIT Press. Nass, C., & Yen, C. (2010). The man who lied to his laptop: What machines teach us about human relationships. New York: Current. Paivio, A. (1986). Mental representations: A dual coding approach. Oxford, UK: Oxford University Press. Reichelt, M., Kammerer, F., Niegemann, H. M., & Zander, S. (2014). Talk to me personally: Personalization of language style in computer-based learning. Computers in Human Behavior, 35, 199–210. https://doi.org/10.1016/j.chb.2014.03.005. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont, CA: Wadsworth Cengage Learning. Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunications. London: Wiley. Sorden, S. (2012). The cognitive theory of multimedia learning. In B. Irby, G. Brown, R. LaraAlecio, & S. Jackson (Eds.), Handbook of educational theories (1st ed., pp. 155–168). Charlotte, NC: Information Age. Spottswood, E. L., Walther, J. B., Holmstrom, A. J., & Ellison, N. B. (2013). Person-centered emotional support and gender attributions in computer-mediated communication. Human Communication Research, 39, 295–316. https://doi.org/10.1111/hcre.12006. Stiller, K. D., & Jedlicka, R. (2010). A kind of expertise reversal effect: Personalization effect can depend on domain-specific prior knowledge. Australasian Journal of Educational Technology, 26(1), 133–149. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312. Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 19–30). New York: Cambridge University Press. Wei, C., Chen, N., & Kinshuk. (2012). A model for social presence in online classrooms. Educational Technology Research & Development, 60(3), 529–545. https://doi.org/10.1007/ s11423-012-9234-9.

An Instrumentalized Framework for Supporting Learners’ Self-Regulation in Blended Learning Environments

25

Stijn Van Laer and Jan Elen

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Blended Learning and Blended Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Need for Systematic Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Self-regulation and Its Influencing Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conceptual Foundations of the Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Authenticity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Personalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learner Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scaffolding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cues for Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cues for Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Phase 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Phase 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Phase 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Conceptual Framework and Current Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Instrumentalized Framework and Designing Blended Learning . . . . . . . . . . . . . . . . . . . . . . . Limitations and Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 1: Visual Representation of the Instrument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

576 577 578 579 582 582 582 585 587 588 589 590 592 593 594 594 595 598 601 603 604 605 606 607 608

S. Van Laer (*) · J. Elen Centre for Instructional Psychology and Technology, KU Leuven, Leuven, Belgium e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_121

575

576

S. Van Laer and J. Elen

Abstract

The premise in instructional design theory is that, in order to identify and target instructional shortcomings, designers should conduct a thorough analysis of the various elements involved in the instructional process. This is also the case for technology-rich means of instruction such as online and blended learning. Nevertheless it often seems that insufficient attention is directed to the description of learning environments when redesigning them. In the case of blended learning, studies suggest, for example, that this type of learning often challenges learners’ self-regulation. Existing research provides little insight into how blended environments can support learners’ self-regulation. These observations are problematic since such insights are needed for effective (re)designs. Therefore, the aim of this chapter is to present an instrumentalized framework which can be used to describe and thus characterize support for learners’ self-regulation in blended learning environments as a basis for investigations and empirical trials to uncover effective redesigns and guidelines. The instrumentalized framework elaborates on seven attributes of learning environments that may be expected to support selfregulation according to the current literature on self-regulation. The framework is operationalized in an instrument that facilitates the description of any blended learning environment from the perspective of learners’ self-regulation support. We demonstrate the validity and reliability of the instrument in two empirical research cycles which included six blended learning environments. The instrument can be used to describe and characterize environments as a starting point for their redesign and, consequently, improve support for self-regulation. Keywords

Self-regulation · Instructional design · Blended learning · Descriptive instrument · Design guidelines

Introduction In recent decades, interest in the use of blended forms of learning has increased considerably. This type of learning happens in an instructional context which is characterized by the deliberate combination of online- and classroom-based interventions to instigate and support learning (Boelens, Laer, De Wever, & Elen, 2015). Recent research on the effectiveness of blended learning has led to a proliferation of studies that emphasize the importance of learners’ self-regulation in such environments. Results show, for instance, that if learners are to succeed in blended learning environments, a greater amount of self-regulation is often required (e.g., Kuo, Walker, Schroder, & Belland, 2014). This finding raises questions about how blended environments can be (re)designed to overcome this issue and how learners’ self-regulation can be supported in such environments.

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

577

In response to these questions, design guidelines have been derived from syntheses of research on particular elements of self-regulation, such as monitoring, self-efficacy, and metacognition. Such guidelines suggest embedding self-regulation training into instruction by, for example, modeling self-regulation, using cognitive apprenticeships, and providing attributional feedback to identify appropriate strategies (e.g., Ley & Young, 2001; Perry & Drummond, 2002; Perry, Nordby, & VandeKamp, 2003). Although these studies seem to agree on the importance of selfregulation for learning and provide guidelines for embedding it in learning environments, they are rarely generalizable nor have they been operationalized as (validated) instruments for describing learning environments. Consequently, no frameworks or systems are available (let alone instruments) for describing support for learners’ self-regulation in blended learning environments. This observation is problematic since without such frameworks and instruments, the systematic description and (re)design of a (blended) learning environment are almost impossible. The aim of this chapter is therefore to present an instrumentalized framework for the systematic description of support for learners’ self-regulation in blended learning environments. This instrumentalized framework consists of a conceptual framework and an instrument, validated here in two empirical research cycles. The conceptual framework originates from an analysis of the literature (by Van Laer and Elen (2016)) on support for self-regulation and provides seven attributes that characterize blended learning environments’ potential support for learners’ self-regulation. The seven attributes in the conceptual framework are authenticity, personalization, learner control, scaffolding, interaction, cues for reflection, and cues for calibration. The conceptual framework and the instrument constructed around it can assist in the description and characterization of blended learning environments but do not propose empirical guidelines on (re)design. The aim of the conceptual framework and instrument is to facilitate research and practice by taking a systematic approach to investigating and supporting learners’ self-regulation in blended learning environments. Such an approach can serve as a starting point for redesign and, consequently, improved support for self-regulation. Before elaborating on the conceptual foundations of the framework presented by Van Laer and Elen (2016), we discuss the blended learning concept and its challenges, explain the need for systematic descriptions in environment design, and elaborate on how self-regulation is developed and can potentially be supported.

Blended Learning and Blended Learning Environments Blended learning is a popular concept. A common aspect in many definitions of blended learning is that it combines online and face-to-face learning. Hence, it is assumed to combine the advantages of both modes of delivery (Graham, Henrie, & Gibbons, 2014). In line with this idea, blended learning in this study is defined as learning happening in an instructional context which is characterized by a deliberate combination of online- and classroom-based interventions to instigate and support learning (Boelens et al., 2015). Nonetheless, the relation of blended learning to

578

S. Van Laer and J. Elen

concepts such as the flipped classroom and hybrid learning is unclear, and the instantiation of the blend remains vague (Oliver & Trigwell, 2005). Despite this, blended learning as a notion is widely used in higher and adult education, K-12 education, and corporate training (Bonk, 2017). Over the years, blended learning has been the focus of many research studies. The majority of studies on blended learning have focused either on comparing blended and face-to-face learning or on the characteristics that learners need to thrive in such environments (Deschacht & Goeman, 2015). With regard to the latter, research has identified that learners with high amounts of verbal ability and self-efficacy and learners with high self-regulatory capabilities often perform better in blended learning environments compared to learners who lack these capabilities (Kizilcec, PerezSanagustin, & Maldonado, 2017). Despite the importance of these types of research, hardly any research has discussed how to propel the quest for empirical evidence to support the design of blended learning environments in which less “capable” learners can also find success.

The Need for Systematic Descriptions To be able to design appropriate solutions for educational problems, stakeholders supporting learners’ self-regulation are advised to use a systematic approach. Instructional design, as the design of learning environments is often referred to, emphasizes a systematic approach and is concerned with understanding, improving, and applying methods of instruction to shape learning environments (Reigeluth, 2013). It is the process of selecting and configuring methods for bringing about the desired changes in learners’ behavior. The results of instructional design are often a blueprint for the development of the actual course (van Merriënboer & Kirschner, 2017). This blueprint prescribes which methods, and in which configuration, can be used in a specific context to support learners in their attempts to achieve instructional goals. To be able to advance in the (re)design of learning environments, it is necessary to evaluate the effectiveness of current instructional designs. There are two main reasons for this. The first one is to describe the instructional conditions under which learning is supported. The second one is to make systematic adaptations to these conditions to strive toward increased learning. Although the necessity of considering the learning environment in the design of instruction is widely recognized by instructional design theory, no significant attention seems to be given to describing them and using these descriptions to formulate questions and a context for verification and hypothesis testing (Shavelson, Phillips, Towne, & Feuer, 2003). This finding might be explained by the observation that most models approach the designs of learning environments as blank canvases to be drawn on. This is rarely the case in practice, however. Without a system to describe and characterize the learning environment, instructional design theory produces theoretically sound but practically unusable results, meaning no practical (re)design is possible.

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

579

Self-regulation and Its Influencing Conditions By definition, effective learners are self-regulating ones (Butler, 1998). Selfregulation is the process that unfolds when learners use metacognitive skills, in a particular context, to achieve goals both internal and external to themselves. Many models of self-regulation include or are constructed around four main cyclic stages: (1) task identification, (2) goal-setting and planning, (3) enacting, and (4) adaptation (for an overview see: Puustinen & Pulkkinen, 2001). When learners encounter a new task for the first time, they try to (1) identify or categorize it. While doing this, they develop perceptions of the task concerned. Based on these perceptions, learners (2) set goals and plan how to achieve them. Once goals are set, learners use their (3) metacognitive knowledge and skills and act to achieve the goals set. Finally, when the learners are confronted with their actual achievements (e.g., through summative feedback), self-regulating learners may (4) adapt their studying techniques, keeping the freshly acquired experiences and their future needs in mind. Each of these stages of self-regulation is influenced by conditions within and external to the learner (e.g., Winne & Hadwin, 2013).

Variables Within the Learner Research identified three major sets of variables in relation to differences in selfregulation: cognitive, metacognitive, and motivational ones. With respect to cognitive variables (e.g., Zimmerman & Schunk, 2006), two frequently investigated concepts are (a) learners’ intelligence and (b) learners’ prior domain knowledge. With regard to the latter, research showed that learners who had more prior domain knowledge used fewer information sources and focused more on sources related to appropriate strategies to regulate one’s learning toward achieving the desired learning outcomes (Song, Kalet, & Plass, 2016). Intelligence proved to be positively related to metacognitive skillfulness, with learners with higher scores on intelligence being better able to select the desired metacognitive skills and thus self-regulate toward the desired learning outcome (Veenman, Elshout, & Meijer, 1997). A second set of variables relates to metacognition (e.g., Borkowski, Carr, Rellinger, & Pressley, 1990). Two metacognitive domains related to learners’ selfregulation can be extracted from literature. On the one hand, there is metacognitive knowledge, which is the information needed to be able to select appropriate metacognitive skills. On the other hand, there are the metacognitive skills themselves, which reflect learners’ ability to make actual changes to their own behavior. Results show that learners with a wide array of metacognitive skills use more varied strategies while studying compared to learners who are less skilled (Pintrich, 2002). According to the researcher, this difference can be attributed to a more skillful analysis of the situation which may result in the selection of more appropriate strategies. The last set of variables relates to motivation (e.g., Schraw, Crippen, & Hartley, 2006). As self-regulation within educational psychology refers to settings in which goals are set not by learners alone, but also by formal institutions (and in ideal scenarios in mutual agreement), motivation is often seen in the light of learners’

580

S. Van Laer and J. Elen

goal-orientation. Learners’ goal-orientation encompasses different ways learners deal with the goals they receive and eventually appropriate. Learners can approach or avoid either performance or mastery. When learners have a mastery-approach orientation, they internalize the goal as their own and are motivated to master the goal. When learners have a performance-avoidance orientation, however, they attempt to avoid appearing incompetent compared to others. One finding which illustrates the impact of goal-orientation on self-regulation is that learners who want to master a task consult materials outside of the course content, whereas performance-avoidance learners will stick to the task more rigidly and regulate their learning toward the desired outcome (Pintrich, 2002). Similar findings were retrieved in relation to Deci and Ryan’s (2010) notion of internal and external motivation.

Variables External to the Learner Different stages, dimensions, and processes of self-regulation may be influenced by specific instructional interventions (Ifenthaler, 2012). As pointed out by Ley and Young (2001), several self-regulation interventions have been tailored to specific content, learners, or media. Interventions have been suggested for writing, reading comprehension, and mathematics (e.g., Schunk, 1998). Others have incorporated support for self-regulation into college learning-to-learn courses or in computermediated instruction (e.g., Winne & Hadwin, 2013). The literature contains only a limited number of studies that have focused on support for self-regulation in blended learning environments (Kassab, Al-Shafei, Salem, & Otoom, 2015). Some approaches have been directed toward specific populations such as children, adolescents, and learning-disabled learners (Butler, 1998). Although there is a substantial amount of research available that describes ways to support learners’ self-regulation, there are several remaining issues that make the practical application of these guidelines impossible. First, while much research does consider self-regulation as an inherent part of learning, research that takes this perspective and presents concrete design guidelines is scarce. The guidelines formulated often see self-regulation as a specific goal (to design for) instead of as an inherent attribute of learning (Zimmerman & Schunk, 2006). This results in descriptions of interventions that focus on increasing specific elements of self-regulation (e.g., task definition, monitoring, etc.). Only a few studies attempted to combine findings from different backgrounds into a set of guidelines or principles toward a conceptual framework or emphasized the inconclusiveness of guidelines for learners with particular characteristics. Among those who attempted to come up with guidelines to support self-regulation were Ley and Young (2001), Perry and Drummond (2002), and Perry et al. (2003). Ley and Young (2001) proposed guidelines to design learning environments that support self-regulation: (a) To help learners prepare and structure an effective learning environment (b) To organize instruction and activities to facilitate cognitive and metacognitive processes

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

581

(c) To use instructional goals and feedback to present the learner with monitoring opportunities (d) To provide learners with continuous evaluation information and opportunities to self-evaluate With regard to the conceptualization of self-regulation used in this chapter, the guidelines formulated by Ley and Young (2001) seem to relate most closely to the enacting and modifying phases of self-regulation (phases 3 and 4, respectively). No indications are provided about how to support learners in identifying the task at hand or in setting appropriate goals and planning to achieve them, however. Perry and Drummond (2002) and Perry et al. (2003) approached support for selfregulation in a broader, more general fashion. They suggested that: (a) Learners and instructors should function as a community of learners. (b) Learners and instructors should be engaged in complex, cognitively demanding activities. (c) Increasingly, learners should take control of learning by making choices, controlling challenge, and evaluating their work. (d) Evaluation should be nonthreatening. It is embedded in ongoing activities, emphasizes processes as well as products, focuses on personal progress, and encourages learners to view errors as opportunities to learn. (e) Instructors should provide instrumental support to learners’ learning, combining explicit instruction and extensive scaffolding to help learners acquire the knowledge and skills they need to complete complex tasks. The guidelines of Perry and Drummond (2002) and Perry et al. (2003) seem to take a more holistic approach than those of Ley and Young (2001) and focus on interventions that trigger the four different phases of self-regulation through specific interventions (e.g., community of practice, assessment, etc.). The literature review revealed no models for the design of learning environments that support learners’ self-regulation dating from after 2003. After 2003, educational-psychological research focused on specific metacognitive strategies and skills, rather than on learning environments as a whole. Although Ley and Young (2001), Perry and Drummond (2002), and Perry et al. (2003) established sets of guidelines for supporting self-regulation, to the best of our knowledge, none of these guidelines have been either (a) translated into a generalizable conceptual framework for the support of self-regulation or (b) operationalized to describe and characterize (blended) learning environments in a systematic way. This observation is problematic since without such approaches, no systematic investigations or empirical attempts at more effective (re)designs are possible (van van Merriënboer & Kirschner, 2017). Without a systematic approach and framework for describing and characterizing learning environments, such guidelines might do more harm than good. This can be illustrated with the case of learner control, for example: depending on the learners’ characteristics, increased learner control may be either beneficial or detrimental to effective self-directed behavior (Duffy & Azevedo, 2015).

582

S. Van Laer and J. Elen

Problem Statement Research on self-regulation in blended learning environments shows that learners need to have specific characteristics and self-regulation abilities to be successful in such environments. Literature seems to provide only a limited set of guidelines on how to design blended learning environments in this respect. Although some fruitful attempts have been made to come up with sets of guidelines, more recent literature (e.g., Lallé, Taub, Mudrick, Conati, & Azevedo, 2017) has begun to acknowledge that insufficient empirical insights are currently available to distinguish which guidelines are most effective for which learners in which contexts. Yet, to be able to advance in our investigations of which support in blended learning environments is best for which learners, we do need conceptual frameworks, instruments, and methods to describe and thus to characterize learning environments. Such methods can serve as a starting point for empirical and more experimental investigations and might enable the field to provide guidelines and models on how to design blended learning environments that support learners’ self-regulation. In the next section, we discuss the conceptual framework before explaining the development and validation of an instrument and method for describing and characterizing support for selfregulation in blended learning environments.

Conceptual Foundations of the Framework Using a (n = 95) systematic literature review (see original study for methodological details), Van Laer and Elen (2016) identified seven attributes that support selfregulation in blended learning environments. The results of this literature review provided the conceptual foundations for the framework developed here. For each of the attributes, (i) a definition and (ii) evidence from the literature that demonstrate a potential link between the attribute and self-regulation were provided in Van Laer and Elen (2016). In what follows we first summarize this information before describing (iii) the attributes’ operationalization and illustrating them with examples. Finally we (iv) instrumentalize each attribute as a number of questions. Table 1 presents a summary of the conceptual framework (see original study for references and in-depth theoretical background). In the second step, we elaborate on the development of the instrument and method for describing and characterizing blended learning environments.

Authenticity Definitions of authenticity range from real-world relevance, needed in real-life situations, and of important interest to the learner for later professional life. Taking into account these definitions, Van Laer and Elen (2016) define authenticity as the real-world relevance of both the learning environment and the task.

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

583

Table 1 Overview of the conceptual framework presented, based on the Van Laer and Elen’s (2016) seven attributes (see original study for references and in-depth theoretical background)

Attribute and definition Authenticity The real-world relevance of on the learning environment and the task

Personalization The modification of the learning environment to the inherent needs of each individual learner

Learner control Learners having or having not control over the pacing, content, learning activities, and sequences

Scaffolding Changes in the task and learning environment, so learners can accomplish tasks that would otherwise be out of their reach

Interaction The involvement of learners with elements in the learning environment

Manifestation in learning environments • Authentic context • Authentic activities • Expert performance • Multiple roles • Collaborative knowledge construction • Tacit knowledge made explicit • Authentic assessment • Name recognition • Self-described • Cognitionbased

• Control over pacing • Control over content • Control over learning activities • Control over sequence • Contingency • Fading over time • Transfer of responsibility

• Learnercontent interaction • Learnerinstructor interaction • Learnerlearner interaction

Relation to self-regulation • Exploration of tactics for learning • Planning and self-monitoring • Information-seeking behavior • Need to self-regulate, adopting strategies and attuning to the goal, adopted forms of SR, and learning as identity development • Intrinsic goal-orientation, task value, use of elaboration strategies, critical thinking, and metacognition

• Interest value • Self-representation, selfefficacy, relatedness, and social approval • Goal-setting and planning, performance, and selfreflection • Self-control • Instructional selfmanagement • Metacognitive skillfulness • Development of learning strategies • Planning and goal-setting • Rehearsal and self-checking • Planning and monitoring • Strategy use • Self-management, information seeking, and adaptive behavior • Self-structuring and problematizing • Self-evaluation • Strategy use • Metacognitive knowledge • Self-efficacy and test anxiety • Modeling

(continued)

584

S. Van Laer and J. Elen

Table 1 (continued)

Attribute and definition

Cues for reflection Prompts that aim to activate learners’ purposeful critical analysis of knowledge and experience, in order to achieve deeper meaning and understanding

Cues for calibration Triggers for learners to test learners’ perceptions of achievement against their actual achievement and their perceived use of study tactics against their actual use of study tactics

Manifestation in learning environments • Learnerinterface interaction • Vicarious interaction • Reflectionbefore-action • Reflection-inaction • Reflection-onaction • Delayed metacognitive monitoring • Forms for summarizing • Timed alerts • Review of “right” information • Effective practice tests

Relation to self-regulation

• Cognitive structures and abilities • Self-explanation • Metacognitive knowledge and metacognitive control • Awareness of learning process • Self-reflection ability • Reassessment • Goal-orientation • Task identification • Problem-solving • Cognitive strategies, problemsolving strategies, and critical thinking skills, knowledge of cognition, regulation of cognition, self-efficacy, and epistemology

Authenticity and Self-regulation The relation of authenticity to self-regulation has to do with the realistic and ill-structured nature of the learning environment and tasks presented to learners. Well-structured tasks (which are common in education) rarely challenge learners to explore tactics for learning, which may hinder their ability to reach their full potential (Perry & Drummond, 2002). More specifically, they are likely to undermine self-regulation, encourage only shallow processing, and limit performance (Salomon & Perkins, 1998). With regard to the learning environment, authenticity in the learning environment helps learners to develop adequate perceptions of their future professional context, improving their understanding of what is expected (Ley & Young, 2001). While authenticity is very important for self-regulation, moderation is essential. Not all learners will benefit equally from ill-structured authentic tasks and environments. Poorly structured authentic tasks and environments may increase learners’ anxiety and may also be too challenging, leading to withdrawal instead of engagement (Winne & Hadwin, 2013). Authenticity in Learning Environments A large body of literature has investigated the design of authentic tasks and environments (e.g., Reeves & Reeves, 1997; Wiggins, 1993). According to this

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

585

research, authentic learning environments are characterized by (a combination of) the following: 1. Authentic contexts or contexts which reflect how knowledge will be applied in real life. Research on authentic contexts shows that it is not sufficient to provide real-world examples to illustrate what is being taught. Instead, the value of authentic contexts lies in their complex, all-enveloping nature. 2. Authentic activities or ill-defined activities which present a single complex task to be completed over a sustained period of time, instead of a series of shorter disconnected examples. 3. Expert performance, which entails the facilitation of access to expert thinking, the modeling of processes, and access to the social periphery. 4. Multiple roles or different perspectives which enable learners to investigate the learning environment from more than one viewpoint, enabling and encouraging them to explore the learning environment repeatedly. 5. Collaborative knowledge construction, which refers to knowledge construction opportunities for learners to collaborate and thus to estimate their own perceptions of learning. Consequently, tasks assigned to a group instead of to an individual seem to be key to establishing such knowledge construction. 6. Tacit knowledge made explicit or opportunities for learners to articulate knowledge already available within to foster planning of further learning. 7. Authentic assessment of learners’ learning which provides the opportunity for learners to be effective performers with acquired knowledge and to craft polished performances or products in collaboration with others. This requires the assessment to be seamlessly integrated with the activity and to provide appropriate criteria for scoring varied products. To instrumentalize the abovementioned characteristics with the aim of determining whether a given learning environment contains authentic real-world relevance, the following questions were formulated: • Is an authentic context provided that reflects the way the knowledge will be used in real life? • Are authentic activities provided? • Is access to expert performances and the modeling of processes provided? • Are multiple roles and perspectives provided? • Is support for collaborative construction of knowledge provided? • Is articulation provided to enable tacit knowledge to be made explicit? • Is authentic assessment of learning within the tasks provided?

Personalization Personalization in this chapter is defined as the modification of the learning environment to the inherent needs of each individual learner. This definition is in line

586

S. Van Laer and J. Elen

with current definitions that describe personalization as nonhomogeneous experiences related directly to the learner, associated with elements of inherent interest to the learner, and connected to topics of high interest value (e.g., Wilson et al., 2007).

Personalization and Self-regulation Current research literature emphasizes the connection between personal agency and self-regulation and argues that personalized instruction results in a change of selfrepresentation based on psychological needs such as competence (perceived selfefficacy), relatedness (sense of being a part of the activity), and acceptance (social approval), which are components of learners’ self-regulation (e.g., Türker & Zingel, 2008). Evidence has also been provided of a relationship between personalization, learners’ goal-setting and planning, performance, and self-reflection (e.g., Dabbagh & Kitsantas, 2004). By receiving modified instruction related to one’s current skill level, it might be possible for learners to monitor their learning more accurately and thus boost their learning. When the learning environment is aligned with learners’ personal preferences, their interest might increase and thus self-regulation will be impacted positively. In conclusion there seems to be a clear link between personalization and self-regulation particularly in the task identification and goal-setting and planning phase of learners’ self-regulation. Personalization in Learning Environments Three major ways to incorporate personalization into learning environments could be identified (e.g., Devedžić, 2006; Martinez, 2002): 1. Name recognition: This type of personalization aims at the acknowledgment of the learner as an individual. For example, the learner’s name can appear in the instruction or previous activities or accomplishments that have been collected and stored can later be presented when appropriate. 2. Self-description: Self-described personalization enables learners (using questionnaires, surveys, registration forms, and comments) to describe preferences and common attributes. For example, learners may take a pre-course quiz to identify existing skills, preferences, or past experiences. Afterward, options and instructional experiences appear based on the learner-provided answers. 3. Learners’ cognitive needs: Cognitive-based personalization uses information about cognitive processes, strategies, and ability to deliver content specifically targeted to specific types (defined cognitively) of learners. For example, learners may choose to use an audio option because they prefer hearing text instead of reading it. Or, a learner may prefer the presentation of content in a linear fashion, instead of an unguided presentation with hyperlinks. To instrumentalize the abovementioned characteristics with the aim of determining whether a given learning environment contains personalization, the following questions were identified:

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

587

• Is the personalization name-recognized? • Is the personalization self-described? • Is the personalization cognitive-based?

Learner Control Learner control refers to the amount of control learners have over support in learning environments. Definitions vary from freedom of task selection by the learner, control of learning sequences, allowing decisions on which contents to receive, allowing decisions on how specific content should be displayed, and control over the pace of information presentation. Van Laer and Elen (2016) define learner control in line with these definitions and see learner control as the degree to which learners have or have not control over the pacing, content, learning activities, and sequences.

Learner Control and Self-regulation Theorists such as Merrill (2012) assert that learners need to be given control. In addition to this control supporting motivation, exercising control over one’s learning can be a valuable educational experience in itself. The results can be experienced, and the best tactics for different instructional situations can be discovered in the process. In this way, the exercise of learner control can be thought of as a precursor to the development of self-regulation. The idea that learner control is the fine-tuned application of self-regulation is based on the assumption that learners who command the greatest range and depth of learning skills will be the best equipped to handle learner control and other forms of instructional self-management (Resnick, 1972). From this perspective, skills in self-regulation become essential for the effective use of learner control, resulting in the importance of the optimal tuning of learner control to the actual level of self-regulation to avoid undirected behavior. As might be expected, learner control can only be granted when learners possess the ability to effectively use it to purposefully direct their learning. If learners are not able to do so, they will lack to ability to regulate their learning and begin to drift (Lawless & Brown, 1997). Learner Control in Learning Environments Literature reports on the manifestation of learner control often in four ways (e.g., Sims & Hedberg, 1995; Williams, 1993): (1) Control over pacing, learners have control over the speed and time at which content is presented; (2) control over content, the learner is permitted to skip over certain instructional units. This option generally refers to the selection of topics or objectives associated with a specific lesson, although it does not extend to a choice of which content items are displayed. This component of learner control does not focus on the micro level of interaction, in which the learner must make certain choices in response to questions or problems. Therefore, while the learner has control over the content selected for study, the actual presentation of that content has generally remained instructor driven. Thus, there would appear to be two levels of content control: one at which the learner chooses

588

S. Van Laer and J. Elen

a module of study and one at which the presentation and associated display elements are also controlled by the learner; (3) control over learning activities, this type of control includes options for the learner to see examples, do exercises, receive information, consult a glossary, ask for more explanation, or take a quiz; and finally there is the (4) control over sequence, learners can skip forward or backward through activities or they are allowed to retrace a route through the material and options to control when to view such features as content indexes or content maps. Sequence control refers to the order in which the content is viewed and often is defined in terms of being able to move to and from content items. To instrumentalize the abovementioned characteristics with the aim of determining whether a given learning environment contains learner control, the following questions were identified: • • • •

Is control over pacing allowed? Is control over content allowed? Is control over learning activities allowed? Is control over sequence allowed?

Scaffolding Scaffolding in this chapter is defined as changes in tasks and the learning environment, so learners can accomplish tasks that would otherwise be out of their reach. This definition derives from a collection of different approaches to scaffolding which mainly emerged from design research on interactive learning environments. These approaches all emphasized that scaffolding involves providing assistance to learners on an as-needed basis, fading the assistance as learner competence increases (e.g., Wood, Bruner, & Ross, 1976). Based on these approaches, a variety of design guidelines or principles have been proposed.

Scaffolding and Self-regulation According to Vygotsky (1978), learners improve when they are assisted by more advanced or knowledgeable sources of instruction (e.g., instructors or peers). The concept of zone of proximal development refers to “the distance between the actual developmental level as determined by independent problem-solving and the level of potential development as determined through problem-solving under adult guidance or in collaboration with more capable peers” (p. 86). This external guidance or support helps learners monitor their current abilities and calibrate their further actions. Scaffolding also contributes to the planning and monitoring of learners. By providing them with suggestions for potential next steps, learners will be able to direct (regulate) their own learning more toward desired goals (Moos & Azevedo, 2009). The same goes for the use of metacognitive strategies, selfmanagement, information seeking, and adaptive behavior. Finally, scaffolding might also provide the necessary tools to support learners in making adaptations to one’s personal learning environment and define the problems that need to be

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

589

overcome (Feng & Chen, 2014). In conclusion, scaffolding can be seen as the temporal replacement of learners’ self-regulation. Over time the responsibility for it will shift toward the learner.

Scaffolding in Learning Environments Three major ways in which scaffolding is represented in learning environments are as follows (e.g., Garza, 2009; Puntambekar & Hubscher, 2005): (1) Contingency of support, support is adapted to the current level of the learners’ performance and should either be at the same or a slightly higher level. A tool for contingency is the application of diagnostic strategies. Such strategies often encompass small, recurring formative tasks to be able to monitor learners’ current level. To provide appropriate support, it is key to determine the learners’ current level of competence; (2) fading of support over time, as the ability of the learner increases, the support fades over time with regard to the level and/or the amount of it. Examples of such fading support are the elaborate explanation or instruction at the beginning of a course, and there were later in the course fewer instructions that are given for a similar task; finally, (3) transfer of responsibility, as support fades, responsibility for the learner’s performance of a task is gradually transferred to the learner. Responsibility can refer to cognitive (e.g., responsibility for the correctness of the task) and metacognitive activities (e.g., responsibility for the approach of the task) as well as to learners’ affect. To instrumentalize the abovementioned characteristics with the aim of determining whether a given learning environment contains scaffolding, the following questions were identified: • Is support tailored to the learner through continuous monitoring? • Does the support fade over time? • Is there a transfer of responsibilities over time?

Interaction Van Laer and Elen (2016) describe interaction as the involvement of learners with elements in the learning environment. In this chapter we adopt the same definition. This definition encompasses the nature of interaction in various forms of learning environments and in a variety of ways, considering the learners’ level of involvement in a specific learning opportunity and as the objects of interaction such as other participants or content materials. The nature of interaction is also dependent upon the contexts in which interaction occurs, in a face-to-face situation or at a distance.

Interaction and Self-regulation Previous research (e.g., Zimmerman & Schunk, 2006) emphasizes the importance of interaction in providing (a) modeling, (b) opportunities for guided practice, and (c) instrumental feedback to impact learners’ self-regulation. Through these processes, learners develop competence with the task, content, and context, thereby

590

S. Van Laer and J. Elen

becoming self-regulated learners. Self-regulated learners rely on internal standards, self-reinforcement, self-regulatory processes, and self-efficacy beliefs. Subsequently, by interacting with elements of the learning environment, learners get to reflect and judge on their own performance. By interacting with peers, content, etc., self-evaluation, the use of metacognitive skills and the production of metacognitive knowledge, one’s self-efficacy and test anxiety, and modeling capabilities are likely to increase and impact how learners regulate their learning.

Interaction in Learning Environments Five types of interaction were observed in learning environments (e.g., Sutton, 2001; Woo & Reeves, 2007): 1. Learner-content interaction: This type of interaction is interaction between the learner and the content or subject of study. This type of interaction is often limited to a big portion of one-way communication with a subject expert (or medium), intended to help learners in their study of the subject. 2. Learner-instructor interaction: This type of interaction is interaction between the learner and the expert who prepared the subject material or some other expert acting as instructor. 3. Learner-learner interaction: This type of interaction is between one learner and other learners, alone or in group settings, with or without the real-time presence of an instructor. 4. Learner-interface interaction: This type of interaction describes the interaction between the learner and the tools needed to perform the required task. 5. Vicarious interaction: This final type of interaction takes place when a learner actively observes and processes both sides of a direct interaction between two other learners or between another learner and the instructor. To instrumentalize the abovementioned characteristics with the aim of determining whether a given learning environment contains interaction, the following questions were identified: • • • • •

Is learner-content interaction facilitated? Is learner-instructor interaction facilitated? Is learner-learner interaction facilitated? Is learner-interface interaction facilitated? Is vicarious interaction facilitated?

Cues for Reflection Dewey (1958) defined reflection as “active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusion to which it tends” (p. 9). Moon (1999) describes reflection as “a form of mental processing with a purpose and/or anticipated outcome that is

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

591

applied to relatively complex or unstructured ideas for which there is not an obvious solution” (p. 23). Boud, Keogh, and Walker (2013) define reflection as “a generic term for those intellectual and affective activities in which individuals engage to explore their experiences in order to lead to a new understanding and appreciation” (p. 19). All three definitions emphasize purposeful critical analysis of knowledge and experience, in order to achieve deeper meaning and understanding; therefore, reflection cues in this chapter will be identified as prompts that aim to activate learners’ purposeful critical analysis of knowledge and experience, in order to achieve deeper meaning and understanding.

Reflection and Self-regulation By reflecting on one’s own learning, learners become aware of their learning processes and possible alternative strategies. This is important because the perception of choice is a critical aspect of self-regulation, and the awareness of alternatives is a prerequisite for changing less than optimal study habits (Boud et al., 2013). On the one hand, reflection promotes the development of the necessary cognitive structure; on the other hand, it makes this structure available for learning activities. Reflection can thus be conceived of as the bridge between metacognitive knowledge and metacognitive control (self-regulation), facilitating the transfer of metacognitive knowledge to new situations (Ertmer, Newby, & MacDougall, 1996). These processes impact not only learners’ cognitive structures but also their ability to deal with them. Learners’ self-explanation capabilities, their awareness of the learning process, and their self-reflection ability (Michalsky & Kramarski, 2015) also seem to be related to how reflection impacts learners’ self-regulation. Cues for Reflection in Learning Environments Three types of cues for reflection can occur during instruction (e.g., Davis & Linn, 2000; Farrall, 2007): (1) Cues for reflection-before-action, these types of cues aim to trigger learners’ proactive reflection. For example, learners are asked about what they think the upcoming task will be about; (2) cues for reflection-in-action, these types of cues aim to trigger learners’ reflection, while they are performing a task and aim at encouraging learners to reflect upon if they needs to alter, amend, change what they are doing and being in order to adjust to changing circumstances, to get back into balance, to attend accurately, etc. Learners might benefit from checking with themselves if they are on the right track, and if not, what are better ways? For example, an instructor asks to review the actions they are undertaking; finally (3) cues for reflection-on-action, these types of cues attempt to trigger learners to systematically and deliberately think back over their actions. In other words this type cues encourages learners to think back on what they have done to discover how knowing-in-action might have contributed to unexpected action. For example, learners are asked about their experiences regarding a task that is just finished. The more cues for reflection are given, the more likely it is that learners will actually use them. Diminishing the number of cues over time ensures that learners do not become cue-dependent.

592

S. Van Laer and J. Elen

To instrumentalize the abovementioned characteristics with the aim of determining whether a given learning environment contains cues for reflection, the following questions were identified: • Does the reflection-for-action approach apply? • Does the reflection-in-action approach apply? • Does the reflection-on-action approach apply?

Cues for Calibration Calibration cues are defined by Van Laer and Elen (2016) as triggers for learners to test their perceptions of achievement against their actual achievement and their perceived use of study tactics against their actual use of study tactics. This definition is comparable to others who see these cues as prompts to compare learners’ perceptions of achievement to the achievement compared with external standards and perceived use of study tactics and actual use of study tactics. Calibration concerns on the one hand the deviation of a learner’s judgment from more objective measures, introducing notions of bias and accuracy and on the other hand metric issues regarding the validity of cues’ contributions to judgments and the grain size of cues (e.g., Azevedo & Hadwin, 2005).

Calibration and Self-regulation If a learner encounters an impediment while pursuing a goal, the interruption triggers a reassessment of the situation (Carver & Scheier, 1990). Engaging in this reassessment leads learners to estimate how probable it is that they can achieve their goal if they invest further effort, modify their plan, or both. If confidence or hopefulness exceeds an idiosyncratic threshold, then the learner is likely to persevere and when a deficit between estimated performance and actual performance is identified to adapt the plan that has been guiding engagement and continues working toward the initial goal. At this point in the stream of cognitive processing, selfregulation has been exercised (Bandura, 1993). For learners to be able to achieve the desired learning outcomes, they need to calibrate their perception of the task at hand and the goals to be achieved. Cues for Calibration in Learning Environments With regard to the design of cues for calibration in learning environments, five methods could be identified (e.g., Nietfeld, Cao, & Osborne, 2006; Thiede, Anderson, & Therriault, 2003): (1) Cues for delayed metacognitive monitoring, this type is based on a phenomenon labeled “the delayed judgment of learning effect” that shows improved judgments after a learning delay similar to improved achievement associated with distributed sessions over time. For example, learners might be first asked to highlight a text and at a later time evaluate the highlighted content relative to how well it is understood, how easily it can be retrieved, and how it relates to the learning objective. In this case, learners are asked to evaluate previous made

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

593

judgments; (2) forms for summarizing, summarizing information seems to improve calibration accuracy. It is suggested that the summaries are more effective when forms and guidelines are provided. For example, an instructor gives the learners the task to summarize a specific content component and to review it using a correction key; (3) timed alerts, Thiede et al. (2003) state that summarizing information after a delay improved calibration accuracy; (4) review of “right” information, learners have a tendency to select “almost learned” or more interesting content for restudy. If learners were to rate test items on judgment of learning and interest, they could be provided feedback indicating that selection of content for restudy based on interest and minimal challenge may not be the best choices. For example, an instructor advices the learners to select exercises that are challenging for them; and finally, (5) effective practice tests, learners might need to be aware of the change in behavior they should make. By informing them on the mistakes they already made, learners might direct further attempts. For example, an instructor gives the results of the previous test as guideline for the completion of the next test. To instrumentalize the abovementioned characteristics with the aim of determining whether a given learning environment contains cues for calibration, the following questions were identified: • Is a strategy applied to guide learners to delay metacognitive monitoring? • Is a strategy applied for the provision of forms that guide learners to summarize content? • Are timed alerts given that guide learners to summarize content? • Is a strategy applied for helping learners review the “right” information? • Is a strategy applied for effective practice tests that provide learners with records of their performance on past tests as well as items (or tasks) on those tests? In summary, based on the description of each of the attributes presented above, an instrumentalized version of the framework was developed. This instrument was used to describe support for self-regulation in blended learning environments. The instrument can be found in Appendix 1. In the following section, we will discuss the validity and reliability of the instrument.

Methods In the introduction of this chapter, the conceptual foundations of the framework based on Van Laer and Elen’s (2016) attributes that support self-regulation were described resulting in an instrument to describe the support for learners’ selfregulation by learning environments. For each of the attributes, we (i) formulated a definition, (ii) gathered findings from the literature that demonstrate a link between that attribute and self-regulation, (iii) elaborated the attribute’s operationalization and illustrated it with examples, and finally (iv) instrumentalized each attribute as a number of questions. Based on this instrument, two empirical research cycles were

594

S. Van Laer and J. Elen

used to investigate the suitability of the instrument for both research and practice. Below the procedure followed is presented.

Research Design To construct the instrument and achieve high reliability and validity, a validating approach was used. According to Andersson and Bach (2005) such an approach includes the following elements. First there is the design and development phase of the instrument, based on a conceptual framework. The result of this is the translation of the conceptual framework into an instrument which is then embedded in a methodology to assure high reliability and validity. Finally both the instrument and the methodology are tested and optimized when needed by continuously challenging them to assure further replicability. For the design, development, and validation of the instrument presented in this chapter, Andersson and Bach’s (2005) elements were captured in three phases. The first phase included the elaboration of the seven attributes as defined by Van Laer and Elen (2016) into a conceptual framework and the translation of this framework into an instrument to describe the support of learners’ self-regulation in blended learning environments. This phase was reported upon in the previous section of this chapter. The second phase (first empirical research cycle) included embedding the instrument in a methodology and the first time use of this methodology. In the third and final phase (second empirical research cycle), modifications were made and the methodology was applied for the second time in a different context.

Phase 1 Based on the work of Van Laer and Elen (2016), a conceptual framework was constructed. This framework consists of seven attributes that support learners’ selfregulation in blended learning environments: • Authenticity: real-world relevance of the learning experience to learners’ lives and professional context • Personalization: tailoring of the learning environment to the inherent preferences and needs of each individual learner • Learner control: degree to which learners have control over the content and activities within the learning environment • Scaffolding: changes in the task or learning environment which assist learners in accomplishing tasks that would otherwise be beyond their reach • Interaction: learning environment stimulating learners’ involvement with elements of and in the learning environment • Reflection cues: triggers aiming at activating learners’ purposeful critical analysis of knowledge

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

595

• Calibration cues: triggers for learners to test their perceptions against their actual performance and study tactics The combination of these attributes characterizes the support system of learners’ self-regulation in the learning environment from different but related theoretical perspectives. The validation of the instrumentation of the conceptual framework entailed: (a) The formulation of questions to test for indicators of each attribute (b) The use of these questions to describe and hence characterize support for selfregulation in blended learning environments. By operationalizing the conceptual framework as guidance questions, an instrumentalized framework was created. This instrumentalized framework was used during the testing and optimization phases (Phases 2 and 3).

Phase 2 The second phase of the instrument validation took place in a study aiming at the identification of the relation between designs of blended learning environments that support self-regulation and learners’ learning outcomes. During this study the instrument was used for the first time.

Sample We used four blended learning courses taught in two Flemish centers for adult education. The four courses were categorized as blended learning courses as they deliberately combined online- and classroom-based interventions to instigate and support learning (Boelens et al., 2015). All the courses covered the same subject, “Introduction to basic statistics.” The topics included means, modes, frequency tables, etc. Each course had an identical length of 8 weeks. Learners took the course in the first semester of the school year. Both schools were similar in size and context. They were located near a major city and had similar heterogeneous target groups and institutional needs. Both of them were among the largest of their kind and leading institutions of adult education in Flanders, with each providing over 50 courses and catering for over 1000 learners. Blended Learning Courses In line with the operationalization of blended learning, all of the courses involved in the study used a deliberate mix of face-to-face lessons and online lessons. Each faceto-face lesson lasted 3 hours either from 09:00 h to 12:00 h or from 14:00 h to 17:00 h. The learners were expected to spend the same amount of time on the online lessons. In the first school, each course addressed five topics: quantitative and qualitative characteristics of data, representative surveys, descriptive tables, presentation of statistical data using ICT, and the use of grouped data. Course 1 contained

596

S. Van Laer and J. Elen

two face-to-face meetings, one at the start and one on the day of the examination. During the first lesson, the instructor introduced the materials and methodology of the course. Following this introduction, eight online lessons were provided. Course 2 included five face-to-face lessons and five online lessons. It started and ended with a face-to-face lesson. Between these, a face-to-face or online lesson took place every other week. During the face-to-face lessons, the instructor briefly summarized the content of the instructional videos and presentations provided in the online lessons. In both courses, each topic started with the presentation of “Theory,” including general definitions and different examples. At the end of the conceptual part, an individual research project was introduced both via the online learning environment and by the instructor. The conceptual part was followed by “Exercises”; each of the exercises was framed in a different context. After the completion of the last exercise of each topic, a test followed. In the second school, Course 3 was divided into seven weekly meetings. The course contained three topics: data collection, data presentation, and statistical key concepts, in that order. Five of the weekly meetings were in a face-to-face format during which both the instructor and learners used online materials. Finally, Course 4 started with a face-to-face session, during which the instructor introduced the individual research project, the learning materials, and the methodology of the course and gave a brief overview of the entire course. Seven online lessons were then provided.

Procedure First, a backup was made of all online components (virtual learning environment) of the four blended learning environments. These backups were uploaded to our servers for description. Subsequently, each offline component (classroom environment) was registered using an audiovisual recorder. These recordings were also uploaded onto the server for description. The researcher and a (non-domain-expert) colleague functioned as raters. In blended learning environments, it is a challenging task to find a unit of analysis that encompasses both the online and offline context. It is almost impossible to parse these media into comparable “chunks.” The use of equal chunks of information is important because changes to the size of this unit will affect description decisions and comparability of outcome between different models (Cook & Ralston, 2003). To assure that different raters using the same instrument, see the blended learning environment under investigation in the same fashion literature suggest to identify a consistent “theme” or “idea” (unit of meaning) as the unit of analysis (e.g., Henri, 1992). This is because themes and topics can carry on over the boundaries of the online and offline contexts and often entail the same elements (see Strijbos, Martens, Prins, and Jochems (2006) for a more in-depth discussion of the issue of unitization). To overcome the issue of varying units of analysis due to differences in proportion online and offline components, the raters agreed to use a unit of analysis which related to the topics addressed. The two raters coded the four environments completely and independently by reviewing both the online and offline components of the course and filling out the descriptive instrument for each of the topics addressed.

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

597

The instrument was translated into an Excel document to ease the data collection. Raters were not able to answer the conceptual questions at once. A mean score would be calculated based on the answers gathered for each of the questions. To be able to apply such an approach, a scoring scheme based on a Likert-type scale was introduced. For each question a score needed to be assigned. Important is that the scoring is based on occurrence, not on, for example, the quality of the materials as assumed by the scorer. The scoring possibilities were 1 = Never, 2 = Little, 3 = Somewhat, 4 = Much, and 5 = Always. When a score of 1 (Never) is given, this means that there is not the slightest relation between the unit observed and the question raised. When score 2 (Little) is given, there are minimal (implicit) references to the question. Score 3 (Somewhat) is answered when there are clear (explicit) references to the question. Score 4 (Much) is answered when there is a systematic integration of the characteristic addressed in the question throughout the unit. Finally, the score 5 (Always) is given when consistent throughout the unit every element contains the characteristic addressed in the question. As the instrument was developed in Excel, the observations (and scoring) could be summarized in a bar chart. This chart makes it easy to interpret the scores with regard to the inclusion of the different attributes, the individual scores for each of the units, and even the entire course. Based on the raters’ scoring, the inter-rater score was calculated. In this chapter inter-rater reliability is defined as the extent to which different raters, each describing the same content, come to the same decisions (Rourke, Anderson, Garrison, & Archer, 2001). In this phase among other coefficients, the Kendall’s W (also known as Kendall’s coefficient of concordance) is used to investigate inter-rater reliability. This coefficient is particularly of interest for the purpose of the method presented. It is a normalization of the statistic of the Friedman test and can be used for assessing agreement of ordinal variables (Likert-type scale) among multiple raters. Kendall’s W ranges from 0 (no agreement) to 1 (complete agreement) and is reported with a significance score. After the inter-rater score was calculated, both raters met to discuss features of the instrument that were not clear.

Results Based on the reports of the two raters, the Kendall’s W was calculated. It became clear that there was high agreement (Fleiss, 1993) between both raters, W = 0.683, p < 0.033. Both raters agreed (Schmidt, 1997) that with regard to the attributes that support self-regulation, authenticity of the different learning environments differed depending on the nature of the topic. Authenticity was observed more when the topic was in direct relation to applications of a task (e.g., the individual “research” project learners had to do). Personalization in the online learning environment was primarily focused on the presentation of different contextualized exercises and on the choice learners had in selecting a topic to work on in their individual project. Personalization in the face-to-face context was mostly done by addressing learners by their name or by presenting examples from learners’ professional or private life. Further, the instructors delivered instruction mainly in a one-size-fit all approach. Learners were allowed much more learner control in the online learning environment compared to

598

S. Van Laer and J. Elen

the face-to-face environments. In the online learning environments, they were free to select the sequence of topics; all topics were visible from the first lesson onward. Nonetheless, learners did not have control over what activity to do in which topic; these were selected by the instructors. In the face-to-face context, learners were allowed to take control over additional exercises when others were still working on previous ones. Scaffolding throughout the duration of the course was done by shifting responsibility toward the learner. A lot of support was provided when learners solved exercises; the individual project received the least support. In the face-to-face context, instructors tailored support to the learners’ capabilities by giving personal (verbal) feedback. In the online learning environments, instructors did not tailor support to the learners. The difference in interaction between the faceto-face and online contexts was remarkable. In the online learning environment, interaction focused on learner-content and learner-interface interaction. In the faceto-face context, interaction was more focused on learner-instructor and learner-peer interaction. Finally, both cues for reflection and for calibration were addressed the least in all environments described. Reflection cues for one’s own learning were not provided, neither before, during, nor after one’s actions. If reflection cues were given, they entailed hypothetical mistakes learners could make while solving a specific exercise. Finally, some feedback was provided on specific content elements. In either case no action was expected from the learners. The graphical representation of the four different courses can be found in Fig. 1. Based on their experiences, both raters discussed issues related to the use of the instrument and formulated recommendations for further development. Two issues arose. The first issue was the lack of concrete guidelines on how to interpret each attribute and its questions. The second issue related to the use of the Likert-type scales to report the occurrence of each of the attributes in the targeted blended learning environments. Although the Kendall’s W was high, both researchers had the feeling that they did not had the same understanding of when to give what score on the Likert-type scale. To overcome these issues and to increase the inter-rater reliability further, we aimed in phase 3 to improve the methodology used.

Phase 3 In the third phase, which took place in a study investigating the impact of learners’ characteristics on their behavior in blended learning environments, modifications were made based on the experiences of the first application of the instrument. These modifications included the integration of a 2-hour rater training session and a rater manual to the methodology. Secondly, the instrument was applied for the second time in a different context.

Sample We used two blended learning environments employed during a freshmen’s business communication course taught at a university in the Philippines. The courses entailed three different modules. Topics addressed in the course were writing letters, giving

Score on Likert scale

t en

ity

ity

ic

ic

nt

e th

h ut

A

0

1

2

3

4

5

Au

0

1

2

3

4

5

Pe

r

na

o

rs

Pe

so

a

Le

rn

on

c er

tro

l S

ld

fo

f ca

in

g I

n

io

ct

ra

e nt R

n

io

ct

le

ef

n

Le

n

ar

er

tro

n co

l S

ca

ng

di

l ffo I

n

tio

c ra

e nt

Course 3

R

e

n

io

ct

fle

Attributes that support self-regulation

io

at

liz

n

Attributes that support self-regulation

io

at

iz

l na

Course 1

n

tio

a br

i

al

C

C

n

tio

ra

ib al

ity tic en h ut A

0

1

2

3

4

5

ity

ic

nt

e th

Au

0

1

2

3

4

5

ar Le

l ro

t on rc ne

g in ld fo af Sc

n io ct ra te In

n io ct le ef R

ar Le

l ro t on rc ne

g in ld ffo a Sc

n tio ac er t In

n io ct le ef R

Attributes that support self-regulation

n io at

Course 4

Attributes that support self-regulation

iz l na o rs Pe

r

Pe

so

n

io

at

iz

l na

Course 2

n tio ra ib al C

n tio ra ib al C

An Instrumentalized Framework for Supporting Learners’ Self. . .

Fig. 1 Mean score of attributes identified for each course in schools 1 and 2

Score on Likert scale

Score on Likert scale Score on Likert scale

25 599

600

S. Van Laer and J. Elen

presentations, etc. Each course ran for 8 weeks. The courses had different constellations of both online and offline components.

Blended Learning Environments The first business communication course contained two face-to-face meetings, one at the start and one on the day of the examination. During the first lesson, the instructor introduced the materials and methodology of the course. Following this introduction, five online lessons were provided. The second course included four face-to-face lessons and two online lessons. It started and ended with a face-to-face lesson. In-between of these, every other week a face-to-face or online lesson took place. During the face-to-face lessons, the instructor provided the learners with the needed information and exercises to master the goals of the course. During the online lessons, the learners received extra exercises and small tasks. For both environments each module started with the presentation of “Theory,” including general definitions and several examples. The conceptual part was followed by “Exercises”; each of the exercises was framed in a similar context. After the completion of the last exercise of each topic, a practical test followed. Procedure Comparable to the previous phase, all online components (virtual learning environment) and offline components (classroom environment) were registered. All the data was also uploaded onto the server for description. In contrast to the previous phase, four raters were involved in the description of each module of the two blended learning environments. The raters were the researcher, the instructor, one domainexpert-colleague, and one non-domain-expert colleague of the instructor. Based on the reflections of phase 2, two extra tools (rater training and rater manual) were used to improve the reliability of the descriptive instrument. On the one hand, a 2-hour rater training session was developed based on the conceptual background of the framework and the procedures to ensure reliability and validity. Three main actions were undertaken during the training. The first relates to the identification of a unit of analysis among the raters. During the training, the raters identified and agreed on a suitable unit of analysis for the description of the targeted blended learning environments. A second action related to the scoring of the different attributes. The raters discussed and agreed on when they would give each score to a question using the same scoring guidelines as used during phase 1. The last action was the discussion of each of the attributes and their operationalized concepts and examples. In addition all the guidelines formulated were also described in a rater manual developed to support the raters during the scoring. This manual contained the conceptual background of the framework and the procedures to ensure reliability and validity. After the training the four raters coded the two environments completely and independently by reviewing both the online and offline components of the course and filling out the descriptive instrument for each of the topics addressed. Finally the inter-rater score was calculated in a similar fashion as in phase 2.

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

601

Results Based on the reports of the four raters, the Kendall’s W was calculated. The results showed that in phase 2, there was high agreement (Schmidt, 1997) between the four independent raters, W = 0.776, p < 0.000. These results show that the four raters highly agreed that both courses could be rated equally for all seven attributes. Authenticity of the both courses was similar but differed depending of the nature of the topic. Authenticity was observed more when the topic was in direct relation to applications of a task (e.g., writing a complaint letter, compared to the introduction of the formal rules of writing a letter). Personalization in the online learning environment was primarily focused on the presentation of different contexts exercises could take place in. In both the online and offline environment, learners were, for example, free to choose what product they wrote a complaint letter for. Further, the instructors (both online and offline) delivered instruction mainly in a one-size-fit all approach. Learners were allowed equal learner control in the online learning environment compared to the face-to-face environments. In the online learning environments, they only were able to start exercises of each topic once the instructor made the exercises available. Similar, in the offline environment, learners did not have control over what activity to do in which topic either. Scaffolding throughout the duration of the course was done by shifting responsibility toward the learner. A lot of support was provided when learners solved exercises in both contexts. In the face-to-face context, instructors tailored support to the learners’ capabilities by giving personal (verbal) feedback. In the online learning environments, instructors did tailor support to the learners via the discussion forum. Although interaction was facilitated differently the two courses, the face-to-face and online contexts contained similar amounts of interaction. In the online learning environment as well as in the offline environment, interaction focused on learner-learner and learner-instructor interaction. Finally, both cues for reflection and for calibration were addressed the least in both courses. Reflection cues for one’s own learning were provided, neither before nor during learners’ exercises. If reflection cues were given, they focused only on prior-performance, they never looked into the future nor did they investigate ongoing action. Some feedback was provided on specific content elements. In either case no action was expected from the learners. It was concluded that all attributes were manifested equally in both course so that it was possible to compare both courses throughout. The graphical representation of the two different courses can be found in Fig. 2.

Discussion and Conclusions This chapter presented an instrumentalized framework based on which support for learners’ self-regulation in blended learning environments can be described and characterized. A conceptual framework was provided, consisting of seven attributes that support learners’ self-regulation in blended learning environments. The combination of these attributes comprises a support system for learners’ self-regulation in

602

S. Van Laer and J. Elen Course 1 5

Score on likert scale

4

3

2

1

0

Authenticity

Personalization Learner control

Scaffolding

Interaction

Reflection

Calibration

Reflection

Calibration

Attributes that support self-regulation Course 2 5

Score on likert scale

4

3

2

1

0 Authenticity

Personalization Learner control

Scaffolding

Interaction

Attributes that support self-regulation

Fig. 2 Mean score of attributes identified for each course

the learning environment. For each of the attributes, (i) a definition was formulated, (ii) findings from the literature that demonstrate a potential link between the attribute and self-regulation were gathered, (iii) the attribute’s operationalization was developed and illustrated via examples, and finally (iv) each attribute was

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

603

instrumentalized as a number of questions which together make up the overall instrument. The next step was the development and validation of an instrument (and methodology) for describing and characterizing the support of learners’ self-regulation in blended learning environments. In phase 2 the instrument achieved high reliability (W = 0.683, p < 0.033) for the entire sample coded, without the need for substantial rater training or guidance. In phase 3 the results showed that when the rater training (unit of analysis, scoring, and discussion of the attributes) was applied and the raters were provided with a raters manual, inter-rater reliability among four learners increased significantly W = 0.776, p < 0.000. The results of both phases indicate that the instrument is a reliable and valid instrument for its purpose.

The Conceptual Framework and Current Guidelines Based on the outcomes of the instrumentalization of the conceptual framework, we compared the conceptual framework presented in this chapter to on the one hand two highly regarded sets of guidelines related to the support of self-regulation (i.e., Ley & Young, 2001; Perry & Drummond, 2002; Perry et al., 2003) and on the other hand to more up-to-date literature focusing on guidelines for each of the attributes separately, often taking into account learner characteristics. We found that the conceptual framework presented in this chapter encompasses (though on different levels) the same elements as both sets of guidelines and current literature. In a post hoc qualitative analysis, it became clear that: Authenticity as perceived in our framework relates closely to elements described by Perry and Drummond (2002) and Steiner (2016), who emphasize that learners should be engaged in authentic, complex, cognitively demanding activities. Complex authentic tasks address multiple goals and result in the production of extensive knowledge and strategies (McCardle & Hadwin, 2015). Ley and Young (2001) and Oxford (2016) place similar emphasis on the nature of the learning environment in maximizing learners’ learning. With regard to personalization, Ley and Young (2001) and Guerra, Hosseini, Somyurek, and Brusilovsky (2016) indicate that when learners relate to the learning environment, they will be able to identify information needed for their learning more appropriately. Perry and Drummond (2002) and Tabuenca, Kalz, Drachsler, and Specht (2015), on the other hand, add to this that personalization provides support for each learner’s strengths and weaknesses. Regarding learner control, Perry and Drummond (2002) and more recent Duffy and Azevedo (2015) suggest that learners should take control of learning by making choices, controlling the level of challenge, and evaluating their work by doing so they are more likely to persist when difficulties arise (Stevenson, Hartmeyer, & Bentsen, 2017). For Ley and Young (2001), learner control relates to giving learners the possibility to deal with distractions and organize the learning environment according to their own needs (Murray, 2014).

604

S. Van Laer and J. Elen

With regard to scaffolding, both Ley and Young (2001) and Perry and Drummond (2002) and many more in recent years (e.g., Lin, Lai, & Chang, 2016) indicate that the key to scaffolding is decreasing instrumental support for learners’ learning and combining explicit instruction and extensive scaffolding to help learners acquire the knowledge and skills they need to complete complex tasks. Regarding interaction, Perry and Drummond (2002) and Järvelä, Järvenoja, Malmberg, Isohätälä, and Sobocinski (2016) indicate that learners should actively interact with others to construct new insights and strategies to deal with changes. Ley and Young (2001) and Kuo et al. (2014) emphasize that learners should constantly be exposed to examples and interactions showing a variety of strategies they are able to apply. Finally, in line with the conceptual framework presented in this chapter, the sets of guidelines proposed by Perry and Drummond (2002) and Ley and Young (2001) and current literature (e.g., Bannert, Sonnenberg, Mengelkamp, & Pieger, 2015; Verpoorten, Westera, & Specht, 2017) also considers cues for reflection (triggers for monitoring) and cues for calibration (effective monitoring) to be essential. This body of literature emphasizes the organization of instruction and activities to support metacognitive processes and the use of instructional goals and feedback to present the learner with monitoring opportunities. The investigation of current guidelines to support learners’ self-regulation in the light of the conceptual framework described in this chapter shows that the framework presented here contains similar elements as included in the guidelines proposed by Perry and Drummond (2002), Perry et al. (2003), and Ley and Young (2001) and by more to date literature on each of the attributes separately. This finding not only validates the conceptual basis of our framework but may also serve as a starting point for its further elaboration toward guidelines for designing blended learning environments supporting learners’ self-regulation based on ongoing empirical trials.

The Instrumentalized Framework and Designing Blended Learning Despite its conceptual similarities to the models described above, the instrumentalized framework presented in this study does not suggest any guidelines on how to (re)design blended learning environments to support learners’ self-regulation. Nor does it provide a clear demarcation between the different attributes, as to date no such demarcation has been established conclusively in the literature. Instead, the aim of the instrumentalized framework presented in this chapter is to help researchers and practitioners characterize and describe blended learning environments by identifying these attributes and operationalizing them as an instrument. This is achieved using a systematic approach to investigating and supporting self-regulation in blended learning environments. An important remark with regard to the latter is that while using this instrumentalized framework can improve the design of the blended learning environment with regard to support for self-regulation, this can only be achieved through continuous redesign and testing against empirical trials.

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

605

Limitations and Considerations While the instrumentalized framework presented has proven its value for research and practice, certain considerations and limitations should be pointed out. A first set of considerations relates to the underlying conceptual framework. As argued by theorists of the self-regulation concept, self-regulation is influenced by variables within and external to the learner (Winne & Hadwin, 2013). This chapter focuses on the latter type of variables, yet self-regulation theory hypothesizes that combinations of the two types determine the self-regulatory behavior of learners, resulting in different impacts for learners with different learner variables. Although the conceptual framework presented does not give guidelines on how to operationalize each of the attributes in relation to learner variables, it provides the means to investigate them in relation to each attribute. A good example of such an investigation might be a study investigating the amounts of learner control beneficial for skilled learners versus inexperienced learners (e.g., Niemiec, Sikorski, & Walberg, 1996). Further research is still needed to clarify the relation of each of the attributes to learner variables. With regard to the attributes themselves, it must be acknowledged that the conceptual framework only provides principles for describing and thus characterizing blended learning environments, not for designing them. Although this gives a clear idea of how each attribute can be defined and identified within blended learning environments, it also leads to unclear demarcations and overlap. It should be kept in mind that the aim of this instrumentalized framework is to enable the characterization of the blended learning environment, not its redesign. To be able to identify and analyze new research and insights which could contribute to the characterization of blended learning environments, Van Laer and Elen’s (2016) systematic review should be repeated over time. The way in which the attributes are currently described and illustrated is conceptually up-to-date, but the literature is rather dated. In addition, the framework ignores the possibility that certain combinations of attributes might be more beneficial than others. Such hypotheses are often illustrated by research findings that highlight, for example, that a combination of cues for both reflection and calibration have a higher impact on learning than when only cues for reflection are provided (Sonnenberg & Bannert, 2015). Further research is needed to investigate which combinations of attributes are most suitable for which types of learner. The final consideration in relation to the conceptual framework is the focus on blended learning. As each of the attributes was derived from research deliberately aimed at combined online and offline interventions, we expect the framework to be applicable to blended learning environments. Based on previous findings about the impact of technology on learning (e.g., Tamim, Bernard, Borokhovski, Abrami, & Schmid, 2011), we hypothesize that the conceptual framework can also be applied in purely online and purely offline contexts. Further research on the applicability of the framework in purely online or offline environments would improve the generalizability of the conceptual framework.

606

S. Van Laer and J. Elen

A second set of considerations relates to the instrument itself. On the one hand, there is the grain size used for the description of the occurrence of each of the attributes. In contrast to frameworks which focus on detailed descriptions of learning environments (per minute or hour), we adopted the “principle” approach in a context where units of analysis are the themes, modules, or topics covered over a number of weeks. Although this approach is common practice in instructional design theory (e.g., Merrill, 2012), it is rarely applied to investigating support for learners’ selfregulation from an educational-psychological perspective (Winne & Hadwin, 2013). However, as in Perry et al. (2003), such an approach might also be desirable, especially for feasible course (re)design. On the other hand, there is what appears to be the quantitative, summative approach to the instrument. Because we used an Excel spreadsheet to score each attribute, it might appear that each unit of the course is scored based on quality. As mentioned above, however, this is not the aim of this instrument (see Likert-type scale used). The sole goal of the instrument is to map the actual character of a blended learning environment. Based on this state redesigns can be made, which can be evaluated using any subsequent changes in learners’ self-regulation.

Conclusions While the instrumentalized framework presented has its limitations and further research is needed to optimize its capabilities, this chapter has attempted to illustrate that it contributes to existing literature and practice in two ways. Firstly, by providing both a conceptual framework and an instrument focusing on the characterization of support for learners’ self-regulation, we are to the best of our knowledge the first to focus on support for self-regulation in blended learning environments. The instrumentalized framework makes it possible (a) to describe and thus characterize blended learning environments in terms of self-regulation; (b) to provide, based on this characterization, a starting point for investigations to overcome design issues related to learners’ self-regulation in blended learning environments (e.g., Kuo et al., 2014); and (c) to advance the design of blended learning environments more systematically by monitoring its characteristics. Secondly, research and practice will benefit from this more systematic approach to describing and characterizing support for self-regulation in blended learning environments. The ability to describe (blended) learning environments in a systematic and replicable way opens doors to an array of research opportunities and practical interventions and thus facilitates further investigation of self-regulatory issues. It also makes it possible to investigate the impact of each attribute and combinations of attributes in relation to differences in learner variables, thus allowing designers to target learners’ self-regulation more accurately. Acknowledgments We would like to acknowledge the support of Bicol University and the project “Adult Learners Online,” financed by the Agency for Science and Technology (Project Number: SBO 140029), which made this research possible.

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

607

Appendix 1: Visual Representation of the Instrument Course: … Topic: …

Does the learning environment contain authentic real-world relevance?

1

2

3

4

5

Never

Little

Somewhat

Much

Always

0

Comments

0

Comments

0

Comments

0

Comments

0

Comments

0

Comments

0

Comments

Is an authentic context provided that reflects the way the knowledge will be used in real life? Are authentic activities provided? Is there access to expert performances and the modelling of processes provided? Are there multiple roles and perspectives provided? Is there support for collaborative construction of knowledge provided? Is articulation provided to enable tacit knowledge to be made explicit? Is authentic assessment of learning within the tasks provided?

Does the learning environment contain personalization? Is the personalization name-recognized? Is the personalization self-described? Is the personalization cognitive-based?

Does the learning environment allow learner control? Is control of pacing allowed? Is control of content allowed? Is control of learning activities allowed? Is control of content sequence allowed?

Does the learning environment scaffold support? Is support tailored to the learner through continuous monitoring? Is the support fading over time? Does the support fade over time?

Does the learning environment entail interaction? Is learner-content interaction facilitated? Is learner-instructor interaction facilitated? Is learner–learner interaction facilitated? Is learner-interface interaction facilitated? Is vicarious interaction facilitated?

Does the learning environment contain reflection cues? Does the reflection-for-action approach applies? Does the reflection-in-action approach applies? Does the reflection-on-action approach applies?

Does the learning environment contain calibration cues? Is a strategy applied to guide learners to delay metacognitive monitoring? Is a strategy applied for the provision of forms that guide students to summarize content? Are timed alerts given that guide students to summarize content? Is a strategy applied for helping learners review the ‘‘right’’ information? Is a strategy applied for effective practice tests, that provide students with records of their performance on past tests as well as items (or tasks) on those tests?

608

S. Van Laer and J. Elen

References Andersson, B., & Bach, F. (2005). On designing and evaluating teaching sequences taking geometrical optics as an example. Science Education, 89(2), 196–218. https://doi.org/ 10.1002/sce.20044. Azevedo, R., & Hadwin, A. F. (2005). Scaffolding self-regulated learning and metacognition–implications for the design of computer-based scaffolds. Instructional Science, 33 (5), 367–379. Bandura, A. (1993). Perceived self-efficacy in cognitive-development and functioning. Educational Psychologist, 28(2), 117–148. https://doi.org/10.1207/s15326985ep2802_3. Bannert, M., Sonnenberg, C., Mengelkamp, C., & Pieger, E. (2015). Short-and long-term effects of students’ self-directed metacognitive prompts on navigation behavior and learning performance. Computers in Human Behavior, 52, 293–306. Boelens, R., Van Laer, S., De Wever, B., & Elen, J. (2015). Blended learning in adult education: towards a definition of blended learning (pp. 1–3). Brussels: Genth University. Bonk, C. (2017). Best practices for online and blended learning: Introducing the R2D2 and TEC-VARIETY models. Paper presented at the Fall Faculty Development Conference, Indianapolis, IN. Borkowski, J. G., Carr, M., Rellinger, E., & Pressley, M. (1990). Self-regulated cognition: Interdependence of metacognition, attributions, and self-esteem. Dimensions of thinking and cognitive instruction, 1, 53–92. Boud, D., Keogh, R., & Walker, D. (2013). Reflection: Turning experience into learning. New York: Routledge. Butler, D. L. (1998). The strategic content learning approach to promoting self-regulated learning: A report of three studies. Journal of Educational Psychology, 90(4), 682–697. https://doi.org/ 10.1037/0022-0663.90.4.682. Carver, C. S., & Scheier, M. (1990). Principles of self-regulation: Action and emotion. New York, NY: Guilford Press. Cook, D., & Ralston, J. (2003). Sharpening the focus: Methodological issues in analysing on-line conferences. Technology, Pedagogy and Education, 12(3), 361–376. Dabbagh, N., & Kitsantas, A. (2004). Supporting self-regulation in student-centered web-based learning environments. International Journal on E-Learning, 3(1), 40–47. Davis, E. A., & Linn, M. C. (2000). Scaffolding students’ knowledge integration: Prompts for reflection in KIE. International Journal of Science Education, 22(8), 819–837. https://doi.org/ 10.1080/095006900412293. Deci, E. L., & Ryan, R. M. (2010). Self-determination. New Jersey: Wiley Online Library. Deschacht, N., & Goeman, K. (2015). The effect of blended learning on course persistence and performance of adult learners: A difference-in-differences analysis. Computers & Education, 87, 83–89. https://doi.org/10.1016/j.compedu.2015.03.020. Devedžić, V. (2006). Semantic web and education, Integrated series in information systems (Vol. 12). Boston: Springer. Dewey, J. (1958). Experience and nature (Vol. 1). New York: Courier Corporation. Duffy, M. C., & Azevedo, R. (2015). Motivation matters: Interactions between achievement goals and agent scaffolding for self-regulated learning within an intelligent tutoring system. Computers in Human Behavior, 52, 338–348. https://doi.org/10.1016/j.chb.2015.05.041. Ertmer, P. A., Newby, T. J., & MacDougall, M. (1996). Students’ responses and approaches to casebased instruction: The role of reflective self-regulation. American Educational Research Journal, 33(3), 719–752. Farrall, S. (2007). Desistance studies vs. cognitive-behavioural therapies: Which offers most hope for the long term. In Dictionary of probation and offender management (p. 178). Cullompton, UK: Willan Publishing.

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

609

Feng, C. Y., & Chen, M. P. (2014). The effects of goal specificity and scaffolding on programming performance and self-regulation in game design. British Journal of Educational Technology, 45(2), 285–302. Fleiss, J. (1993). Review papers: The statistical basis of meta-analysis. Statistical Methods in Medical Research, 2(2), 121–145. Garza, R. (2009). Latino and white high school Students' perceptions of caring behaviors are we culturally responsive to our students? Urban Education, 44(3), 297–321. Graham, C. R., Henrie, C. R., & Gibbons, A. S. (2014). Developing models and theory for blended learning research. Blended learning: Research perspectives, 2, 13–33. Guerra, J., Hosseini, R., Somyurek, S., & Brusilovsky, P. (2016). An intelligent interface for learning content: Combining an open learner model and social comparison to support selfregulated learning and engagement. Paper presented at the Proceedings of the 21st International Conference on Intelligent User Interfaces, Sonoma, California. Henri, F. (1992). Computer conferencing and content analysis. In Collaborative learning through computer conferencing (pp. 117–136). Berlin, Germany: Springer. Ifenthaler, D. (2012). Determining the effectiveness of prompts for self-regulated learning in problem-solving scenarios. Educational Technology & Society, 15(1), 38–52. Järvelä, S., Järvenoja, H., Malmberg, J., Isohätälä, J., & Sobocinski, M. (2016). How do types of interaction and phases of self-regulated learning set a stage for collaborative engagement? Learning and Instruction, 43, 39–51. Kassab, S. E., Al-Shafei, A. I., Salem, A. H., & Otoom, S. (2015). Relationships between the quality of blended learning experience, self-regulated learning, and academic achievement of medical students: A path analysis. Advances in Medical Education and Practice, 6, 27–34. https://doi.org/10.2147/Amep.S75830. Kizilcec, R. F., Perez-Sanagustin, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in massive open online courses. Computers & Education, 104, 18–33. https://doi.org/10.1016/j.compedu.2016.10.001. Kuo, Y.-C., Walker, A. E., Schroder, K. E., & Belland, B. R. (2014). Interaction, internet selfefficacy, and self-regulated learning as predictors of student satisfaction in online education courses. The Internet and Higher Education, 20, 35–50. Lallé, S., Taub, M., Mudrick, N. V., Conati, C., & Azevedo, R. (2017). The Impact of Student Individual Differences and Visual Attention to Pedagogical Agents During Learning with MetaTutor. Paper presented at the International Conference on Artificial Intelligence in Education, Wuhan, China. Lawless, K. A., & Brown, S. W. (1997). Multimedia learning environments: Issues of learner control and navigation. Instructional Science, 25(2), 117–131. https://doi.org/10.1023/ A:1002919531780. Ley, K., & Young, D. B. (2001). Instructional principles for self-regulation. Etr&D – Educational Technology Research and Development, 49(2), 93–103. https://doi.org/10.1007/Bf02504930. Lin, J. W., Lai, Y. C., & Chang, L. C. (2016). Fostering self-regulated learning in a blended environment using group awareness and peer assistance as external scaffolds. Journal of Computer Assisted Learning, 32(1), 77–93. Martinez, M. (2002). Designing learning objects to personalize learning. In D. A. Wiley (Ed.), The instructional use of learning objects (pp. 151–171). Bloomington, Indiana: Association for Educational Communications & Technology. McCardle, L., & Hadwin, A. F. (2015). Using multiple, contextualized data sources to measure learners’ perceptions of their self-regulated learning. Metacognition and Learning, 10(1), 43–75. Merrill, M. D. (2012). First principles of instruction. New York: John Wiley & Sons. Michalsky, T., & Kramarski, B. (2015). Prompting reflections for integrating self-regulation into teacher technology education. Teachers College Record, 117(5), 1–38. Moon, J. (1999). Reflection in learning and professional development. Abingdon, UK: Routledge Falmer.

610

S. Van Laer and J. Elen

Moos, D. C., & Azevedo, R. (2009). Learning with computer-based learning environments: A literature review of computer self-efficacy. Review of Educational Research, 79(2), 576–600. https://doi.org/10.3102/0034654308326083. Murray, G. (2014). The social dimensions of learner autonomy and self-regulated learning. Studies in Self-Access Learning Journal, 5(4), 320–341. Niemiec, R. P., Sikorski, C., & Walberg, H. J. (1996). Learner-control effects: A review of reviews and a meta-analysis. Journal of Educational Computing Research, 15(2), 157–174. https://doi. org/10.2190/Jv1u-Eq5p-X2pb-Pdba. Nietfeld, J. L., Cao, L., & Osborne, J. W. (2006). The effect of distributed monitoring exercises and feedback on performance, monitoring accuracy, and self-efficacy. Metacognition and Learning, 1(2), 159–179. Oliver, M., & Trigwell, K. (2005). Can ‘blended learning’ be redeemed. e-Learning, 2(1), 17–26. https://doi.org/10.2304/elea.2005.2.1.17. Oxford, R. L. (2016). Teaching and researching language learning strategies: Self-regulation in context. New York: Taylor & Francis. Perry, N., & Drummond, L. (2002). Helping young students become self-regulated researchers and writers. Reading Teacher, 56(3), 298–310. Perry, N. E., Nordby, C. J., & VandeKamp, K. O. (2003). Promoting self-regulated reading and writing at home and school. Elementary School Journal, 103(4), 317–338. https://doi.org/ 10.1086/499729. Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory Into Practice, 41(4), 219–225. https://doi.org/10.1207/s15430421tip4104_3. Puntambekar, S., & Hubscher, R. (2005). Tools for scaffolding students in a complex learning environment: What have we gained and what have we missed? Educational Psychologist, 40(1), 1–12. Puustinen, M., & Pulkkinen, L. (2001). Models of self-regulated learning: A review. Scandinavian Journal of Educational Research, 45(3), 269–286. https://doi.org/10.1080/ 00313830120074206. Reeves, T. C., & Reeves, P. M. (1997). Effective dimensions of interactive learning on the World Wide Web. In Web-based instruction (pp. 59–66). Englewood Cliffs, NJ: Educational Technology. Reigeluth, C. M. (2013). Instructional-design theories and models: A new paradigm of instructional theory (Vol. 2). New York, NY: Routledge. Resnick, L. B. (1972). Open education – some tasks for technology. Educational Technology, 12(1), 70–76. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education (IJAIED), 12, 8–22. Salomon, G., & Perkins, D. N. (1998). Chapter 1: Individual and social aspects of learning. Review of Research in Education, 23(1), 1–24. Schmidt, R. C. (1997). Managing Delphi surveys using nonparametric statistical techniques. Decision Sciences, 28(3), 763–774. https://doi.org/10.1111/j.1540-5915.1997.tb01330.x. Schraw, G., Crippen, K. J., & Hartley, K. (2006). Promoting self-regulation in science education: Metacognition as part of a broader perspective on learning. Research in Science Education, 36 (1–2), 111–139. https://doi.org/10.1007/s11165-005-3917-8. Schunk, D. H. (1998). Teaching elementary students to self-regulate practice of mathematical skills with modeling. New York, NY: The Guilford Press. Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32(1), 25–28. Sims, R., & Hedberg, J. (1995). Dimensions of Learner Control A Reappraisal for Interactive Multimedia Instruction. Paper presented at the Australasian Society for Computers in Learning in Tertiary Education, Melbourne, VIC.

25

An Instrumentalized Framework for Supporting Learners’ Self. . .

611

Song, H. S., Kalet, A. L., & Plass, J. L. (2016). Interplay of prior knowledge, self-regulation and motivation in complex multimedia learning environments. Journal of Computer Assisted Learning, 32(1), 31–50. Sonnenberg, C., & Bannert, M. (2015). Discovering the effects of metacognitive prompts on the sequential structure of SRL-processes using process mining techniques. Journal of Learning Analytics, 2(2015), 72–100. Steiner, H. H. (2016). The strategy project: Promoting self-regulated learning through an authentic assignment. International Journal of Teaching and Learning in Higher Education, 28(2), 271–282. Stevenson, M. P., Hartmeyer, R., & Bentsen, P. (2017). Systematically reviewing the potential of concept mapping technologies to promote self-regulated learning in primary and secondary science education. Educational Research Review, 21, 1–16. https://doi.org/10.1016/j. edurev.2017.02.002. Strijbos, J.-W., Martens, R. L., Prins, F. J., & Jochems, W. M. (2006). Content analysis: What are they talking about? Computers & Education, 46(1), 29–48. Sutton, L. A. (2001). The principle of vicarious interaction in computer-mediated communications. International Journal of Educational Telecommunications, 7(3), 223–242. Tabuenca, B., Kalz, M., Drachsler, H., & Specht, M. (2015). Time will tell: The role of mobile learning analytics in self-regulated learning. Computers & Education, 89, 53–74. https://doi.org/ 10.1016/j.compedu.2015.08.004. Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4–28. https://doi.org/10.3102/ 0034654310393361. Thiede, K. W., Anderson, M. C. M., & Therriault, D. (2003). Accuracy of metacognitive monitoring affects learning of texts. Journal of Educational Psychology, 95(1), 66–73. https://doi.org/ 10.1037/0022-0663.95.1.66. Türker, M. A., & Zingel, S. (2008). Formative interfaces for scaffolding self-regulated learning in PLEs. elearning Papers, 14(9, July). Van Laer, S., & Elen, J. (2016). In search of attributes that support self-regulation in blended learning environments. Education and Information Technologies, 22(4), 1395–1454. https://doi.org/10.1007/s10639-016-9505-x. van Merriënboer, J. J., & Kirschner, P. A. (2017). Ten steps to complex learning: A systematic approach to four-component instructional design. New York: Routledge. Veenman, M. V. J., Elshout, J. J., & Meijer, J. (1997). The generality vs domain-specificity of metacognitive skills in novice learning across domains. Learning and Instruction, 7(2), 187–209. https://doi.org/10.1016/S0959-4752(96)00025-4. Verpoorten, D., Westera, W., & Specht, M. (2017). Effects of isolated versus combined learning enactments in an online course. International Journal of Technology Enhanced Learning, 9(2–3), 169–185. https://doi.org/10.1504/Ijtel.2017.10005187. Vygotsky, L. (1978). Interaction between learning and development. Readings on the development of children, 23(3), 34–41. Wiggins, G. P. (1993). Assessing student performance: Exploring the purpose and limits of testing. San Francisco, CA: Jossey-Bass. Williams, M. D. (1993). A Comprehensive Review of Learner-Control: The Role of Learner Characteristics. Paper presented at the Association for Educational Communications and Technology, New Orleans, LA. Wilson, S., Liber, O., Johnson, M., Beauvoir, P., Sharples, P., & Milligan, C. (2007). Personal learning environments: Challenging the dominant design of educational systems. Journal of E-Learning and Knowledge Society, 3(2), 27–38. Winne, P. H., & Hadwin, A. (2013). nStudy: Tracing and supporting self-regulated learning in the internet. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (Vol. 28, pp. 293–308). New York, NY: Springer.

612

S. Van Laer and J. Elen

Woo, Y., & Reeves, T. C. (2007). Meaningful interaction in web-based learning: A social constructivist interpretation. The Internet and Higher Education, 10(1), 15–25. Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100. Zimmerman, B. J., & Schunk, D. (2006). Competence and control beliefs: Distinguishing the means and ends. In Handbook of educational psychology (pp. 349–367). Mahwah, NJ: Erlbaum.

Stijn Van Laer is a full-time Ph.D. researcher at KU Leuven, Department of Educational Sciences, Center for Instructional Psychology and Technology. His interest lies in the use of educational media and technologies in teaching and learning practices at course level. He is specifically interested in uncovering the relation between course design, learners’ characteristics, and learners’ self-regulation. Before starting his Ph.D., he worked as an instructional designer and educational technologist in the central support services of KU Leuven. As a junior researcher, he published multiple articles and obtained a number of Ph.D. grants, including one from the Association for Educational Communications and Technology. He is also involved in the conference committee of the European Association for Research on Learning and Instruction’s Junior Researcher Conference. Jan Elen is a Full Professor at KU Leuven, Department of Educational Sciences, Center for Instructional Psychology and Technology and former Vice Dean of Education in the Faculty of Psychology and Educational Sciences. For several years he was the head of the educational support office at KU Leuven. He has been the coordinator of the Special Interest Group on Instructional Design of the European Association for Research on Learning and Instruction for a number of years. In addition to introductory courses on learning and instruction, he teaches courses on educational technology and the design of learning environments. He is currently the Senior Editor of Instructional Science.

Digital Technologies and Adults: Social Networking, Holding Environments, and Intellectual Development

26

Smith M. Cecil and Lindstrom Denise L.

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adult Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Constructive-Developmental Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Holding Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adults’ Uses of Digital Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How Online Social Networking Sites Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Online Social Networking Sites as Holding Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Social Networking Sites and Cognitive Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications for Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

614 615 617 619 620 621 622 625 626 627 628

Abstract

This chapter is concerned with understanding and describing how digital technology tools for communication – specifically, online social networking sites (e.g., Facebook, Twitter) – may support and contribute to adults’ cognitive development (i.e., verbal skills, reasoning, and problem-solving abilities). We draw upon a theory of adult intellectual development – constructive developmentalism (Kegan, R. In over our heads: The mental demands of modern life. Cambridge, MA: Harvard University Press, 1994) – that describes how adults make sense of increasingly complex environments and how their interactions with digital environments may support or enhance meaning-making in S. M. Cecil (*) College of Education and Human Services, West Virginia University, Morgantown, WV, USA e-mail: [email protected] L. Denise L. Department of Curriculum and Instruction/Literacy Studies, College of Education and Human Services, West Virginia University, Morgantown, WV, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_88

613

614

S. M. Cecil and L. Denise L.

everyday life. The concept of “holding environments” is used to explain the digital space created both by what the social networking site affords and the users’ decisions about how to design and use their social network (i.e., for information, entertainment, discussions). The qualities of the holding environment play a significant role in users’ participation in online networking sites and what they may gain from such participation, in terms of cognitive skills. Implications for education around learners’ uses of digital tools are discussed. Keywords

Adult development · Social networking sites · Holding environment

Introduction As the chapters throughout this handbook amply illustrate, the widespread introduction and availability of digital technologies and tools, over the past 25 years in particular, has ushered in an era of unprecedented access to an incredible amount of information and diverse ways of communicating with others around the world – instantaneously, in many cases. Digital technologies – including personal computers and mobile devices such as electronic notebooks and cell phones, the Internet, and varieties of software programs and applications (“apps”) – enable ways for individuals to access and use information to increase their knowledge, hone their skills, perform their jobs, communicate with and socialize with others, and be entertained and informed. Further, these digital technologies, including online social networking sites (e.g., Facebook), can be used to not only consume but to create and disseminate ideas, knowledge, and sounds (music) and images (photos, videos) with other people and groups in real time. Digital games create imaginary worlds and new environments for learning, competition and teamwork, and experimenting with different identities and roles. There has been extensive study of the effects of digital games on learning within educational settings (Clark, Tanner-Smith, & Killingsworth, 2016), and we direct the interested reader to that work. Some applications enable users to monitor and evaluate their skills and performance, such as tracking how many steps they take each day, or physical activities and health status, such as the number of hours of sleep per night or number of calories consumed each day (Fanning, Mullen, & McAuley, 2012). Social networking sites (SNSs), in particular, may be particularly important to individuals and social groups in regard to building trust, fostering tolerance, increasing social support, and encouraging community and political engagement (Greenhow, 2011; Hampton, Goulet, Rainie, & Purcell, 2011). Affordable personal computers have been available for about 35 years. People have been “on” the Internet for about half of that time. It is only within the past decade or so that large numbers of people have owned “smart” mobile phones that can function as handheld computers. By now, digital technologies are infused throughout everyday life and activities, having been adopted for business, industry,

26

Digital Technologies and Adults: Social Networking, Holding. . .

615

and retail and consumer use and within educational institutions, government institutions and agencies, by community and social groups, and among individuals. But there has not been much time for scholars to assess the effects of using digital technology on adults’ intellectual or social development. We know of no studies in the literature, for example, that compare randomized groups of adults who have and use, or do not use, digital technologies and which assess the individual development of members of both groups over significant periods of time. What impacts do digital tools have on individuals’ intellectual, moral and ethical, and social development, for example? Given the phenomenal adoption and uses of digital technologies around the world, this chapter is concerned with understanding and describing how uses of a specific digital technology – online social networking sites – may support and contribute to adults’ cognitive development (i.e., growth in knowledge and verbal skills, reasoning, and problem-solving abilities). While there is a significant body of research that addresses how and how well adults learn within digital environments and through the uses of digital technologies, there has been much less attention devoted to understanding how developmental processes of growth, maturity, and aging might be impacted and influenced by digital technology. Thus, much of this chapter is, of necessity, speculative. The outline of this chapter is as follows. We begin by defining and describing adult development as it generally occurs across different domains of human growth and change. In particular, we summarize a specific theory of adult intellectual development – constructive developmentalism (Kegan, 1994) – that may be helpful in understanding both how adults make sense of an increasingly digital intellectual and social environment and how digital environments may support or enhance adults’ meaning-making in everyday life. Such meaning-making is assumed to substantially contribute to intellectual development. For example, digital data gathering and storage tools can extend and enhance memory and judgment by assisting individuals in performing more complex analyses than they could without these digital tools. Then, we turn to describing one prominent form of a digital technological tool – online social networking sites (SNSs) – and the potential influence of participation in online social networks on adults’ cognitive development. We conclude with implications for educators of young students and adult learners.

Adult Development Adult development has been described as a “process of qualitative change in attitudes, values, and understandings that adults experience as a result of ongoing transactions with the social environment, occurring over time but not strictly as a result of time (Taylor, Marienau, & Fiddler, 2000, p. 10). “[U]nderstandings” conveys the acquisition of knowledge by individuals, and “transactions with the social environment” includes – among other activities –adults’ uses of digital tools to think, learn, create, problem-solve, and communicate with others.

616

S. M. Cecil and L. Denise L.

Development refers to intraindividual change processes that occur over some period of time – typically, months, years, or decades. Development is also sometimes referred to as growth, maturation (typically, in the first few decades of life), or aging (in the latter half of life). Learning, in contrast to development, takes place more episodically and rapidly and occurs through individuals’ social and cultural experiences and activities. Development plays a role in how a person learns (i.e., as the cognitive system matures, individuals may be able to learn more efficiently, given an appropriate and supportive environment); learning, in turn, may influence how a person develops. People change over time. Change appears to be very rapid early in life and through adolescence, but then slows down during the more stable, “settled” years of adulthood – although change is a constant of adult life. Physical bodies grow and mature, and the effects of aging become more apparent as years go by. Individuals may attempt to offset the more undesirable effects of aging – weight gain, deterioration of sensory abilities such as vision, and declines in strength and vigor – through diet and exercise regimens, but some declines in intellectual, sensory, and physical abilities are inevitable as people grow older. Also, as people acquire information and knowledge through experience in social and cultural institutions such as schools (i.e., “crystallized intelligence,” Cattell, 1971), their new knowledge can often lead to changes in their beliefs, perspectives, and values. Despite growth in knowledge and experience, certain genetically determined aspects of intellectual abilities (i.e., “fluid intelligence,” Cattell, 1971) begin to decline by the middle years of adulthood (Schaie & Zanjani, 2006), such as speed of perceptual processing and working memory. Adults’ social environments also change over time, leading to new life roles and experiences, interests, and activities – and these social changes may contribute to other developmental changes that individuals experience. The uses of online social networking tools may be helpful to individuals in coping with and adapting to these life changes, by connecting people to one another and sharing information and providing support. A variety of theories have been proposed that attempt to explain the ways in which adults develop as they age and mature and acquire experience in the world and how development in the adult years differs from that of children and youth. Developmental change may be considered in regard to the self (i.e., how one’s self-perceptions and self-concept change over time), one’s social roles, cognitive and intellectual abilities, as well as physical changes and health status. These domains of development overlap and influence one another. For example, physical changes with age (i.e., diminished visual acuity, loss of muscle tone) contribute to changes in self-perceptions (e.g., understanding that one cannot do certain physical activities that once were easy). Declines in health and associated loss of physical capacities may attenuate opportunities for social interactions and contribute to declines in cognitive abilities (Schaie & Zanjani, 2006). In this chapter, we focus on a specific theory of adult cognitive development to illustrate the ways in which digital technologies may work to influence cognitive development over the adult life course. Kegan’s (1994) constructivedevelopmental theory of adult development describes qualitative changes in the ways in which adults make meaning from their experiences in the world.

26

Digital Technologies and Adults: Social Networking, Holding. . .

617

Constructive-Developmental Theory Adult cognitive development is characterized by increasingly complex forms of thinking, according to Kegan (1994). Two premises are at the center of Kegan’s constructive-developmental theory. First, people actively make sense of their experiences in the world (i.e., constructivism). Second, the ways in which people make sense of, or draw meaning from, their experiences change and grow more complex over time (i.e., developmentalism) (Drago-Severson, 2008). Constructivedevelopmentalists such as Kegan (1994) believe that the systems by which people make meaning of their lived experiences grow and change over time. This ability to make meaning occurs through transformation; that is, the cognitive system becomes increasingly more complex and better equipped to deal with multiple task demands, problems, and uncertainties in life. This cognitive transformation occurs through the mutual interactions of biologically determined maturational processes and lived experience in social environments. Cognitive transformation is similar to but not synonymous with conceptual change, which is defined as any learning that changes an existing belief, idea, or way of thinking (Posner, Strike, Hewson, & Gertzog, 1982). Kegan (1994) further suggests that the “hidden curriculum” of life is constantly changing and increasing in its demands – in ways that may be developmentally challenging for many adults. That is, most adults in modern societies face a dizzying number of life tasks to be completed; decisions to be made; information to be taken in, evaluated, and utilized; and personal and professional relationships to navigate on any given day. Adults who are better able to understand and draw meaning from their experiences in extraordinarily complex, challenging environments are likely to thrive and will develop more sophisticated intellectual skills, self-reflective abilities, and nuanced social skills. A salient example of the complexity of the modern social environment and its’ concomitant demands is rooted in the notion that individuals should be responsible for themselves. This is a neoliberal perspective that has permeated social and public policy work over the past two decades in the United States and other Western societies (Treas & Hill, 2008). Social policies emerging from this perspective advocate for shifting the role of government from assisting – even taking care of – people’s economic and social welfare to government simply acting as an intermediary between the marketplace and the individual. Significant changes in social welfare policies in a number of areas have directly affected people’s lives, resulting in greater challenges for many. These include welfare reform in the United States during the Clinton Administration in the 1990s (Personal Responsibility and Work Opportunity Reconciliation Act of 1996), reductions to or elimination of employer-provided pension plans for employees, and most recently, the Patient Protection and Affordable Care Act (PPACA, colloquially known as “Obamacare”) of 2010. As a consequence, there is demand and greater need for individuals to be highly knowledgeable about, and competent to take charge of and manage, for example, their retirement plans and portfolios, financial investments, and healthcare coverage plans. Corresponding to the rise of this ideological viewpoint is the popularization and availability of the Internet, which provides an “information highway” for people to

618

S. M. Cecil and L. Denise L.

access and use all kinds of information via computers and smartphones. People can “go online” to do banking, check the results of medical tests, buy and sell stocks, and develop and manage entrepreneurial activities – all without the need for assistance from others. Thus, much of adult life today is lived and managed within the context of digital environments. In the remainder of the chapter, we further explore the extent to which digital technologies – specifically, social networking sites – create “holding environments” that can (when appropriately designed and utilized) support cognitive development. Within such online holding environments, adults can grow to better manage the complexity of their lives and their roles as workers, learners, and citizens. Drawing upon Piaget’s (2008) classic constructivist theory of cognitive development, constructive-developmental theory holds that the interactions between a biological organism (i.e., the human brain) and its’ environment – in all physical, social, and cultural manifestations – lead to more advanced forms of cognitive ability and sophisticated meaning-making. According to Kegan (1994), meaning-making has both cognitive and affective qualities and is a physical (in terms of the senses), social (in interaction with others), and survival (meaning-making is essential) activity. Intellectual development occurs in a series of qualitative changes in the form of knowing (i.e., understanding the world), which Kegan refers to as “orders of consciousness.” Kegan describes five orders of consciousness, three of which pertain most directly to adulthood. In infancy and childhood, individuals first learn to distinguish themselves from objects and objects from themselves, but remain highly egocentric in their interactions with parents and others, and are impulsive in their desires (first order). Kegan then describes adult knowers as instrumental (second order), socializing (third order), and self-authoring (fourth order). The fifth order of consciousness, transforming (fifth order) knowing, is thought to be very rare, in Kegan’s scheme. Each order of consciousness is more advanced than the preceding order in regard to the individual’s meaning-making and abilities to see beyond the self and consider others’ perspectives and integrate these perspectives with one’s own. Instrumental knowers are described as having a concrete orientation to life (Drago-Severson, 2008) and are characterized as believing that knowledge is a collection of facts and ideas that is portable, relatively concrete and inflexible, and once obtained, permanent. Instrumental knowers tend to view knowledge in terms of black and white or right or wrong. Knowledge is prescriptive and can be applied to any problem regardless of context. Further, knowledge is generated by and obtained from sources of authority – teachers, experts, and those who rule or govern – rather than developed through one’s own action and self-reflection. Instrumental knowers are limited in their ability to understand others’ perspectives. This form of knowing has significant limitations for the instrumental learner in dealing with the complexity of the modern world. Instrumental knowers thus might restrict their digital social networking to a narrow range of activities and with similar, like-minded persons; Kegan (1994, p. 193) estimated that only a tiny percentage of adults – fewer than 5% – are purely instrumental in their orientation to the world. Socializing knowers, in contrast, view knowledge as something that one needs in order to meet others’ expectations and fulfill one’s social roles (i.e., parent, worker,

26

Digital Technologies and Adults: Social Networking, Holding. . .

619

citizen) and responsibilities to others. According to Drago-Severson (2008), the approval and acceptance of other people is important to socializing knowers, and as Taylor (2006) observed, socializing knowers are therefore vulnerable to critical feedback. Knowledge is considered to be objective, and like instrumental knowers, socializing adults understand knowledge as that which is largely derived from experts rather than one’s own in-depth study and learning. Though a more advanced form of knowledge because of developed capacity for abstract reasoning and ability to consider others’ points of view, socializing knowledge is nonetheless constrained. In the digital social networking environment, socializing knowers will look to others to provide confirmation for their ideas. According to Kegan (1994), most adults in Western societies have developed the capacity for socialized meaning-making. Self-authoring knowers understand that knowledge is something that is selfconstructed through one’s action and experiences in the world. Knowledge comes from one’s interpretations and self-determined evaluation of standards, values, and perceptions. “Right answers” – to the extent that they exist – may depend on the specific context in which information, facts, ideas, and experiences are obtained. Self-authoring learners are not relativists, but have an appreciation for nuance and context in the ways that knowledge is constructed. According to Drago-Severson (2008), self-authoring knowers “take responsibility for and ownership of their own internal authority” when learning something new (p. 61). More so than instrumental or socializing knowers, self-authoring adults can compare their own internal standards and judgments to those of others, taking these other perspectives into consideration. And, rather than finding new information and situations as threatening, selfauthoring knowers see possibilities for new understandings. In the digital social networking environment, self-authoring knowers view themselves and others as both generators of information and co-constructors of knowledge. They are able to critically evaluate and integrate a variety of opinions, facts, and ideas. Kegan (1994) argues that self-authoring knowing is the minimum level needed to deal with the complexity of contemporary life; yet, less than half of adults have achieved selfauthoring meaning-making. Finally, transformative knowers are able to understand perspectives beyond themselves and others. They are systemic in their understanding of how people are interdependent within larger, more inclusive systems of relationships where affiliation and nurturance are essential qualities. However, environments rarely provide the necessary conditions that foster such meaning-making, relationship building, and transformative thinking, according to Kegan (1994), and thus, transformative knowers are exceedingly rare.

Holding Environments Given these different ways of knowing, perceiving, and learning described by Kegan, different individuals need supportive, as well as sufficiently challenging, environments in which to explore, take risks, and experiment in order to grow, change, and transform their thinking. Kegan refers to these contexts as “holding

620

S. M. Cecil and L. Denise L.

environments”– a concept borrowed from psychotherapy (Winnicott, 1973/1992) – and he argues that such environments function to (1) meet individuals’ needs by providing appropriate supports and accommodations, (2) challenge and provoke individuals to grow beyond their existing self-perceptions, and (3) provide continuity and stability as individuals undergo growth and change. Digital technologies are not only a part of an adult’s environment, but according to Backonja, Hall, and Thielke (2014), they produce their own microenvironments for the user. As Backonja et al. note, a digital technology user’s local world is changed, in some manner, by communication with other people (i.e., text messaging), finding and using information (i.e., on the Internet), or participating in simulated worlds (i.e., playing digital games). Thus, depending on how an individual uses a given digital device and its software features or applications, the digital microenvironment may function as a holding environment that can offer challenges and opportunities for the user to take intellectual risks within a “safe” space. We suggest, then, that social networking sites such as Facebook can also function as intellectual holding environments, as their various built-in features (e.g., privacy controls, discussion space, and personalization, among others) provide accommodations and potential challenges to users while also yielding continuity and stability over time. Before discussing ways in which social network users can create digital holding environments that can contribute to their intellectual development, in terms of growth in knowledge, critical reasoning and problem-solving, or creative activities, we describe some of the ways that adults utilize digital (online) tools and platforms and provide a brief primer on how online social networking is performed. We next summarize several studies that report on adults’ uses of digital technologies, in particular, social media, to get a sense of the scope of adults’ digital practices that might support growth toward self-authoring and, ultimately, further cognitive development. We conclude with implications for educators – in K-12, postsecondary, and adult education – in regard to teaching students about creating, managing, and benefitting from their digital social networking sites’ intellectual holding environments.

Adults’ Uses of Digital Technologies As of 2015, there appears to be a widespread adoption of digital technologies by adults in the United States and other Western nations. For instance, 97% of millennial adults (ages 18–34) are Internet users, as are 83% of adults in their 50, and 61% of adults age 70 and older (Perrin & Duggan, 2015). More than half of US adult age groups have broadband access in their homes and cell phone ownership, and its use is nearly universal among US adults, ranging from 74% among those ages 70–87 to 98% of young adults, ages 18–34. Likewise, slightly more than half of US adults, age 50 and younger, use tablet computers, and more than half of US adults under age 60 use some kind of SNSs (Perrin & Duggan, 2015). According to a recent report from the Pew Research Center, the vast majority of Americans (87%) believe that their use of the web and cell phones helps them learn new things, stay better

26

Digital Technologies and Adults: Social Networking, Holding. . .

621

informed on topics of personal interest, and increases their capacity to share ideas and their creations with others (Purcell & Rainie, 2014) – all signs of adults’ intellectual engagement with information and ideas. However, research also shows that older adults (over the age of 65) who do not currently use the Internet are skeptical about the benefits of having access to the Internet and the use of digital tools like cell phones. In one self-report survey, over half of older adults (53%) disagreed or strongly disagreed with the statement that people lacking Internet access are at a real disadvantage because of all the information they might be missing (Smith, 2014). There is also a divide present between older adults with and without college degrees, as a strong majority of college educated older adults (87%) go online, while only 40% of those lacking a college degree do so (Smith, 2014). Thus, many older adults, in particular, may be disadvantaged when it comes to managing their lives and roles as learners and citizens in an increasingly digitized world. Overall, nearly two-thirds of American adults (65%) use SNSs, up from 7% when the Pew Research Center began systematically tracking SNS usage in 2005 (Perrin & Duggan, 2015). Young adults (ages 18–29) are the most likely group to use SNSs – fully 90% do. However, usage among adults 65 and older has more than tripled since 2010 when only 11% of older US adults used SNSs. Today, 35% of all adults 65 and older report using SNSs, compared with 2% in 2005. Over the past decade, adults having some college experience have been consistently more likely than those with a high school degree or less to use SNSs (Perrin & Duggan, 2015).

How Online Social Networking Sites Work According to Boyd and Ellison (2007), a social networking site is a “web-based service that allow[s] users to 1) construct a public or semipublic profile within a bounded system, 2) articulate a list of other users with whom they share a connection, and 3) view and traverse their list of connections and those made by others within the system” (p. 211). This definition highlights “the public display of connections” as a distinguishing feature of SNSs. There are numerous SNS platforms, the most popular of which are Facebook, Pinterest, Instagram, LinkedIn, and Twitter (Pew Research Center, Internet and Technology, 2017). Although these platforms differ from one another in their technical features and users adopt them for somewhat different purposes, they all enable users to connect with other people. When an individual joins a SNS such as Facebook, he or she creates a personal profile that employs texts and images to convey information (which may be authentic or fabricated) about himself or herself (Donath & boyd, 2004), such as age, education level, gender, personal interests, and occupation (Ellison, Steinfield, & Lampe, 2007). Some of this personal information can remain inaccessible to others and, thereby, private. Most SNSs have a variety of privacy control features to limit other parties’ access to a user’s information. After creating and signing in (or “logging on”), the SNS user can proceed to develop a social network by “friending” or “following” other users whom they know or with whom they want

622

S. M. Cecil and L. Denise L.

to establish a connection (e.g., a friend of a friend). This process gives individuals access to others’ profiles and status updates so relationships can be sustained or newly formed based on this information sharing (Boyd, 2008; Jones, Millermaier, Goya-Martinez, & Schuler, 2008). SNS platforms such as Facebook are characterized by a continuous, chronological timeline or stream of information. These are the status updates (Facebook) where the user sees links to news stories and information, facts, and ideas, personal information updates, commentary, and accolades and affirmations from friends, as well as “likes,” or sharing and re-tweets of these and other updates. Photos, videos, and music files, among other contents, are also shared among users. Alongside the SNS privacy features, users have some control over the information that appears in their stream of information. For example, users can determine whom they “friend” and which organizations, social and community groups, businesses, news outlets, and other entities to “like” or follow online. Users can change their SNS settings to limit the updates in their Facebook newsfeed, they can prioritize the information that is seen, and they can block or prevent status updates and comments from other users, friends, and followers. Thus, SNS users can create an online microenvironment in which the information that appears in the status updates and tweets is either open and unlimited or constrained to a few friends or family members who have similar interests and hold like opinions. Alternatively, the status updates may present a diversity of ideas, perspectives, and opinions from a broad cross section of sources (i.e., news sites, community groups, government agencies, friends). In this latter case, the user is willing and sufficiently interested to see and consider ideas and opinions that might be at odds with their own views. While this practice may be more characteristic of self-authoring adults, it is possible that some socializing learners, and perhaps even instrumental ones, do so as well under certain circumstances. But the ways in which these groups of adults respond to the SNS holding environments that they have created are likely to differ. In any case, online social networks are potentially ripe environments for meaning-making to take place.

Online Social Networking Sites as Holding Environments A holding environment is an abstract concept, and it differs from the more obvious structural environment of the SNS user interface. According to Kegan (1982), holding environments that support cognitive transformation are organized so that they challenge individuals to grow beyond their existing self-perceptions (Kegan, 1982). Further, holding environments that are presumed to be most effective in supporting cognitive development are, according to Kegan (1994), those that are carefully constructed so that they provide emotional support to the individual, while also challenging their previously held assumptions. There is some evidence to support the assertion that SNS holding environments can support cognitive growth. Although Internet-mediated communication is often depicted as impoverished or even antisocial (Thurlow, 2006), research shows that online discussions in SNS are frequently hyper-personal and intimate (Walther &

26

Digital Technologies and Adults: Social Networking, Holding. . .

623

Parks, 2002) and help to crystallize relationships that “might otherwise remain ephemeral” (Ellison et al., 2007; Walther & Parks, 2002; Wellman, Haase, Witte, & Hampton, 2001). This may explain why most SNS users characterize the communication present within their online social networks as emotionally supportive. Interestingly, 69% percent of American teenagers who use social media say that, in their experience, people their age are mostly kind to one another on social networking sites – rather than being venues for online bullying of others. Also, 85% of adult users of online social networking sites say that most people are kind to one another on these sites (Lenhart, Madden, Smith, Purcell, Zickuhr & Raine, 2011). The ease with which a user can create and recreate their personal profile means that individuals can play with their identities in ways that can elicit affirmation from others of who they are or want to be. In other words, individuals can present themselves authentically or can portray fictionalized, misleading, and deliberately false information (Gil-Or, Levy-Belz, & Turel, 2015). Several studies show that SNS users often experience both identity affirmation and emotional support online. For instance, young people report receiving overwhelming support for their creative works (i.e., artwork, photography, poetry, and videos), which helps to affirm their identities as creative persons (Greenhow & Robelia, 2009). Other studies report that participation in SNSs enhances feelings of belongingness and affiliation (Shapiro & Margolin, 2014) – emotions associated with highly effective learning environments – further confirming personal identity. Also, some reports suggest that Facebook is a comfortable virtual environment for those individuals who do not normally engage in politics and political discussions. They can, for example, observe their Facebook friends’ participation in political behavior (e.g., sharing articles about political issues, posting photos at political rallies) and might therefore feel safer about expressing an opinion of a political issue – without fear of castigation and disapproval. Still, roughly half of SNS users feel that the political conversations they see on social media are angrier (49%), less respectful (53%), and less civil (49%) than those in other areas of life (Duggan & Smith, 2016). Yet, more than one-third of social network users (39%) observe that these interactions are no less respectful than political interactions they encounter offline. A few users find political debates on social media to be more civil (7%), informative (14%), and focused on important policy issues (10%) than those they see elsewhere (Duggan & Smith, 2016). The research also makes clear that the extent to which users reap benefits from SNS participation depend on how they interact with others within the intellectual space of the social networking site. Communication that is directed to others (i.e., tagging photos, making comments, “liking” others’ posts, and posting one’s own status updates) is associated with greater social bonding and feeling less alone (Burke, Marlow & Lento, 2010). While SNSs reduce reliance on familiar and local ties, they can also increase opportunities to form and maintain more diverse and widespread social ties. Exposure to a wider network of online “friends” is easily possible within social networks, which enables potentially more diverse and differing views in regard to personal, social, political, and lifestyle issues and concerns (Ellison, Steinfield, & Lampe, 2011). But this diversity of viewpoints increases the likelihood that one’s beliefs, attitudes, and values might be interrogated by those

624

S. M. Cecil and L. Denise L.

who are not close associates. However, most people tend to avoid those with whom they strongly disagree in their face-to-face interactions (Duggan & Smith, 2016), and the same behavior may be true online, as SNS users can readily block or “unfriend” those who espouse opinions with which they disagree. Challenging one’s existing way of thinking is one of the necessary conditions of a holding environment – particularly one that is organized such that it can affect intellectual change. Yet, many SNS users do not fully capitalize on opportunities to access and consider alternative viewpoints within the confines of their social network feeds. These are lost opportunities to expand one’s views, consider different perspectives, reflect upon one’s own beliefs and convictions, as well as evaluate the validity of the information that one holds. For example, less than half (47%) of those who identify themselves as consistently conservative, and who discuss politics on their SNSs, could identify one or more discussion partners with whom they disagree at least some of the time (Mitchell, Gottfried, Kiley, & Matsa, 2014). Somewhat more than half (59%) of those persons identifying as consistently liberal could do so. While persons holding liberal views relied more on a range of news sources, suggesting openness to diverse perspectives, they were more likely than their conservative counterparts to block someone within their online social network because of perceived political differences (Mitchell et al., 2014). Also, a majority of SNS users (64%) reported feeling stressed and frustrated by online political discussions with those with whom they disagree (Duggan & Smith, 2016). Only 16% of SNS users say they have changed their views about a political issue after discussing it or reading news articles, essays, or comments about the topic on their SNS (Rainie & Smith, 2012). Transformation in thinking that leads to qualitatively higher forms of meaning-making (i.e., from socializing to self-authoring) appears to be constrained for many participants in online social networks. Many adults on social networking sites also struggle to determine the veracity of much of the information that appears in their newsfeeds. Some people, when using search engines to locate information, also turn to their social networks to attempt to validate the information that they find online (Morris, Teevan, & Panovich, 2010). Almost two-thirds of US adults (64%) say that so-called “fake” news stories cause them a great deal of confusion (Barthel, Mitchell & Holcomb, 2016). It is estimated that nearly a quarter (24%) of Americans have shared fake news online, either knowingly or unknowingly (Barthel et al., 2016). This phenomenon is thought to have at least indirectly affected the outcome of the 2016 US presidential election, as a number of pejorative stories about Democratic candidate Hilary Clinton proliferated (PBS NewsHour, 2016). Yet, rather than taking personal responsibility for preventing the spread of fake news, Barthel et al. (2016) found that a significant portion on US adults (45%) believe that social networking companies, such as Facebook, and elected officials or the government should take action to prevent such news stories from appearing online – a viewpoint that raises important free speech concerns. Recently, social networking platforms have recognized the problems that the proliferation of fake news creates and are taking steps to address the issue. Facebook, for example, recently launched a tool that flags fake stories in its news feed.

26

Digital Technologies and Adults: Social Networking, Holding. . .

625

Facebook users can now easily report suspected fake news that appears in their news feeds; these stories are then reviewed by third-party fact-checkers to determine their veracity (Chowdhry, 2017). Perhaps more significantly, schools and teachers are helping students learn how to distinguish real from fake news by using online tools such as Checkology (Lapowsky, 2017). Students are also learning how to identify and use authentic news and information sources. Still, few classroom studies have yet been conducted to determine the effects of this instruction on students’ abilities to distinguish authentic from fake news sources.

Social Networking Sites and Cognitive Development In much the same ways that reading and writing practices can contribute to knowledge growth and cognitive development (Krashen, 1990; Stanovich, 1993) through exposure to new and challenging vocabulary and facts, opinions, and perspectives that are different from one’s own ideas, online social network participation may also influence adults’ meaning-making abilities and, ultimately, their cognitive growth. There is, in fact, some limited evidence to suggest that using online social networks may make people smarter, in terms of contributing to cognitive abilities such as verbal skills, working memory performance, and academic achievement. Evans, Kairam, and Pirolli (2010), for example, observed that when participants engaged in challenging online search tasks, they received more responses to their queries within their social networking sites (and, thereby, more information to sort through, analyze, and confirm), although more thorough answers were provided within oneon-one private channels. Alloway, Horton, Alloway, and Dawson (2013) assessed and compared adolescents who were either long-term (i.e., more than 1 year) users of online social networks or were relative novices. The experienced social network users had higher verbal abilities, better working memory skills, and greater academic achievement as compared to the novices. Of course, it may be that persons with higher cognitive abilities are more likely to adopt and use online social networks. While these few studies are far from definitive, they point to possibilities and needs for further investigating the potential benefits of online social network participation to adults’ cognitive development. Still, evidence that individuals use SNS to capitalize on opportunities to engage with diverse points of view in ways that move them toward becoming self-authoring individuals is limited. Studies are needed that assess adults’ uses of online social networks over time and how they may benefit. Does online social networking result in learning that is substantive and meaningful for adults? If so, what are the specific characteristics of the intellectual holding environments that are present for those who benefit? Are there specific, common elements to online social networks’ holding environments that lead to cognitive growth? Can these holding environments be easily created by other users, or is some amount of instruction necessary? Does participation in online social networks lead to gains in particular cognitive skills such as memory (both short and long term), inductive and deductive reasoning, and problem-solving or boost performance in creative abilities? Answering these questions will be of great value to educators who

626

S. M. Cecil and L. Denise L.

currently use digital technologies for teaching or who question the value of such tools for learning and instruction.

Implications for Education Although research to date is lacking, it seems likely that adults’ participation in online social networks to communicate with others, learn and become informed, or simply to be entertained has great potential for individuals’ cognitive growth, in terms of knowledge growth, and greater perspective-taking skills, reasoning abilities, and problem-solving. Yet, much depends on the qualities of the holding environment of the social network – and the qualities of that environment are largely within the control of the user. The decisions that users make about how they want to use their online social networks, their purposes for doing so, whom they choose as “friends,” and what information sources (e.g., news sites) they want to appear on their personal site determine, in large part, the characteristics of the online holding environment. It is likely that many social network users do not think very carefully or deliberately about the qualities of the online intellectual environment and the potential power of online social networking for influencing their ideas, beliefs, and behaviors. These conditions suggest that greater efforts are needed to educate learners of all ages about how to use online social networks to their greater benefit. A number of writers have suggested that today’s millennial learners are “digital natives” who have grown up surrounded by digital devices and tools and are, therefore, very adept in their uses (Prensky, 2001). However, there is little empirical evidence in support of this claim, and most studies show that young people need explicit instruction and assistance in the uses of digital tools for the purposes of learning (Facer & Furlong, 2001; Hargittai & Hinnart, 2008; Helsper & Eynon, 2010). Several classroom studies find that students may not be receiving this instruction. One survey found that less than 20% of K-12 teachers allow their students to use social media tools such as blogs or YouTube (MMS Education, 2009). Another large survey determined that fewer than two of five teachers (39%) encourage their students to participate in online discussions or instruct them in the uses of collaborative web-based tool to complete assignments or support students’ learning (Purcell, Heaps, Buchanan, & Friedrich, 2013); only about 5% of teachers have used Twitter for a classroom activity. Given teachers’ reticence to use digital tools to support learning, students are unlikely to develop the cultural competencies and social and technological skills needed to use digital technologies in ways that can support their intellectual development. Jenkins, Purushotma, Weigel, Clinton, and Robinson (2009) argue that, for individuals to use online social networks in ways that can challenge their existing beliefs and perceptions, they must develop new media literacy skills. These new skills include negotiation, which helps adults to move seamlessly across diverse communities while respecting multiple, sometimes competing points of view; understanding and following alternative norms; and judgment, which is the ability to evaluate the reliability and credibility of different sources of information. To be clear,

26

Digital Technologies and Adults: Social Networking, Holding. . .

627

while these new media literacies involve the social and technical skills that are developed through collaboration and networking within digital technologies, they also build on and extend the traditional literacy skills that are taught in school (Jenkins et al., 2009). When taught effectively, media literacy skills may enable learners to establish online holding environments within their social networking sites that effect cognitive change.

Conclusion People who participate in online social networks gain access to a vast store of information and a range of ideas, opinions, and beliefs as held by others with whom they interact or share (Greenhow, 2011). The social network community that an individual establishes within their online network is inherently an intellectual holding environment that has potential to shape and influence the person’s cognitive growth and development. Thus, one’s perspective-taking abilities might be enhanced over time through purposeful interactions with people who represent a diversity of opinions and beliefs. A social network user who is devoted to thinking deeply about problems, dilemmas, and conflicts might benefit from sustained engagement in a community of persons who vary in their knowledge, beliefs, and political orientation. Yet, the available evidence finds that many adults are not well prepared to use online social networks in ways that enable deep intellectual engagement and which will support cognitive growth. Jenkins et al. (2009) claim that contemporary digital technologies have established a participatory culture that minimizes traditional barriers to civic and social engagement. This engaged, participatory culture supports the diversification of cultural expression, individuals’ and groups’ social connections, and peer-to-peer learning. But, what appears missing within this new participatory culture is that many adults are not well positioned to use these digital tools in ways that can best support their cognitive development. The near-constant emergence of innovative digital technologies means that new skills, strategies, dispositions, and social practices must also develop in step so that these tools can be used effectively (Coiro, Knobel, Lankshear, & Leu, 2008). More particularly, a new mindset is required for individuals to fully advantage the affordances of SNSs and the holding environments that can lead to transformations in thinking. Knobel and Lankshear (2007) suggest that individuals who assume that digital technologies have not fundamentally changed society, but have only rendered society more complex, will continue to look to others – individuals and institutions – for expertise and authority. This mindset characterizes instrumental knowers and, often, socialized learners, in Kegan’s (1994) framework. However, those persons who understand that digital technologies have led to expertise being widely distributed, and authority as collectively constructed, are better positioned to fully participate in a digitally connected world. And, such use has positive implications for cognitive development across the life span.

628

S. M. Cecil and L. Denise L.

References Alloway, T. P., Horton, J., Alloway, R. G., & Dawson, C. (2013). Social networking sites and cognitive abilities: Do they make you smarter? Computers and Education, 63, 10–16. Barthel, M., Mitchell, A., & Holcomb, J. (2016) Many Americans believe fake news is sowing confusion. Washington, DC: Pew Internet and American Life. http://www.journalism.org/2016/ 12/15/many-americans-believe-fake-news-is-sowing-confusion/ Backonja, U., Hall, A. K., & Thielke, S. (2014). Older adults’ current and potential uses of information technologies in a changing world: A theoretical perspective. The International Journal of Aging and Human Development, 80(1), 41–63. Burke, M., Marlow, C., & Lento, T. (2010, April). Social network activity and social well-being. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1909–1912). ACM. Boyd, D. (2008). Why youth (heart) social network sites: The role of networked publics in teen social life. In D. Buckingham (Ed.), Youth, identity, and digital media (pp. 119–142). Cambridge: MIT Press. Boyd, D., & Ellison, N. B. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13(1), 210–230. Cattell, R. B. (1971). Abilities: Their structure, growth, and action. New York: Houghton Mifflin. ISBN:0-395-04275-5. Chowdhry, A. (2017, March 3). Facebook launches a new tool that combats fake news. Forbes Magazine. https://www.forbes.com/sites/amitchowdhry/2017/03/05/facebook-fake-news-tool/ #5001bf8f7ec1 Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86(1), 79–122. https://doi.org/10.3102/03465543115582065 Coiro, J., Knobel, M., Lankshear, C., & Leu, D. J. (2008). Central issues in new literacies and new literacies research. In J. Corio, M. Knobel, C. Lankshear, & D. J. Leu (Eds.), The handbook of research on new literacies (pp. 1–21). New York: Lawrence Erlbaum Associates. Donath, J., & Boyd, D. (2004). Public displays of connection. Blue Tuesday Technology Journal, 22 (4), 71–82. Drago-Severson, E. (2008). 4 practices serve as pillars for adult learning. JSD: The Learning Forward Journal, 29(4), 60–63. Duggan, M., & Smith, A., (2016). The political environment on social media. Washington, DC: Pew Research Center. http://www.pewinternet.org/2016/10/25/the-political-environment-onsocial-media/ Ellison, N. B., Steinfield, C., & Lampe, C. (2007). The benefits of Facebook “friends”: Social capital and college students’ use of online social network sites. Journal of Computer-Mediated Communication, 12(4), 1143–1168. Ellison, N. B., Steinfield, C., & Lampe, C. (2011). Connection strategies: Social capital implications of Facebook-enabled communication practices. New Media & Society, 13(6), 873–892. Evans, B. M., Kairam, S., & Pirolli, P. (2010). Do your friends make you smarter? An analysis of social strategies in online information seeking. Information Processing and Management, 46(6), 670–692. Facer, K., & Furlong, R. (2001). Beyond the myth of the ‘Cyberkid’: Young people at the margins of the information revolution. Journal of Youth Studies, 4(4), 451–469. Fanning, J., Mullen, S. P., & McAuley, E. (2012). Increasing physical activity with mobile devices: A meta-analysis. Journal of Medical Internet Research, 14(6), 161–172. https://doi.org/ 10.2196/jmir.2171 Gil-Or, O., Levi-Belz, Y., & Turel, O. (2015). The “Facebook-self”: Characteristics and psychological predictors of false self-presentation on Facebook. Frontiers in Psychology, 6, 99. https:// www.ncbi.nlm.nih.gov/pmc/articles/PMC4330900/

26

Digital Technologies and Adults: Social Networking, Holding. . .

629

Greenhow, C. (2011). Online social networks and learning. On the Horizon, 19(1), 4–2. https://doi. org/10.1108/10748121111107663 Greenhow, C., & Robelia, B. (2009). Old communication, new literacies: Social network sites as social learning resources. Journal of Computer-Mediated Communication, 14(4), 1130–1161. Hampton, K., Goulet, L.S., Rainie, L, & Purcell, K. (2011, June 16). Social networking sites and our lives. Washington, DC: Pew Research Center: Internet, Science & Tech. http://www. pewinternet.org/2011/06/16/social-networking-sites-and-our-lives/ Hargittai, E., & Hinnart, A. (2008). Digital inequality: Differences in young adults’ use of the Internet. Communication Research, 35(5), 602–621. Helsper, E. J., & Eynon, R. (2010). Digital natives: Where is the evidence? British Educational Research Journal, 36(3). https://doi.org/10.1080/01411920902989227 Jenkins, H., Purushotma, R., Weigel, M., Clinton, K., & Robison, A. J. (2009). Confronting the challenges of participatory culture: Media education for the 21st century. Reports on digital media and learning. The John C. and Catherine T. MacArthur Foundation. Cambridge, MA: MIT Press. Jones, S., Millermaier, S., Goya-Martinez, M., & Schuler, J. (2008). Whose space is MySpace? A content analysis of MySpace profiles. First Monday, 13(9). http://firstmonday.org/article/view/ 2202/2024 Kegan, R. (1994). In over our heads: The mental demands of modern life. Cambridge, MA: Harvard University Press. Kegan, R. (1982). The evolving self: Problems and process in human development. Cambridge, MA: Harvard University Press. Krashen, S. (1990). How reading and writing make you smarter, or how smart people read and write. In J. E. Alatis (Ed.), Georgetown University round table on language and linguistics 1990. Linguistics, language teaching and language acquisition: The interdependence of theory, practice and research. Washington, DC: Georgetown University Press. Knobel, M., & Lankshear, C. (2007). Sampling “the new” in new literacies. In M. Knoble & C. Lankshear (Eds.), A new literacies sampler (pp. 1–22). New York: Peter Lang. Lapowsky, I. (2017, June 17). In a fake fact era, schools teach the ABCs of news literacy. Wired Magazine. https://www.wired.com/2017/06/fake-fact-era-schools-teach-abcs-news-literacy/ Lenhart, A., Madden, M., Smith, A., Purcell, K., Zickuhr, K., & Rainie, L. (2011). Teens, kindness and cruelty on social network sites. Washington, DC: Pew Research Center. Retrieved January 30, 2017 at: http://www.pewinternet.org/2011/11/09/teens-kindness-and-cruelty-on-social-network-sites/ Mitchell, A., Gottfried, J., Kiley, J., & Matsa, K. (2014). Political polarization and media habits. Washington, DC: Pew Research Center. MMS Education. (2009). A survey of K-12 educators on social networking and content sharing tools. MCH Strategic Data, MMS Education. Retrieved January 30, 2017 at: http://www.edweb. net/fimages/op/K12Survey.pdf Morris, M. R., Teevan, J., & Panovich, K. (2010, May 23). A comparison of information seeking using search engines and social networks. In Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media, Washington, DC. (pp. 291–294). Menlo Park, CA: AAAI Press Patient Protection and Affordable Care Act, 42 U.S.C. § 18001 (2010). Pew Research Center: Internet and Technology. (2017, January 11). What social media platforms are most popular. http://www.pewinternet.org/chart/which-social-media-platforms-are-most-popular/ PBS NewsHour. (2016). Did fake news influence the outcome of election 2016. http://www.pbs.org/ newshour/extra/daily_videos/why-is-it-important-for-news-sources-to-be-trustworthy/ Perrin, A., & Duggan, M. (2015). Americans’ Internet access: 2000–2015. As Internet use nears saturation for some groups, a look at patterns of adoption. Washington, DC: Pew Research Center. http://www.pewinternet.org/2015/06/26/americans-internet-access-2000-2015/ Personal Responsibility and Work Opportunity Reconciliation Act of. (1996). H.R. 3734 – 104th Congress. Piaget, J. (2008). Intellectual evolution from adolescence to adulthood. Human Development, 51(1), 40–47.

630

S. M. Cecil and L. Denise L.

Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education, 66, 211–227. Prensky, M. (2001). Digital natives: Partnering for real learning. Thousand Oaks, CA: Corwin Press. Purcell, K., & Rainey, L. (2014). Americans feel better informed thanks to the Internet. Pew Internet Center Research and Technology. http://www.pewinternet.org/2014/12/08/better-informed/ Purcell, K., Heaps, A., Buchanan, J., & Friedrich, L. (2013). How teachers are using technology at home and in their classrooms. Washington, DC: Pew Research Center. Rainie, L., & Smith, A. (2012). Politics on social networking sites. Washington, DC: Pew Research Center. Schaie, K. W., & Zanjani, F. A. K. (2006). Intellectual development across adulthood. In C. Hoare (Ed.), Handbook of adult development and learning (pp. 99–122). New York: Oxford University Press. Shapiro, L. A. S., & Margolin, G. (2014). Growing up wired: Social networking sites and adolescent psychosocial development. Clinical Child and Family Psychology Review, 17(1), 1–18. https://doi.org/10.1007/s10567-013-0135-1. Smith, A. (2014). Older adults and technology use. Washington, DC: Pew Research Center. Stanovich, K. E. (1993). Does reading make you smarter? Literacy and the development of verbal intelligence. Advances in Child Development and Behavior, 24, 133–180. Taylor, K. (2006). Autonomy and self-directed learning: A developmental journey. In C. Hoare (Ed.), Handbook of adult development and learning (pp. 196–218). New York: Oxford University Press. Taylor, K., Marienau, C., & Fiddler, M. (2000). Developing adult learners: Strategies for teachers and trainers. San Francisco: Jossey-Bass. Thurlow, C. (2006). From statistical panic to moral panic: The metadiscursive construction and popular exaggeration of new media language in the print media. Journal of Computer-Mediated Communication, 11(3), 667–701. Treas, J., & Hill, T. (2008). Social trends and public policy in an aging society. In M. C. Smith & N. DeFrates-Densch (Eds.), Handbook of adult learning and development (pp. 763–783). New York: Routledge. Walther, J. B., & Parks, M. R. (2002). Cues filtered out, cues filtered in: Computer-mediated communication and relationships. In M. L. Knapp & J. A. Daly (Eds.), Handbook of interpersonal communication (3rd ed.pp. 529–563). Thousand Oaks, CA: Sage. Winnicott, D. W. (1973/1992). The child, the family, and the outside world. New York: Perseus Books. Wellman, B., Haase, A. Q., Witte, J., & Hampton, K. (2001). Does the Internet increase, decrease, or supplement social capital? Social networks, participation, and community commitment. American Behavioral Scientist, 45(3), 436.

M. Cecil Smith, Ph.D. (University of Wisconsin-Madison), is Professor of Learning Sciences and Human Development and Associate Dean for Research and Graduate Education, in the College of Education and Human Services at West Virginia University. Prior to coming to WVU in 2013, he was Professor of Educational Psychology at Northern Illinois University. His primary research interests pertain to adults’ literacy skills and practices and adult education. This research has been funded by the National Center for the Study of Adult Learning and Literacy and the International Literacy Association. Denise L. Lindstrom, Ph.D. (Iowa State University), is an Assistant Professor in the Department of Curriculum and Instruction/Literacy Studies at West Virginia University. Her research interests focus on in-school practices with emerging technologies, teacher attitudes toward and perceptions of technology integration, effective professional development with digital media, and integrating STEAM (science, technology, engineering, arts, and mathematics) in K-12 classrooms.

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

27

Beat Adrian Schwendimann

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Concept Maps and Knowledge Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Knowledge Integration Maps and Task Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Concept Maps as Versatile Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Generating Concept Maps to Elicit and Add Ideas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Concept Maps as Tools to Distinguish and Sort Out Alternative Ideas . . . . . . . . . . . . . . . . . . . . . . . . Case Study: Critiquing as an Alternative to Generating Concept Maps . . . . . . . . . . . . . . . . . . . Concept Maps as Assessment Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Forms of KIM Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Case Study: Concept Maps to Distinguish Expert and Novice Knowledge . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

632 634 636 639 639 643 646 655 657 668 674 677 678 678

Abstract

Concept maps can serve as versatile tools for learning, teaching, and assessment to support integrating complex concepts. Research suggests that concept maps can be successfully implemented in a wide variety of settings, from K12 to higher and professional education. However, the effectiveness of concept maps depends on different factors, such as concept map training, choosing a suitable form of concept map to match the task and learner, and how to evaluate concept maps. This chapter presents two case studies that use a particular form of concept map, a Knowledge Integration Map, to illustrate different concept mapping tasks and

B. A. Schwendimann (*) University of California, Berkeley, CA, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_86

631

632

B. A. Schwendimann

evaluations. This chapter concludes that, if implemented thoughtfully, concept maps can be versatile tools to support knowledge integration processes toward a deeper understanding of the relations and structures of complex concepts. Keywords

Concept map · Assessment tool · Learning tool · Knowledge integration · Knowledge Integration Map

Introduction As the complexity of knowledge continues to increase at an unprecedented pace (Barnett, 2000), teachers and students require powerful tools that support connecting complex concepts. Making sense of complex problems requires connecting concepts and eliciting relations between concepts. Sense-making refers to the processes of creating a structure of related concepts, such as placing “items into frameworks” (Weick, 1995, p. 6) and continually seeking “to understand connections” (Klein, Moon, & Hoffman, 2006, p. 71) that allow solving authentic complex problems. When trying to make sense of concepts, learners of all ages, from young children to adults, hold a rich repertoire of dynamically connected, coexisting, and often conflicting alternative concepts about the world around them (Davis, 2003; Davis & Linn, 2000; diSessa, 2008; Linn, 2002; Slotta, Chi, & Joram, 1995; Songer, 2006) rather than a consistent understanding. Conflicting alternative concepts can coexist because they are often contextualized (Davis, 2004). Consequentially, students often fail to connect concepts from one context to another (diSessa, 1988). Prior concepts are not simply exchanged for new concepts because concepts are embedded in a dynamic network where they define and constrain each other (Demastes, Good, & Peebles, 1995; diSessa, 2008; Park, 2007). Research suggests that in order to form more integrated knowledge, learners need to add and distinguish new concepts and connections to their existing repertoire of concepts rather than replace existing concepts (Demastes et al., 1995; Linn, 2008; Strike & Posner, 1992). Instead of seeing existing concepts as obstacles that need to be replaced, constructivist approaches to learning aim to add new concepts and, through application in different contexts, help learners develop criteria to distinguish which and when concepts are relevant (Linn, 2008). To facilitate learning processes, concept maps can serve as tools to elicit relations between concepts within and across contexts. A concept map can be described as a node-link diagram showing the semantic relationships between concepts. The technique for constructing concept maps is called “concept mapping.” A concept map consists of nodes, arrows as linking lines, and linking phrases that describe the relationship between nodes. Two nodes connected with a labeled arrow are called a proposition (see Fig. 1). Like other node-link diagrams, concept maps consist of visuospatially arranged nodes and links, but additionally they also present semantic information in the form of link labels.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

633

Fig. 1 Concept map of a concept map

Concept maps are versatile graphic organizers that can represent many different forms of relationships between concepts. The relationship between concepts can be articulated in linking phrase, for example, “leads to” (causal), “consists of” (partwhole), “follows” (temporal), “is inside of” (spatial), “increases” (quantified), or “is different than” (comparison). Nodes (usually nouns) and linking phrases (usually verbs) form a semantic network of propositions (see Table 15). The theoretical framework of concept mapping is based on David Ausubel’s assimilation theory of meaningful learning, which stresses the importance of individuals’ existing cognitive structures (prior knowledge) in being able to learn new concepts. Knowledge needs to be structured to be meaningful (Bransford, Brown, & Crocking, 2000). David Ausubel (1963; Ausubel, Novak, & Hanesian, 1978) discussed the importance of the hierarchical arrangement of information within organizational tools. Inspired by this framework, Joseph D. Novak and his research team at Cornell University in the 1970s developed concept mapping as a way of graphical representation of concepts, based on their research on understanding changes in children’s knowledge of science (1984). With its emphasis on actively engaging learners in eliciting and connecting existing and new concepts, concept mapping is considered to be consistent with constructivist epistemology. Concept map activities can support eliciting existing and missing concepts and connections through the process of visualizing them as node-link diagrams. Research indicates that re-representing text in a concept mapping format can be done in a fairly automated way without requiring construction of new or revision of existing connections between concepts (Holley, Dansereau, & Harold, 1984; Karpicke & Blunt, 2011). Greater benefit may arise if the concept map activity constrains concepts and relations to a novel format (see sections “Knowledge Integration Maps and Task Design” and “Concept Maps as Versatile Tools”).

634

B. A. Schwendimann

Concept Maps and Knowledge Integration To make sense of complex concepts, learners need to connect and distinguish concepts. The “knowledge integration” pattern operationalizes constructivist learning processes by describing learning as the process of integrating concepts through the cognitive processes of eliciting, adding, connecting, critiquing, distinguishing, sorting out, refining, and applying concepts in a broad range of contexts (Bransford et al., 2000; Linn & Eylon, 2006). The “knowledge integration” pattern provides an operational framework for concept mapping activities as it focuses on eliciting existing concepts and connections through the process of visualizing them as nodes and links (see Table 1). “Knowledge integration” uses the term “ideas,” while concept mapping refers to them as “concepts.” Concepts can be defined as a generalized, perceived regularity or pattern, similar to Plato’s essence of an object, for example, “house” or “tree.” A concept refers to a shared, abstract pattern – in contrast to a concrete example, such as “this particular tree” (Novak & Gowin, 1984). The explicitness and compactness of concept maps can help keeping a big-picture overview (Kommers & Lanzing, 1997). The “gestalt effect” of concept maps allows viewing many concepts at once, increasing the probability of identifying gaps and making new connections. Generating concept maps requires learners to represent concepts in a new form which can pose desirable difficulties (Bjork & Linn, 2006; Linn, Chang, Chiu, Zhang, & McElhaney, 2010) – a condition that introduces difficulties for the learner to slow down the rate of learning and enhance long-term learning outcomes, retention, and transfer. The process of translating concepts from texts and images to a node-link format may foster deeper reflection about concepts and their connections (Weinstein & Mayer, 1983) and prevent rote memorization (Scaife & Rogers, 1996). Throughout a curriculum, learners can add new concepts to Table 1 Concept mapping for knowledge integration Knowledge integration process Eliciting existing ideas Adding new ideas and connecting to existing ideas in learners’ repertoires

Distinguishing/critiquing ideas

Sorting out ideas/refining

Applying ideas

Concept mapping activity Concept maps can be used as a pretest activity to elicit existing concepts New concepts can be added to existing propositions in a concept map. If several alternative relations between two concepts are possible, learners have to decide which one to use in the map. If applicable, learners decide which concepts to add to the map After adding new concepts, concepts can be rearranged into new groups, and the concept map network structure might need revision to reflect the new concepts Different sources of evidence can be used as references to sort out concepts and further refine the concept map Concept maps can be used as resources to generate explanations of scientific phenomena

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

635

their existing concept map. Unlike textbooks, concept maps have no fixed order and may thereby encourage knowledge integration strategies. For example, a student may decide to add the most important or most central concept first. Developing criteria to select concepts requires deeper processing than the student might normally exercise when reading text. Students need to develop metacognitive strategies to distinguish alternative concepts, for example, through predicting outcomes and explanation generation (Bransford et al., 2000). The scaffolded process of adding and revising concept maps requires students to decide which concepts and connections to include. The decision-making process may foster the generation of criteria to distinguish pivotal concepts. Clustering related concepts in spatial proximity can support learners’ reflections on shared properties of and relationships between concepts. Links between concepts from different areas can be seen as indication for knowledge integration across different contexts. Concept maps may support making sense of concepts by eliciting semantic relations between concepts (see Table 1). Concept maps can change students’ understanding beyond remembering isolated concepts to constructing meaningful connections of organized knowledge (Bransford et al., 2000). Mason (1992) observed that students exposed to “mapping” during instruction demonstrated “insight into the interrelatedness of concepts” (p. 60), instead of seeing scientific knowledge as a collection of isolated facts. Knowledge integration suggests that a successful curriculum starts by eliciting existing alternative ideas about scientific phenomena. Learners need tools to elicit their existing ideas and distinguish alternative ideas. Ideas cannot be understood in isolation but need to be connected to existing ideas (Bruner, 1960). Learning an idea means seeing it in relation to other ideas, distinguishing it from other ideas, and being able to apply it in specific contexts. To learn a subject is to have actively integrated key ideas and the relations between them. Knowledge integration activities are designed to help learners construct more coherent understanding by developing criteria for the ideas that they encounter. Research suggests that concept mapping is especially efficient, in comparison to other interventions such as outlining or defining ideas, for learning about the relations between ideas (Cañas, 2003). Concept maps as knowledge integration tools elicit ideas (concepts) as nodes and relations between ideas as labeled arrows. The visual format of concept maps can foster critical distinctions between alternative ideas and relationships, either individually or through collaboration in communities of learners Cognitive science research (e.g., see Bransford et al., 2000) indicates that new ideas need to be connected to existing ideas to be stored in the long-term memory. Eliciting existing ideas brings them from long-term memory to working memory. Learners make sense of new ideas by integrating them into their existing repertoire of ideas. Knowledge integration suggests that ideas should be presented in multiple contexts and support generation of connecting ideas across contexts. Multiple representations of ideas (e.g., dynamic visualizations, animations, pictures, or diagrams) can facilitate learning and performance supporting different accounts of scientific phenomena (Ainsworth, 2006; Pallant & Tinker, 2004), for example, by complementing each other or constraining interpretations (Ainsworth, 1999).

636

B. A. Schwendimann

However, learners making connections between different representations can be challenging as the representations are connected through multiple relations that are often not intuitively obvious to the learner (Duncan & Reiser, 2005).

Knowledge Integration Maps and Task Design Applying knowledge integration patterns to concept mapping lead to the development of a novel form of concept map, called Knowledge Integration Map (KIM), which aims to elicit and scaffold cross-field connections by categorizing concepts through the spatial arrangement into domain-specific levels (see Table 2). Bruner stated that “virtually all cognitive activity involves and is dependent on the process of categorizing” (Bruner, Goodnow, & Austin, 1986, p. 246). Providing such scaffolding for sorting out and grouping related concepts into categories can support knowledge integration. Concept mapping tasks are found in many different forms and provide different amounts of constraints. The task range from low-directed maps where students can freely choose their concepts and labels to highly directed tasks where students fill in concepts out of a given list into blanks in a given skeletal network structure (Novak & Cañas, 2006). Highly constrained maps can be beneficial for low performance and younger students, although they provide less insight into students’ partial knowledge. Free drawing concept maps provide the most insight, but do not allow for standardized comparisons between students. Constraining students by providing them with a set of concepts or link labels allows for standardized or even automated comparison across students on the exact same content but appears to be more challenging for many students than working from memory. They must discipline themselves to use the given concepts rather than to freely follow their thought patterns (Fisher, Wandersee, & Moody, 2000). KIMs aim for a balanced design by providing students with a small set of concepts but allowing them to generate their own connections and labels. This design allows comparing maps of different students with each other. KIM worksheets consist of five elements: (1) focus question, (2) evolution-specific levels (genotype and phenotype), (3) instructions, (4) given list of concepts, and (5) starter map (see Fig. 2). KIM tasks are created through the following process: 1. Generate focus question. 2. Based on domain experts and textbooks, identify key concepts for the map that allow answering the focus question adequately. 3. Structure concept map into field-specific levels, for example, in biology, genotype/phenotype, or individual/population, and in chemistry, micro/macro/ symbolic. 4. Create a starter map. 5. Create a concept map training activity.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

637

Table 2 Characteristics of Knowledge Integration Maps (KIM) Domain-specific levels

Given list of concepts but free labels and links

Concept map training activity Starter map

This characteristic combines aspects of concept mapping with aspects of Venn diagrams. The KIM drawing area is divided into several domain-specific vertical levels, for example, in biology into the micro-level “genotype” and the macro-level “phenotype.” This arrangement requires learners to (a) generate criteria and categorize concepts, (b) sort out concepts into according levels (clustering), and (c) generate connections between concepts within and across levels. Sorting out and grouping concepts spatially according to semantic similarity requires learners to generate criteria and make decisions about information structure that is latent in texts (Nesbit & Adesope, 2006). This is expected to support knowledge integration by showing concepts in contexts to other concepts and eliciting existing (and missing) connections within and across levels. Cross-links are especially desirable as they can be interpreted as “creative leaps on the part of the knowledge producer” (Novak & Cañas, 2006) and support reasoning across ontologically different levels (Duncan & Reiser, 2007) Many students have difficulties distinguishing important concepts in a text, lecture, or other form of presentation. Part of the reason is that many students learn only to memorize but not distinguish and sort out concepts. They fail to construct propositional frameworks and see learning as “blur of myriad facts, dates, names, equations, or procedural rules to be memorized, especially in science mathematics and history” (Novak & Cañas, 2006). Ruiz-Primo et al. (2000) compared concept mapping tasks with varying constraints and found that constructing a map using a given list of concepts (forced-choice design) reflected individual student differences in connected understanding better than more constrained fill-the-map forms. Complex topics, such as evolution, consist of large numbers of concepts that often make it challenging for novices to identify key concepts. Providing students with a list of expert-selected key concepts can serve as signposts and model expert understanding. Concept maps generated from the same set of concepts allow for better scoring and comparison. Students’ alternative concepts are captured in the concept placement, link labels, and link direction. Knowledge Integration Maps can help students elicit relations between concepts, distinguish central concepts, and make sense of complex science topic such as evolution Students need initial training activities to learn the concept mapping method and generate criteria for concept map critique Building a KIM from scratch can be challenging. Providing a starter map as a partially worked example could reduce anxiety (Czerniak & Haney, 1998). Critiquing and revising concept maps with starter maps requires a completion strategy (Chang, Chiao, Chen, & Hsiao, 2000; Sweller, Van Merrienboer, & Paas, 1998) (continued)

638

B. A. Schwendimann

Table 2 (continued) Collaborative concept map activity

Focus question

Feedback and revision

Tools

KIMs are generated collaboratively in dyads. As each proposition is constrained to only one link, students are required to negotiate which connection to revise or generate. Students are required to generate criteria and negotiate with their partner The domain-specific focus question guides the construction of the KIM as learners select concepts and generate links to answer the focus question (Derbentseva, Safayeni, & Canas, 2007) Feedback and revision supports students’ knowledge integration through revisiting, reflecting, and revising existing and new concepts. Concept maps often need several revisions to adequately answer the focus question. Kinchin, De-Leij, and Hay (2005) suggested that generating several new concept maps could support revisiting concepts better than continuously revising one concept map. Starting new maps allows reviewing superordinate structures that otherwise persists without revision KIMs can be generated using paper-and-pencil or digital concept mapping tools such as Cmap (Cañas, 2004)

Fig. 2 Knowledge Integration Map worksheet

KIMs model what experts consider important concepts by providing a list of expert-selected concepts. Kinchin (2000a) noted that the number of given concepts should be kept small (around 10–20) to reduce complexity and time consumption. Based on an evaluation of major textbooks, state standards, and interviews with experts (for a discussion on expertise, see, e.g., Chi, Glaser, and Rees (1982), Schvaneveldt et al. (1985), Scardamalia and Bereiter (1991), and Hoffman (1998)), a small number of concepts (11) have been selected for the forced-choice design of the Knowledge Integration Map. The number of concepts was kept low in order to keep to size and complexity of the KIM reasonable for the given time constraints for its creation. A total of 55 connections are possible between the given 11 concepts, but not all propositions are of equal importance. (Considering each

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

639

direction individually and allowing for circular links to same concept, 11
11 = 121 connections are possible.) Students need to decide which connections are essential to represent their understanding. Additionally, each connection can go in either direction and be described by many different labels. Students need to match the directionality of the connection with the label and construct a label that accurately describes the nature of relations. As the map constrains students to only one connection for each relation, the students need to develop decision-making criteria. Students are free to generate their own links and labels. To model expert understanding, the given list of concepts includes only expert concepts, but no alternative concepts such as “need,” “intentionality,” or “want.” Alternative concepts can be expressed through concept placement and link labels.

Concept Maps as Versatile Tools Since their first conception, concept maps have been used by a wide variety of users in a broad range of settings (Cañas, 2003). Daley and Torre concluded that concept maps are used mainly in four different ways: (1) by promoting meaningful learning, (2) by providing an additional resource for learning, (3) by enabling instructors to provide feedback to students, and (4) by conducting assessment of learning and performance (Daley & Torre, 2010). As illustrated in Fig. 3, concept maps can be generated by curriculum designers (teachers or researchers) or students. Curriculum designers can use concept maps to identify core ideas and knowledge structures when designing or revising curricula (e.g., Edmondson, 1995; Martin, 1994; Starr & Krajcik, 1990). Concept maps can be used as assessment tools, for example, concept maps can serve as pretests or as embedded formative assessments to identify students’ prior ideas, which can be used to design curricula that connect to existing alternative ideas and provide feedback. Concept maps can be used as summative assessments to track changes in students’ understanding (see section “Concept Maps as Assessment Tools”). Concept maps can be used as advance organizers to provide an overview of core ideas prior to instruction (e.g., see Mistades, 2009) and illustrate the (otherwise often hidden) structures of knowledge. In technology-enhanced learning environments, concept maps can serve as dynamic user interfaces to navigate through activities (e.g., see Puntambekar, Stylianou, & Huebscher, 2003) (see more in section “Concept Maps as Assessment Tools”).

Generating Concept Maps to Elicit and Add Ideas The first steps of knowledge integration are eliciting existing ideas and connecting them to new ideas. Complex fields of knowledge, such as different areas of science, consist of a large number of ideas that are connected in different ways. In the context of biology, Schmid and Telaro commented that “The schools’ favored approach to teaching unfamiliar material is rote learning. Rote learning predictably fails in the

Fig. 3 Different uses of concept maps

640 B. A. Schwendimann

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

641

face of multilevel, complex interactions involved in biology. Concept mapping ... stresses meaningful learning, and appears to be ideally suited to address biological content” (Schmid & Telaro, 1990, pp. 78–79). As a learning tool, concept maps can support knowledge integration processes by eliciting core ideas and connections and making possible clusters or hierarchies visible. Watson (2005) found that graphic organizers such as concept maps can scaffold integrating students’ isolated ideas toward an organized interconnected network of ideas. Research indicates that the implementation of concepts maps can shift the epistemological authority from the teacher to the student, reduce emphasis on right and wrong answers, and create visual entry points for learners of varying abilities (O’Donnell, Dansereau, & Hall, 2002; Roth 1993). Several meta-analyses reviewed the effects of concept maps as learning tools. Horton et al. (1993) compared the effects of concept mapping reported in 19 classroom-implemented quantitative studies. The meta-analysis found that concept maps as learning tools produced generally medium-sized positive effects on student’s achievement and large positive effects on student’s attitudes. The mean effect size for studies using pre-made maps was 0.59. Concept maps generated by students in groups produced a mean effect size of 0.88. Nesbit and Adesope (2006) conducted a meta-analysis of 55 experimental and quasi-experimental studies in which students learned how to use concept maps. The study included 5,818 students ranging from fourth grade to postsecondary in fields such as science, psychology, statistics, and nursing. Across different conditions and settings, the study found that the use of concept maps was associated with increased knowledge retention, with mean effect sizes varying from small to large depending on how the concept maps were used. Cañas, Sanchez, and Brenes (2003) found concept maps to be effective learning tools with generally positive effects on knowledge acquisition. Kinchin critically reviewed recent studies on concept maps as learning tools in higher education and pointed out that review studies need to distinguish different forms of concept map activities (Kinchin, 2014). The effectiveness of concept maps as learning tools depends to some degree on finding the right degree of freedom to match the task and the abilities of learners. Concept maps range from very constrained forms (fillthe-blanks) to no constrictions (blank worksheet) (Cañas, Novak, & Reiska, 2012). Concept mapping tasks with few constraints can provide learners with a focus question while giving them free choice to select their own concepts and links. Medium constraint forms can provide learners with pre-made lists of concepts or linking phrases but give free choice of which concepts to connect. Highly constrained forms of concept maps can provide learners with a skeletal network structure and pre-made lists of concepts or linking phrases to be filled into blanks in the structure. Concept map can provide a how or why question as a “focus question” to describe the purpose of a concept map and guide concept map generation. Concept maps can be generated by hand using paper and pencil, flashcards, and post-its or by using computer software (exemplars are the freeware tool Cmap (http://cmap.ihmc.us/ or commercial tools such as “Inspiration” (http://www.inspira tion.com). Research indicates that using concept mapping software can facilitate construction, revision, and addition of hyperlinks and multimedia (Cañas, 2003).

642

B. A. Schwendimann

Independent of the setup or technique used to generate concept maps, concept mapping requires initial training to familiarize learners with the concept mapping generation principles and criteria for concept map evaluation. Concept maps have been investigated as learning tools in a wide variety of different fields from K-12 to higher education. Concept mapping research has mainly focused on science classrooms (see Table 3 for an overview of concept maps in science education) but has been extended to include a wide variety of disciplines and contexts, for example, language, mathematics, and history education (Kinchin & Hay, 2007). Study participants have ranged from elementary to higher education students, for example, middle school students (Coleman, 1998; Sizmur & Osborne, 1997), high school students (Reiska, Soika, & Cañas, 2018; Stensvold & Wilson, 1990), university students (Heinze-Fry & Novak, 1990; Kinchin, 2014; Pearsall et al., 1997), and pre-service teacher students (Mason, 1992). Concept maps can represent very simple partial ideas to complex connected networks of ideas, which make them usable for a wide range of learners. For example, Kern and Crippen (2008) used embedded Table 3 List of studies of concept maps as science learning tools by subject Subject Chemistry

Physics

Earth Science

Biology

Ecology Astronomy Medicine/ nursing

References Stensvold & Wilson, 1990; Markow & Lonning, 1998; Brandt et al., 2001; Nicoll, Francisco, & Nakhleh, 2001b; Liu, 2004; Uzuntiryaki & Geban, 2005; DeMeo, 2007; Oezmen, Demircioglu, & Coll, 2007; BouJaoude & Attieh, 2008; Kaya, 2008; Aydin, Aydemir, Boz, Cetin-Dindar, & Bektas, 2009; Mun, Kim, Kim, & Krajcik, 2014; Turan-Oluk & Ekmekci, 2018; Shawli, 2018 Bascones & Novak, 1985; Moreira, 1987; Pankratius, 1990; Carey & Spelke, 1994; Roth, 1994a; Roth, 1994b; Adamczyk & Willson, 1996; Pushkin, 1999; Reiska, Dahncke, & Behrendt, 1999; Anderson, Lucas, & Ginns, 2000; Van Zele, Lenaerts, & Wieme, 2004; Mistades, 2009; Suprapto, Prahani, Jauhariyah, & Admoko, 2018; Bakri & Muliyati, 2018 Ault, 1985; Hoz, Tomer, Bowman, & Chayoth, 1987; Rebich & Gautier, 2005; Snead & Snead, 2004; Englebrecht, Mintzes, Brown, & Kelso, 2005; Hsu, Wu, & Hwang, 2008; Hsu, 2008; Morfidi, Mikropoulos, & Rogdaki, 2018 Stewart, 1979; Novak, 1980; Heinze-Fry & Novak, 1990; Schmid & Telaro, 1990; Wallace & Mintzes, 1990; Okebukola, 1992; Trowbridge & Wandersee, 1996; Wandersee, 1996; Pearsall, Skipper, & Mintzes, 1997; Fisher, 2000; Kinchin, 2000a; Cakir & Crawford, 2001; Chang, Sung, & Chen, 2001; Kinchin, 2001; Mintzes, Wanderersee, & Novak, 2001; Odom & Kelly, 2001; Tsai & Huang, 2002; Brown, 2003; Preszler, 2004; Kinchin et al., 2005; Buntting, Coll, & Campell, 2006; Keraro, Wachanga, & Orora, 2007; Chang, 2007; Hmelo-Silver, Marathe, & Liu, 2007; Mintzes & Quinn, 2007; Kern & Crippen, 2008; Byrne & Grace, 2010; Cathcart, Stieff, Marbach-Ad, Smith, & Frauwirth, 2010; Grossschedl & Tröbst, 2018; Hamdiyati, Sudargo, Redjeki, & Fitriani, 2018 Brody, 1993; Heinze-Fry, 1998 Zeilik et al., 1997 Mahler, Hoz, Fischl, Tov-Ly, & Lernau, 1991; Edmondson, 1993; Edmondson, 1995; Irvine, 1995; Bressington, Wong, Lam, & Chien, 2018; El-Hay, El Mezayen, & Ahmed, 2018

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

643

concept maps in a 1-month-long biology unit. Using the electronic concept mapping tool Cmap (Cañas, 2004), students individually generated concept maps from a given list of ideas and revised them three more times throughout the curriculum. Students received feedback from peers and the teacher. Findings indicate that embedded concept maps can support students’ integration of biology ideas and reveal conceptual changes in students’ understanding. To track conceptual changes of students’ ideas in a university course, Trowbridge and Wandersee (1994) asked college students to individually generate concept maps to summarize specific lectures. Students generated ten different concept maps from a given list and selfchosen ideas. The instructor graded all concept maps and provided feedback. Results suggest that changes in superordinate core ideas can indicate conceptual changes in students’ understanding of complex ideas. Research indicates that concept mapping as learning tools may be particularly beneficial for lower-performing students (O’Donnell et al., 2002; Snead & Snead, 2004; Spaulding, 1989; Stice & Alvarez, 1987; Wise, 2007) and students with learning disabilities (Crank & Bulgren, 1993). Concept map activities can help low-performing students to a greater degree because they model the active inquiring approach often found in higher-performing students (Cañas, 2003), and it can provide scaffolds for a more organized and deliberative approach to learning. The minimal number of words and propositional forms used to represent ideas in a concept map might be beneficial especially for English language learners (ELL) and students of low academic abilities (Schmid & Telaro, 1990).

Concept Maps as Tools to Distinguish and Sort Out Alternative Ideas The next steps in knowledge integration are reflecting critically on alternative ideas and sorting them out. Several studies found that monitoring your own learning progress through reflection encourages students to revisit and reorganize their ideas (Chiu, 2008, 2009). Concept maps can be used as metacognitive tools that support learners by eliciting existing connections and reveal missing connections between ideas, especially cross-connections (Shavelson, Ruiz-Primo, & Wiley, 2005). This can help students to reflect and contrast their existing ideas with new ideas in the learning material. It can encourage students to build on their own ideas, rather than isolate new ideas from existing knowledge. Eliciting one’s understanding can promote student self-monitoring of their learning progress and support generating self-explanations. Self-explanations as an attempt to make sense of new ideas have been found beneficial for the integration of ideas (Chi, 2000). Ritchhart, Turner, and Hadar (2009) found that concept maps as a metacognitive tool can support student self-reflection about their conceptions of thinking and thinking processes. The reflection on links in concept maps can contribute to the development of reasoning skills (McMillan, 2010). Especially in less constrained concept map tasks, learners need to make decisions about which ideas and/or links to include in their map. Concept maps do not aim to include every possible idea and connection

644

B. A. Schwendimann

but a careful selection. Students need to generate criteria to identify and distinguish core ideas and their connections from alternative ideas and connections. Concept map generation and revision activities can encourage learners to revisit, reflect on, and revise their existing ideas. Critiquing is the process of creating a set of criteria, applying criteria to compare one’s own or other’s alternative ideas against each other, reflecting on how those ideas apply to alternative ideas, and selecting supported ideas based on evidence (Shen, 2010). Critique activities require students to use or develop criteria to reflect, revise their work, and self-monitor their learning progress (Chi, 2000) that can foster the development of metacognitive skills for lifelong autonomous learning. Critique activities encourage the elaboration of ideas and conjectures. Asking students to critique has been found to facilitate the development of coherent and generative criteria (Slotta & Linn, 2000). Critique is often applied in collaborative settings. In science, peer critique is a central aspect of the nature of science (Ford, 2008). Scientific knowledge is collaboratively constructed by the scientific community, which evaluates each other’s theories and findings (Wenger, 1998). Learners’ views of the nature of science influence their willingness to critique (Schwarz & White, 2005; Tabak, Weinstock, & Zvilling-Beiser, 2009). Many students seem to hold the objectivist view that scientific knowledge is discovered and static (Marcum, 2008) rather than consisting of constructed tentative models. When scientific ideas are understood as immutable products, there is little reason to critique. Linn and Eylon (2006) noted that critique activities can engage students to “question scientific claims and explore the epistemological underpinnings of scientific knowledge” (p. 536). From a situated learning perspective, critique activities in the classroom can mimic what professionals do in their communities (Lave & Wenger, 1991). Critiquing peer work can provide a driving force for revising one’s own work (Lehrer & Schauble, 2004). The social process of reaching agreement is critical in shaping one’s ideas (Clark & Sampson, 2008; Enyedy, 2005). In science education, collaboratively critiquing ideas requires learners to argue, negotiate, and make informed decisions (Berland & Reiser, 2009). Finding common ground can be a driving force for critique. To reach such common ground, students need to pose questions, make revisions, accept propositions, defend against criticism, and improve their criteria (Shen & Confrey, 2007). Brown and Campione (1996) showed that elementary students can form communities of learners that constructively share resources and review each other’s work. Students need authentic opportunities to develop criteria to distinguish valid alternative ideas based on evidence and scrutinize the reliability of sources (Cuthbert & Slotta, 2004; Davis & Kirkpatrick, 2002). DiSessa (2002, 2004) found that students are able to develop their own criteria to critique representations. A meta-study by Falchikov and Goldfinch (2000) found that student-generated criteria work better for peer assessment than using a set of given criteria. However, students have usually little opportunity to critique (Clark & Slotta, 2000; Grosslight, Unger, Jay, & Smith, 1991; Shen & Confrey, 2010). Students can (a) critique their own ideas, (b) a peer’s ideas, (c) common alternative ideas, or (d) experts’ ideas.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

645

(a) Critiquing one’s own ideas: Research indicates the difficulty of critiquing one’s own work, for both experts and novices (Guindon, 1990). People tend to discount ideas that contradict their existing ideas (Chinn & Brewer, 2001; Kuhn, 1962; Schauble, Glaser, Duschl, Schulze, & John, 1995). For example, students as well as professional engineers often stick to their initial design strategies and resist alternative ideas (Cuthbert & Slotta, 2004). (b) Critiquing a peer’s ideas: Analyzing a peer’s work may be easier than evaluating expert-generated work. Critiquing peer work can motivate students to improve their own work and better understand what needs to be revised. Comparing one’s own ideas against those of a peer can help students to value their own ideas while developing criteria to critically review them. However, critiquing peers can be socially difficult as students tend to give overly generous or overly critical feedback (Hoadley & Kirby, 2004). Schwendimann found that critiquing peergenerated concept maps anonymously can facilitate productive feedback and improve the quality of concept maps in subsequent revision steps (Schwendimann, 2014b). (c) Critiquing common alternative ideas: Providing students common alternative ideas can serve as a starting point for critique. Critiquing and revising concept maps with deliberate flaws are partial solutions that require a completion strategy (Chang et al., 2000; Sweller et al., 1998; Van Merriënboer, 1990). Giving all students the same artifact equalizes conditions, compared to a peer-critique activity where each student receives different ideas from peers. On the negative side, having to compare, critique, and select ideas from three different sources (e.g., two collaborating group members and a given concept map) could increase cognitive load in some students. (d) Comparing one’s own ideas to expert ideas could help students identify gaps in their understanding. Previous studies using expert-made concept maps often presented maps to students as a form of summary to be studied (O’Donnell et al., 2002). In these settings, students did not actively generate their own connections or critically evaluate presented propositions. A meta-analysis (Horton et al., 1993) found that studying expert-made and student-generated concept maps seemed to have an equally positive effect on improving students’ achievement. On the other hand, Cliburn (1990) noted that teacher-generated concept maps could support integrative understanding. O’Donnell et al. (2002) found that students could recall more central ideas when they learned from expert-made knowledge maps than when they learned from texts. Students with low verbal ability or low prior knowledge often benefited the most. Chang et al. (2001) compared generating concept maps to critiquing them using a computer-based tool that provided feedback by comparing student-generated maps to an expertgenerated benchmark map. Generating and critiquing concept maps led to similar results, both better than a control group that did not use concept maps. However, Novak (1980) observed that studying pre-made expert maps in genetic instruction could be confusing to some students as expert-generated concept maps could be seen by students as the one correct solution. According to the

646

B. A. Schwendimann

underlying constructivist view of concept maps, expert-generated maps can be useful but should not be presented as final answers but as one of many possible solutions. The following case study serves to illustrate the use of concept maps as learning tools by distinguishing generating and critiquing concept maps.

Case Study: Critiquing as an Alternative to Generating Concept Maps Previous studies suggested that a combination of generating and critiquing concept maps can support integrating evolution concepts within and across levels but also that the combination of activities can be time-consuming (Schwendimann & Linn, 2015). As time in science classrooms is limited and valuable, this study aims to identify and develop a more efficient concept mapping activity by distinguishing the time requirements and learning effects from either collaboratively generating or critiquing concept maps that integrate phenotype- and genotype-level concepts. Both cogeneration and co-critique of concept maps are expected to facilitate learning gains, but they might differ in their time requirements. A week-long technology-enhanced unit on evolution, delivered through the WISE platform (Linn & Hsi, 2000), was implemented by a science teacher in four classes in a high school with an ethnically and socioeconomically diverse student population of 9th and 10th grade students (n = 93). The high school had an enrolment of around 2000 students and was located in the urban fringe of a large city. The participating teacher was an experienced master teacher with nine years of teaching experience. The teacher implemented the unit as an introduction to the subsequent topic of evolution after completing several weeks of introduction to genetics. The teacher randomly grouped students into dyads. Students worked collaboratively in dyads by sharing a computer throughout the project. Student dyads in each class were randomly assigned to the concept map generation (n = 41) or critique (n = 52) task. Student dyads in the generation group created their own connections from a given list of concepts. Generating their own connections allows students to elicit their existing and missing connections and organize concepts in context to each other. Student dyads in the critique group received a concept map that included errors in connections and concept placements commonly found in student-generated concept maps and the literature. The concept maps of the generation and critique groups consisted of the same concepts. Students were instructed to generate their own criteria to review the presented concept map and negotiate with their partner how to critique and revise the map. Knowledge Integration Map task: Knowledge Integration Maps used in this study divide the drawing space into the evolution-specific levels “genotype” and “phenotype.” KIMs were designed to provide a balance between constraints (usage of given list of concepts) and openness that allows expressing a variety of concepts (studentgenerated connections and placement of concepts). Learners received a list of concepts that needed to be categorized and placed in the corresponding areas.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

647

Procedure: The teacher introduced all students to the concept mapping method and the Cmap software. Students individually received identical pretests and posttests delivered through the WISE environment. The WISE unit consisted of five consecutive activities. The first three activities focused on changes in the genotype caused by mutations. The third activity presented an overview of the connections between mutations and genetic variability in the gene pool. Student dyads either generated or critiqued a genotype-level KIM using the provided six concepts DNA, gene, allele, genetic drift, genetic diversity, and mutation. The second section focused on phenotype-level concepts. The fourth activity presented two guided inquiry activities to explore the connections between mutations, natural selection, and genetic diversity. The fifth activity introduced the concept “genetic drift” as an additional selection process and explored the effects of small population sizes on genetic drift. Finally, student dyads either generated or critiqued a phenotype-level KIM, using the provided five concepts population size, natural selection, environment, adaptation, and fitness (see Table 4). Data sources: This case study used a pretest-posttest design to measure changes in knowledge integration. The assessment consisted of five identical two-tier items (combining multiple choice and a subsequent explanation), three short essay items, and a KIM generation task. The first tier presents students with multiple choices of common misconceptions, followed by the second tier that asks students to provide an explanation for their choice in the first tier. The two-tier item design lowered the chances for random selection in the multiple-choice tier as students had to justify their choice (Tsui & Treagust, 2010). The pre-/posttest items underwent systematic revisions after pilot testing with biology teachers, students, and assessment experts. Liu, Lee, and Linn (2010) reported the validity and reliability of knowledge integration test construction and analysis. The KIM generation task provided students with the combined list of concepts from the embedded KIM 1 and 2 (11 concepts: natural selection, adaptation, DNA, mutation, genetic drift, genetic diversity, population size, gene, fitness, allele, and environment). Students were instructed to place Table 4 KIM tasks

KIM task Generation group

Critique group

Training (individual and in dyads) KIM generation and critique activity KIM generation and critique activity

Pretest (individual) Genotype and phenotype KIM generation and critique activity Genotype and phenotype KIM generation and critique activity

Embedded KIM 1: genotype level (in dyads) KIM generation map 1

Embedded KIM 2: phenotype level (in dyads) KIM generation map 2

KIM critique map 1

KIM critique map 2

Posttest (individual) Genotype and phenotype KIM generation and critique activity Genotype and phenotype KIM generation and critique activity

648

B. A. Schwendimann

Table 5 Knowledge integration rubric: sample item – “What changes occur gradually over time in groups of finches that live in different environments?”

No answer (blank) Offtask Irrelevant/ incorrect Partial Basic Complex

KI Score 0

Sample answers None

1 2

I don’t know Finches develop new beaks to adapt to a new environment

3 4

Finches inherit traits from their parents Finches have differently shaped beaks that give them different chances to survive natural selection Natural selection causes those finches with helpful mutations to their beaks to be more genetically fit and adapt to the environment better. Therefore, the finches with the beaks adapted to their environment are more likely to reproduce, and the trait gradually becomes dominant in the group

5

the concepts in the corresponding area (genotype or phenotype) and generate connections (within and across levels). Analysis: The two-tier items were scored using a five-level knowledge integration rubric (see Table 5) (Linn, Lee, Tinker, Husic, & Chiu, 2006). Higher knowledge integration scores indicate more complex normative links among different concepts relevant to the genetic basis of evolution. Paired t-tests, chi-square tests, and effect sizes were calculated. Multiple regression analysis and ANOVA were used to investigate whether the two groups (generation and critique) differed from each other in learning gains and KIM usage. This study used a multi-tier KIM analysis method (Schwendimann, 2014a), including presences or absence of connections, quality of connections, network density, and spatial placement of concepts. The main goal of the KIM analysis was to identify students’ nonnormative concepts about evolution and track changes throughout the sequence of concept maps.

KIM Generation Analysis • Propositional level: A five-level knowledge integration rubric for KIM propositions (Schwendimann, 2014a) was used to determine changes in link quality. The propositional analysis focused on overall and essential connections. Essential connections were identified from a benchmark KIM generated by a group of experts. • Network level: An analysis method that focuses only on isolated propositions does not account for the network character of a whole map. To capture this information, network analysis methods were used to identify changes in the prominence (incoming and outgoing connections) of expert-selected indicator concepts: “mutation” for the genotype-level and “natural selection” for the phenotype level. Multiplied with the KI score for each connection, a “weighted

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

649

prominence score” for each of the two indicator concepts was calculated (see Figs. 4 and 5). A better integration of genotype and phenotype concepts would be expected to lead to a more frequent use of the normative concept “mutation.”

To describe semantic changes in the relationships between concepts, qualitative variables are needed. This study used the structure-behavior-function (SBF) framework to create the super-categories of the taxonomy. The SBF framework was originally developed by Goel (Goel & Chandrasekaran, 1989), to describe complex systems in computer science, and then applied to complex biological systems by Hmelo-Silver and colleagues (2007; Liu & Hmelo-Silver, 2009). The taxonomy is both theory-driven and informed by empirical data from previous studies 97

100 92 90 in percent (of each test)

80 70 60 50

Pretest KIM

40

PosCest KIM

30 20 10

3

5

1

2

0

0

0 Undefined

Structure

Behavior

Function

Fig. 4 Changes in KIM relation super-categories from pretest to posttest

60 54

in percent (of each test)

50

47 38

40 30

27 Pretest KIM

20

Posttest KIM

12 8

10 3

1

1 1

0 1

4

2 0

1 0

0 0

0 0

0

0 0

0 0

Ca u

sa

er m et -d

l-p

in i

pe ro /P te

Ca us al

ib u At tr

st ic ro ba bi lit y Qu an tif ie d M ec Pr ha oc ni ed st ur ic al -T Fu em nc po tio ra na l l(n Te ee le d ed ol og ) ica l(w an ts )

rty

ity ro xim

lp ia at Sp

y/ rit ila Sim

ris on pa

te la re

t-W Pa r

Co m

d

l be la Un

No

ho le

0

Fig. 5 Changes in KIM relationship subcategories from pretest to posttest

650

B. A. Schwendimann

(Schwendimann, 2014a; Schwendimann & Linn, 2015). The taxonomy distinguishes between structure (what is the structure/static relation?); behavior (what is the dynamic relation between concepts?); and function (what are the functional relations between concepts?) (see Table 16: Categories of different types of KIM relations). The subcategories (e.g., part-whole, deterministic, probabilistic, quantified, procedural-temporal) for the taxonomy emerged from KIM analysis and were reported by Schwendimann (2014a). Results: Pretest-posttest results – Findings indicate that students overall made significant learning gains from pretest to posttest. Paired t(93) = 6.08, p < 0.0001 (two-tailed)]; effect size (Cohen’s d)=0.63 (SD pretest=2.24, SD posttest=2.41). Results indicate a shift toward higher knowledge integration scores (KI score 3 or higher). Students in both KIM task groups (critique and generation) used significantly fewer nonnormative concepts in the posttest than in the pretest (t(96) = 2.67, p < 0.01). For example, in the pretest, a student chose three nonnormative options in the multiple-choice item and focused only on phenotype-level concepts in the explanation. In the posttest, the same student chose only the normative option and provided an explanation that used the normative genotype-level concept “mutation.” Students can improve their KIM performance not only quantitatively (the number of links and knowledge integration score of KIM connections) but also qualitatively change the types of relationships (see Fig. 4). Using the structure-behavior-function (SBF) framework to categorize different types of relations, students most frequently generated links in the “behavior” category (to describe dynamic relations). A more detailed analysis of the relation types in each super-category revealed that students generated fewer causal-deterministic (7%) and more causal-probabilistic (+4%) (e.g., “could lead to”) and quantified (+11%) (e.g., “increases”) KIM relations in the posttest (see Fig. 5). The increase in causal-probabilistic relationships can be seen as shift toward more statistical thinking on the gene pool level. The increase of quantified relations could indicate a shift toward thinking more in dynamic relationships that reflects the functional interdependency of evolution concepts [28]. Multiple regression analysis indicates that both groups gained significantly in their average KIM knowledge integration scores, R2 = 2.013, F(2, 88) = 11.09, p = 0.000. Both KIM task groups significantly increased the number of cross-links between genotype and phenotype concepts from pretest to posttest, (N = 94): pretest mean = 2.52 (SD = 1.66); posttest mean = 1.03 (SD = 1.15). t(93) = 7.49, p < 0.001; effect size (Cohen’s d) = 1.04. This indicates that students gained in integrating genotype and phenotype concepts after the WISE unit (see Fig. 6). In accordance with gains in prominence of the KIM indicator concepts “mutation” and “natural selection,” multiple regression analysis suggests that students overall used normative evolution concepts more often than nonnormative concepts in the posttest than in the pretest (R2 = 0.18, F(1,94) = 20.18, p < 0.001 (see Fig. 7). Findings from network analysis suggest that students in both groups created significantly more links to and from the two indicator concepts in the posttest: “mutation” (t(93) = 5.39, p = 0.00) and “natural selection” (t(93) = 5.83, p = 0.00) (see Fig. 7). These observations indicate that the two normative indicator

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

651

Fig. 6 Example of a student’s pretest and posttest KIM (critique group). The posttest KIM shows a higher number of cross-connections and more connections to/from the indicator concepts “mutation” and “natural selection”

concepts gained in explanatory strength in students’ repertoire of evolution concepts. The KIM variables “weighted prominence score” for the indicator concepts “mutation” and “natural selection” (see Fig. 7) are strongly correlated with the overall KIM KI score: “mutation” r(94) = 0.75, p < 0.001, and “natural selection” r(94) = 0.70, p < 0.001. Regarding time, students in both KIM task groups spent about the same average amount of time on the pretest KIM (14 min) and posttest KIM (13 min) (see Table 6). Both groups showed equal KIM posttest performance. However, the critique group needed significantly less time for the embedded KIM activities, p < 0.05.

652

B. A. Schwendimann

Weighted Prominence Score of KIM genotype level Indicator Idea "Mutation"

12

10

8

6

Pretest Posttest

4

2

0 Generation

Critique

Weighted Prominence Score of KIM phenotype level Indicator Idea "Natural Selection"

10 9 8 7 6 5

Pretest

4

Posttest

3 2 1 0 Generation

Critique

Fig. 7 Weighted prominence scores of the KIM indicator concepts “mutation” (left) and “natural selection” (right) Table 6 Time spent on KIM tasks In minutes KIM 1 (genotype) KIM 2 (phenotype) Pretest Posttest

Generation group mean 11.94 8.62 14.95 13.65

Critique group mean 6.55 5.39 13.39 13.06

Total mean (median) 9.03 (8) 6.54 (4) 14.12 (16) 13.35 (12.5)

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

653

(t(27) = 2.72, p = 0.01. These results indicate that KIM critique tasks were more time-efficient than the KIM generation tasks while leading to the same posttest gains. Discussion: Overall, findings suggest that the combination of collaborative KIM tasks and technology-enhanced inquiry tasks in the WISE unit improved students’ integration of evolution concepts. Findings indicate that the collaborative technology-enhanced concept mapping form “Knowledge Integration Map” facilitated students’ generation of connections between and across genotype and phenotype levels. As anticipated, students in both KIM task groups showed equal improvement in the posttest tasks. Results from this study suggest that generating or critiquing KIMs can effectively support knowledge integration of evolution concepts. This could be explained by the relatively short duration of the two embedded KIM activities (only about 20 min out of a week-long unit), the similarities of the tasks (same given concepts, same drawing areas), and that both generation and critique activities can support knowledge integration (Schwendimann & Linn, 2015). Both KIM tasks were designed to support students’ knowledge integration through adding, connecting, distinguishing, sorting out, and contrasting evolution concepts on different levels. Sorting concepts into domain-specific levels can provide scaffolds for students to think about why concepts belong into a specific category. This aligns with Marzano’s findings that the identification of similarities and differences is one of the most effective learning strategies (Marzano, Pickering, & Pollock, 2001). Students collaboratively used their existing knowledge to compare categories, generate criteria for each category, and negotiate where to place concepts. Students in both KIM task groups increased the number of cross-links between genotype- and phenotype-level concepts, used more normative evolution concepts, and generated more connections to/from indicator concepts in the posttest. The increase of prominence of the normative indicator concepts “mutation” and “natural selection” coincided with a decreased usage of nonnormative concepts in posttest explanations. Results suggest that building more connections between genotype- and phenotype-level concepts can reduce the usage of nonnormative concepts when generating explanations of evolutionary change. Students not only generated more connections in the KIM posttest, but they generated more quantified relationships, which can be seen as an indicator for deeper understanding (Derbentseva et al., 2007). Working on KIMs collaboratively in pairs required students to negotiate and make their criteria explicit. The visual knowledge representation of KIMs can support collaborative work by enabling efficient information retrieval and exchange. Multiple embedded KIMs can allow students to self-monitor their learning progress by making existing and absent connections explicit. Previous studies (Schwendimann, 2009; Schwendimann & Linn, 2015) suggested that the combination of generating and critiquing KIMs can support students’ integration of concept-related evolution. However, the combination of both tasks can be time-consuming. This study developed and investigated critiquing as a more time-effective KIM activity design. Despite similar learning outcomes, the

654

B. A. Schwendimann

critique group required significantly less time to complete their two embedded KIM tasks. The critique group might have been faster because generating new links from scratch can be more challenging than revising existing connections, and it requires in-depth reflection of smaller selection of propositions. Critique-task KIMs can reduce the demanding decision-making process as students only revise connections they disagree with. On the other hand, students in the KIM generation group needed to make decisions for each proposition and placement. As time in the classroom is limited and precious, this study suggests that collaborative KIM critique tasks can be a beneficial and more time-efficient alternative to generating concept maps from scratch. Critiquing and sorting out of alternative concepts is a central process of knowledge integration and an important skill for autonomous, lifelong learning. Implications: This study demonstrates that Knowledge Integration Map tasks embedded within a technology-enhanced evolution unit focused on knowledge integration have the potential to transform learning in biology classrooms where time is limited and precious. This study developed and explored critiquing KIMs as a time-effective form of collaborative concept mapping task that can foster knowledge integration of complex scientific knowledge, such as evolution. Additionally, critiquing KIMs can elicit criteria to distinguish nonnormative concepts and provide students with a genuine opportunity to negotiate and apply critique. For Knowledge Integration Maps to provide maximum benefit to students, KIM activities should be integrated with a variety of other learning activities, such as scaffolded inquiry activities. Learning how to generate KIMs and how to revise KIMs takes time and practice. Ideally, KIMs would be introduced early in a student’s academic career rather than later, so they can integrate it into their developing study strategies (e.g., see Santhanam, Leach, & Dawson, 1998). Critiquing KIMs that include common nonnormative concepts can generate genuine opportunities for students to reflect on their own knowledge and apply critique. Critique activities can support criteria generation as student dyads have to negotiate which elements they want to revise. Critiquing KIMs can encourage knowledge integration by fostering self-monitoring of learning progress, identifying gaps in knowledge, and distinguishing nonnormative concepts. A forcedchoice design for KIM concepts but free choice for concept placement, connections, and labels was found to be an effective and balanced form of concept mapping. To capture the wide variety of students’ nonnormative concepts, this study suggests using a combination of quantitative and qualitative KIM analysis methods in addition to traditional assessment methods (Linn & Hsi, 2000). KIMs can contain different forms of information: presence or absence of connections, quality of connections, different types of link labels, different types of networks, and spatial placement of concepts. To account for these different aspects of KIMs, several different analysis strategies need to be applied to triangulate changes over time. Findings from this study are valuable for the design of effective collaborative learning environments to support more integrated understanding of biology.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

655

Concept Maps as Assessment Tools Many conventional forms of assessment, such as multiple-choice, true/false, and fillthe-blanks, focus on recall of isolated ideas (Ruiz-Primo, 2009). Hyerle (1996) has called for a shift in the focus of future teaching, learning, and assessment away from rote recall of “isolated things” toward recognition of “how students interactively construct the pattern that connects” (p. 20). Markham, Mintzes, and Jones(1993) found that major differences in content knowledge of novices and experts are a lack of integration, lack of cross-links between concepts, and a limited number of hierarchical levels. Integrating complex concepts in fields requires connecting concepts from different domains and levels. Concept maps can be used as assessment tools to elicit students’ connections between ideas (Edmondson, 2000; Hay, 2008; Mintzes et al., 2001; Popova-Gonci & Lamb, 2012; Ruiz-Primo, 2000; Stoddart, Abrams, Gasper, & Canaday, 2000) and track changes in students’ understanding of relations between ideas (Ruiz-Primo & Shavelson, 1996). Quantitative or qualitative concept map indicators can track changes in students’ knowledge integration of complex ideas (Schwendimann, 2014a). Concept map assessments have been found to show varying correlations with conventional tests – depending on the type of conventional test, the concept map activity design, and the concept map scoring system (Stoddart et al., 2000). More constrained forms of concept map assessment have been found to be highly correlated with multiplechoice tests (Liu & Hinchey, 1993, 1996; Rice, Ryan, & Samson, 1998; Schau, Mattern, Weber, Minnick, & Witt, 1997). Course grades in a university biology course showed moderate correlation to concept mapping scores (Farrokh & Krause, 1996). Osmundson reported a moderate correlation between middle school essays and concept maps (Osmundson, Chung, Herl, & Klein, 1999). Since 2009, concept maps have been used in standardized large-scale assessments in the US National Assessment of Educational Progress (NAEP) to measure changes in conceptual understanding of science ideas (Ruiz-Primo, Iverson, & Yin, 2009). Concept maps as assessment tools have been used to assess prior ideas and/or changes in conceptual understanding in a wide variety of contexts (Edmondson, 2000; Mintzes et al., 2001; Ruiz-Primo, 2000; Ruiz-Primo & Shavelson, 1996). Table 7 shows a selection of concept map activities implemented with different age groups in different science class settings. The studies shown in Table 7 serve to illustrate the range of concept map implementations and do not aim to represent a comprehensive review. Concept mapping can offer several advantages over conventional assessment forms: 1. Unlike recall-oriented assessment forms, concept maps are generative forms of assessment that can also reveal partial understanding. 2. To understand and use ideas, ideas need to be connected to existing ideas. Interconnection between ideas is an essential property of knowledge. One aspect of competence in a field is well-integrated and structured knowledge (e.g., see

656

B. A. Schwendimann

Table 7 A selection of research on concept maps as assessment tools (in science education) School level Kindergarten

Science General Science

Elementary School Middle School

General Science

High School

Biology

General Science

Physics Earth/ Environmental Science Chemistry Undergraduate

Biology

Graduate/ Post-Graduate

Chemistry Computer Science Earth/ Environmental Science Physics Mathematics/ Statistics Medical/Nursing school

Engineering Research Methods Vocational education (VET) Science Teachers

Science

References Stice & Alvarez, 1987; Mancinelli, Gentili, Priori, & Valitutti, 2004; Birbili, 2006 ; Sundararajan, Adesope, & Cavagnetto, 2018 González, 1997 Rice et al., 1998; Osmundson et al., 1999; Guastello, Beasley, & Sinatra, 2000; Snead & Snead, 2004; Gerstner & Bogner, 2009 Novak, Bob Gowin, & Johansen, 1983; Demastes et al., 1995; Kinchin, 2000a; Banet & Ayuso, 2003; Royer & Royer, 2004; Chang, 2007; Wise, 2007 Rye & Rubba, 2002; Yin, Vanides, Ruiz-Primo, Ayala, & Shavelson, 2005 Hsu et al., 2008

Ruiz-Primo, Schultz, Li, & Shavelson, 2001; Liu, 2004; Uzuntiryaki & Geban, 2005 Pearsall et al., 1997; Buntting et al., 2006; Cathcart et al., 2010; Hamdiyati et al., 2018 Nicoll, Francisco, & Nakhleh, 2001a Acton, Johnson, & Goldsmith, 1994 Rebich & Gautier, 2005; Metcalf et al., 2018

Mistades, 2009; Koponen & Nousiainen, 2018 Schau & Mattern, 1997; Syarifuddin, 2018 Irvine, 1995; Van Neste-Kenny, Cragg, & Foulds, 1998; West, Pomeroy, Park, Gerstenberger, & Sandoval, 2000; Bruechner & Schanze, 2004; Vilela, Austrilino, & Costa, 2004; Veo, 2010; Chen, Liang, Lee, & Liao, 2011; Maneval, Filburn, Deringer, & Lum, 2011; Nejat, Kouhestani, & Rezaei, 2011; Sarhangi et al., 2011; Schuster, 2011; Taylor & Littleton-Kearney, 2011; Tseng et al., 2011; Nijman, Sixma, Triest, Keus, & Hendriks, 2012; Atay & Karabacak, 2012; Gerdeman, Lux, & Jacko, 2013; Silva, Foureaux, Sá, Schetino, & Guerra, 2018; Roessger, Daley, & Hafez, 2018; Garwood, Ahmed, & McComb, 2018; Sun et al., 2018; Chand, Sowmya, & Silambanan, 2018 Walker & King, 2003; Fang, 2018 Hay, 2007 Koopman, Teune, & Beijaard, 2011; Schaap, Van der Schaaf, & De Bruijn, 2011; Van Bommel, Kwakman, & Boshuizen, 2012; Nugrahani, Prasetyo, & Iswari, 2018 Rutledge & Mitchell, 2002; Nehm & Schonfeld, 2007; Koponen & Pehkonen, 2010; Koc, 2012; Suprapto et al., 2018

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

657

Bransford et al., 2000; Glaser, Chi, & Farr, 1985; Novak & Gowin, 1984). Cognitive psychologists postulated that “the essence of knowledge is structure” (Anderson, 1984, p. 5). Unlike traditional forms of assessment that focus on recall of isolated ideas (isolated nodes in a concept map), concept maps represent connections between ideas (links between nodes). 3. Experts and successful students develop well-differentiated and highly integrated frameworks of related ideas (Chi, Feltovich, & Glaser, 1981; Mintzes, Wandersee, & Novak, 1997; Pearsall et al., 1997). Concept maps can reveal students’ knowledge organization by showing connections, clusters of ideas, hierarchical levels, and cross-links between ideas from different levels (Shavelson et al., 2005). Cross-links are of special interest as they can indicate creative leaps on the part of the knowledge producer (Novak & Cañas, 2006). 4. The form of assessment directs students’ learning. Concept mapping can foster students’ learning for conceptual understanding instead for memorization of isolated ideas (see Concept Maps as Learning Tools). 5. Research indicates that concept maps can assess different kinds of knowledge than conventional assessment forms (Ruiz-Primo, 2000; Shavelson et al., 2005; Yin et al., 2005).

Forms of KIM Analysis Literature indicates that concept map analysis is no trivial task and can use a wide variety of scoring methods (see the following discussion of quantitative and qualitative analysis methods). Concept maps can be analyzed either qualitatively or quantitatively. Figure 8 provides an overview of different KIM analysis methods.

Quantitative Concept Map Analysis The inclusion of concept maps as large-scale assessment tools, for example, those used in the 2009 NAEP exam in science (Ruiz-Primo et al., 2009), requires economical as well as reliable and valid scoring methods. Several studies reported the validity and reliability of quantitatively evaluating concept maps as assessment tools (e.g., Ifenthaler, 2010; Markham, Mintzes, & Jones, 1994; Ruiz-Primo, 2000; RuizPrimo, Schultz, & Shavelson, 1997; Ruiz-Primo & Shavelson, 1996; Ruiz-Primo et al., 2001, 2009; Stoddart et al., 2000; Yin et al., 2005). Concept maps contain several elements that can be quantitatively evaluated: concepts, hierarchy levels, propositions, and the overall network structure. Links and concepts can be easily counted, but their amount provides little insight into a student’s understanding. A higher number of links does not necessarily mean that the student understands the topic better as many links might be invalid or trivial (Austin & Shore, 1995; Herl, 1999). Evaluating the number of hierarchy levels has been suggested by Novak (Novak & Gowin, 1984). The existence of hierarchies is linked to a higher level of expertise, but hierarchy levels can be difficult to differentiate, and

Fig. 8 Overview of KIM analysis methods

Total KI uncorrected crosslink score (weighted)

Uncorrected cross-link score

Structural links Behavioral links Functional links

Standardized ratios

used for

Benchmark map

Simple Linear Circular Hub/Spoke Tree Network

Link type classification

Quantitative

Topoligical classification

distinction of

Selected indicator concepts

distinction of

All concepts

of

Prominence score

Prestige

Networks analysis

Centrality

Total KI essential connections score (weighted)

Essential connections

Network density score

Total number of essential connections

Total KI corrected cross-link score (weighted)

Corrected cross-link score

Selected propositions

Total number of correctedcross links

Crossconnections

Propositional analysis

Total number of propositions

All propositions

Total number of uncorrectedcross links

Total KI concept map score (weighted)

Concept Placement score

Concepts (Ideas)

Map components

Quantitative

KIM Scoring Methods

658 B. A. Schwendimann

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

659

some concept maps can be non-hierarchical but still valid maps. Propositions, the composite of two concepts, a link label, and an arrow, can be evaluated in order to learn about students’ understanding. It can be decided to evaluate all propositions equally, to weight certain propositions more than others (Rye & Rubba, 2002), or to analyze only certain indicator propositions (Ruiz-Primo et al., 2009). Yin (2005) showed that scoring each individual proposition on a four-point individual proposition scale, summed up to a “total accuracy score,” provided the best validity: 0 for scientifically wrong or irrelevant propositions, 1 for partially incorrect propositions, 2 for correct but scientifically “thin” propositions, and 3 for scientifically correct and strong propositions. The “total accuracy score” allows comparing the overall quality of students’ concept maps. The disadvantage of this method is its time consumption, and equal evaluation of links that show deeper understanding and trivial links. Yin (2005) compared the total accuracy score to a second concept map scoring method, the convergence score. Propositions of the students’ concept map are compared to an expert-generated benchmark map. The convergence score is the proportion of accurate propositions out of all possible propositions in the benchmark map. Concept maps can contain large numbers of rather trivial connections. An alternative to scoring all links is to focus only on a small number of selected links (Yin et al., 2005). Ruiz-Primo et al. (2009) suggest that scoring only essential links is more sensitive to measuring change because it focuses only on the key concepts of the concept map. However, analyzing only isolated propositions does not account for the network characteristics of a concept map. Quantitative propositional alone could lead to the same score for a list of isolated propositions and a network of the same propositions. Network analysis can be used to describe the connectedness of a KIM’s overall density and prominence of selected indicator concepts.

Benchmark KIM To understand and use concepts, concepts need to be connected to existing concepts. The degree of interconnections between concepts is an essential property of knowledge. One aspect of competence in a field is well-integrated and structured knowledge (Bransford et al., 2000; Glaser et al., 1985; Novak & Gowin, 1984). Cognitive psychologists postulate that “the essence of knowledge is structure” (Anderson, 1984, p. 5). An expert-generated KIM can be used to identify the overall structure, central concepts, and essential connections (see Fig. 9). However, a benchmark map should not be interpreted and used as the single correct solution but as an expertgenerated suggestion that allows identifying central concepts and connections for a detailed analysis. A benchmark KIM can be used to standardize variables to compare different student-generated KIMs against one another. The benchmark KIM indicates how many and which connections experts generate. To calculate standardized KIM variables, student-generated KIM variables are divided by the benchmark KIM variables.

is the basis of

is the basis of

DNA

randomly changes Mutation

increases (+)

Diversity in Gene Pool

lowers (–)

are random changes of

leads to new

reason for

leads to loss of

Genetic Drift

can lead to

can lead to (+/–)

does not lead to

Adaptation

always leads to

Increase Fitness of Population

Natural Selection

can lead to

always leads to

always leads to (+)

does not affect

Population Size

lowers (–)

does not affect

can lead to (+/–)

does strongly affect

Fig. 9 KIM benchmark map. Indicator concepts (gray), essential connections (bold)

Gene

is variation of

Allele

Acts upon Genotype/ Genotype level

Environment

defines

Acts upon Phenotype/ Phenotype level

660 B. A. Schwendimann

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

661

Indicator Concepts Ruiz-Primo suggested that knowledge within a content field is organized around central concepts, and to be knowledgeable in the field implies a highly integrated conceptual structure (Ruiz-Primo et al., 1997). Graphic organizers can enhance student learning by representing complex concepts in an organized structure reflecting the importance of each concept (Plotnick, 1997; Romance & Vitale, 1999). To reverse this finding, learners’ understanding of the importance of concepts can be identified by analyzing how connected selected concepts are in a KIM. For the KIM network analysis, one concept from each level (genotype/phenotype) has been selected as the “indicator concept.” Indicator concept analysis describes the number and kind of connections to other concepts. The criteria for selecting indicator concepts were (1) centrality in the expert benchmark KIM and (2) importance according to evolutionary theory literature: • For the genotype level, mutation has been identified as the indicator concept. • For the phenotype level, natural selection has been identified as the indicator concept.

Essential Connections Ruiz-Primo (2009) found that a KIM analysis that focuses on preselected “essential links” instead of all links can reveal a greater variety of maps while being more timeefficient. KIM analysis used ten essential connections (see Fig. 9). The criteria for selecting the essential connections were (1) connections between the indicator concepts and the newly introduced concept “gene pool” and “genetic drift” and (2) cross-connections between genotype and phenotype levels. An increased number of cross-connections can be interpreted as a more connected understanding of genotype and phenotype concepts. KIMs differ from classical concept maps in several characteristics (see Table 8).

KI-Rubric for Concept Maps To quantitatively describe changes in KIMs from pretest to posttest, primary and secondary analysis variables were used. Primary variables are based directly on the KIMs, while secondary variables are calculated from primary variables. Primary propositional scoring included (1) scoring of all propositions and (2) scoring of only essential propositions. Table 8 Comparison between classical concept maps and KIMs Classical concept map No weighted concepts No weighted relations Hierarchical arrangement of concepts

Knowledge integration map Weighted concepts (indicator concepts) Weighted relations (essential connections) Non-hierarchical placement of concepts in domain-specific levels

662

B. A. Schwendimann

Score All Propositions

KIM propositions consist of two concepts and their relation (indicated by a labeled line with an arrowhead). Propositions are the elementary units of Knowledge Integration Maps. Individual propositions were analyzed using a five-level knowledge integration rubric (see Table 9). All propositions were weighted equally. Score Only Essential Propositions

Using the same five-level knowledge integration rubric, only essential propositions were scored. Concept Placement Analysis KIMs ask students to sort out concepts into domain-specific levels (e.g., genotype and phenotype). Concept placement is an additional level of information that indicates how students categorize concepts. Connecting concepts within a level indicates students’ understanding of the relations between closely related concepts. Connecting concepts across levels (cross-links) indicates students’ understanding across ontologies and levels of space and time. Cross-links are of particular interest as they can indicate “creative leaps on the part of the knowledge producer” (Novak & Cañas, 2006) and reasoning across ontologically different levels (Duncan & Reiser, 2007). Cross-links are relations between concepts in different levels. Cross-connections are of particular interest as they indicate if students see connections between genotype- and phenotype-level concepts. As concepts might be wrongly placed by students, an observed cross-connection might actually be a connection between two concepts of the same level (“uncorrected cross-link”). To account for such cases, a “corrected cross-link” variable indicates intra-domain connections even if the concepts were wrongly placed. Table 9 KIM knowledge integration rubric KI Score 0 1 2

3 4 5

Link label quality None (No connection) Wrong label (a) No label (b) Correct label (c) Incorrect label No label Partially correct label Fully correct label

Link arrow None (No connection) Wrong arrow direction (a) Only line (b) Wrong arrow direction (c) Correct arrow direction Correct arrow direction Correct arrow direction Correct arrow direction

Sample propositions None Genetic variability includes mutation (a) Mutation – genetic variability (b) Genetic variability –contributes to > mutation (c) Mutation – includes > genetic variability Mutation –> genetic variability Mutation – increases -> genetic variability Mutation – causes random changes in the genetic material which in turn increases -> genetic variability

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

663

Primary Analysis Variables Two different sets of primary variables were created: non-weighted number of links and links weighted by their respective knowledge integration (KI) scores. Primary Variables: Number of Links (See Table 10)

As propositions may differ not only in quantity but also quality, propositions were weighted by multiplying them with their respective KI scores. Primary Variables: Knowledge Integration (KI) Scores (See Table 11)

KIM Secondary Analysis Variables

Another way to describe quantitative changes in KIMs is density variables and ratios (calculated from primary analysis variables). Ratios and densities can be relative or standardized (see Table 12). Table 10 KIM primary variables: number of links Variable name Total number of links Total number of essential links Total number of uncorrected cross-links

Total number of corrected cross-links

Description Number of links in the KIM Number of essential links in the KIM Uncorrected cross-links are connections that cross the line between the genotype and phenotype level. Because of falsely placed concepts, the connection might not be a true crossconnection between a genotype- and phenotype-level concept. However, the uncorrected cross-link can be seen as an indicator for students’ motivation to connect concepts across levels Corrected cross-links count connections between genotype- and phenotype-level concepts, even if the concepts were wrongly placed

Table 11 KIM primary variables Variable name Total KI score of all links (total accuracy score) KI score essential links KI score genotype level only KI score phenotype level only KI score uncorrected crossconnections KI score corrected crossconnections

Description Product of total number of links and their respective KI scores Product of total number of essential links and their respective KI scores Product of number of links in the genotype-level area (not counting cross-links) and their respective KI scores Product of number of links in the phenotype-level area (not counting cross-links) and their respective KI scores Product of number of uncorrected cross-connections and their respective KI scores Product of number of corrected cross-connections and their respective KI scores

664

B. A. Schwendimann

Table 12 KIM secondary variables Variable name Relative density Standardized density Relative essential link ratio Standardized essential link ratio Corrected crossconnections ratio KI score ratio Standardized KI score ratio

Description Total number of student-generated connections divided by total number of possible connections (=55) Total number of student-generated connections divided by total number of links in benchmark map (=23) Total number of essential student-generated connections divided by total number of student-generated connections Total number of essential student-created connections divided by total number of essential connections in benchmark map (=10) Total number of student-generated cross-connections (corrected) divided by total number of cross-connections in benchmark map Total KI score in student-generated map divided by total KI score in expert-generated benchmark map (=126) Total KI score of essential connections in student-generated map divided by total KI score of essential connections in benchmark map (=50)

KIM Network Analysis Research suggests that concept maps can assess different forms of knowledge than conventional assessment forms (Ruiz-Primo, 2000; Shavelson et al., 2005; Yin et al., 2005), for example, knowledge structure and cross-connections. However, the commonly used quantitative propositional method of analysis does not capture changes in the overall network structure. Network analysis uses the frequency of usage of essential concepts as indicators for a more integrated understanding. The network analysis method is based on social network analysis (Wasserman & Faust, 1994). As students develop a more complex understanding, they might also identify certain concepts as more important and connect them more often. In the KIM example used in this chapter, the indicator concepts “mutation” (genotype level) and “natural selection” have been selected (see Fig. 9). Two measurements were used to capture changes in connection frequencies to the indicator concepts. Network analysis method can identify changes in “centrality” (outgoing connections) and “prestige” (incoming connections) of expert-selected indicator concepts (mutation for genotype level and natural selection for phenotype level). • Centrality: Outgoing connections from the indicator concept. This variable describes how many relations lead away from the indicator concept. • Prestige: Incoming connections to the indicator concept. This variable describes how many relations from other concepts lead to the indicator concept. The two network variables centrality and prestige can be combined to a total “prominence score” (importance indicator) for each indicator concept. Multiplied with the KI score for each connection, a “weighted prominence score” for each of the two indicator concepts can be calculated.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

665

An adjacency matrix was used to establish centrality and prestige of each indicator concept. The adjacency matrix, sometimes also called a connection matrix, is a matrix with rows and columns labeled by graph vertices, with a 1 or 0 in position according to whether two concepts are adjacent or not (Chartrand & Zhang, 2004 ; Pemmaraju & Skiena, 2003). The expert-generated KIM benchmark was used to determine benchmark values of centrality and prestige. Qualitative KIM Analysis

Qualitative analysis methods complement quantitative descriptions of concept maps by tracking changes in the geometrical structure (topology) and types of propositions. KIM Topological Analysis Quantitative analysis methods focus only on isolated propositions and can therefore not give an account of the network character of a whole map. Kinchin (2000b, 2001) suggested a framework of four classes (simple, chain/linear, spoke/hub, net) to describe the major geometrical structure of a concept map. A “network” structure indicates a more integrated understanding than a “fragmented” concept map structure. However, a ranking of these categories is only possible at the extreme ends, with “fragmented” at one end and “networks” at the other. All other classes fall in between. Yin (2005) extended Kinchin’s framework by two additional classes (tree and circle) (see Table 13): 1. Simple: Mostly isolated propositions. Table 13 Concept map topological categories. (Adapted from (Yin et al., 2005))

Simple/fragmented

Chain/linear

Tree

Hub/spoke

Circular

Network

666

2. 3. 4. 5. 6.

B. A. Schwendimann

Chain: Propositions are in a linear chain. Tree: Linear chain but with branches. Hub: Connections emanate from a center concept. Circular propositions: Propositions are daisy-chained forming a circle. Network: Complex set of interconnected propositions.

The analysis methods developed for KIMs further extend Yin’s framework. As Knowledge Integration Maps are divided into domain-specific levels (e.g., genotype and phenotype), the geometrical structure of each level needs to be described (see Table 14). Coding includes each possible combination of geometrical structures in the two levels. Changes in the topology of KIMs can indicate changes in students’ knowledge integration. Qualitative Proposition-Type Analysis Learning about relations between concepts is challenging for all learners. When learning a language, students learn nouns before verbs (Gentner, 1978). Typically, KIM concepts are nouns, while link labels are verbs. Learning about the relations between concepts can be more challenging than understanding concepts. However, understanding the relations between concepts is essential to an integrated understanding of biology. Most existing concept map analyses focus on quantitative variables (see section on “Quantitative Concept Map Analysis”). To describe semantic changes in the relations between concepts, qualitative variables are needed. To track changes in relation types, a link label taxonomy has been developed for KIMs (see Table 15). The relation categories also include negations, e.g., “does not lead to” or “is not part of.” The concept mapping literature suggests a number of different link types. For example, Fisher (2000) distinguished three main types of propositional relations in biology that are used in 50% of all instances: whole/part, set/member, and characteristic. O’Donnell distinguished between three types of relations in knowledge maps: dynamic, static, and elaboration (O’Donnell et al., 2002). Lambiotte suggested dynamic, static, and instructional relation types for concept maps (Lambiotte, Dansereau, Cross, & Reynolds, 1989). Derbentseva distinguished Table 14 Topological KIM categories (for a two-area KIM)

First area Empty Fragmented Linear Tree Hub Circular Network

Second area Empty Fragmented Linear Tree Hub Circular Network

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

667

Table 15 Categories of different types of KIM relations Super-category Unrelated

Structure What is the structure (in relation to other parts)?

Behavior What action does it do? How does it work/influence others?

Function Why is it needed?

Subcategory No connection No label (just line) Unrelated label Part-whole (hierarchical)

Code 0 1 2 3

Similarity/comparison/ contrast Spatial proximity

4

Attribute/property/ characteristic (quality (permanent) or state (temporary) Causal-deterministic (A always influences B)

6

Causal-probability (modality)

8

Causal-quantified Mechanistic

9 10

Procedural-temporal (A happens before B)

11

Functional

12

Teleological

13

5

7

Examples

Is a/are a; is a member of; consist of; contains; is part of; made of; composed of; includes; is example of Contrasts to; is like; is different than Is adjacent to; is next to; takes place in Can be in state; is form of

Contributes to; produces; creates; causes; influences; leads to; effects; depends on; adapts to; changes; makes; results in; forces; codes for; determines Leads to with high/low probability; often/rarely leads to; might/could lead to; sometimes leads to Increases/decreases Explains domain-specific mechanism/adds specific details or intermediary steps Next/follows; goes to; undergoes; develops into; based on; transfers to; happens before/during/after; occurs when; forms from Is needed; is required; in order to; is made for Intends to; wants to

between static and dynamic relations in concept maps (Derbentseva et al., 2007; Safayeni, Derbentseva, & Canas, 2005). To create a taxonomy of link types, higher-order variables are needed. KIM analysis used the structure-behavior-function (SBF) framework to create the supercategories of the taxonomy. The SBF framework was originally developed by Goel and Chandrasekaran (1989; Goel, Rugaber, & Vattam, 2008) to describe complex systems in computer science and then applied to complex biological systems by Hmelo-Silver (2004; Hmelo-Silver et al., 2007; Liu & Hmelo-Silver, 2009).

668

B. A. Schwendimann

• Structure: What is the structure (in relation to other parts)? These variables describe static relations between concepts. Static relations between concepts indicate hierarchies, belongingness, composition, and categorization. • Behavior: What action does it do? How does it work/influence others? These variables describe the dynamic relations between concepts. Dynamic relations between concepts indicate how one concept changes the quantity, quality, or state of the other concept. • Function: Why is it needed? These variables describe functional relations between concepts, for example, “want” (intentionality) or “need” (teleological). The subcategories for the taxonomy emerged from KIM analysis (see Table 15). Categorizing link labels allows tracking and describing how connections changed ontologically. The following case study illustrates how concept maps can distinguish expert and novice knowledge as assessment tools.

Case Study: Concept Maps to Distinguish Expert and Novice Knowledge Concept maps can reveal learners’ knowledge organization by showing connections, clusters of concepts, hierarchical levels, and cross-links between concepts from different levels (Shavelson et al., 2005). Connections between concepts can be seen as an indicator for more integrated knowledge (Bransford et al., 2000; Novak & Gowin, 1984). Concept maps can be a helpful metacognitive tool to visualize the interaction between prior and new conceptual understanding of learners. However, concept map analysis often uses only the final product without taking the construction process into account. The ability to construct a concept map illustrates two important properties of understanding: representation and organization of concepts (Halford, 1993). As a representation, concept maps include not all but selected aspects of the represented world. Experts and novices differ in how they structure and connect concepts (Chi et al., 1981; Mintzes et al., 1997) and in their abilities to distinguish salient surface features from structurally important features of a representation (Leinhardt, Zaslavsky, & Stein, 1990). Experts can better decide how a certain external representation allows them to illustrate, communicate, and analyze a certain principle and create new forms of representations, if required. Developing expertise in a domain includes learning how to detect important elements and organize information. This case study investigates how experts and novices differ in their concept map construction using a talk-aloud protocol to distinguish two modes of reasoning, constraint-based and model-based reasoning (Parnafes & diSessa, 2004). Constraint-based reasoning refers to the cognitive process of finding values for a set of variables that will satisfy a given set of constraints. When utilizing this kind of reasoning, learners focus primarily on the constraints, one at a time. The second

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

669

mode is model-based reasoning. Using this holistic approach, learners try to address all or most constraints at the same time to create a global model of the whole scenario. This study aims to answer the research questions: How do novices and experts differ in their concept map construction processes? How do novices of different academic performance levels differ in their concept map construction? How does verbal reasoning (talk aloud) align with concept map construction? Methods: Prior to the concept mapping task, each participant was interviewed about their familiarity with concept mapping in general, their self-assessment of their evolution biology knowledge, and their experience with concept mapping software. Each participant received initial training in basic concept mapping techniques and the software “Inspiration” by a researcher. The training phase included the presentation of a sample concept map and a step-by-step concept map construction protocol. The participants were instructed to (1) group related concepts, (2) link concepts with arrows, (3) label each link, (4) add cross-links, and (5) revise the whole map. All participants were instructed to talk aloud to describe their actions and reasoning while constructing their concept map. The think-aloud technique has been found to reveal thought processes in a variety of tasks (Ericsson & Simon, 1985), for example, concept map construction (Ruiz-Primo et al., 2001), multiplechoice test taking (Levine, 1998), performance assessment (Ayala, Yin, Shavelson, & Vanides, 2002), and problem solving (Baxter & Glaser, 1998). Ericsson suggests that verbalization is a direct encoding of heeded thoughts that reflect their structure (Ericsson & Simon, 1985). Verbalizing one’s inner dialogue does not need translation and does not require a significant amount of additional processing; therefore talking aloud does not slow down task performance – as long as connections between concepts can be recalled from memory. When connections between concepts need to be newly generated, it leads to measurably slower verbalization. Because of their greater existing content knowledge, experts might need to generate fewer new propositions (connections between concepts) when constructing concept maps in their area of expertise than novices. Experts might therefore show more fluent and faster construction of concept maps. Each participant was instructed to construct a concept map from a given list of 18 concepts (see Table 16). These concepts were identified as core elements in the US national educational standards for cell biology, genetics, and evolution. Concepts from all three different areas (DNA, cell, and evolution) were chosen and provided in Table 16 List of given concepts (organized by areas) DNA Cell Evolution

Chromosomes, chromatids, crossing over, random segregation of chromosomes Cell division, random fusion of gametes, clones, diploid, haploid, mitosis, meiosis, body cells, sex cells (gametes), sperm cells, egg cells (ovum) Evolution, genetic variability, natural selection

670

B. A. Schwendimann

a randomly arranged list (without the grouping shown in Table 16). The forcedchoice design constrained participants to use only the provided concepts but allowed them to generate their own links and labels. The important concept “mutation” was deliberately omitted from the list to investigate if participants would introduce the concept on their own as a link label. Schwendimann and Linn (2015) highlighted the importance of iteratively revising concept maps. Therefore, participants received no time limit and were allowed to revise their concept map until satisfied with the final product. Data sources: Three different kinds of data were collected: • Concept maps can be drawn by hand or by using specialized computer software. Royer’s comparison between these two methods indicated significantly more complex concept maps when generated using concept mapping software (Royer & Royer, 2004). This study used the concept mapping tool “Inspiration” (2016). • Screen recording software (Wisdom Soft, 2016) was used to capture the concept map construction process. To describe the concept map construction process, two screenshots of intermediate stages and the final product were captured. • Voice recorders captured the talk-aloud utterances of the participants during the concept map construction process.

Participants This case study included three adult domain experts (two postdoctoral biology researchers and one experienced biology teacher) and three 9th and 10th grade students from a public high school. Following purposive sampling, the experts were selected to represent two different forms of expertise (research and teaching), while the students represented the range of general academic performance levels (high, middle, and low). The students received extra credit from their teacher for their voluntary participation. All three high school students attended the same biology class. Prior to participating in the study, each student completed a weeklong session on cell biology and genetics that included all concepts provided for the concept mapping activity. All three students were familiar with concept mapping techniques, but none of the students used the software “Inspiration” before. Results: The result section describes the concept map construction and critique tasks by the three experts and the three novices. Experts Biology Expert A: Expert A was a postdoctoral fellow in biophysical sciences at a major US research university. A had no prior experience with concept mapping or the “Inspiration” software but frequently used flowcharts in professional presentations. Expert A quickly understood the principles of concept mapping and the handling of the Inspiration software after the training session. Concept map construction task: Expert A began the concept map by dividing the provided concepts into two groups, cell division/meiosis/mitosis/clones and body cells/sex cells/sperm cells/crossing over/random fusion of gametes (see Table 17, stage 2). Expert A placed the most comprehensive concept “evolution” on top and

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

671

Table 17 Concept map development of expert A

Stage 1

Stage 2

Stage 3

Stage 4

“cell division” at the bottom and then grouped related terms around them. In a second arrangement phase, A divided the concepts into the groups “meiosis” and “mitosis.” Only after arranging and clustering all concepts, A began linking them. Expert A said “I am thinking hierarchically, but the connectors are not going to be very hierarchical because sometimes a concept is the subject and sometimes an object,” while pointing at a horizontal chain of concepts (see stage 3). At the end of the systematic construction activity, which took only 15 min, expert A started adding cross-links. This led to the final concept map (see stage 4), which partially followed the “circle of life” model: random fusion of gametes ! fertilized ovum ! mitosis ! meiosis ! new gametes. Expert A did not create a connection between egg cells and sex cells because of A’s interpretation of egg cells as being already fertilized. Expert A also did not connect meiosis with genetic variability, arguing that the central concept “mutation” was missing in the list of given concepts and that without mutation meiosis will not enhance genetic variability. Biology Expert B: Expert B was a postdoctoral fellow in neurogenetics at a major US research university and had no prior experience with concept mapping or “Inspiration.” Expert B understood the principles of concept mapping quickly after the initial training phase. Concept map construction task: B began the concept map by clustering the related concepts “sex cells,” “sperm cells,” and “egg cells.” From this starting point, B developed a temporal chain to illustrate meiotic and mitotic cell division. Like both

672

B. A. Schwendimann

other experts, B noticed the absence of the concept “mutation” in the provided list of concepts. Expert B explained that without mutation there would be no alleles and therefore no variability in meiosis. B stated that a reduction of evolution to the Darwinian view of natural selection and survival of the fittest leads to an inaccurate oversimplification. B suggested that “genetic drift” should be added to the list of concepts. B created an interesting connection between body cells/mitosis/meiosis, by arguing that body cells can undergo either one of these two cell division processes. While working on the concept map, B tried to construct the concept map from the viewpoint of a high school student, as B perceived the given concepts as a constraint that forced making “over-simplifications and large logical stretches.” B made several connections, especially to evolutionary concepts, which implied several sub-steps (which B explained verbally). These sub-steps were only explained orally and could therefore not be detected in the final concept map. After finishing the first phase of connections, B began adding cross-links. Expert B did not connect the concepts “cell division” with “meiosis” and “mitosis.” B’s final map did not show a hierarchical structure but consisted mostly of temporal chains. B invested 27 min on the concept map. Biology Expert C: Expert C was an experienced biology teacher at a US public high school. C has not used concept maps as a personal tool but taught concept mapping techniques to students. Concept map construction task: C started by grouping the concepts into “meiosis” and “mitosis” under the top-level concept “cell division.” Expert C placed chromosomes and chromatids between the two groups, as they belonged to both. The evolutionary concepts “evolution” and “natural selection” were singled out until the end of the activity. C then arranged and connected concepts in each group either according to structure (e.g., cell type, haploid) or function (e.g., crossing over, genetic variability). In a second phase, C rearranged the concepts to follow closely the “lifecycle model” found in biology textbooks (similar to expert A): meiosis ! fusion of gametes ! body cells ! mitosis. C identified this approach as a deliberate strategy. Throughout the construction phase, “chromosomes” remained the connecting element in the center. Finally, C added multiple cross-links and connected the evolution group with the cell division group, through the concept “genetic variability.” Like the other two experts, C noticed the absence of the concept “mutation” and worked around this constraint by referring to mutation in the link label between chromosomes and genetic variability. Concluding, C stated that this activity has been “really hard” and that it provided a better appreciation for tasks assigned to students. Expert C spent 33 min until satisfied with the final concept map. C created the concept map with the most cross-links of all six participants. Novices Novice D: Student D was high-performing 9th grade student. D showed complex and coherent understanding of the topic, despite being in a lower grade than the other two novice participants. D was the most articulate of all three novices and engaged in checking, revising, and investing the most amount of time the concept map (45 min) of all six participants.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

673

Concept map construction task: Like expert C, novice D first grouped all concepts into two groups (“meiosis” and “mitosis”) and placed the concept “chromosomes” in-between them. D then arranged and linked the concepts in each group according to procedural criteria (see Table 18, stage 3). D correctly linked “evolution” to the meiosis cluster, but did not create connections between the related concepts “genetic variability,” “random segregation,” and “random fusion of gametes.” D created a proposition that genetic variability leads to natural selection, which would have to be considered incorrect at first. However, after prompting, D provided a comprehensive oral description of the relations between meiosis, genetic variability, natural selection, and evolution. Finally, D added several cross-links and checked each proposition again (see Table 18, stage 4). D revised the validity of every proposition again each time after adding another concept. D’s approach was thorough and systematic. Novice E: Student E was a 10th grade student classified as an average student. Table 18 Concept map development of student D

Stage 1

Stage 2

Stage 3

Stage 4

674

B. A. Schwendimann

Concept map construction task: Novice E first divided all concepts into two groups (“mitosis” and “meiosis”). Like expert C, E placed “chromosomes” between the two cell division subgroups. E singled out “evolution” and “natural selection” and did not connect them until the end of the activity (also similar to expert C). E was not sure about the meaning of the concepts “haploid” and “diploid,” but nevertheless used them correctly. E did not use the concepts “chromatids” and “crossing over” as E could not recall their meaning (these two concepts remained unconnected). Like all three experts, E noticed the absence of the important concept “mutation.” E worked systematic and fast, finishing the concept map in only 12 min. This supports the assumption that E had an existing understanding of the connections between the given concepts and did not have to newly generate them. Novice F: Student F was a 10th grade student described as a low-performing student by the teacher. F was unfamiliar with a majority of the provided concepts and needed more support by the experimenter than the other five participants. Concept map construction task: F started by creating three different groups: cell division/meiosis/mitosis, evolution/natural selection, and sex cells/sperm cells/egg cells. F expressed confusion regarding the meaning of the concepts “mitosis” and “meiosis” and could not remember the meaning of “haploid” and “diploid.” F began to connect concepts in a rather hesitant and unsystematic way. F’s three initial groups evolved first into pairs (see Table 19, stage 2), which were then prolonged into three independent chains. Each chain represented a temporal flow (Table 19, stage 3). F’s labels were mostly very short, for example, and, or, or then. F did not create an overarching order in the map. Even after prompting by the researcher, F failed to identify any cross-links between the three separate chains (see Table 19, stage 4). F spent 25 min on constructing the map and expressed satisfaction after all links were “somehow connected.” The map, as well as F’s knowledge of the domain, seemed to be very fragmented and incomplete.

Discussion This section discusses observations made during each stage of the concept map construction process, proceeding from initial layouts and revisions to the final product. During the initial construction process, the three experts and novices D and E (high-performing participants) fluidly generated their concept maps, which suggests that they had previously existing knowledge of propositions. All high-performing participants demonstrated their ability to move between the two modes of reasoning by switching back and forth between the big-picture view of model-based reasoning (“gestalt” effect) to arrange and rearrange their concepts into clusters and the more detailed view of constraint-based reasoning when creating individual propositions. Experts and knowledgeable students demonstrated their awareness of given constraints by noting that the provided concepts allowed only a limited representation of their actual understanding. However, they found ways to work around this limitation, for example, by introducing the omitted concept “mutation” in a link label. It is noteworthy that all three experts, but only one of the students, mentioned that the

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

675

Table 19 Concept map development of student F

Stage 1

Stage 2

Stage 3

Stage 4

important concept “mutation” was missing. Identifying central concepts (or noticing their absence) can be seen as an indicator of expertise. In contrast, the academically weakest student F progressed slowly and struggled creating connections, which suggests that F’s knowledge of biology concepts was not well integrated and that connections had to be newly constructed. F showed the greatest difficulties and created a fragmented, mostly linear concept map. F accepted the given constraints without questioning and used a constraint-based approach by adding one concept at a time. F seemed more focused on task completion than using concept maps to creatively express one’s understanding. These observations suggest that concept map construction can allow for both constraint-based reasoning and model-based reasoning, depending on the level of expertise of the participant. More knowledgeable participants were able to move fluidly back and forth between constraint-based and model-based modes of reasoning. During the revision process, the high-performing participants commended that they hesitated at times adding more links to avoid “making a mess.” This suggests that aesthetic reasoning (in addition to constraint-based and model-based reasoning) can also influence concept map construction. Initial groupings and hierarchies disappeared during the further development of the concept map. These intermediate stages are not accessible in the final concept

676

B. A. Schwendimann

map. During concept map construction and revision, some initially correct propositions were changed to invalid propositions and vice versa. Interestingly, the final concept maps constructed by the participants differed much less from each other than anticipated. The construction processes and final concept map of high-performing students did not noticeably differ from expert-generated maps. Teacher-expert C created the most complex map, followed by novices D and E. Experts and novices did not significantly differ regarding their ability to create clusters and hierarchies. Maps of knowledgeable students showed as many crosslinks and network complexities as maps created by experts. Comparing talk-aloud utterances to the developing concept maps provided valuable insights. Several times, oral explanations clarified concept map propositions that would otherwise have to be considered invalid (e.g., expert A created the proposition “mitosis contributes to evolution” but then argued that without mitosis there would be no higher organisms and their evolution, as their bodies developed through mitotic cell division). Such an additional elaboration may reveal more extensive conceptual knowledge than the condensed and constrained form of the concept map propositions reflect. Expert participants provided more detailed oral explanations than novice participants. Experts’ link labels were often shorthand for several intermediate steps (which they explained orally). Further analysis of talkaloud utterances also revealed several noticeable differences between the experts. The research-experts expressed greater difficulties generating their concept maps than the teacher-expert or the students. Several factors could contribute to this observation: the research experts had only limited prior experience in generating concept maps. The two research experts experienced it as a challenge to express their complex and sophisticated understanding in the constrained format of a concept map. In contrast, teacher-expert C showed fewer difficulties representing conceptual understanding in the shorthand form of concept maps because concept maps are frequently found in biology textbooks. Overall, this case study suggests that concept maps can reveal differences in knowledge of experts and (low-performing) novices. High-performing participants (experts and novices) demonstrated their ability to fluidly move back and forth between a big-picture view (model-based reasoning) and a more detailed view when creating individual links (constraint-based reasoning). Concept map construction processes and the final products indicated few differences between highperforming novices and experts. Nevertheless, experts expressed their deeper understanding orally, because they could not adequately express their extensive knowledge due to task and aesthetic constraints of concept maps. The shorthand form used to describe relations between concepts allows keeping an aesthetic big-picture view but limits capturing explanatory depth. Experts and novices often used the same link labels to describe a relation between concepts, but oral elaboration revealed that experts often compressed more knowledge into a link label (called higher “epistemic density” by Maton & Doran (2016) and used the same linking words to represent different meanings (Ariew, 2003). Accompanying explanations are needed to further explain understanding represented by a proposition.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

677

Implications and Limitations As a case study with a small sample size, analysis can offer only limited insights. However, several suggestions can be offered. Concept maps are used as assessment tools to track changes in students’ understanding, for example, in standardized large-scale assessments in the US National Assessment of Educational Progress (NAEP) (Ruiz-Primo et al., 2009). Usually, only the final concept maps are evaluated. Results from this study highlight the possible divergence between the concept map construction process and the finished product. During the construction process, most participants created meaningful clusters of related concepts and/or followed a temporal flow. However, these clusters or temporal flows were often no longer identifiable in the final product. A teacher or researcher who evaluates only the final product will often lack this additional information. Final concept maps elicit only a limited snapshot of a learner’s integrated knowledge. Participants’ oral explanations of their thought processes often diverged or expanded the reasoning leading to certain propositions. Some link labels might even have to be considered incorrect without the accompanying oral explanation. One way to triangulate this hidden understanding could be looking at written assessments (e.g., essays) or oral explanations that cover the same concepts. In such longer explanations, learners can express their understanding in more detail and provide supporting evidence. This study used a concept map form that represents a compromise between open and heavily constrained formats by providing a list of concepts but leaving link generation to the participant. Open-ended concept maps, where students can choose their own concepts and links, might reflect students’ knowledge structures more accurately, but they are more difficult to compare, require more time, and could be more challenging especially for weaker students (Cañas et al., 2012). On the other hand, more constrained forms of concept maps can lead to ceiling effects (Yin et al., 2005). Due to the constraints of the concept mapping task (e.g., provided list of concepts; only one relation between two concepts, short link labels), a highperforming student’s map can be difficult to distinguish from an expert’s map. Many participants generated only short link labels (maybe due to aesthetical graphical restrictions (limited space between two nodes) that did not represent the same understanding as their oral elaborations). Accompanying explanations (oral or written) could provide valuable insights into the meaning of propositions and the reasoning process during construction. Experts were very selective about which propositions to include. Experts’ selection processes could serve as scaffolds for novices to support their critical reflection and informed decision-making on which connections are relevant to include in their maps. Expert-generated concept maps are often used as references for evaluation. Using expert-generated maps benchmark maps might falsely suggest that there is only one correct answer (Kinchin, 2000b). Findings from this study suggest that there is no single expert reference map. Each expert in this study generated a valid map but constructed different propositions and structures. Expert maps can differ from one

678

B. A. Schwendimann

another (Acton et al., 1994), even when using a limited number of provided concepts. This also raises the question of who is considered an “expert.” There are many different kinds of experts, for example, researchers, practitioners, proficient amateurs, and science teachers (Hmelo-Silver et al., 2007). More research is needed to address the “expert problem” by providing better descriptions of what constitutes an “expert” and distinguishing different types and levels of experts. As a compromise, an expert reference map could be created as an aggregate of several expert-generated maps (Ruiz-Primo et al., 2001). However, even an aggregated expert map represents only one of many possible valid solutions and should only be used with caution for a direct comparison with novice-generated maps. Multiple concept map analysis strategies can be used to complement each other and triangulate changes in learners’ understanding (Schwendimann, 2014a). Concept map generation and analysis should reflect the constructivist perspective that knowledge can and should be constructed and represented in many different ways.

Conclusions This chapter presented an overview of concept maps as learning, teaching, and assessment tools. Studies in different settings indicate that concept maps can serve as versatile tools. Central questions regarding concept maps are how to design concept mapping task and how to evaluate concept maps. Concept mapping tasks range from highly constrained to open, while evaluation can build on quantitative and/or qualitative approaches. However, despite such promising instantiations, concept maps are still not as widely implemented as learning and assessment tools (Kinchin, 2001). The research community is tasked to develop guidelines how to best design concept mapping tasks. If implemented thoughtfully, concept maps can be powerful tools to support knowledge integration processes toward a deeper understanding of the relations and structures of complex ideas. Acknowledgments The research for this chapter was supported by the National Science Foundation grant DRL-0334199 (“The Educational Accelerator: Technology Enhanced Learning in Science”). Disclaimer Parts of this chapter have been previously published by the author in the form of a dissertation, journal articles, and book chapters.

References Acton, W. H., Johnson, P. J., & Goldsmith, T. E. (1994). Structural knowledge assessment – Comparison of referent structures. Journal of Education & Psychology, 86(2), 303–311. Adamczyk, A., & Willson, M. (1996). Using concept maps with trainee physics teachers. Physics Education, 31(6), 374–381. Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183–198.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

679

Ainsworth, S. E. (1999). A functional taxonomy of multiple representations. Computers & Education, 33(2/3), 131–152. Alters, B. J., & Nelson, C. E. (2002). Perspective: Teaching evolution in higher education. Evolution, 56(10), 1891–1901. Anderson, D., Lucas, K. B., & Ginns, I. S. (2000). Development of knowledge about electricity and magnetism during a visit to a science museum and related post-visit activities. Science Education, 84, 658–679. Anderson, R. C. (1984). Some reflections on the acquisition of knowledge. Educational Researcher, 13(9), 5–10. Ariew, A. (2003). Ernst Mayr’s ‘ultimate/proximate’ distinction reconsidered and reconstructed. Biology and Philosophy, 18(4), 553–565. Atay, S., & Karabacak. (2012). Care plans using concept maps and their effects on the critical thinking dispositions of nursing students. International Journal of Nursing Practice, 18(3), 233–239. Ault, C. R. (1985). Concept mapping as a study strategy in earth science. Journal of College Science Teaching, 15, 38–44. Austin, L. B., & Shore, B. M. (1995). Using concept mapping for assessment in physics. Physics Education, 30, 41. Ausubel, D. P. (1963). The psychology of meaningful verbal learning: An introduction to school learning. New York, NY: Grune & Stratton. Ausubel, D. P., Novak, J. D., & Hanesian, H. (1978). Educational psychology – A cognitive view. London, UK: Holt, Rienhart and Winston. Ayala, C. C., Yin, Y., Shavelson, R. J., & Vanides, J. (2002). Investigating the cognitive validity of science performance assessment with think alouds: Technical aspects. New Orleans, LA: American Educational Researcher Association. Aydin, S., Aydemir, N., Boz, Y., Cetin-Dindar, A., & Bektas, O. (2009). The contribution of constructivist instruction accompanied by concept mapping in enhancing pre-service chemistry teachers’ conceptual understanding of chemistry in the laboratory course. Journal of Science Education and Technology, 18, 518–534. Bakri, F., & Muliyati, D. (2018). Design of multiple representations e-learning resources based on a contextual approach for the basic physics course. Proceedings from Journal of Physics: Conference Series, 1013(1), 1–7. Banet, E., & Ayuso, G. E. (2003). Teaching of biological inheritance and evolution of living beings in secondary school. International Journal of Science Education, 25(3), 373–407. Barnett, R. (2000). Realizing the university in an age of supercomplexity. Buckingham, UK/Philadelphia, PA: Society for Research into Higher Education & Open University Press. Bascones, J., & Novak, J. D. (1985). Alternative instructional systems and the development of problem-solving skills in physics. International Journal of Science Education, 7(3), 253–261. Baxter, G. P., & Glaser, R. (1998). Investigating the cognitive complexity of science assessments. Educational Measurement: Issues and Practice, 17(3), 37–45. Berland, L. K., & Reiser, B. J. (2009). Making sense of argumentation and explanation. Science Education, 93(1), 26–55. Birbili, M. (2006). Mapping knowledge: Concept maps in early childhood education. Early Childhood Research and Practice, 8(2). Bjork, R. A., & Linn, M. C. (2006). The science of learning and the learning of science – Introducing desirable difficulties. APS Observer, 19(3), 1–2. BouJaoude, S., & Attieh, M. (2008). The effect of using concept maps as study tools on achievement in chemistry. Eurasia Journal of Mathematics, Science and Technology Education, 4(3), 233–246. Brandt, L., Elen, J., Hellemans, J., Heerman, L., Couwenberg, I., Volckaert, L., & Morisse, H. (2001). The impact of concept mapping and visualization on the learning of secondary school chemistry students. International Journal of Science Education, 23(12), 1303–1313. Bransford, J., Brown, A. L., & Crocking, R. R. (2000). How people learn: Brain, mind, experience, and school (Expanded edition). Washington, DC: National Academy Press

680

B. A. Schwendimann

Bressington, D. T., Wong, W.-K., Lam, K. K. C., & Chien, W. T. (2018). Concept mapping to promote meaningful learning, help relate theory to practice and improve learning self-efficacy in Asian mental health nursing students: A mixed-methods pilot study. Nurse Education Today, 60, 47–55. Brody, M. J. (1993). Student misconceptions of ecology: Identification, analysis and instructional design. In J. D. Novak (Ed.), Proceedings of the third international seminar on misconceptions and educational strategies in science and mathematics. Ithaca, NY: Cornell University. Brown, A. L., & Campione, J. C. (1996). Psychological learning theory and the design of innovative environments: On procedures, principles and systems. In L. Shauble & R. Glaser (Eds.), Contributions of instructional innovation to understanding learning. Hillsdale, NJ: Lawrence Erlbaum Associates. Brown, D. S. (2003). High school biology: A group approach to concept mapping. American Biology Teacher, 65(3), 192–197. Bruechner, K., & Schanze, S. (2004). Using concept maps for individual knowledge externalization in medical education. In First international conference on concept mapping. Pamplona, Spain. Bruner, J. S. (1960). The process of education. New York, NY: Vantage. Bruner, J. S., Goodnow, J. J., & Austin, G. A. (1986). A study of thinking. New Brunswick, NJ: Transaction Publishers. Buntting, C., Coll, R. K., & Campell, A. (2006). Student views of concept mapping use in introductory tertiary biology classes. International Journal of Science and Mathematics Education, 4(4), 641–668. Byrne, J., & Grace, M. (2010). Using a concept mapping tool with a photograph association technique (compat) to elicit children’s ideas about microbial activity. International Journal of Science Education, 32(4), 37–43. Cakir, M., & Crawford, B. (2001). Prospective biology teachers’ understanding of genetics concepts. Paper presented at the annual meeting of the Association for the education of Teachers in Science, Costa Mesa, CA. Cañas, A. J. (2003). A summary of literature pertaining to the use of concept mapping techniques and technologies for education and performance support. The Institute for Human and Machine Cognition 40 S. Alcaniz St. Pensacola FL 32502 http://www.ihmc.us/users/aCañas/Publica tions/ConceptMapLitReview/ Cañas, A. J. (2004). Cmap tools – Knowledge modeling kit [Computer Software]. Pensacola, FL: Institute for Human and Machine Cognition (IHMC). Canas, A. J. (2016). Cmap tools – Knowledge modeling kit [Computer Software]. Pensacola, FL: Institute for Human and Machine Cognition (IHMC). Cañas, A. J., Novak, J. D., & Reiska, P. (2012). Freedom vs. Restriction of content and structure during concept mapping – Possibilities and limitations for construction and assessment. In Proceedings of the fifth international conference on concept mapping, Proc. of the Fifth Int. Conference on Concept Mapping Valletta, Malta 2012 (pp. 247–257). Cañas, Suri, Sanchez, Gallo, & Brenes. (2003). Synchronous collaboration in cmap tools. IHMC. Carey, S., & Spelke, E. (1994). Domain-specific knowledge and conceptual change. In Mapping the mind: Domain specificity in cognition and culture (pp. 169–200). Cambridge, MA/New York, NY: Massachusetts Institute of Technology, Department of Brain & Cognitive Sciences/Cambridge University Press. Cathcart, Laura, Stieff, Mike, Marbach-Ad, Gili, Smith, Ann, & Frauwirth, Kenneth. (2010). Using knowledge structure maps as a foundation for knowledge management. ICLS. Chand, L., Sowmya, K., & Silambanan, S. (2018). Meaningful learning in medical science by selfdirected approach of concept mapping. Journal of Education Technology in Health Sciences, 5(1), 31–35. Chang, K. E., Chiao, B. C., Chen, S. W., & Hsiao, R. S. (2000). A programming learning system for beginners-a completion strategy approach. IEEE Transactions on Education, 43(2), 211–220. Chang, K. E., Sung, Y. T., & Chen, S. F. (2001). Learning through computer-based concept mapping with scaffolding aid. Journal of Computer Assisted Learning, 17(1), 21–33.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

681

Chang, S.-N. (2007). Externalising students’ mental models through concept maps. Journal of Biological Education, 41(3), 107–112. Chartrand, G., & Zhang, P. (2004). Introduction to graph theory. Boston, MA: McGraw-Hill Higher Education. Chen, S.-L., Liang, T., Lee, M.-L., & Liao, I.-C. (2011). Effects of concept map teaching on students’ critical thinking and approach to learning and studying. The Journal of Nursing Education, 50(8), 466–469. Chi, M. T. H. (2000). Self-explaining: The dual processes of generating inference and repairing mental models. In Advances in instructional psychology: Educational design and cognitive science (Vol. 5, pp. 161–238). Mahwah, NJ: Lawrence Erlbaum Associates Publishers. Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121–151. Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In Advances in the psychology of human intelligence (pp. 7–75). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Chinn, C. A., & Brewer, W. F. (2001). Models of data: A theory of how people evaluate data. Cognition and Instruction, 19(3), 323–393. Chiu, J. (2008). Examining the role of self-monitoring and explanation prompts on students’ interactions with dynamic molecular visualizations. In Poster presented at the 8th international conference of the learning sciences, international perspectives in the learning sciences: Cre8ting a learning world, Utrecht, The Netherlands. Chiu, J. L. (2009). The impact of feedback on student learning and monitoring with dynamic visualizations. Annual meeting of the American Educational Research Association, San Diego, CA. Cicagnani. (2000). Concept mapping as a collaborative tool for enhancing online learning. Educational Technology & Society, 3(3). Clark, D. B., & Sampson, V. (2008). Assessing dialogic argumentation in online environments to relate structure, grounds, and conceptual quality. Journal of Research in Science Teaching, 45(3), 293–321. Clark, D. B., & Slotta, J. (2000). Evaluating media-enhancement and source authority on the Internet: The knowledge integration environment. International Journal of Science Education, 22(8), 859–872. Cliburn, J. W., Jr. (1990). Concept maps to promote meaningful learning. Journal of College Science Teaching, 19(4), 212–217. Cline, B. E., Brewster, C. C., & Fell, R. D. (2009). A rule-based system for automatically evaluating student concept maps. Expert Systems with Applications, 37, 2282. Coleman, E. B. (1998). Using explanatory knowledge during collaborative problem solving in science. Journal of the Learning Sciences, 7(3), 387–427. Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making thinking visible. American Educator, 15(3), 6–11. Crank, J. N., & Bulgren, J. A. (1993). Visual depictions as information organizers for enhancing achievement of students with learning disabilities. Learning Disabilities Research and Practice, 8(3), 140–147. Cuthbert, A., & Slotta, J. (2004). Fostering lifelong learning skills on the World Wide Web: Critiquing, questioning and searching for evidence. International Journal of Science Education, 27(7), 821–844. Czerniak, C. M., & Haney, J. J. (1998). The effect of collaborative concept mapping on elementary preservice teachers’ anxiety, efficacy, and achievement in physical science. Journal of Science Teacher Education, 9(4), 303–320. Daley, B. J., & Torre, D. M. (2010). Concept maps in medical education: An analytical literature review. Medical Education, 44(5), 440–448. Davis, E. A. (2003). Prompting middle school science students for productive reflection: Generic and directed prompts. The Journal of the Learning Sciences, 12(1), 91–142. Davis, E. A. (2004). Knowledge integration in science teaching: Analysing teachers’ knowledge development. Research in Science Education, 34(1), 21–54.

682

B. A. Schwendimann

Davis, E. A., & Kirkpatrick, D. (2002). It’s all the news: Critiquing evidence and claims. Science Scope, 25(5), 32–37. Davis, E. A., & Linn, M. C. (2000). Scaffolding students’ knowledge integration: Prompts for reflection in KIE. International Journal of Science Education, 22(8), 819–837. Demastes, S. S., Good, R. G., & Peebles, P. (1995). Students’ conceptual ecologies and the process of conceptual change in evolution. Science Education, 79(6), 637–666. DeMeo, S. (2007). Constructing a graphic organizer in the classroom: Introductory students’ perception of achievement using a decision map to solve aqueous acid-base equilibria problems. Journal of Chemical Education, 84(3), 540–546. Derbentseva, N., Safayeni, F., & Canas, A. J. (2007). Concept maps: Experiments on dynamic thinking. Journal of Research in Science Teaching, 44(3), 448–465. diSessa, A. (2004). Metarepresentation: Native competence and targets for instruction. Cognition and Instruction, 22, 293–331. diSessa, A. A. (1988). Knowledge in pieces. In G. Forman & P. Pufall (Eds.), Constructivism in the computer age. (pp. 49–70). Hillsdale, NJ: Lawrence Erlbaum Associates. diSessa, A. A. (2002). Students’ criteria for representational adequacy. In K. Gravemeijer, R. Lehrer, B. Van Oers, & L. Verschaffel (Eds.), Synbolizing, modeling, and tool use in mathematics education (pp. 105–129). Boston, MA: Kluwer. diSessa, A. A. (2006). A history of conceptual change research: Threads and fault lines. In K. Sawyer (Ed.), The cambridge handbook of the learning sciences (pp. 265–282). New York, NY: Cambridge University Press. diSessa, A. A. (2008). A bird’s eye view of the “pieces” vs. “Coherence” controversy. In S. Vosniadou (Ed.), International handbook of research on conceptual change. Mahwah, NJ: Lawrence Erlbaum Associates. Duncan, R. G., & Reiser, B. J. (2005). Designing for complex system understanding in the high school biology classroom. Annual meeting of the National Association for Research in Science Teaching. Duncan, R. G., & Reiser, B. J. (2007). Reasoning across ontologically distinct levels: Students’ understandings of molecular genetics. Journal of Research in Science Teaching, 44(7), 938–959. Edmondson, K. M. (1993). Concept mapping for meaningful learning in veterinary education. In J. D. Novak (Ed.), Proceedings of the third international seminar on misconceptions and educational strategies in science and mathematics. Ithaca, NY: Cornell University. Edmondson, K. M. (1995). Concept mapping for the development of medical curricula. Journal of Research in Science Teaching, 32(7), 777–793. Edmondson, K. M. (2000). Assessing science understanding through concept maps. In Assessing science understanding (pp. 15–40). Academic Press. El-Hay, S. A. A., El Mezayen, S. E., & Ahmed, R. E. (2018). Effect of concept mapping on problem solving skills, competence in clinical setting and knowledge among undergraduate nursing students. Journal of Nursing Education and Practice, 8, 34. Englebrecht, A. C., Mintzes, J. J., Brown, L. M., & Kelso, P. R. (2005). Probing understanding in physical geology using concept maps and clinical interviews. Journal of Geoscience Education, 53(3), 263. Enyedy, N. (2005). Inventing mapping: Creating cultural forms to solve collective problems. Cognition and Instruction, 427–466. Ericsson, K. A., & Simon, H. A. (1985). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press. Falchikov, N., & Goldfinch, J. (2000). Student peer assessment in higher education: A metaanalysis comparing peer and teacher marks. Review of Educational Research, 70(3), 287–322. Fang, N. (2018). An analysis of student experiences with concept mapping in a foundational undergraduate engineering course. International Journal of Engineering Education, 34(2), 294. Farrokh, K., & Krause, G. (1996). The relationship of concept-mapping and course grade in cell biology. Meaningful Learning Forum, 1.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

683

Fisher, K. M. (2000). SemNet software as an assessment tool. In Assessing science understanding: A human constructivist view (pp. 197–221). San Diego, CA: Academic. Fisher, K. M., Wandersee, J. H. M., & Moody, D. E. (2000). Mapping biology knowledge. Dordrecht, The Netherlands: Kluwer Academic Publishers. Ford, M. J. (2008). Disciplinary authority and accountability in scientific practice and learning. Science Education, 92, 404. Gaines, B. R., & Shaw, M. L. G. (1995). Collaboration through concept maps. In CSCL 1995 proceedings, 95, 135–138. Gallenstein, N. L. (2005). Never too young for a concept map. Science and Children, 43(1), 44–47. Garwood, J. K., Ahmed, A. H., & McComb, S. A. (2018). The effect of concept maps on undergraduate nursing students’ critical thinking. Nursing Education Perspectives, 39(4), 208–214. Gentner, D. (1978). On relational meaning: The acquisition of verb meaning. Child Development, 49, 988. Gerdeman, J. L., Lux, K., & Jacko, J. (2013). Using concept mapping to build clinical judgment skills. Nurse Education in Practice, 13(1), 11–17. Gerstner, S., & Bogner, F. X. (2009). Concept map structure, gender and teaching methods: An investigation of students’ science learning. Educational Research, 51(4), 425–438. Glaser, R., Chi, M. T. H., & Farr, M. J. (1985). The nature of expertise. Columbus, OH: National Center for Research in Vocational Education, The Ohio State University. Goel, A., & Chandrasekaran, B. (1989). Functional representation of designs and redesign problem solving. In Proceedings of the 11th international joint conference on artificial intelligence, 2, 1388–1394. Goel, A. K., Rugaber, S., & Vattam, S. (2008). Structure, behavior, and function of complex systems: The structure, behavior, and function modeling language. Artificial Intelligence for Engineering Design, Analysis and Manufacturing, 23, 23. González, F. M. (1997). Diagnosis of spanish primary school students’ common alternative science conceptions. School Science and Mathematics, 97(2), 68–74. Grosslight, L., Unger, C., Jay, E., & Smith, C. (1991). Understanding models and their use in science: Conceptions of middle and high school students and experts. Journal of Research in Science Teaching (Special Issue: Students’ Models and Epistemologies of Science), 28(9), 799–822. Grossschedl, J., & Tröbst, S. (2018). Biologie lernen durch Concept Mapping: Bedeutung eines Lernstrategietrainings für kognitive Belastung, kognitive Prozesse und Lernleistung–Kurzdarstellung des DFG–Projekts. Zeitschrift für Didaktik der Biologie (ZDB)-Biologie Lehren und Lernen, 22(1), 20–30. Grundspenkis, J., & Strautmane, M. (2009). Usage of graph patterns for knowledge assessment based on concept maps. Scientific Journal of Riga Technical University. Computer Sciences, 38 (38), 60–71. Guastello, E. F., Beasley, T. M., & Sinatra, R. C. (2000). Concept mapping effects on science content comprehension of low-achieving inner-city seventh graders. Remedial and Special Education, 21(6), 356–364. Guindon, R. (1990). Designing the design process: Exploiting opportunistic thoughts. Human Computer Interaction, 5(2), 305–344. Halford, G. S. (1993). Children’s understanding: The development of mental models. Australia Hillsdale, NJ: Lawrence Erlbaum Associates. Hamdiyati, Y., Sudargo, F., Redjeki, S., & Fitriani, A. (2018). Using concept maps to describe undergraduate students’ mental model in microbiology course. Proceedings from Journal of Physics: Conference Series, 1013(1), 1–5. Hay, D. B. (2007). Using concept maps to measure deep, surface and non-learning outcomes. Studies in Higher Education, 32(1), 39–57. Hay, D. B. (2008). Developing dialogical concept mapping as an e-learning technology. British Journal of Educational Technology, 39, 1057–1060.

684

B. A. Schwendimann

Heinze-Fry, J. A. (1998). Concept mapping: Weaving conceptual connections. In Weaving connections: Cultures and environments – Selected papers from the 26th annual North American association of environmental education conference (NAAEE) (pp. 138–147), Troy, OH. Heinze-Fry, J. A., & Novak, J. D. (1990). Concept mapping brings long-term movement toward meaningful learning. Science Education, 74(4), 461–472. Herl, H. E. (1999). Reliability and validity of a computer-based knowledge mapping system to measure content understanding. Computers in Human Behavior, 15(3-4), 315–333. Herl, H. E., O’Neil, H. F. J., Chung, G. K., Dennis, R. A., & Lee, J. J. (1997, March). Feasibility of an on-line concept mapping construction and scoring system. Report: ED424233. 27pp. Hmelo, C. E., Holton, D. L., & Kolodner, J. L. (2000). Designing to learn about complex systems. The Journal of the Learning Sciences, 9(3), 247–298. Hmelo-Silver, C. (2004). Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions. Cognitive Science, 28, 127–138. Hmelo-Silver, C. E., Marathe, S., & Liu, L. (2007). Fish swim, rocks sit, and lungs breathe: Expert–novice understanding of complex systems. Journal of the Learning Sciences, 16(3), 307–331. Hoadley, C., & Kirby, J. (2004). Socially relevant representations in interfaces for learning. In Y. B. Kafai, W. A. Sandoval, N. Enyedy, A. S. Nixon, & F. Herrera (Eds.), Embracing diversity in the learning sciences: Proceedings of the sixth international conference of the learning sciences (pp. 262–269). Mahwah, NJ: Lawrence Erlbaum Associates. Hoffman, R. R. (1998). How can expertise be defined? Implications of research from cognitive psychology. In R. Williams, W. Faulkner, & J. Fleck (Eds.), Exploring expertise (pp. 81–100). Edinburgh, Scotland: University of Edinburgh Press. Holley, C. D., Dansereau, D. F., & Harold, F. O. N. (1984). Spatial learning strategies: Techniques, applications, and related issues. New York, NY: Academic. Hook, P. A., & Boerner, K. (2005). Educational knowledge domain visualizations: Tools to navigate, understand, and internalize the structure of scholarly knowledge and expertise. In New directions in cognitive information retrieval (pp. 187–208). Springer, Dordrecht. Hoppe, H. U., Engler, J., & Weinbrenner, S. (2012). The impact of structural characteristics of concept maps on automatic quality measurement. In J. van Aalst, K. Thompson, M. J. Jacobson, & P. Reimann (Eds.), Proceedings of the 10th international conference of the learning sciences (ICLS). Sydney, NSW: ISLS. Horton, P. B., McConney, A. A., Gallo, M., Woods, A. L., Senn, G. J., & Hamelin, D. (1993). An investigation of the effectiveness of concept mapping as an instructional tool. Science Education, 77(1), 95–111. Hoz, R., Tomer, Y., Bowman, D., & Chayoth, R. (1987). The use of concept mapping to diagnose misconceptions in biology and earth sciences. In J. D. Novak (Ed.), Proceedings of the second international seminar misconceptions and educational strategies in science and mathematics (Vol. I, pp. 245–256). Ithaca, NY: Cornell University. Hsu, Y. S. (2008). Learning about seasons in a technologically enhanced environment: The impact of teacher-guided and student-centered instructional approaches on the process of students’ conceptual change. Science Education, 92(2), 320–344. Hsu, Y. S., Wu, H., & Hwang, F. (2008). Fostering high school students’ conceptual understandings about seasons: The design of a technology-enhanced learning environment. Research in Science Education, 38(2), 127–147. Hyerle, D. (1996). Visual tools for constructing knowledge. Alexandria, VA: Association for Supervision and Curriculum Development. Ifenthaler, D. (2010). Relational, structural, and semantic analysis of graphical representations and concept maps. Educational Technology Research and Development, 58(1), 81–97. https://doi. org/10.1007/s11423-008-9087-4 Inspiration. (2016). Inspiration. Irvine, L. (1995). Can concept mapping be used to promote meaningful learning in nurse education? Journal of Advanced Nursing, 21(6), 1175–1179. Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331, 772.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

685

Kaya, O. N. (2008). A student-centred approach: Assessing the changes in prospective science teachers’ conceptual understanding by concept mapping in a general chemistry laboratory. Research in Science Education, 38(1), 91–110. Keraro, F. N., Wachanga, S. W., & Orora, W. (2007). Effects of cooperative concept mapping teaching approach on secondary school students’ motivation in biology in Gucha district. International Journal of Science and Mathematics Education, 5(1), 111–124. Kern, C., & Crippen, K. J. (2008). Mapping for conceptual change. The Science Teacher, 75(6), 32–38. Kinchin, I. M. (2000a). Concept mapping in biology. Journal of Biological Education, 34(2), 61–68. Kinchin, I. M. (2000b). From ‘ecologist’ to ‘conceptual ecologist’: The utility of the conceptual ecology for teachers of biology. Journal of Biological Education, 34(4), 178–183. Kinchin, I. M. (2001). If concept mapping is so helpful to learning biology, why aren’t we all doing it? International Journal of Science Education, 23(12), 1257–1269. Kinchin, I. M. (2014). Concept mapping as a learning tool in higher education: A critical analysis of recent reviews. The Journal of Continuing Higher Education, 62(1), 39–49. Kinchin, I. M., De-Leij, F. A. A. M., & Hay, D. B. (2005). The evolution of a collaborative concept mapping activity for undergraduate microbiology students. Journal of Further and Higher Education, 29(1), 1–14. Kinchin, I. M., & Hay, D. B. (2007). The myth of the research-led teacher. Teachers and Teaching, 13(1), 43–61. Klein, G., Moon, B. M., & Hoffman, R. R. (2006). Making sense of sensemaking 1: Alternative perspectives. IEEE Intelligent Systems, 21(4), 70–73. Koc, M. (2012). Pedagogical knowledge representation through concept mapping as a study and collaboration tool in teacher education. Australasian Journal of Educational Technology, 28(4), 656–670. Kommers, P., & Lanzing, J. (1997). Students’ concept mapping for hypermedia design: Navigation through world wide web (WWW) space and self-assessment. Journal of Interactive Learning Research, 8(3–4), 421–455. Koopman, M., Teune, P., & Beijaard, D. (2011). Development of student knowledge in competence-based pre-vocational secondary education. Learning Environments Research, 14(3), 205–227. Koponen, I. T., & Nousiainen, M. (2018). Concept networks of students’ knowledge of relationships between physics concepts: Finding key concepts and their epistemic support. Applied Network Science, 3(1), 1–21. Koponen, I. T., & Pehkonen, M. (2010). Coherent knowledge structures of physics represented as concept networks in teacher education. Science & Education, 19(3), 259–282. Kuhn, T. S. (1962). The structure of scientific revolutions (1st ed.). Chicago, IL: University of Chicago Press. Lambiotte, J. G., Dansereau, D. F., Cross, D. R., & Reynolds, S. B. (1989). Multirelational seminatic maps. Educational Psychology Review, 1(4), 331–367. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. In R. Pea & J. S. Brown (Eds.), Learning in doing: Social, cognitive, and computational perspectives (pp. 29–129). Cambridge, MA: Cambridge University Press. Lehrer, R., & Schauble, L. (2004). Modeling natural variation through distribution. American Educational Research Journal, 41(3), 635–679. Lehrer, R., Schauble, L., Carpenter, S., & Penner, D. (2000). The interrelated development of inscriptions and conceptual understanding. In Symbolizing and communicating in mathematics classrooms: Perspectives on discourse, tools, and instructional design (pp. 325–360). Madison, WI/Mahwah, NJ: University of Wisconsin/Lawrence Erlbaum Associates Publishers. Leinhardt, G., Zaslavsky, O., & Stein, M. K. (1990). Functions, graphs, and graphing: Tasks, learning, and teaching. Review of Educational Research. Special Issue: Toward a Unified Approach to Learning as a Multisource Phenomenon, 60(1), 1–63.

686

B. A. Schwendimann

Levine, R. (1998). Cognitive lab report (report prepared for the national assessment governing board). Palo Alto, CA: American Institutes for Research. Linn, M. C. (2000). Designing the knowledge integation environment. International Journal of Science Education, 22(8), 781–796. Linn, M. C. (2002). Science education: Preparing lifelong learners. In N. J. Smelser & P. B. Baltes (Eds.), International encyclopedia of the social and behavioral sciences. New York, NY: Pergamon. Linn, M. C. (2008). Teaching for conceptual change: Distinguish or extinguish ideas. In S. Vosniadou (Ed.), International handbook of research on conceptual change. New York, NY: Routledge. Linn, M. C., Chang, H.-Y., Chiu, J., Zhang, H., & McElhaney, K. (2010). Can desirable difficulties overcome deceptive clarity in scientific visualizations? In A. Benjamin (Ed.), Successful remembering and successful forgetting: A Festschrift in honor of Robert A. Bjork. London, UK: Psychology Press. Linn, M. C., Davis, E. A., & Eylon, B.-S. (2004). The scaffolded knowledge integration framework for instruction. In M. C. Linn, E. A. Davis, & P. Bell (Eds.), Internet environments for science education (pp. 47–72). Mahwah, NJ: Lawrence Erlbaum Associates. Linn, M. C., & Eylon, B. S. (2006). Science education: Integrating views of learning and instruction. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 511–544). Mahwah, NJ: Lawrence Erlbaum Associates. Linn, M. C., & Hsi, S. (2000). Computers, teachers, peers: Science learning partners. Mahwah, NJ: Lawrence Erlbaum Associates. Linn, M. C., Lee, H.-S., Tinker, R., Husic, F., & Chiu, J. L. (2006). Teaching and assessing knowledge integration in science. Science, 313(5790), 1049–1050. Liu, L., & Hmelo-Silver, C. E. (2009). Promoting complex systems learning through the use of conceptual representations in hypermedia. Journal of Research in Science Teaching, 46, 1023. Liu, O. L., Lee, H. S., & Linn, M. C. (2010). Multifaceted assessment of inquiry-based science learning. Educational Assessment, 15(2), 69–86. Liu, X. (2004). Using concept mapping for assessing and promoting relational conceptual change in science. Science Education, 88(3), 373–396. Liu, X., & Hinchey, M. (1993). The validity and reliability of concept mapping as an alternative science assessment. In The proceedings of the third international seminar on misconceptions and educational strategies in science and mathematics. Ithaca, NY: Misconceptions Trust. Liu, X., & Hinchey, M. (1996). The internal consistency of a concept mapping scoring scheme and its effect on prediction validity. International Journal of Science Education, 18(8), 921–937. Mahler, S., Hoz, R., Fischl, D., Tov-Ly, E., & Lernau, O. Z. (1991). Didactic use of concept mapping in higher education: Applications in medical education. Instructional Science, 20(1), 25–47. Mancinelli, C., Gentili, M., Priori, G., & Valitutti, G. (2004). Concept maps in kindergarten. In Concept maps: Theory, methodology, technology. Proceedings of the first international conference on concept mapping. Pamplona, Spain: Universidad pública de navarra. Maneval, R. E., Filburn, M. J., Deringer, S. O., & Lum, G. D. (2011). Concept mapping: Does it improve critical thinking ability in practical nursing students? Nursing Education Perspectives, 32(4), 229–233. Marcum, J. (2008). Instituting science: Discovery or construction of scientific knowledge? International Studies in the Philosophy of Science, 22, 185–210. Markham, K. M., Mintzes, J. J., & Jones, M. G. (1993). The structure and use of biological knowledge about mammals in novice and experienced students. Paper presented at the third international seminar on misconceptions and educational strategies in science and mathematics, Cornell University, Ithaca, NY, August 1–4, 1993 Markham, K. M., Mintzes, J. J., & Jones, M. G. (1994). The concept map as a research and evaluation tool: Further evidence of validity. Journal of Research in Science Teaching, 31(1), 91–101.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

687

Markow, P. G., & Lonning, R. A. (1998). Usefulness of concept maps in college chemistry laboratories: Students’ perceptions and effects on achievement. Journal of Research in Science Teaching, 35(9), 1015–1029. Martin, D. J. (1994). Concept mapping as an aid to lesson planning: A longitudinal study. Journal of Elementary Science Education, 6(2), 11–30. Marzano, R. J., Pickering, D., & Pollock, J. E. (2001). Classroom instruction that works: Researchbased strategies for increasing student achievement. Alexandria, VA: ASCD. Mason, C. L. (1992). Concept mapping: A tool to develop reflective science instruction. Science Education, 76(1), 51–63. Maton, K., & Doran, Y.J. (n.d.in press, 2016) Semantic density: A translation device for revealing complexity of knowledge practices in discourse, part 1 – Wording, Onomázein, August. Mayr, E. (1988). Toward a new philosophy of biology. Cambridge, MA: Harvard University Press. McClure, J. R., Sonak, B., & Suen, H. K. (1999). Concept map assessment of classroom learning: Reliability, validity, and logistical practicality. Journal of Research in Science Teaching, 36(4), 475–492. McMillan, W. J. (2010). Teaching for clinical reasoning – Helping students make the conceptual links. Medical Teacher, 32, 436–442. Metcalf, S. J., Reilly, J. M., Kamarainen, A. M., King, J., Grotzer, T. A., & Dede, C. (2018). Supports for deeper learning of inquiry-based ecosystem science in virtual environmentsComparing virtual and physical concept mapping. Computers in Human Behavior, 87, 459–469. Michael, R. S. (1995). The validity of concept maps for assessing cognitive structure. Dissertation Abstracts International Section A: Humanities and Social Sciences, 55(10-A), 3141. Mintzes, J., & Quinn, H. J. (2007). Knowledge restructuring in biology: Testing a punctuated model of conceptual change. International Journal of Science and Mathematics Education, 5, 281–306. Mintzes, J. J., Wanderersee, J. H., & Novak, J. D. (2001). Assessing understanding in biology. Journal of Biological Education, 35. Mintzes, J. J., Wandersee, J. H., & Novak, J. D. (1997). Meaningful learning in science: The human constructivist perspective. In Handbook of academic learning: Construction of knowledge. The educational psychology series (pp. 405–447). Wilmington, NC/San Diego, CA: University of North Carolina, Department of Biological Science/Academic. Mintzes, J. J., Wandersee, J. H., & Novak, J. D. (2000). Assessing science understanding: A human constructivist view. San Diego, CA: Educational Psychology Press/Academic. Mistades, V. M. (2009). Concept mapping in introductory physics. Journal of Education and Human Development, 3(1), 177. Moreira, M. A. (1987). Concept mapping as a possible strategy to detect and to deal with misconceptions in physics. In J. D. Novak (Ed.), Proceedings of the second international seminar “misconceptions and educational strategies in science and mathematics” (Vol. III, pp. 352–360). Ithaca, NY: Cornell University. Morfidi, E., Mikropoulos, A., & Rogdaki, A. (2018). Using concept mapping to improve poor readers’ understanding of expository text. Education and Information Technologies, 23(1), 271–286. Mun, K., Kim, J., Kim, S.-W., & Krajcik, J. (2014). Exploration of high school students concepts about climate change through the use of an issue concept map (ic-map). In International conference on science education 2012 proceedings (pp. 209–222). Springer, Berlin, Heidelberg Nehm, R. H., & Schonfeld, I. S. (2007). Does increasing biology teacher knowledge of evolution and the nature of science lead to greater preference for the teaching of evolution in schools? Journal of Science Teacher Education, 18(5), 699–723. Nejat, N., Kouhestani, H. R., & Rezaei, K. (2011). Effect of concept mapping on approach to learning among nursing students. HAYAT, 17(2), 22–31. Nesbit, J. C., & Adesope, O. O. (2006). Learning with concept and knowledge maps: A metaanalysis. Review of Educational Research, 76(3), 413–448. Nicoll, G., Francisco, J. S., & Nakhleh, M. (2001a). An investigation of the value of using concept maps in general chemistry. Journal of Chemical Education, 78(8), 1111.

688

B. A. Schwendimann

Nicoll, G., Francisco, J.S., & Nakhleh, M.B. (2001b). A three-tier system for assessing concept map links: A methodological study. Nijman, J. L., Sixma, H., Triest, B. V., Keus, R. B., & Hendriks, M. (2012). The quality of radiation care: The results of focus group interviews and concept mapping to explore the patients perspective. Radiotherapy and Oncology, 102(1), 154–160. Norton, P. B., McConney, A. A., Gallo, M., Woods, A. L., Senn, G. J., & Hamelin, D. (1993). An investigation of the effectiveness of concept mapping as an instructional tool. Science Education, 77(1), 95–111. Novak, J. D. (1980). Meaningful reception learning as a basis for rational thinking. In The psychology of teaching for thinking and creativity. Columbus, Oh: ERIC Clearinghouse for Science, Mathematics and Environmental Education. Novak, J. D., Bob Gowin, D., & Johansen, G. T. (1983). The use of concept mapping and knowledge vee mapping with junior high school science students. Science Education, 67(5), 625–645. Novak, J. D., & Cañas, A. J. (2006). The theory underlying concept maps and how to construct them. Pensacola, FL: IHMC. Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. Cambridge, UK: Cambridge University Press. Nugrahani, R., Prasetyo, A. P. B., & Iswari, R. S. (2018). Authentic assessment of fungi for vocational school student: concept map, self assessment and performance test. Journal of Innovative Science Education, 7(1), 11–24. O’Donnell, A. M., Dansereau, D. F., & Hall, R. H. (2002). Knowledge maps as scaffolds for cognitive processing. Educational Psychology Review, 14(1), 71–86. Odom, A. L., & Kelly, P. V. (2001). Integrating concept mapping and the learning cycle to teach diffusion and osmosis concepts to high school biology students. Science Education, 85(6), 615–635. Oezmen, H., Demircioglu, G., & Coll, R. K. (2007). A comparative study of the effects of a concept mapping enhanced laboratory experience on turkish high school students’ understanding of acid-based chemistry. International Journal of Science and Mathematics Education. Okebukola, P. A. (1992). Concept mapping with a cooperative learning flavor. The American Biology Teacher, 54(4), 218–221. Okebukola, P. A., & Jegede, O. J. (1989). Students’ anxiety towards and perception of difficulty of some biological concepts under the concept-mapping heuristic. Research in Science & Technological Education, 7(1), 85–92. Osborne, R. J., & Wittrock, M. C. (1983). Learning science: A generative process. Science Education, 67(4), 489–508. Osmundson, E., Chung, G., Herl, H., & Klein, D. (1999). Knowledge mapping in the classroom: A tool for examining the development of students’ conceptual understandings. Los Angeles, CA: University of California Los Angeles. Pallant, A., & Tinker, R. F. (2004). Reasoning with atomic-scale molecular dynamic models. Journal of Science Education and Technology, 13(1), 51–66. Pankratius, W. J. (1990). Building an organized knowledge base: Concept mapping and achievement in secondary school physics. Journal of Research in Science Teaching, 27(4), 315–333. Park, H. J. (2007). Components of conceptual ecologies. Research in Science Education, 37(2), 217–237. Parnafes, O., & diSessa, A. A. (2004). Relations between types of reasoning and computational representations. International Journal of Computers for Mathematical Learning, 9(3), 251–280. Pearsall, N., Skipper, J., & Mintzes, J. J. (1997). Knowledge restructuring in the life sciences: A longitudinal study of conceptual change in biology. Science Education, 81(2), 193–215. Pemmaraju, S. V., & Skiena, S. S. (2003). Computational discrete mathematics: Combinatorics and graph theory with mathematica. Cambridge, UK: Cambridge University Press. Penner, D. E. (2000). Explaining systems: Investigating middle school students’ understanding of emergent phenomena. Journal of Research in Science Teaching, 37(8), 784–806.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

689

Pirnay-Dummer, P., & Ifenthaler, D. (2010). Automated knowledge visualization and assessment. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge (pp. 77–115). New York, NY: Springer. Plotnick, E. (1997). Concept mapping: A graphical system for understanding the relationship between concepts: An ERIC digest. Clearinghouse on Information & Technology. Popova-Gonci, V., & Lamb, M. C. (2012). Assessment of integrated learning: Suggested application of concept mapping to prior learning assessment practices. The Journal of Continuing Higher Education, 60, 186–191. Preszler, R. (2004). Cooperative concept mapping: Improving performance in undergraduate biology. Journal of College Science Teaching, 33(6), 30–35. Puntambekar, S., Stylianou, A., & Huebscher, R. (2003). Improving navigation and learning in hypertext environments with navigable concept maps. Human Computer Interaction, 18(4), 395–428. Pushkin, D. (1999). Concept mapping and students, physics equations and problem solving. In M. Komorek, H. Behrendt, H. Dahncke, R. Duit, W. Graeber, & A. Kross (Eds.), Research in science education – Past, present, and future (Vol. 1, pp. 260–262). Kiel, Germany: IPN Kiel. Rebich, S., & Gautier, C. (2005). Concept mapping to reveal prior knowledge and conceptual change in a mock summit course on global climate change. Journal of Geoscience Education, 53(4), 355. Reiska, P., Dahncke, H., & Behrendt, H. (1999). Concept maps in a research project on “learning physics and taking action”. In M. Komorek, H. Behrendt, H. Dahncke, R. Duit, W. Graeber, & A. Kross (Eds.), Research in science education – Past, present, and future (Vol. 1, pp. 257–259). Kiel, Germany: IPN Kiel. Reiska, P., Soika, K., & Cañas, A. J. (2018). Using concept mapping to measure changes in interdisciplinary learning during high school. Knowledge Management & E-Learning: An International Journal (KM&EL), 10(1), 1–24. Rice, D. C., Ryan, J. M., & Samson, S. M. (1998). Using concept maps to assess student learning in the science classroom: Must different methods compete? Journal of Research in Science Teaching, 35(10), 1103–1127. Ritchhart, R., Turner, T., & Hadar, L. (2009). Uncovering students’ thinking about thinking using concept maps. Metacognition and Learning, 4, 145–159. Roessger, K. M., Daley, B. J., & Hafez, D. A. (2018). Effects of teaching concept mapping using practice, feedback, and relational framing. Learning and Instruction, 54, 11. Romance, N. R., & Vitale, M. R. (1999). Concept mapping as a tool for learning: Broadening the framework for student-centered instruction. College Teaching, 47(2), 74–79. Roth, W. M. (1993). Using Vee and concept maps in collaborative settings: Elementary education majors construct meaning in physical science courses. School Science and Mathematics, 93(5), 237–244. Roth, W. M. (1994a). Student views of collaborative concept mapping: An emancipatory research project. Science Education, 78(1), 1–34. Roth, W. M. (1994b). Science discourse through collaborative concept mapping – New perspectives for the teacher. International Journal of Science Education, 16(4), 437–455. Roth, W. M., & McGinn, M. K. (1998). Inscriptions: Toward a theory of representing as social practice. Review of Educational Research, 68(1), 35–59. Roth, W. M., & Roychoudhury, A. (1993). The concept map as a tool for the collaborative construction of knowledge: A microanalysis of high school physics students. Journal of Research in Science Teaching, 30(5), 503–534. Royer, R., & Royer, J. (2004). Comparing hand drawn and computer generated concept mapping. Journal of Computers in Mathematics and Science Teaching, 23(1), 67–81. Ruiz-Primo, M. A. (2000). On the use of concept maps as an assessment tool in science: What we have learned so far. Revista Electrónica De Investigación Educativa, 2(1), 30. Ruiz-Primo, M. A., Iverson, H., & Yin, Y. (2009). Towards the use of concept maps in large-scale assessments: Exploring the efficiency of two scoring methods. NARST conference 2009, Garden Grove (CA).

690

B. A. Schwendimann

Ruiz-Primo, M. A., Schultz, S. E., Li, M., & Shavelson, R. J. (2001). Comparison of the reliability and validity of scores from two concept-mapping techniques. Journal of Research in Science Teaching, 38(2), 260–278. Ruiz-Primo, M. A., Schultz, S. E., & Shavelson, R. J. (1997). Concept map-based assessment in science: Two exploratory studies (CSE report, 436). Ruiz-Primo, M. A., & Shavelson, R. J. (1996). Problems and issues in the use of concept maps in science assessment. Journal of Research in Science Teaching, 33(6), 569–600. Rutledge, M. L., & Mitchell, M. A. (2002). High school biology teachers’ knowledge structure, acceptance and teaching of evolution. American Biology Teacher, 64(1), 21–28. Rye, J. A., & Rubba, P. A. (2002). Scoring concept maps: An expert map-based scheme weighted for relationships. School Science and Mathematics, 102(1), 33–44. Safayeni, F., Derbentseva, N., & Canas, A. J. (2005). A theoretical note on concepts and the need for cyclic concept maps. Journal of Research in Science Teaching, 42(7), 741–766. https://doi. org/10.1002/tea.20074 Santhanam, E., Leach, C., & Dawson, C. (1998). Concept mapping: How should it be introduced, and is there evidence for long term benefit? Higher Education, 35(3), 317–328. Sarhangi, F., Masoumy, M., Ebadi, A., Seyyed Mazhari, M., Rahmani, A., & Raisifar, A. (2011). Effect of concept mapping teaching method on critical thinking skills of nursing students. Iranian Journal of Critical Care Nursing (IJCCN). 143–148. Scaife, M., & Rogers, Y. (1996). External cognition: How do graphical representations work? International Journal of Human Computer Studies, 45(2), 185–213. Scardamalia, M., & Bereiter, C. (1991). Literate expertise. In Toward a general theory of expertise. Prospects and limits (pp. 172–194). Cambridge, UK: Cambridge University Press. Schaap, H., Van der Schaaf, M. F., & De Bruijn, E. (2011). Development of students’ personal professional theories in senior secondary vocational education. Evaluation & Research in Education, 24(2), 81–103. Schau, C., & Mattern, N. (1997). Assessing students’ connected understanding of statistical relationships. From Gal, I. & Garfield, J. B. (editors). The Assessment Challenge in Statistics Education. IOS Press, 1997 (on behalf of the ISI). ISBN 90 5199 333 1, (pp. 91–104). Schau, C., Mattern, N., Weber, R., Minnick, K., & Witt, C. (1997). Use of fill-in concept maps to assess middle school students’ connected understanding of science. AERA annual meeting, Chicago, IL. Schauble, L., Glaser, R., Duschl, R. A., Schulze, S., & John, J. (1995). Students’ understanding of the objectives and procedures of experimentation in the science classroom. The Journal of the Learning Sciences, 4(2), 131–166. Schmid, R. F., & Telaro, G. (1990). Concept mapping as an instructional strategy for high school biology. Journal of Educational Research, 84(2), 78–85. Schuster, P. M. (2011). Concept mapping: A critical thinking approach to care planning. Philadelphia, PA: FA Davis. Schvaneveldt, R. W., Durso, F. T., Goldsmith, T. E., Breen, T. J., Cooke, N. M., Tucker, R. G., & DeMaio, J. C. (1985). Measuring the structure of expertise. International Journal of Man-Maschine Studies, 23, 699–728. Schwarz, C. V., & White, B. Y. (2005). Metamodeling knowledge: Developing students’ understanding of scientific modeling. Cognition and Instruction, 23(2), 165–205. Schwendimann, B. A. (2007). Integrating interactive genetics visualizations into high school biology. Annual meeting of the American Educational Research Association, Chicago, IL. Schwendimann, B. A. (2009). Scaffolding an integrated understanding of biology through dynamic visualizations and critique-focused concept mapping. Annual meeting of the American Education Research Association (AERA), San Diego, CA. Schwendimann, B. A. (2011a). Mapping biological ideas: Concept maps as knowledge integration tools for evolution education (Dissertation). Retrieved from http://search.proquest.com/ docview/928947890?accountid=1475 Schwendimann, B. A. (2011b). Integrating genotypic and phenotypic ideas of evolution through critique-focused concept mapping. AERA annual meeting 2011, New Orleans, LA.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

691

Schwendimann, B. A. (2011c). Linking genotypic and phenotypic ideas of evolution through collaborative critique-focused concept mapping. In Proceedings of the 9th international conference on computer-supported collaborative learning (CSCL). Hong Kong, China: CSCL Conference. Schwendimann, B. A. (2014a). Making sense of knowledge integration maps. In D. Ifenthaler & R. Hanewald (Eds.), Digital knowledge maps in education: Technology enhanced support for teachers and learners. New York, NY: Springer. Schwendimann, B. A. (2014b). Comparing two forms of concept map critique activities to support knowledge integration in biology education. In Proceedings of the sixth international conference on concept mapping. Santos, Brazil: International Conference on Concept Mapping. Schwendimann, B. A., & Linn, M. C. (2015). Comparing two forms of concept map critique activities to facilitate knowledge integration processes in evolution education. Journal of Research in Science Teaching, 4, 70–94 Shavelson, R. J., Ruiz-Primo, M. A., & Wiley, E. W. (2005). Windows into the mind. Higher Education, 49(4), 413–430. Shawli, A. S. (2018). Concept mapping as an assessment of cognitive load and mental effort in complex problem solving in chemistry (Doctoral thesis). Montana State University. Shen, J. (2010). Nurturing students’ critical knowledge using technology-enhanced scaffolding strategies in science education. Journal of Science Education and Technology, 19(1), 1–12. https://doi.org/10.1007/s10956-009-9183-1 Shen, J., & Confrey, J. (2007). From conceptual change to transformative modeling: A case study of an elementary teacher in learning astronomy. Science Education, 91(6), 948–966. Shen, J., & Confrey, J. (2010). Justifying alternative models in learning the solar system: A case study on K-8 science teachers’ understanding of frames of reference. International Journal of Science Education, 32(1), 1–29. Silva, J. H. D., Foureaux, G., Sá, M. A. D., Schetino, L. P. L., & Guerra, L. B. (2018). The teaching and learning of human anatomy: The assessment of student performance after the use of concept maps as a pedagogical strategy. Ciência & Educação (Bauru), 24(1), 95–110. Sizmur, S., & Osborne, J. (1997). Learning processes and collaborative concept mapping. International Journal of Science Education, 19(10), 1117–1135. Slotta, J. D., Chi, M. T. H., & Joram, E. (1995). Assessing students’ misclassifications of physics concepts: An ontological basis for conceptual change. Cognition and Instruction, 13(3), 373–400. Slotta, J. D., & Linn, M. C. (2000). How do students make sense of Internet resources in the science classroom? In M. J. Jacobson & R. Kozma (Eds.), Learning the sciences of the 21st century (pp. 193–226). Hillsdale, NJ: Lawrence Erlbaum & Associates. Snead, D., & Snead, W. L. (2004). Concept mapping and science achievement of middle grade students. Journal of Research in Childhood Education, 18(4), 306–320. Songer, N. B. (2006). Biokids: An animated conversation on the development of curricular activity structures for inquiry science. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 355–369). New York, NY: Cambridge University Press. Spaulding, D. T. (1989). Concept mapping and achievement in high school biology and chemistry. Dissertation. Florida. Institute of Technology. Starr, M. L., & Krajcik, J. S. (1990). Concept maps as a heuristic for science curriculum development: Toward improvement in process and product. Journal of Research in Science Teaching, 27(10), 987–1000. Stensvold, M. S., & Wilson, J. T. (1990). The interaction of verbal ability with concept mapping in learning from a chemistry laboratory activity. Science Education, 74(4), 473–480. Stewart, J. (1979). Concept maps: A tool for use in biology teaching. American Biology Teacher, 41(3), 171–175. Stice, C. F., & Alvarez, M. C. (1987). Hierarchical concept mapping in the early grades. Childhood Education, 64(2), 86–96.

692

B. A. Schwendimann

Stoddart, T., Abrams, R., Gasper, E., & Canaday, D. (2000). Concept maps as assessment in science inquiry learning-a report of methodology. International Journal of Science Education, 22(12), 1221–1246. Strike, K. A., & Posner, G. J. (1992). A revisionist theory of conceptual change. In R. A. Duschl & R. J. Hamilton (Eds.), Philosophy of science, cognitive psychology, and educational theory and practice. Albany, NY: State University of New York Press. Sun, J. C.-Y., Hwang, G.-J., Lin, Y.-Y., Yu, S.-J., Pan, L.-C., & Chen, A. Y.-Z. (2018). A votable concept mapping approach to promoting students’ attentional behavior: An analysis of sequential behavioral patterns and brainwave data. Journal of Educational Technology & Society, 21(2), 177–191. Sundararajan, N., Adesope, O., & Cavagnetto, A. (2018). The process of collaborative concept mapping in Kindergarten and the effect on critical thinking skills. Journal of STEM Education, 19(1), 5–13. Suprapto, N., Prahani, B. K., Jauhariyah, M. N. R., & Admoko, S. (2018). Exploring physics concepts among novice teachers through CMAP tools. Proceedings from Journal of Physics: Conference Series, 997(1), 1–7. Sweller, J., Van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296. Syarifuddin, H. (2018). The effect of using concept maps in elementary linear algebra course on students’ learning. Proceedings from IOP Conference Series: Materials Science and Engineering, 335(1), 1–4. Tabak, I., Weinstock, M., & Zvilling-Beiser, H. (2009). Epistemology and learning in the disciplines: Cross-domain epistemological views of science versus humanities students. In J. Shen (Ed.), Critique to learn science. Symposium conducted at the meeting of the national association for research in science teaching, Garden Grove, CA. Taylor, L. A., & Littleton-Kearney, M. (2011). Concept mapping: A distinctive educational approach to foster critical thinking. Nurse Educator, 36(2), 84–88. Trowbridge, J. E., & Wandersee, J. H. (1994). Identifying critical junctures in learning in a college course on evolution. Journal of Research in Science Teaching, 31(5), 459–473. Trowbridge, J. E., & Wandersee, J. H. (1996). How do graphics presented during college biology lessons affect students’ learning? Journal of College Science Teaching, 26(1), 54–57. Tsai, C.-C., & Huang, C.-M. (2002). Exploring students’ cognitive structures in learning science: A review of relevant methods. Journal of Biological Education, 36(4), 163–169. Tseng, H.-C., Chou, F.-H., Wang, H.-H., Ko, H.-K., Jian, S.-Y., & Weng, W.-C. (2011). The effectiveness of problem-based learning and concept mapping among Taiwanese registered nursing students. Nurse Education Today, 31(8), 41–46. Tsui, C., & Treagust, D. (2010). Evaluating secondary students’ scientific reasoning in genetics using a two-tier diagnostic instrument. International Journal of Science Education, 32(8), 1073–1098. Turan-Oluk, N., & Ekmekci, G. (2018). The effect of concept maps, as an individual learning tool, on the success of learning the concepts related to gravimetric analysis. Chemistry Education Research and Practice, 19, 819–833. Uzuntiryaki, E., & Geban, O. (2005). Effect of conceptual change approach accompanied with concept mapping on understanding of solution concepts. Instructional Science, 33(4), 311–339. van Amelsvoort, M., Andriessen, J., & Kanselaar, G. (2005). Using representational tools to support historical reasoning in computer-supported collaborative learning. Technology, Pedagogy and Education, 14(1), 25–41. Van Bommel, M., Kwakman, K., & Boshuizen, H. P. (2012). Experiences of social work students with learning theoretical knowledge in constructivist higher vocational education: A qualitative exploration. Journal of Vocational Education & Training, 64(4), 529–542. Van Merriënboer, J. J. G. (1990). Strategies for programming instruction in high school: Program completion vs. Program generation. Journal of Educational Computing Research, 6(3), 265–285. Van Neste-Kenny, J., Cragg, C. E. B., & Foulds, B. (1998). Using concept maps and visual representations for collaborative curriculum development. Nurse Educator, 23(6), 21–25.

27

Concept Maps as Versatile Learning, Teaching, and Assessment Tools

693

Van Zele, E., Lenaerts, J., & Wieme, W. (2004). Improving the usefulness of concept maps as a research tool for science education. International Journal of Science Education, 26(9), 1043–1064. Veo, P. (2010). Concept mapping for applying theory to nursing practice. Journal for Nurses in Professional Development, 26(1), 17–22. Vilela, R., Austrilino, L., & Costa, A. (2004). Using concept maps for collaborative curriculum development. In Proceedings of the first international conference on concept mapping, Pamplona, Spain. Walker, J. M. T., & King, P. H. (2003). Concept mapping as a form of student assessment and instruction in the domain of bioengineering. Journal of Engineering Education, 92(2), 167–178. Wallace, J. D., & Mintzes, J. J. (1990). The concept map as a research tool: Exploring conceptual change in biology. Journal of Research in Science Teaching, 27(10), 1033–1052. Wandersee, J. H. (1996). Bioinstrumentation: Tools for understanding life. Reston, WA: National Association of Biology Teachers. Wasserman, S., & Faust, K. (1994). Social network analysis: Methods and applications (p. 825). Cambridge, UK: Cambridge University Press. Watson, C. E. (2005). Graphic organizers: Toward organization and complexity of student content knowledge (Dissertation). Weick, K. E. (1995). Sensemaking in organizations (Vol. 3). Thousand Oaks: Sage. Weinstein, C. E., & Mayer, R. E. (1983). The teaching of learning strategies. Innovation Abstracts, 5, 4. Wenger, E. (1998). Communities of practice: Learning, meaning and identity. Cambridge, UK: Cambridge University Press. West, D. C., Pomeroy, J. R., Park, J. K., Gerstenberger, E. A., & Sandoval, J. (2000). Critical thinking in graduate medical education: A role for concept mapping assessment? JAMA, 284 (9), 1105. Wisdom Soft. (2016). Autoscreenrecorder 2.0. Autoscreenrecorder 2.0 [Computer Software]. Wise, A. M. (2007). Map it: How concept mapping affects understanding of evolutionary processes (Thesis). Yin, Y., Vanides, J., Ruiz-Primo, M. A., Ayala, C. C., & Shavelson, R. J. (2005). Comparison of two concept-mapping techniques: Implications for scoring, interpretation, and use. Journal of Research in Science Teaching, 42(2), 166–184. Zeilik, M., Schau, C., Mattern, N., Hall, S., Teague, K. W., & Bisard, W. (1997). Conceptual astronomy: A novel model for teaching postsecondary science courses. American Journal of Physics, 65, 987.

Beat A. Schwendimann is an accomplished educator and researcher with a passion for scienceeducation. He holds an M.Sc. in biology from the Swiss Federal Institute of Technology (ETH) in Zurich, Switzerland. He also holds a master’s in advanced studies in secondary and higher education (MAS-SHE) from ETH and a Ph.D. in Science and Mathematics Education from UC Berkeley. His doctoral research focused on concept mapping as tools to enhance technology-based STEM learning. Beat A. Schwendimann has extensive teaching experience with a wide range of learners and taught teacher training courses. For his post-doctoral work, he held positions as a research scientist at the École polytechnique fédérale de Lausanne (EPFL) in Lausanne, Switzerland, and the University of Sydney, Australia, where he developed, implemented, and studied innovative learning technologies. Beat A. Schwendimann has received several awards and honors, including a Fulbright scholarship to conduct his Ph.D. research in the USA. He is currently working at the Swiss teacher federation (Dachverband Lehrerinnen und Lehrer Schweiz LCH) as a member of the executive board and as the head of pedagogical office.

Technology and Feedback Design

28

Phillip Dawson, Michael Henderson, Tracii Ryan, Paige Mahoney, David Boud, Michael Phillips, and Elizabeth Molloy

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Is Feedback? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Educator to Student Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Digital Recordings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Digital Text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Collaborative Writing Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bug in Ear Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Computer to Student Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Computer-Assisted Language Learning (CALL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Student Response Systems (SRS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Automated Feedback on Online Multiple Choice Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Automated Writing Evaluation Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Intelligent Tutoring Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peer to Student Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Blogs and Discussion Boards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Collaborative Writing Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peer Feedback Software and Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

696 697 699 700 703 703 706 707 708 709 709 711 712 713 715 716 716 718 719

P. Dawson (*) · P. Mahoney · D. Boud Centre for Research in Assessment and Digital Learning, Deakin University, Geelong, VIC, Australia e-mail: [email protected]; [email protected]; [email protected] M. Henderson · T. Ryan · M. Phillips Faculty of Education, Monash University, Melbourne, VIC, Australia e-mail: [email protected]; [email protected]; [email protected] E. Molloy Department of Medical Education, University of Melbourne, Melbourne, VIC, Australia e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_124

695

696

P. Dawson et al.

Self-Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Digital Recordings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . e-Portfolios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Benefits, Challenges, and Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Future of Feedback and Digital Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

721 721 723 724 731 733

Abstract

This chapter provides a synthesis of recent research into how technology can support effective feedback. It begins by adopting a definition of feedback in line with recent advances in feedback research. Rather than viewing feedback as mere information provision, feedback is viewed as an active process that students undertake using information from a variety of sources. The results of a systematic literature search into technology and feedback are then presented, structured around the parties involved in feedback: students, their peers, educators, and computers. The specific feedback technologies focused on include digital recordings; bug in ear technologies; automated feedback; and intelligent tutoring systems. Based on this synthesis of the literature, benefits, challenges and design implications are presented for key feedback technologies. The chapter concludes with a discussion of improved feedback approaches that are likely to be enabled by technology in the future. Keywords

Feedback · Educational technology · Modality · Learning

Introduction Feedback about student learning is important, often misunderstood and complex. Technology can enable current practices, offer new opportunities, but can also complicate and challenge feedback. This chapter reviews the literature on the use of digital technology in student feedback practices and highlights established and emerging trends, as well as the diversity in approaches. These approaches are thematically organized according to the source of feedback comments, namely: educator, computer, peer, and self. However, within these categories there is a wide range of technology mediated feedback practices, from digital multimedia recordings and text annotations to intelligent tutors and student response systems. Overall, these approaches are reported to lead to positive student perceptions or other outcomes. However, this chapter also highlights a number of challenges for educators and educational designers who seek to implement these designs and concludes with consideration of future practices.

28

Technology and Feedback Design

697

What Is Feedback? Feedback is such a commonly used term in educational contexts, so we might imagine that it is clearly understood and used well. Unfortunately, this is not the case. This is particularly problematic in the context of feedback and digital technology, where feedback is commonly and unnecessarily used quite differently in the fields of education and technology. From the point of view of education, feedback commonly refers to information provided to learners about their work by teachers or other agents. It is seen as an input into an educational process which is left in the hands of the learner to do with whatever they wish. Teachers may hope that the information provided is productively used, but there is little follow through to track it or ensure that this happens. In the technology discourse, feedback is a process, not an input, which regulates a system, necessarily influencing the output of that system. Feedback has not occurred if the system is not influenced. Input without effect is not feedback, it is merely input. This gap in how feedback is understood might provide part of the explanation of why feedback in educational contexts has been subject to such relentless criticism by students. In higher education, feedback is often revealed as the number one concern of students across institutions, across disciplines, and over time. Students complain that they do not get enough information about their work, that what they do get is not useful and they do not get it in a timely fashion (see Li & De Luca, 2014, for a review of assessment feedback). Is there then some way of bridging the divide which provides a way of understanding feedback that is consistent with its longstanding use in technology and offers useful directions for education? We suggest that firstly there is and secondly that we can build on this conception to establish ways of thinking about feedback in the digital context which respects the fact that learners are humans with their own volition and that an educational view of feedback must fit with this view, rather than with a more technical view as the learner as one component of a technical system. Consequently, we argue that many of the current feedback traditions in education should be challenged. We should critically consider dimensions such as the agency of the student, the ability to measure effects, feedback’s location in a learning sequence, feedback’s goals, and how information flows. Such a framework is offered by Boud and Molloy (2013) who described three ways of thinking about feedback which they labeled Feedback Mark 0, Feedback Mark 1, and Feedback Mark 2. Table 1 provides a succinct comparison of these conceptions. They called the first of these conceptions Mark 0 because they regarded it as having so little of the characteristics of feedback used in other disciplines that calling it feedback at all was a problem. Unfortunately, Mark 0 reflects the most common feedback practices in education. Feedback in such a view is initiated by teachers, it normally occurs at the end of a sequence of teaching following an occasion of assessment, there is no process to detect whether information provided has any effect, and student involvement in feedback as such is minimal. Students may independently choose to do something as a result of the information available, but that is not in this conception an integral and necessary part of the process called feedback.

698

P. Dawson et al.

Table 1 Comparison of feedback conceptions Approach

Locus Features

Location Effects Learner involvement Information provided

Goal

Feedback mark 0 Conventional – teachers provide comments without monitoring effects Teacher Taken-for-granted act of teacher/ assessor At end of teaching sequence Effects not detected directly No student involvement needed Information provided not influenced by effects Study improvement

Feedback mark 1 Agentic – teachers monitor the effects of comments/inputs. The students’ role is to respond to teachers’ input Teacher Closed system (e.g., teacher and student) During learning Effects monitored by teachers Students respond to input from teachers Information provided changes in response to immediate effects Task performance improvement

Feedback mark 2 Participatory – both students and teachers have the role of monitoring and responding to effects Teacher and learner Open system (multiple sources of input) Adaptive/responsive During learning and beyond Effects monitored by teachers and learners Students respond, question, seek, and evaluate input Information provided changes in response to immediate and long-term effects Judgment performance improvement

The second of Boud and Molloy’s (2013) conceptions, called Feedback Mark 1, took key ideas of feedback as used in science and technology and applied them to educational contexts. In this conception, feedback was still driven by teachers or embedded in the learning management system, but it incorporated the fundamental idea of feedback as a process which necessarily leads to effects. In the case of learners, an effect would be some detectable change in their practices or learning outcomes. Feedback is not seen as an add on at the end of a process of teaching and learning but intrinsic to the learning process, leading to changes in what students do as they progress. These effects are monitored and the inputs varied in the light of the effects. There is always a feedback loop in which the information provided to learners is designed to lead to some change in learning behavior which is then monitored and the inputs changed to produce the effects desired. Feedback Mark 1 characterizes what is or should be commonplace in instructional design: system performance is referenced to effects on learners. The important move from Mark 0 to Mark 1 is the emphasis on effects and the necessary actions which learners must take if the process is identified as feedback. Such a conception of feedback is not enough, however, to provide a robust basis on which to ground educational activity. The most important limitation is that it positions learners in a contingent relationship in which they have little volition: the system is modified to maximize outputs regardless of the desires of learners. They are supposed to learn despite themselves! How then could learners have a more

28

Technology and Feedback Design

699

active role in assessment without them being reduced to just one element in a physical system? This concern led to Boud and Molloy’s third conception of feedback, Feedback Mark 2. The importance of effects and the feedback loop is retained from Mark 1, but the learner is placed in a more agentic position. A key element is that feedback in this conception is dialogic, that is, information is exchanged as a two-way process between learner and teacher, students express a view about what they want to know and information moves to and fro throughout the learning process. Feedback is not associated only with acts of assessment but is a key feature of the entire learning system. Effects are monitored by both teachers and learners through a learning management system. Student engagement is not an add-on but an intrinsic feature of the feedback process. Consideration of Feedback Mark 2 led Boud and Molloy to define feedback in a more learner-centered way as: a process whereby learners obtain information about their work in order to appreciate the similarities and differences between the appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work. (Boud & Molloy, 2013, p. 6)

Boud and Molloy’s definition raises several interesting questions for educators, technologists, and educational designers: What information is most useful? How can learners best obtain the information, and from whom or what? How do learners come to know the applicable standards? What is quality and how is it best communicated? And finally, how can learners action the information for improvement? In addition to these questions, Boud and Molloy’s conceptions of Mark 0, 1, and 2 should also challenge us to question the role of the teachers and students in monitoring effects. These questions continue to be applicable when we consider the role of digital technologies. For instance, how might technologies enable access, change roles, mediate delivery, offer new ways of creating, manipulating, or experiencing the input, and enable the tracking of effects? Unfortunately, it is common for researchers and practitioners to take for granted what feedback is and to report their work accordingly without revealing their assumptions. Certainly, in the context of the literature review in this chapter, there was a predominance of studies that treated feedback as if it were solely an input and failed to track effects. Nevertheless, the review of technology enabled feedback practices can inform our designs, but at the same time should be critically appraised in light of how they help achieve feedback as defined above.

Literature Review What is the current state of feedback with technology? What promising new feedback technologies are there, and how are they being incorporated into feedback approaches, especially those described above? To explore such questions, a structured review of the literature was conducted. The process involved in the literature review is described below, as are the key findings relating to the most commonly

700

P. Dawson et al.

researched types of technology used in feedback design and delivery. We have focused on feedback about student learning, rather than feedback about teaching or curricula; however, we note overlaps in some sections where the two are closely related.

Method Systematic literature searches were performed by two experienced researchers (TR and PM) between December 2016 and February 2017. The searches were conducted in three stages: the first stage aimed to establish the scope of the field, the second aimed to ascertain the validity of search terms, and the third stage refined the search results and identified likely articles. Stage one searches were guided by themes proposed by the members of the research team experienced in feedback design and the use of technology (PD, MH, DB, MP, LM). The chosen themes were media effects and outcomes, issues of timing, artificial intelligence, automated feedback, peer feedback, peer assessment, systems managing feedback flow, self-feedback, self-assessment, academic integrity, and stealth feedback. Stage one searches were performed by one author (PM) using three databases which provide access to a large number of scientific academic articles from education researchers: ProQuest Education, ERIC, and PsycINFO. These initial searches were kept deliberately broad in order to get a sense of the data before more exhaustive and targeted secondary searches were conducted. All primary and secondary search terms were recorded in a spreadsheet, along with the number of results. Prior to the second stage searches, four members of the research team (PD, MH, TR, DB) examined the spreadsheet of search results and identified viable topics for further searches. These topics centered on the use of technological tools in the creation, mediation, tracking, or experience of feedback “inputs,” or performance related information. The research team felt that these topics were most likely to result in a breadth of feedback designs in which technologies played a variety of roles in the creation or mediation of performance information. These topics were then inductively organized into clusters based on the source of the feedback “inputs” (i.e., educator-to-student, computer-to-student, peer-to-student, and self-feedback) and were used as the basis of the search terms for the secondary searches (see column 2 of Table 2). Stage two searches were conducted by one author (TR) and were limited to peerreviewed scholarly journal articles that were (a) written in English and (b) published between 1st January 2012 and 1st January 2017. These searches were guided by the need to establish which types of technology were used to provide feedback input by each of the four identified sources. To assist in the return of highly relevant research, searches were also restricted to articles that featured the search terms in the abstract, rather than anywhere in the entire document. In an effort to reduce the labor of sorting through the potential hundreds of abstracts that could have been returned for each topic, a pragmatic decision was made to limit the more focused secondary

Computerto-student feedback

Source of feedback Educatorto-student feedback

ab(feedback) AND ab (automated OR automation OR automatic)

Stage 2 search terms to determine main technology types used by sources ab(feedback) AND ab (teacher OR educator) AND ab(technology OR online)

81

Number of search results 234

Bug in ear technology Computer assisted language learning Student response systems Automated feedback on MCQs Automated writing

Collaborative writing tools

Digital text

Form of technology mediated feedback practice Digital recordings

8

6

8

17

10

9

ab(feedback) AND ab(“response system” OR “feedback device”) ab(“automated feedback” OR “automatic feedback” OR “online feedback”) AND ab (quiz OR test OR exam) ab(feedback) AND ab(“automated writing”)

4

8

12

Number of abstracts selected 30*

12

7

39

46

Number of search results 521

21

Stage 3 search terms to identify key literature ab(feedback) AND ab(audio OR video OR screencast OR multimodal OR podcast) NOT ab(peer) NOT ab(self) NOT ab (automated) ab(“electronic feedback” OR “online feedback” OR “digital feedback”) NOT ab (peer) NOT ab(self) NOT ab(automated) ab(feedback) AND (ab(“collaborative writing”) OR ab(wiki) OR ab(“google doc”)) NOT ab(peer) NOT ab(self) NOT ab (automated) ab(feedback) AND ab(“in ear”) NOT ab (peer) NOT ab(self) NOT ab(automated) ab(feedback) AND ab(“computer assisted language learning”)

Table 2 Search terms, number of results returned, and number of papers selected and reviewed

Technology and Feedback Design (continued)

4

5

3

5

3

6

11

Number of papers reviewed 27

28 701

140

103

ab(feedback) AND ab (peer) AND ab (technology OR online)

ab((“self feedback” OR “self evaluation” OR “self assessment”) AND (technology OR online))

Source of feedback

Peer-tostudent feedback

Selffeedback

e-portfolios

evaluation tools Intelligent tutors Blogs and discussion boards Collaborative writing software Peer feedback software and tools Digital recordings

Form of technology mediated feedback practice

ab(feedback) AND ab(“intelligent tutor” OR “cognitive tutor”) ab(“peer feedback” OR “feedback by peer”) AND (ab(blog) OR (ab(journal) OR ab(“discussion board”) OR ab(forum))) ab(“peer feedback” OR “feedback by peer”) AND (ab(“collaborative writing”) OR ab(wiki) OR ab(“google doc”)) ab(“peer feedback” OR “feedback by peer”) AND ab(software OR tool OR program OR application) ab(“self feedback” OR “self evaluation” OR “self assessment”) AND ab(audio OR video OR screencast OR multimodal OR podcast) ab(“self feedback” OR “self evaluation” OR “self assessment”) AND ab(“eportfolio” OR “web-based portfolio” OR “online portfolio” OR “digital portfolio”)

Stage 3 search terms to identify key literature

9

62

5

5

13

8

15

55

9

4

Number of abstracts selected

29

5

Number of search results

4

9

4

4

7

3

Number of papers reviewed

NOTE: Stage one searches were performed using the ProQuest Education, ERIC, and PsycINFO databases; however, the results of these searches were extensive and are not provided here. Stage two searches were performing using the ProQuest Education database, while stage three searches were performed using the ProQuest Education and ERIC databases. Items marked with * indicate where the number of abstracts reviewed was limited when it was deemed that a saturation point had been reached

Number of search results

Stage 2 search terms to determine main technology types used by sources

Table 2 (continued)

702 P. Dawson et al.

28

Technology and Feedback Design

703

searches to one database. The ProQuest Education database was selected for this purpose, as it includes a vast catalogue of research focused on primary, secondary, and higher education. Abstracts were sorted by relevance and reviewed to assess (a) their relevance to the topic of interest and (b) the type of technology used in the feedback design. Search terms were then refined as necessary and recorded (see column 4 of Table 2). Search terms that did not result in the return of at least four articles published within the last 5 years were abandoned. Stage three searches were conducted by two authors (TR and PM). These searches were performed on two databases simultaneously (ERIC and ProQuest Education); however, duplicate search records were omitted. The same search settings used in the stage two searches were applied again during stage three. Abstracts were sorted by relevance and read by at least one author to ascertain their relevance to the specific topic of interest. Although the search terms were designed to be as targeted as possible, not all of the search results were found to be relevant. As such, a decision was made to omit articles from further consideration if their abstracts did not fit within the scope of the search; namely, technology mediated feedback practices. The final column in the table provides the number of papers that met the search criteria, and actually informed the literature review. It should be noted that the search for educator generated digital recordings resulted in a particularly large number of papers. In this case (as indicated by an asterisk in Table 2), the papers were filtered as above, but only the first 30 papers that met the criteria were analyzed at the abstract level. It was deemed at the point of 30 abstracts that a saturation point had been achieved with regards to the main benefits, challenges, and design implications. The papers that were selected for review were read in full. Some of these papers were then discarded as their findings were not empirically based. The key findings, particularly relating to the reported effectiveness (or not) of the design, and implications for future design were summarized. The following sections present a synthesis of those results, organized according to the source of feedback input and the form of technology practice as indicated in Table 2.

Educator to Student Feedback The results of the initial searches indicated that digital recordings, digital text, collaborative writing tools, and bug in ear technology were the most commonly researched forms of technology used in feedback design.

Digital Recordings As shown in Table 2, digital recordings were the most prevalent form of technology mediated feedback design to emerge from the literature review. The bulk of this research centers on the use of audio (e.g., Bourgault, Mundy, & Joshua, 2013; Carruthers et al., 2015; Cavanaugh & Song, 2014), video (e.g., Borup, West, &

704

P. Dawson et al.

Thomas, 2015; Hawkins, Osborne, Schofield, Pournaras, & Chester, 2012), and screencast (e.g., Elola & Oskoz, 2016; Jones, Georghiades, & Gunson, 2012) recordings to provide asynchronous performance-related comments to students after submission of written assessment tasks. Through using audio-visual media to deliver performance information to students, educators can provide detailed comments to students in a relatively short recording. It is generally argued that it is faster to communicate orally than it is through typing or writing (e.g., Denton, 2014; Orlando, 2016). Due to this affordance, educators tended to positively appraise the use of audio-visual media to provide performance information. For example, in a study comparing the use of text, audio, or screencast recordings to provide comments to students, Orlando (2016) discovered that four out of the six educators preferred using screencasts, two preferred audio, and none preferred using text. Other studies have reported that educators appreciate the increased efficiency afforded by recorded comments, indicating that the practice may be relatively sustainable compared to marking up electronic documents or writing handwritten comments on assessment tasks (Borup et al., 2015; Jonsson, 2013; Knauf, 2016; Morris & Chikwa, 2016; Portolese Dias & Trumpy, 2014). Interestingly, several studies have also shown that the content of recorded comments is more often focused on providing holistic suggestions for improvement, rather than the targeted and specific comments often seen in digital text-based comments (Borup et al., 2015; Cavanaugh & Song, 2014; Elola & Oskoz, 2016). Performance information created using audio-visual digital recordings has been associated with enhanced student engagement (Hung, 2016; Morris & Chikwa, 2016; West & Turner, 2016) and performance (Denton, 2014; Elola & Oskoz, 2016). The majority of research confirms that students feel positively toward receiving audio-visual recordings from educators, finding the content to be individualized (Carruthers et al., 2015; Knauf, 2016) and detailed (Gould & Day, 2013; Jonsson, 2013; Morris & Chikwa, 2016). In studies that have directly compared audio-visual recorded comments with text, students generally have a strong preference for the former (Chew, 2014; Johnson & Cooke, 2016; McCarthy, 2015; Moore & Wallace, 2012; West & Turner, 2016). They also perceive recorded comments to be more supportive (Borup et al., 2015; Gould & Day, 2013), personal (Gould & Day, 2013; Knauf, 2016; Mathieson, 2012; West & Turner, 2016), and easy to understand (Bourgault et al., 2013; Turner & West, 2013) than text. On the other hand, some students are initially skeptical about receiving performance information in this way (Fawcett & Oldfield, 2016; Henderson & Phillips, 2015), while others note that textbased comments can be more efficient to scan through than digital recordings (Borup et al., 2015; Morris & Chikwa, 2016). This is because it is often necessary to listen to or watch a full recording to find the relevant information. Overwhelmingly, students recognize that audio-visual recordings of performance information are personal and supportive; therefore, this modality of feedback can be highly effective in educational contexts in which the affective relationship between

28

Technology and Feedback Design

705

students and educators needs bolstering. Audio-visual media facilitate the communication of rich cues like tone and expression (Cavanaugh & Song, 2014), which allows educators to provide more enriched performance information than they can with text (West & Turner, 2016). Educators also tend to communicate in their recordings using a more personal and informal style, and students appreciate their teachers relating to them in a relaxed manner (Borup et al., 2015). Furthermore, many students hold the opinion that audio-visual feedback recordings reflect a greater investment of time and effort by the educator than text comments (Anson, 2015; Chew, 2014), even though the opposite is generally true (Knauf, 2016). For example, in a study of 99 students, Portolese Dias and Trumpy (2014) found that those who received screencast feedback were more likely to believe that their instructor had genuine concern for their learning than those who only received text comments. This may be because students interpret the increased level of detail as reflecting a deeper level of care from educators. As such, this modality may be particularly advantageous when students and educators are presented with limited opportunities for face-to-face dialogue, such as at the beginning of the year or in courses that involve online instruction (Anson, 2015; Borup et al., 2015). Accessibility is one of the key design considerations when creating audio-visual recordings of performance information (Orlando, 2016). It is therefore recommended that educators create recordings that are of a manageable size for students to receive and download, and in a format that students can open without having to install additional applications. McCarthy (2015) recommends using programs that offer the ability to export to mp3 for audio and mp4 for video. Assuming the recordings are not excessively long (3–5 min is recommended), these formats compress files to a sharable size without significant loss of quality. They are also able to be opened automatically by native applications on most computers, smartphones, and tablets. Small files, such as audio recordings, can be sent to the student via email (Bourgault et al., 2013; Munro & Hollingworth, 2014) or returned within an electronic copy of students’ assignments (Orlando, 2016). Richer forms of audiovisual media (e.g., video and screencasts) can be shared using a video hosting website (Mathieson, 2012) or a virtual learning environment (Carruthers et al., 2015; Henderson & Phillips, 2014; Jones et al., 2012; Knauf, 2016; McCarthy, 2015). The method used to return the recordings is also an important factor. As Orlando (2016) notes, embedding the recordings directly into the relevant section of the assessment task has the benefit of allowing students to easily connect comments to the specific section of the work to which they refer. Of course, this is only possible with smaller files, such as audio. For larger files, it may be most beneficial to upload to the virtual learning environment, as this allows students to store their feedback together with other course-related learning materials (Parkin, Hepplestone, Holden, Irwin, & Thorpe, 2012). It also avoids issues associated with using video hosting websites, such as potential breaches of privacy and security (Henderson & Phillips, 2014).

706

P. Dawson et al.

Digital Text With an increasing number of written assessment tasks being submitted electronically, digital text has unsurprisingly become a common modality of technology mediated feedback used by educators (Chang et al., 2012). This is likely due to its simplicity and convenience; educators can employ online tools such as discussion boards and email to provide generalized comments to wider groups of students, or create digital text comments directly on a student’s electronic assessment tasks using easily accessible and user-friendly software such as word processing and PDF annotation programs. Furthermore, by utilizing simple tools such as tracked changes, sticky notes, comment boxes, or annotations, educators can link performance information directly to the applicable section on students’ assessment tasks (Beach, 2012). This leads to targeted and specific comments (Borup et al., 2015), which may aid in comprehension and enable students to take the information on board more readily. Research suggests that most students are comfortable receiving digital text-based comments on their written work, as it aligns with their prior experiences and expectations of feedback (McCarthy, 2015). To compare student preferences for handwritten or digital text comments, N. Chang et al. (2012) recruited 250 undergraduate students to complete an online survey. The majority of students preferred digital text over handwritten comments and provided open-ended responses citing reasons such as timeliness, enhanced accessibility, and legibility. However, in a similar comparison study, Sopina and McNeill (2015) surveyed 335 first-year students who received performance information on subsequent assessment tasks via handwritten comments and digital text. Their results indicated that students were more satisfied with digital text than handwritten comments when it came to timeliness of return, but there were no significant differences in satisfaction for quality or format. The use of digital text comments can be a timely and highly accessible method of providing performance information to students, particularly in comparison to handwritten comments. Digital text offers students the convenience of being able to access performance related comments on any personal computing device quickly and easily, no matter where they are located (Borup et al., 2015). Students can then store the comments permanently on their own devices, or on the university’s learning management systems (Parkin et al., 2012). Educators also appreciate the benefits of digital text-based comments; in a study by Borup et al. (2015), teaching staff noted that digital text provides the ability to easily review and edit comments, as well as the flexibility to complete assessment duties off site (assuming they have a portable computing device). However, despite the convenience of digital text-based comments, it is perhaps not the most efficient means of providing technology mediated performance information (see the audio-visual media subsection for more information on this topic). To increase efficiency when creating digital text comments, some educators utilize electronic rubrics (Gabaudan, 2013) or statement banks (Borup et al., 2015; Denton & Rowe, 2015), as these modalities avoid the need to type similar comments

28

Technology and Feedback Design

707

repeatedly. Statement banks can be created by the educator themselves using wordprocessing software (Leibold & Schwarz, 2015) or with the help of digital mark-up tools such as GradeMark® (Watkins et al., 2014). However, students tend to prefer feedback that offers a high level of detail and personalization, and this is not always possible when providing “one-size fits all” comments (Denton & Rowe, 2015). In addition, statement banks and electronic rubrics may be most appropriate for tasks in which there is a clear or model answer, rather than more complex and open ended forms of assessment, especially those where the criteria involve considerable tacit knowledge. For those types of assessment tasks, audio-visual media may offer a better alternative, providing rich and detailed information in a relatively short timeframe.

Collaborative Writing Tools Collaborative writing tasks are commonly used for educational purposes (Mauri, Ginesta, & Rochera, 2014), often with the goal of students constructing knowledge by engaging in the mutual exchange of opinion, concepts, and thoughts (Alvarez, Espasa, & Guasch, 2012; Zheng, Lawrence, Warschauer, & Lin, 2014). It has been argued that this process is valuable, as it can enhance the role of students as active learners, such as through reflection on their own ideas and abilities (Zheng et al., 2014). While there are various technological tools that support the act of collaborative writing, the literature in this review primarily relating to educator-provided performance information has primarily focused on the use of wikis (Eddy & Lawrence, 2012; Rott & Weber, 2013). Wikis are web-based platforms that allow multiple users to author and edit written content, share files, and post multimedia content, either synchronously or asynchronously. In general, wikis allow various levels of privacy so that the content can either be viewable to the public or restricted to a specified group of users (Eddy & Lawrence, 2012). As Israel and Moshirnia (2012) point out, wikis are highly appropriate for use in educational assessment as they are designed to be userfriendly and flexible, and they allow students to act as both author and reviewer. In particular, wikis may be beneficial for engaging students in the process of authentic assessment, such as building informational resources for the public or clients (Eddy & Lawrence, 2012). Students generally perceive wikis to be easy to use and agree that they are a useful way to learn information (Israel & Moshirnia, 2012). Due to their collaborative design, wikis are most commonly used in tasks incorporating peer feedback. As such, their utility as a tool for feedback design will be discussed in more detail in a later subsection. However, when it comes to educator to student feedback, it should be noted that wikis provide a potentially valuable platform for effective feedback processes. For example, the built-in editing tools allow educators to offer formative comments directly onto the relevant sections of the wiki, both during and after completion (Eddy & Lawrence, 2012). Students can then reflect and respond to the performance information they receive from

708

P. Dawson et al.

educators. Furthermore, wikis allow educators the ability to view the entire history of author modifications, making it possible to monitor student progression and improvements over time. For group-based work, educators also have the ability to view each individual student’s degree of contributions over time. When using wikis to provide performance information to students, Eddy and Lawrence (2012) recommends that the feedback process can be enhanced if educators design checkpoints where they monitor student progress and provide formative comments throughout the creation of the wiki.

Bug in Ear Technology This form of technology has historically involved the use of a two-way communication device, such as a radio transmitter or a Bluetooth communication device, placed in the ear of the student. This is coupled with a microphone used by an instructor who is observing the student, either from a distance within the same room or via webcam from a remote location (Gibson & Musti-Rao, 2015; Rock et al., 2012). While the feedback modalities discussed above have primarily focused on technology that aids in the delivery of asynchronous performance information on written assessment tasks, the use of bug in ear technology is more appropriate for the provision of real-time comments on certain performance or skill-based assessments. Much of the literature relating to the use of bug in ear technology appears to be focused on training preservice teachers as they work in the classroom (Gibson & Musti-Rao, 2015; Kelly, O’Neil, & Kwon, 2014; Rock et al., 2012). As Gibson and Musti-Rao (2015) note, the provision of real-time performance information encourages preservice teachers to immediately correct erroneous teaching behaviors. This is a useful means of preventing errors from becoming a routine part of the teaching practice, which is a valuable component of teacher training (Gibson & Musti-Rao, 2015). One interesting study of bug in ear technology was performed by Rock et al. (2012). These scholars designed an assessment process whereby supervisors used a webcam and microphone to provide real time feedback to 13 masters of teaching students as they were working in the classroom. Supervisors viewed the performance of students in real time from a remote location (e.g., their offices) by using the Skype videoconferencing program. The teachers-in-training were also running Skype on a webcam-enabled computer within their classrooms, as well as a Bluetooth-enabled earpiece which allowed them freedom to walk around while being able to hear their supervisor’s comments. Qualitative analysis of the student teachers’ written reflections revealed that they all highly valued the method by which they had obtained real-time performance information and were able to articulate how the comments had helped them to reflect on and improve their in-classroom strategies and academic delivery. However, almost half of the teaching students also mentioned having technical troubles with the bug in ear technology during the process.

28

Technology and Feedback Design

709

Based on the research, it seems that one of the main affordances of bug in ear technology is that is allows educators to provide real-time performance information as students work on tasks in situ. Students are provided performance information in real-time while not disrupting the flow of the class. The timing of these comments has been described as very powerful, as it helps students immediately understand how they can improve their performance. Bug in ear technology is also relatively cost effective, as educators can provide supervision from their own offices using readily available technology (Kelly et al., 2014). However, one of the drawbacks with using bug in ear technology is that it is unlikely to be sustainable, especially in large classes, due to the amount of time required by educators to provide real time feedback to multiple individual students. Furthermore, due to the multiple pieces of hardware and software needed to run such activities, there is a high risk of technical failure. On this topic, Gibson and Musti-Rao (2015) note that these types of technology are rapidly improving, which may make bug in ear technology a viable option for certain types of assessment in the future.

Computer to Student Feedback Initial searches revealed that there are six commonly researched forms of technology used to provide feedback from computers to students: computer assisted language learning software, student response systems, automated feedback on multiple choice quizzes, automated writing evaluation tools, and intelligent tutors. The subsections below expound on each of these topics.

Computer-Assisted Language Learning (CALL) It is commonly agreed that language students receive regular feedback on their written and spoken proficiencies (Ghahri, Hashamdar, & Mohamadi, 2015; Penning de Vries, Cucchiarini, Bodnar, Strik, & van Hout, 2015). However, opportunities to practice speaking and writing can be limited due to large class sizes and time constraints, and feedback may focus on meaning rather than accuracy, particularly when addressing spoken proficiency (Lee, Cheung, Wong, & Lee, 2013; Penning de Vries et al., 2015). Within this context, digital tools and software for language learning – collectively known as computer-assisted language learning (CALL) – have emerged as a possible means of improving students’ access to language practice opportunities. CALL is a broad field of research and practice that encompasses a diverse range of digital technologies from email, and simple audio systems, to more complex voice recognition, digital games, and immersive learning environments such as virtual worlds. However, much of the current research has focused on web-hosted software which provides students with automated feedback on their language skills (Choi, 2016; Lee et al., 2013; Penning de Vries et al., 2015). This software uses automatic text analysis or speech recognition to offer students immediate feedback on areas

710

P. Dawson et al.

such as content, structure, and grammar (Lee et al., 2013; Penning de Vries et al., 2015). CALL is often described as a convenient and flexible tool with which to practice written or spoken language skills, and its automated nature means students may practice as frequently and as intensively as they choose (Lee et al., 2013; Penning de Vries et al., 2015). It has also been reported that students feel less anxious about making language errors when using a CALL system than in a classroom or face-to-face context (Penning de Vries et al., 2015). CALL is typically used to offer students formative feedback, rather than to conduct summative assessment (Lee et al., 2013). Text-based CALL systems can facilitate a range of language tasks and often provide automated feedback on drafts to allow students to revise their work prior to submission (Lee et al., 2013). CALL systems for early language learners allow students to complete short translation tasks by filling gaps in dialogue or building sentences (Choi, 2016). CALL systems providing feedback on spoken language proficiency may require students to respond to onscreen prompts, such as pronouncing a word, or to answer a question by assembling an assortment of sentence components (Penning de Vries et al., 2015; Wang & Young, 2015). Penning de Vries et al. (2015) note that for more complex spoken language tasks, limiting students’ possible responses can improve the accuracy of automated speech recognition. However, this also limits the potential for practicing complex language tasks. Depending on the sophistication of the software, the performance information provided by CALL systems varies from limited, corrective feedback to suggestions on content, structure, and elaboration (Lee et al., 2013; Penning de Vries et al., 2015). Students generally feel that CALL systems are beneficial to their language proficiency, perceiving them as helpful, easy to use, and motivating (Lee et al., 2013; Penning de Vries et al., 2015). However, evidence of the effectiveness of CALL systems on student learning outcomes is less conclusive. Studies assessing text-based CALL systems typically report significant improvements in student writing and language acquisition after using CALL (Choi, 2016; Lee et al., 2013). By contrast, preliminary findings on the efficacy of speech-based CALL systems suggest that CALL-facilitated speech practice may assist students’ pronunciation, but is no more beneficial to grammar development than students self-monitoring their spoken practice (Penning de Vries et al., 2015; Wang & Young, 2015). CALL systems may also present users with technical and practical challenges. Insufficient or unstable Internet connections, or poor recording technology, can be particularly detrimental to the optimal operation of speech-based CALL systems (Penning de Vries et al., 2015). In addition, the automated nature of CALL systems means that feedback is typically targeted to specific areas – for instance, content rather than grammar – which can limit the usefulness of CALL (Lee et al., 2013). Lee et al. (2013) conclude that CALL systems should be used to supplement rather than supplant educator guidance and feedback. It is also recommended that students should be trained in how to use CALL systems and implement the resultant feedback; for example, students may be provided with revision strategies, examples, and opportunities to practice using the CALL system under educator supervision.

28

Technology and Feedback Design

711

Student Response Systems (SRS) Interactive student response systems (SRS) are commonly used in educational settings to attain and collate students’ responses to a question or topic in real time (Klein & Kientz, 2013; Voelkel & Bennett, 2014). SRS can be used for a range of tasks, including recording attendance and tracking students’ participation frequency; however, SRS are most frequently used as a dual feedback mechanism (Chui, Martin, & Pike, 2013). Student responses provided in class via a SRS provide educators with a snapshot of students’ levels of knowledge and understanding of content, which allows them to instantly alter their teaching to address gaps in understanding (Chui et al., 2013; Klein & Kientz, 2013). In turn, students receive immediate feedback on their own understanding of content, enabling them to reflect on their own learning and identify areas for revision (Chui et al., 2013). This approach is beneficial, as it creates a real-time feedback loop between educator and student (Voelkel & Bennett, 2014). While SRS are available in a variety of formats, perhaps the most common is the traditional “clicker.” This in-class response system involves the use of handheld devices, which students use to select their answer to a multiple-choice or open-ended question posed by their educator (Chui et al., 2013; Klein & Kientz, 2013; Voelkel & Bennett, 2014). Aggregated responses are then sent to the educator’s receiving clicker, who may choose to display and discuss the distribution of the results with the class (Chui et al., 2013; Klein & Kientz, 2013). Educators’ questions and aggregated student results may be displayed on a webpage or embedded into common programs such as PowerPoint (Klein & Kientz, 2013; Voelkel & Bennett, 2014); however, the increasing prevalence of smartphones and wireless Internet access across educational contexts has seen alternatives to clicker devices emerge (Chui et al., 2013; Voelkel & Bennett, 2014). Web-based SRS such as Poll Anywhere allow students to respond via SMS, through online voting via a smartphone or laptop, or even via Twitter (Voelkel & Bennett, 2014). Such web-based SRS are low-cost and easy to use, requiring minimal training and setup times (Voelkel & Bennett, 2014). Research suggests that students generally feel positively toward the use of SRS and consider it to be a valuable means of receiving feedback input. For example, students report that SRS are an engaging and thought-provoking learning tool, which allow them to learn more compared with non-SRS lectures (Klein & Kientz, 2013; Voelkel & Bennett, 2014). Students also feel more confident in their understanding after using SRS (Chui et al., 2013). While educator perceptions of SRS have received limited attention, one study found that web-based SRS were simple and quick to set up, easy for students to use, and offered good opportunities for student engagement and feedback (Voelkel & Bennett, 2014). Findings relating to the effectiveness of SRS in improving student learning outcomes remain unclear. While some studies report that the use of SRS may improve student understanding and performance (for example, Klein & Kientz, 2013; Voelkel & Bennett, 2014) others suggest such gains may be temporary. For example, Chui et al. (2013) found that students who completed in-class SRS quizzes

712

P. Dawson et al.

performed better than students who completed quizzes at the end of class and received feedback in the following lesson; however, overall course performance for both student cohorts remained similar. In addition, student rates of participation in SRS appear erratic; response rates may vary from 20% to 75%, with an average of 50% participation (Voelkel & Bennett, 2014). It has also been noted that SMS voting may discourage students from participating due to the cost involved. Although researchers theorize that the increasing prevalence of mobile phone plans that offer unlimited SMS may alleviate this difficulty, it is recommended that free SRS options such as online voting are prioritized (Voelkel & Bennett, 2014). Students also report that they may not always carry their smartphone or laptop, which may inhibit participation where web-based SRS are used (Voelkel & Bennett, 2014). Researchers also caution that students can potentially correctly answer questions without fully understanding content, which may inculcate a false sense of confidence among students and lead to reduced studying and effort by students (Chui et al., 2013).

Automated Feedback on Online Multiple Choice Questions Educators are increasingly using online multiple choice questions (MCQs) to provide formative assessment in educational contexts (Marden, Ulman, Wilson, & Velan, 2013). Online MCQs are typically made available to students via learning management systems, such as Moodle or Blackboard, which offer simple inbuilt templates (Bälter, Enström, & Klingenberg, 2013; DePaolo & Wilkinson, 2014; Sancho-Vinuesa, Escudero-Viladoms, & Masià, 2013). MCQs are typically completed by students outside of class, without restrictions on the use of study aids such as lecture notes or textbooks, although some studies have explored the use of invigilated, closed book online MCQs (Marden et al., 2013) The online delivery of MCQs offers students flexible and convenient access (Bälter et al., 2013; Marden et al., 2013), while allowing them to receive immediate performance information through the automated marking and feedback process (Bälter et al., 2013). This may improve the efficiency and feasibility of formative feedback for educators, particularly for large cohorts (Marden et al., 2013). Online MCQs also offer a degree of flexibility with regard to feedback. For example, they allow educators to provide feedback of different types, including basic corrective indicators (i.e., correct/incorrect) (Bälter et al., 2013), generic comments that indicate possible errors (Sancho-Vinuesa & Viladoms, 2012), or longer and more detailed explanations or clarifications (DePaolo & Wilkinson, 2014). Online MCQs may also be designed to alert educators to students who have made repeated errors or numerous failed attempts to complete a quiz, helping them identify and support students who may be having difficulties with content (Bälter et al., 2013; Sancho-Vinuesa & Viladoms, 2012). Formative online MCQs generally allow students to test their knowledge of a topic by repeatedly retaking a quiz. Questions may be constructed around a set of variables to allow repeated attempts by students and to limit students sharing answers among themselves (Bälter et al., 2013).

28

Technology and Feedback Design

713

Students are generally positive about the use of online MCQs for feedback purposes and consider them to be challenging, motivating, and valuable study tools (Bälter et al., 2013; DePaolo & Wilkinson, 2014; Marden et al., 2013). Students particularly appreciate that MCQs can be completed multiple times to test their knowledge, and report using this function to revise for summative assessments such as exams (Bälter et al., 2013; Marden et al., 2013). Moreover, the regular use of online MCQs can positively impact students’ study habits, allowing them to gain confidence and insight into their own learning (Bälter et al., 2013). Research suggests that online MCQs can potentially impact student approaches to learning. Sancho-Vinuesa and Viladoms (2012) found that students using online MCQs with generic automated feedback tended to adjust their use of MCQs in accordance with how difficult they had found a topic of study and that students who had made regular use of formative MCQs tended to pass the corresponding summative MCQs. There is emerging evidence that the regular use of formative, online MCQs can lead to improved learning outcomes among students and a reduced rate of students failing or dropping out of a course (Marden et al., 2013; Sancho-Vinuesa et al., 2013; Sancho-Vinuesa & Viladoms, 2012). Improved learning outcomes, such as end-of-semester exam results, are particularly associated with online MCQs which offer students unlimited attempts and are completed outside of class (Marden et al., 2013). It is recommended that the formative nature of online MCQs is clearly communicated to students, to increase the likelihood that students will use MCQs to test their own knowledge. Marden et al. (2013) suggest that students should be advised to first attempt quizzes under exam conditions, in order to provide a realistic indication of their knowledge, as completing MCQs using study resources may lead to quiz scores which do not accurately reflect a student’s understanding of content. It is also recommended that students make a note of which questions they answer incorrectly so as to revise these topics later (Marden et al., 2013). In addition, educators may consider incentivizing participation in online MCQs by allocating a small percentage of credit for undertaking the quizzes (DePaolo & Wilkinson, 2014; Marden et al., 2013).

Automated Writing Evaluation Tools First developed in the 1960s, automated systems for assessing student writing have primarily been used to score student work (Link, Dursun, Karakaya, & Hegelheimer, 2014). The last decade has seen the emergence of automated writing evaluation (AWE) tools which not only assess writing, but provide students with formative feedback on language components such as grammar and structure (Chapelle, Cotos, & Lee, 2015; Link et al., 2014; Ranalli, Link, & Chukharev-Hudilainen, 2017). Feedback generated by AWE systems is instant and specific to individual student submissions, and generally focuses on diagnosing sentence-level errors in language mechanisms. However, AWE tools aimed at providing feedback on discourse characteristics, such as components of an introduction, have also been developed

714

P. Dawson et al.

(Chapelle et al., 2015). Recent research relating to AWE tools has largely emerged from language disciplines, particularly English as a second or foreign language, and indeed marketing of AWE tools has increasingly targeted language disciplines (Bai & Hu, 2017; Ranalli et al., 2017). It is suggested that AWE tools can support educators by providing feedback on sentence mechanics, enabling educators to address higher-level writing components such as content and audience awareness (Ranalli et al., 2017). AWE tools are typically used to aid students in drafting and revising their written work (Chapelle et al., 2015). In particular, AWE tools may be used to assist students in developing a multi-stage writing process, as the automated system means students can receive feedback comments on multiple drafts before submitting their work for final assessment by their educator (Chapelle et al., 2015). AWE tools are usually web-based platforms which offer students flexible access and multiple opportunities to receive feedback on their work (Bai & Hu, 2017; Chapelle et al., 2015). Feedback comments provided by AWE tools can be in a number of forms, including a score, and can highlight errors in a generic formulation (e.g., “You may be using the wrong preposition”) or locate feedback specifically within a student’s work (e.g., “You have used quiet in this sentence. You may need to use quite instead”) (Link et al., 2014; Ranalli et al., 2017). Some AWE tools can also assess and provide numeric indicators for the presence of content such as relevance, vocabulary, and structure (Bai & Hu, 2017). Common AWE systems include Criterion and Pigai (Bai & Hu, 2017; Chapelle et al., 2015). Research relating to AWE tools has primarily sought to establish the accuracy of feedback. AWE tools are typically found to offer acceptable overall levels of feedback accuracy (between 71% and 77%), although there are significant variations between error types, which raise concerns as to their usefulness (Bai & Hu, 2017; Ranalli et al., 2017). In particular, AWE feedback may fail to recognize common second language written errors, significantly undermining claims of AWE’s usefulness in language learning (Ranalli et al., 2017). In addition, students may have difficulty in correctly applying AWE feedback to their work and have been shown to disregard up to 50% of the feedback (Chapelle et al., 2015; Ranalli et al., 2017). However, Bai and Hu (2017) found that student uptake of AWE feedback generally corresponds with the accuracy of AWE corrections, suggesting that students critically evaluate automated feedback and apply it as they consider appropriate. Ranalli et al. (2017) contend that inaccuracies in AWE feedback may damage students’ confidence in AWE tools. While research into student perceptions of AWE tools is limited, students generally consider AWE feedback on sentence mechanics and grammar to be helpful (Bai & Hu, 2017), while AWE feedback on discourse components is largely considered by students to be somewhat or mostly helpful in identifying discrepancies between intended meaning and written output (Chapelle et al., 2015). Educator perceptions of AWE tools are similarly mixed. While educators typically agree that AWE tools promote student autonomy by offering flexible access to feedback, they also consider them to be largely ineffective for providing sufficient levels of high-quality, reliable feedback on student writing (Link et al., 2014). Educators are particularly

28

Technology and Feedback Design

715

concerned that inaccurate AWE feedback can be confusing and even misleading for students (Link et al., 2014); however, they do recognize their utility as an out-ofclass assistant and grammar checker and note that they may help reduce workload in some instances (Link et al., 2014). It has been argued that comprehensive training in AWE systems and features is essential for both educators and students (Chapelle et al., 2015; Link et al., 2014). Learning activities during class are recommended to ensure students can receive assistance if encountering difficulties with the AWE system (Link et al., 2014). In addition, it is recommended that students are advised of potential limitations of AWE-generated feedback and encouraged to critically evaluate all feedback recommendations made by the system (Bai & Hu, 2017; Link et al., 2014). Educators seeking to integrate AWE feedback into their teaching need to also maintain a degree of caution since the degree of accuracy varies depending on the context and complexity of text. Consequently, AWE tools are recommended as a complement to educator or peer feedback, rather than a primary feedback source (Link et al., 2014). While technologies will increasingly improve, the current value of AWE lies in providing students with diagnostic feedback on language mechanics at a basic, sentence level (Ranalli et al., 2017).

Intelligent Tutoring Systems Many educational tasks require direct attention from an educator, from marking written tests to providing one-to-one support to students. However, large class sizes, coupled with increasing time pressures and staffing costs, can limit the ability of educators to provide this personal support (Chu, Yang, Tseng, & Yang, 2014; Hung, Smith, & Smith, 2015). As computer technology develops, intelligent tutoring systems have emerged as a means of providing students with interactive, flexible, and focused personal learning support (Hung et al., 2015; Steif, Fu, & Kara, 2016). Such support is particularly valuable as one-to-one tutoring from an educator has been shown to improve student achievement (Chu et al., 2014). Intelligent tutoring systems may guide students through a learning exercise or seek to diagnose learning difficulties and provide corrective feedback (Chu et al., 2014; Hung et al., 2015). While intelligent tutoring systems may be designed around a number of systems, recent research regarding intelligent tutoring systems has focused on cognitive tutoring (Hung et al., 2015; Steif et al., 2016). Cognitive tutoring mechanisms use a model of cognitive behavior to interpret and evaluate student learning behaviors which take place within the tutoring system, typically centering on a problem-solving exercise (Chu et al., 2014; Hung et al., 2015). However, a significant criticism of cognitive tutoring systems has emerged from disciplines such as engineering and mathematics, which require students to undertake problem-solving tasks (Chu et al., 2014; Steif et al., 2016). While students may take any number of reasoning pathways to arrive at an answer (whether correct or incorrect), cognitive tutoring systems typically restrict students’ reasoning strategies by offering limited methods of solving a problem – for instance, offering

716

P. Dawson et al.

pre-mapped intermediate steps in a calculation (Chu et al., 2014; Steif et al., 2016). However, some recent studies have investigated intelligent tutoring systems that reduce this limitation by derestricting reasoning pathways, to allow students to integrate various strategies and even commit pathway errors (Chu et al., 2014; Steif et al., 2016). The design of cognitive tutoring systems varies significantly between disciplines. Students may complete a series of calculations in a mathematics or engineering context or work through a set of interactive, problem-based scenarios (Chu et al., 2014; Hung et al., 2015; Steif et al., 2016). Cognitive tutoring systems typically provide students with immediate feedback, either when students submit an answer or at a series of preselected points in the system (Chu et al., 2014; Steif et al., 2016). Feedback may take a range of forms, from a diagnosis which highlights the cause of an operational error in a mathematical problem to corrective feedback and suggestions in a dialogic, scenario-based system (Chu et al., 2014; Hung et al., 2015). Cognitive tutoring systems may also prevent students from continuing in a program until errors are corrected (Steif et al., 2016). Students generally enjoy the interactivity of intelligent tutoring systems, which they feel positions them as active participants in their own learning (Hung et al., 2015). Students also consider that cognitive feedback systems provide sufficient feedback to benefit their learning (Hung et al., 2015). Research into the effect of intelligent tutoring systems on student learning outcomes is limited, but initial findings suggest students using intelligent tutoring systems may achieve higher learning outcomes than students undertaking simple web-based quizzes (Chu et al., 2014). It is recommended that students are trained in the use of intelligent tutoring systems before commencing any learning activity (Chu et al., 2014).

Peer to Student Feedback Peer feedback is commonly considered to be beneficial to student learning; receiving feedback from peers allows students opportunities to consider their work from alternate perspectives, while providing feedback to peers can challenge students’ understanding of their own work and develop their critical thinking, improving their self-regulatory skills (Ciftci & Kocoglu, 2012; Ekahitanond, 2013; Wu, Petit, & Chen, 2015). The subsections below present research relating to the use of blogs and discussion boards, collaborative writing software, and specialized peer feedback software.

Blogs and Discussion Boards As online learning has become more common, online text platforms such as blogs and online discussion platforms have become increasingly popular means of facilitating the peer feedback process. Blogs and discussion boards afford a collaborative, interactive, and flexible environment for students to share their work and provide and

28

Technology and Feedback Design

717

receive peer feedback (Ciftci & Kocoglu, 2012; Novakovich, 2016). As they are hosted online, blogs and discussion boards are accessible to students wherever an Internet connection is available, while their asynchronous nature allows students to provide and reflect on peer feedback in their own time (Ekahitanond, 2013; Lee & Markey, 2014; Yoo, 2016). Such social media can also facilitate the sharing of information, including feedback comments, in the form of multimedia such as images, audio, and videos, while the asynchronous nature of the exchange (comments on the blog or posts in the discussion forums) offer the chance for students to engage in a peer feedback dialogue. Peer feedback via blogs and discussion boards follows a similar process to traditional, in-class peer feedback methods. Students providing peer feedback reflect on and comment on their peers’ work, which may take the form of a blog or discussion board post (Ekahitanond, 2013; Novakovich, 2016). In these activities, students are often assigned partners or a number of peers to ensure all students receive feedback (Lee & Markey, 2014). The feedback process may take place in class, such as during peer workshopping sessions or in students’ own time (Ciftci & Kocoglu, 2012; Novakovich, 2016; Wu et al., 2015). Options for blog hosting include established blogging sites, such as Blogger and Qzone (Lee & Markey, 2014; Xianwei, Samuel, & Asmawi, 2016), while discussion forums are typically hosted on native applications within learning management systems such as Moodle (Ekahitanond, 2013). Students have been reported to consider the receipt of peer feedback through blogs and discussion forums to be enjoyable, motivating, and beneficial to their overall learning (Ciftci & Kocoglu, 2012; Ekahitanond, 2013). With regard to blogmediated feedback in particular, students recognize that providing peer feedback helps to improve their writing skills, critical appraisal skills, and learning outcomes, while receiving peer feedback impacts positively on their own work (Ciftci & Kocoglu, 2012; Yoo, 2016). Blog-mediated peer feedback is also considered by students to be convenient and easy to use (Ciftci & Kocoglu, 2012; Xianwei et al., 2016). Students report similar benefits for peer feedback through discussion forums, including improved confidence and teamwork; however, students may also consider discussion forums to be a time-consuming and impersonal means of providing peer feedback (Ekahitanond, 2013). The effectiveness of blog and discussion forum peer feedback in improving student learning outcomes remains underexplored in the literature. However, findings suggest that students who receive blog-mediated peer feedback often receive higher marks than students who receive peer feedback in person or via mark-up (Ciftci & Kocoglu, 2012; Novakovich, 2016). Studies have also found that blogmediated peer feedback comments compare favorably with traditional in-class or electronic mark-up formats; blog mediated peer feedback tends to be of higher overall quality, with students offering increased substantive, critical, and accurate suggestions (Novakovich, 2016; Yoo, 2016). It has also been suggested that students may also feel more comfortable providing critical comments through a blog or discussion forum than in person (Ekahitanond, 2013; Yoo, 2016). However, some studies have found that while students appreciate feedback from their peers, they can

718

P. Dawson et al.

be reluctant to integrate peer comments when revising their work, instead preferring comments from their educators, perceiving this feedback to be more accurate, credible, and “expert” (Ciftci & Kocoglu, 2012; Wu et al., 2015). Blog-mediated and discussion forum peer feedback can present educators with a number of challenges. As with any digital feedback, blogs and discussion forums may be affected by technical difficulties such as inadequate or unreliable Internet access (Ciftci & Kocoglu, 2012; Ekahitanond, 2013). Students may also have difficulty adapting to blog or discussion forum interfaces (Ciftci & Kocoglu, 2012). It is recommended that educators familiarize students with the peer feedback platform through in-class training and provide detailed guidelines to ensure peer feedback is sensitive, constructive, and useful for all recipients (Ciftci & Kocoglu, 2012; Ekahitanond, 2013). In addition to training, it is suggested the educators consider providing student peer feedback exemplars and examples of their own experiences of peer feedback (Ciftci & Kocoglu, 2012). It has also been recommended that to avoid students disengaging from peer feedback participation, peer feedback should be integrated into the curriculum rather than designated an “optional” task; participation may also be incentivized with a small number of marks (Wu et al., 2015).

Collaborative Writing Software Collaborative writing tasks can be a valuable approach to fostering discussion, reflection, and feedback interactions among students (Boldrini & Cattaneo, 2014). During the process of writing and reviewing, students engage in formative feedback among their peers and consider new perspectives and approaches (Boldrini & Cattaneo, 2014; Strobl, 2014). A number of online technologies have emerged as potential platforms for collaborative writing and peer feedback. Wikis and cloudbased text editors such as Google Docs offer simple, accessible, and flexible interfaces for students to draft, edit, and comment on collaborative writing tasks (Andrichuk, 2016; Strobl, 2014; Woo, Chu, & Li, 2013). Collaborative writing software can also be used to facilitate the peer feedback process on non-collaborative tasks; for instance, students may upload their work for review and receive comments or mark-up (Andrichuk, 2016; Boldrini & Cattaneo, 2014). Student perceptions of collaborative writing software for peer feedback vary. It has been reported that students generally enjoy using collaborative writing platforms for peer feedback and agree that the process helps improve their text as well as their writing skills (Andrichuk, 2016; Strobl, 2014). However, two studies indicated that students gained more from receiving than providing peer feedback: Andrichuk (2016) reported that providing peer review did not enhance students’ writing ability as much as the students themselves expected, while Strobl (2014) found that students were more likely to agree that they learned from receiving peer feedback than from providing it. It is worth noting that this finding is in contrast to the general assessment literature, which states that students tend to benefit more from providing than receiving peer feedback (Ertmer et al., 2007; van Popta, Kral, Camp, Martens, & Simons, 2017). This

28

Technology and Feedback Design

719

difference may reflect the particular type of peer feedback that is being provided using collaborative writing software (i.e., written comments on a written task). Students typically appreciate the flexible access of online collaborative writing tools, which allows them to work on their writing and provide feedback from any Internet-connected location (Woo et al., 2013). It has also been noted that some students can feel more comfortable sharing their work in an online environment (Andrichuk, 2016). However, it has been noted that this accessible nature may also contribute to student perceptions that providing peer feedback via collaborative writing mediums can be burdensome and time-consuming (Strobl, 2014). Staff perceptions of the use of collaborative writing platforms for peer feedback remain largely underexplored; however, one study reported that staff found collaborative writing platforms to be an efficient and convenient means for students to co-construct texts and provide peer feedback (Woo et al., 2013). Research has yet to clearly establish the effect of peer feedback via collaborative writing tools on learning outcomes. While studies have found little difference in peer feedback outcomes between paper-based and collaborative writing software, it has been reported that peer feedback through online collaborative platforms can lead to increased revisions at the content level (Boldrini & Cattaneo, 2014; Woo et al., 2013). In addition, collaborative writing software prompts a higher number of peer comments than traditional paper-based peer feedback (Boldrini & Cattaneo, 2014). Comments are also more likely to be at a meaningful, content level than surface-level corrections (Woo et al., 2013). However, a small but significant number of students remain concerned about the possibility for plagiarism to occur in online collaborative writing and peer feedback processes (Andrichuk, 2016). As with most technology enabled learning, it is recommended that students receive appropriate guidance and scaffolding, including technical instructions, roles, and responsibilities such as avoiding plagiarism, and especially how to provide peer feedback comments (Andrichuk, 2016; Strobl, 2014; Woo et al., 2013).

Peer Feedback Software and Tools The literature search for peer feedback software and tools invariably focused on online software and in particular resulted in two main themes: the re-purposing of existing social networking sites, such as Facebook (including specially designed Facebook add-ons), and purpose built learning platforms designed to support peer feedback (Demirbilek, 2015; Ho, 2015; Jiang & Yu, 2014; McCarthy, 2016). As noted in other online feedback systems, it is considered to be an advantage for students to access peers’ work and comments at a time and place convenient to them, thus avoiding the logistical challenges of traditional, paper-based peer review processes (Demirbilek, 2015; Ho, 2015). In addition, it is argued that social networking sites such as Facebook are familiar to most students and contain features which facilitate the provision of a variety forms of peer feedback input, such as commenting and “liking” posts (Demirbilek, 2015). Many social media and purpose built peer feedback tools also allow multimedia, such as images and video, to be

720

P. Dawson et al.

posted, providing a scope for online peer feedback beyond text (Demirbilek, 2015; McCarthy, 2016). This visual element makes online peer feedback tools particularly suited to creative disciplines such as art and design; not only do students produce visual works, but creative work is subjective and typically shaped by multiple perspectives (McCarthy, 2016). It has been reported in several studies that Facebook peer feedback tools make use of students’ existing accounts and capitalize on Facebook’s accessibility on a broad range of Internet-connected devices, including computers and mobile phones (Demirbilek, 2015; McCarthy, 2016). However, online peer feedback interfaces have the potential advantage of a purposefully designed range of functions, such as split-screens simultaneously showing the work, comments, and an instant chat to allow synchronous peer feedback (Ho, 2015; Jiang & Yu, 2014). Regardless of the interface, it is argued that these online interfaces encourage more of a dialogue, where drafts of student work (whether textual or visual) are uploaded, peers provide comments on the work, and students review and respond to feedback on their work (Ho, 2015; McCarthy, 2016). It has been reported that students generally feel that they benefit from receiving peer feedback via these tools and prefer online peer feedback to handwritten, paperbased peer feedback (Demirbilek, 2015; Ho, 2015; McCarthy, 2016). Indeed, it has been noted that students felt that it is easier and more efficient to type comments, rather than handwriting them in a document’s margins (Ho, 2015). Students responded positively to the accessibility and familiar appearance of Facebook peer feedback tools which also made it easy to use (Demirbilek, 2015; McCarthy, 2016). Facebook peer feedback tools has also been shown to facilitate increased social connectivity between students, particularly in out-of-class contexts, and students report enjoying the opportunity to view and comment on their peers’ work (Demirbilek, 2015; McCarthy, 2016). Peer feedback generated in purpose-built online platforms is generally oriented to revision, rather than surface-level comments or generalized praise that are frequently found in face-to-face contexts; however, students are more likely to incorporate peer feedback received in a face-to-face context than online peer feedback (Ho, 2015). Nevertheless, online peer feedback tools have been found to have positive effects on students’ learning outcomes, with low-performing students typically making greater improvements than higher-performing peers (Jiang & Yu, 2014). It is also interesting to note that strong correlations have been found between high levels of online activity and strong academic performance (Demirbilek, 2015; McCarthy, 2016). Online peer feedback platforms also come with a number of challenges for educators, many of which are also relevant to offline methods. Of significant concern is that high-performing students may not accept that lower-performing peers are able to provide useful or accurate feedback (Jiang & Yu, 2014). Students also report anxiety around providing and receiving online peer feedback, particularly when providing critical comments, and it is suggested that offering students anonymity through the use of pseudonyms may alleviate these concerns (Demirbilek, 2015). Training students in using online peer feedback interfaces, and also in providing peer feedback, is recommended as essential to ensuring the success of online peer

28

Technology and Feedback Design

721

feedback for all students; exemplars and practice tasks may also be beneficial to increase students’ understanding of the peer feedback process (Demirbilek, 2015; Ho, 2015). McCarthy (2015) also suggests that educators consider providing appropriate assessment weighting for participation in online peer feedback, to ensure that students receive the benefits which attend strong, consistent participation in the online environment.

Self-Feedback As S.-C. Huang (2016) observes, the distinction between self-feedback and selfassessment is often blurred and difficult to distinguish. Self-assessment is recognized as an important means of developing students’ learning skills and self-regulatory abilities (Boud, 1995; Huang, 2016), and as Hattie and Timperley (2007) argue, the act of questioning and judging oneself necessarily entails “selecting and interpreting information in ways that provide feedback” (p. 94). Thus, self-assessment and selffeedback function as linked and interdependent and are often categorized under the single banner of self-assessment. Initial literature searches revealed that digital recordings and e-portfolios are the most commonly researched forms of technology used to facilitate students’ selffeedback. The subsections below aim to discuss these technological approaches to self-feedback, but at times reflect the indistinct and messy characterizations of selffeedback and self-assessment.

Digital Recordings While self-assessment is recognized as an important component of students’ development as lifelong learners (Hawkins et al., 2012), it has also been shown that selfperceptions can be inaccurate when compared with expert or educator assessment (Hawkins et al., 2012; LeFebvre, LeFebvre, Blackburn, & Boyd, 2015). However, digital recordings have emerged as a means of facilitating and improving students’ self-assessment, a trend that has been facilitated by audio and video recording technologies becoming simpler, cheaper, and more readily available in educational contexts (O’Loughlin, Ní Chróinín, & O’Grady, 2013). Digital recordings allow students to review and critically assess a recording of their own performance (LeFebvre et al., 2015), an affordance of particular value in disciplines which require the development of practical skills, and for transitory assessments such as oral presentations (Barry, 2012; O’Loughlin et al., 2013). Video and audio recordings are the most common digital recording formats and may be used in a range of circumstances. Video is prevalent in many practice-based disciplines, including medicine and physical education, while audio is used in disciplines where visual components of performance are less critical, such as language studies (Barry, 2012; Huang, Chen, Wu, & Chen, 2015; O’Loughlin et al., 2013). Recordings can be hosted via an online platform such as a learning

722

P. Dawson et al.

management system, media sharing site, or simply viewed on the recording device itself. Using digital recordings in the self-assessment process is an opportunity for students to identify discrepancies between their perceived and actual performance (LeFebvre et al., 2015). An iterative assessment design is often implemented, whereby a student is recorded undertaking a task, following which they complete a self-assessment; the recording is then reviewed, and a revised self-assessment takes place (Hawkins et al., 2012; Plant, Corden, Mourad, O’Brien, & van Schaik, 2013). The initial self-assessment stage prior to reviewing the recording may also be omitted (Barry, 2012; O’Loughlin et al., 2013), while semi-structured interviews, during which a student’s recording is viewed and discussed, offer an alternative format to written self-assessments (Plant et al., 2013). Explicitly encouraging students to undertake self-feedback is most common in language and communications disciplines; self-feedback may be directed through prompts such as open- and closed-ended questions, detailed instructions, and asking students to write reflexively on their own recorded performance (Huang, 2016; LeFebvre et al., 2015). Digital recordings are often proposed as a means of improving the accuracy of students’ self-assessments; however, the degree of effectiveness of digital recordings in reducing inaccurate self-assessments remains unclear. While Kachingwe, Phillips, and Beling (2015) and Wittler, Hartman, Manthey, Hiestand, and Askew (2016) found limited improvements to accuracy following the introduction of video recordings for review, Hawkins et al. (2012) reported a significant improvement in selfassessment accuracy when video recordings were introduced in concert with a videorecorded exemplar performance. Indeed, it has been reported in a number of studies that the incorporation of video into the self-assessment process was more likely to improve student accuracy when appropriately scaffolded, whether through exemplars, detailed rubrics, or prompts (Barry, 2012; Hawkins et al., 2012; O’Loughlin et al., 2013; Yoo, 2016). S.-C. Huang (2016) also found that through the careful guiding of students to produce self-feedback also resulted in instances of Hattie and Timperley’s (2007) conceptions of feedback and feedforward, at both task and process levels, along with increased reflections on self-regulation (Huang, 2016). Overall, it has been reported that students generally consider the use of digital recordings to be beneficial for improving both their performance and self-assessment skills (Barry, 2012; Hawkins et al., 2012; O’Loughlin et al., 2013). In particular, language students reviewing audio recordings of themselves speaking valued the opportunity to detect discrepancies between their perceived performance and actual performance, including in pronunciation and fluency (Huang, 2016). While self-assessment is necessarily self-driven, student engagement can be problematic. For example, students may provide vague or generic self-feedback rather than invest the time and effort required to make the process beneficial, regardless of the use of technology (Huang, 2016). Students may not have the insight to compare their own performance to a standard, they may not have the language to express views on their own performance, or may be reticent to reveal deficits in their own performance to an assessor (Boud and Molloy 2013). Students may also be unable to effectively use digital recordings to self-assess if it is not clear

28

Technology and Feedback Design

723

to them how to judge their own performance and against what standard (Hawkins et al., 2012). LeFebvre et al. (2015) therefore recommend the use of exemplars to enable students to recognize effective (or ineffective) practices when reviewing their own recordings. Clear educator guidance, along with structured rubrics, is also recommended to support students in developing their own self-assessment capacity (O’Loughlin et al., 2013).

e-Portfolios Learning portfolios, in this context, are digital repositories of student’s work, including learning products, assessments, and feedback. A key feature of a digital portfolio is the ability for the artifacts within it to be organized, curated, annotated, and portrayed in different ways for different purposes and audiences (see Clarke & Boud, 2016). The literature review reflected this diverse application with frequent examples of digital learning portfolios being used to provide students valuable opportunities to review, reflect on, and curate their own learning and may even be consulted as a reference at a later date (Aguaded Gómez, López Meneses, & Jaén Martínez, 2013; Kabilan & Khan, 2012). It is recognized that learning portfolios can assist students in developing skills in self-regulation and self-assessment; however, traditional, paper-based portfolios have been criticized as impractical and difficult to manage, submit, and assess (Beckers, Dolmans, & van Merriënboer, 2016; Chang, Liang, & Chen, 2013). Increasingly, across the disciplines, educators have turned to online solutions to allow the creation, management, and sharing of students’ learning portfolios, known as “e-portfolios.” e-Portfolios resemble their paper-based counterparts, but their digital format offers a number of advantages over traditional portfolios. e-Portfolios enable students to collate and manage their portfolios over time, stored in a central location that is generally accessible from any Internet-connected device (Beckers et al., 2016; Chang et al., 2013). As the e-portfolio is hosted by a digital platform a range of multimedia and file formats can be accommodated, including images and video (Kabilan & Khan, 2012). e-Portfolios are also more easily shared with peers and educators than their paper equivalents; for instance, students may share their portfolios by e-mail or display them on websites or social media (Kabilan & Khan, 2012). Like traditional portfolios, e-portfolios facilitate self-assessment through the act of curation. Students must review their progress when collating their e-portfolio and reflect on their own work and performance when considering the e-portfolio’s contents (Aguaded Gómez et al., 2013; Chang et al., 2013). Options for e-portfolio management vary and include blogs, online discussion platforms such as Google Groups, and specialized online portfolio assessment systems (Aguaded Gómez et al., 2013; Chang et al., 2013; Kabilan & Khan, 2012). Along with promoting self-assessment, e-portfolios also allow students to share their reflections and progress their peers, and interact with one another’s e-portfolios; this is identified in the research as a significant advantage of the e-portfolio format (Aguaded Gómez et al., 2013; Chang et al., 2013; Kabilan &

724

P. Dawson et al.

Khan, 2012). Educators using e-portfolios typically provide students with detailed instructions on the construction of the e-portfolio but also offer questions that can facilitate reflection and self-assessment. This guidance may be a brief list of discussion points or a 27 point list of questions (Chang et al., 2013; Kabilan & Khan, 2012). Aguaded Gómez et al. (2013) also recommend providing training for students if they may be unfamiliar with the online platform or software used for the e-portfolios. It has been reported that self-assessment used in concert with e-portfolios yields highly consistent results between student self-assessment and educator assessment; furthermore, students’ self-assessment results were also accurately reflected by endof-year exam results (Chang et al., 2013). However, researchers emphasize that selfassessment through e-portfolios is a skill rather than an automatic process for students, and as such must be fostered (Kabilan & Khan, 2012). C.-C. Chang et al. (2013) suggest that ensuring students have a clear understanding of portfolio assessment improves the reliability and validity of e-portfolios. Creating and maintaining e-portfolios is typically concluded to positively promote self-assessment and selfregulation in students (Kabilan & Khan, 2012). Students generally consider e-portfolios to be an effective means of facilitating self-assessment by engaging them in their learning and allowing them to progressively monitor their progress and identify areas for improvement (Aguaded Gómez et al., 2013; Kabilan & Khan, 2012). However, some students found maintaining their e-portfolios to be too time-consuming and disliked the higher level of autonomy or independence required to produce the e-portfolio (Aguaded Gómez et al., 2013; Kabilan & Khan, 2012). As with self-assessment facilitated by digital recordings, concerns have been noted about students reluctant to engage in the e-portfolio process, marked by passivity or generalized responses. Time pressures are also cited as a contributing factor in low-quality self-assessment reflections (Kabilan & Khan, 2012). Technical problems and difficulties in Internet access were also noted as possible barriers to students engaging in the e-portfolio process (Aguaded Gómez et al., 2013; Kabilan & Khan, 2012).

Benefits, Challenges, and Implications Overall, the literature review has revealed that there are various benefits, challenges, and design implications that may shape educators’ decisions about the technology practices that they choose to incorporate into their designs. Table 3 summarizes key findings, organized according to the source of feedback. However, there are also several general observations that apply across sources. First, technology enabled feedback is largely reported to have positive impacts on student perceptions and outcomes and are generally thought to be more engaging. However, these results need to be balanced by the fact that many of the studies were single intervention, often small in scale, and focused on a limited array of outcomes. This caveat is further discussed later in this chapter. Second, successful designs are often linked to technologies that are user-friendly, easy to access, and well supported. This reflects

28

Technology and Feedback Design

725

Table 3 Benefits, challenges, and design implications for the use of technology in feedback design

Source Educatorto-student

Forms of technology mediation Digital recordings: Audio recordings Video recordings Screencasts

Digital text: Annotations/ tracked changes Sticky notes/ comment boxes Discussion boards Email Electronic rubrics Statement banks

Benefits Detailed, clear, and personalized comments Contains rich cues, such as tone and expression Efficient to create Can enhance relationships between educator and student May increase student engagement and performance

Challenges Recordings can be difficult to scan quickly Large file sizes are difficult to share and view Some students may be initially sceptical

Simple and convenient for educators to use Allows for specific comments corresponding to sections of work Students are comfortable with this medium More legible, accessible, and timely than handwritten comments

Providing detailed and personalized comments is not as efficient as audio-visual recordings Electronic rubrics and statement banks are not as detailed as other forms of feedback Learners may equate generic

Design implications Useful for providing post assessment performance information Small files can be embedded directly in to the assessment task Ensure that file sizes are manageable Ensure that file formats are widely compatible Good medium for providing holistic performance information Consider issues of privacy and security Rich cues afforded by this medium necessitate thought with regard to the tone of content delivery Best for comments provided by educators Comments can be linked to the specific parts of the assessment task Limited suitability for performance or skill-based assessments Electronic rubrics and (continued)

726

P. Dawson et al.

Table 3 (continued)

Source

Forms of technology mediation

Benefits

Challenges statements (e.g., statement banks) with lack of educator investment

Collaborative writing tools: Wikis Google Docs

Bug in ear technology: Two-way radio transmitter Bluetooth earpiece and videoconferencing software

Computerto-student

Computer assisted language learning Web-hosted software

User friendly and flexible Offers various privacy levels Educators can view history of modifications and contributions Supports formative feedback during and after assessment Allows for realtime performance information and modification Helps students reflect and improve Avoids public loss of face/ humiliation as the audience (for example clients in industry or pupils in class are not privy to the real time comments Cost effective Offers a range of feedback on content, structure, and grammar Convenient and flexible Students can use as many times as

Monitoring student progress through drafts may take extra time and labor

Laborious process for educators High risk of technological issues May not be user friendly

Reliant on Internet connections and recording technology

Design implications statement banks are suitable for problem-based assessments Best used out of class Useful for authentic assessment Fosters dialogical feedback processes Purposeful checkpoint design is recommended Can be used in class or out of class Suitable for performance and skill-based assessments Observation can be on-site or off-site

Limited to language learning subjects Can be useful to limit number of possible responses for complex tasks Best used to (continued)

28

Technology and Feedback Design

727

Table 3 (continued)

Source

Forms of technology mediation

Benefits

Challenges

necessary May reduce students’ performance anxiety Easy to use

Student response systems Hand held “clickers” Web-based systems (SMS, smartphone/laptop voting, Twitter)

Instantaneous dual feedback to student and educator Allow educators to adjust teaching based on results Has low cost options Easy to use Enhances student engagement

Negligible impact on learning outcomes SMS voting can be costly for students

Automated feedback on MCQs Moodle Blackboard

Convenient delivery Simple for educators to use Flexibility of feedback types Students can test knowledge repeatedly at many points across course

Students may share answers with others if feedback design is not performed carefully

Automated writing evaluation tools Criterion Pigai

Enables instant and specific feedback May enable educators to focus their

Lack of accuracy in feedback can lower student engagement with their use Students may not

Design implications supplement educator feedback Students may need training to get the most out of the activities and feedback Correct answers do not necessarily reflect understanding Web-based SRS necessitates that students bring a digital device to class Is best used for content that has a clear answer (e.g., problembased learning) Only useful for in-class context Useful for summative assessment revision Most suitable for content in which there is a clear answer (e.g., problem-based learning) Questions may be constructed around variables to allow repeated attempts without risk of student sharing answers Recommended for language and writing-based subjects Useful for drafting and (continued)

728

P. Dawson et al.

Table 3 (continued)

Source

Forms of technology mediation

Intelligent tutors Cognitive tutoring mechanisms

Peer-tostudent

Blogs, discussion boards Blogger Qzone Native applications in VLEs

Collaborative writing software: Wikis Google Docs

Peer feedback software and tools: Facebook Other social media

Benefits

Challenges

feedback on higher-level components, such as content

apply feedback if they are not able to interrogate and understand it

Interactive and flexible feedback Corrective feedback Provides students with immediate performance information Collaborative feedback Flexible Interactive Web-based Foster high quality peer interactions Dialogic feedback Peer feedback Engaging for students Foster collective knowledge building for students

Only offers limited methods of problem solving, which can restrict students’ reasoning strategies

Convenient User friendly Supports inclusion of multimedia Use on multiple devices

Students less likely to incorporate peer comments online than face to face context

Discussion boards can be impersonal Require Internet access

Risk of plagiarism when used for peer feedback on drafts

Design implications revising written work Students and educators require training before use Best used as a complement to educator feedback Useful for out of class feedback Most suitable for problem-based disciplines Students need training before use

Can be used for educator or peer feedback Commonly used for languagebased disciplines Useful to assign partners when using in peerbase scenarios Provide in-class training and exemplars Purposeful checkpoint design is recommended Can be used in class or out of class Useful for creative disciplines Best to match students of similar abilities when using for (continued)

28

Technology and Feedback Design

729

Table 3 (continued)

Source

Forms of technology mediation

Benefits

Challenges

Supports recursive drafting Facilitates social connectivity between students

Selffeedback

Digital recordings Audio recordings Video recordings

Can support iterative feedback processes Enables almost instantaneous reflection on performance

It can be difficult engender student engagement and depth of reflection

e-Portfolios Blogs Google Groups Specialized systems

Assist students to develop skills in self-regulation and assessment Students can collate overtime Easy to store Incorporate a range of file formats Easy to share with educators

Students may consider them to be timeconsuming Lack of student engagement

Design implications peer feedback due to credibility judgments Students need training in peer feedback, including exemplars Most appropriate for performance based or language related disciplines Depth of reflection can be enhanced through exemplars Educators should guide students through the process of selfevaluation Consider issues of privacy and security Useful for creative disciplines Students require training to maximize efficiency and effectiveness

recent research in assessment design and in technology enabled learning more broadly (Bennett, Dawson, Bearman, Boud, & Molloy, 2017; Henderson, Selwyn, & Aston, 2015). Third, effective technology enabled feedback practices often fit within well-established traditions of feedback design (e.g., peer feedback software applied to contexts in which educational designs already use peer feedback). In addition to the above benefits, challenges, and design implications that have been identified in the review, there are three important observations regarding the

730

P. Dawson et al.

silences within the literature. First, the literature has revealed a haphazard approach to being explicit about the particular conceptualization of feedback being adopted (e.g., Mark 0, 1 or 2). Implied within many of the papers is the assumption that feedback is simply something that is done to students after an assessment submission. In most cases, there is no clear indication of how the feedback inputs (e.g., comments on the assessment performance) are designed to impact on subsequent assessment or how the impact is to be measured. This calls into question the overall validity and comparability of many studies into technology and feedback; without knowing if a technology was used within a high-quality feedback design or not, it is difficult to conclude if the benefits of an approach are actually related to the technology. In addition, the composition or nature of the comments is sometimes less than clear in the feedback designs. Arguably, the impact of the feedback process is heavily dependent on the nature of the information being provided such as a focus on providing actionable comments and the clarification and use of clear performance standards. These details are frequently unclear despite being critical to the design. In adopting any of the designs identified in this chapter, it is highly recommended to first identify the purpose of feedback, which will in turn help identify what information needs to be conveyed, by whom, for what purpose, and what effects should be monitored. This “output” component, which is so often ignored in the feedback literature, necessitates that the learner will have an opportunity to undertake a subsequent task that shares some properties to the immediate task performed. Second, the research in this review was often focused on the intervention or tool, measuring immediate effects such as student satisfaction or use, without also building into the data collection process a focus on the broader implications or context including the social, cultural, pedagogical, and instructional milieu. Such a limited focus may help explain the invariably positive results of technology enabled feedback reported in the literature. However, it is worth noting that this limited focus is a recognized perennial concern of the broader field of educational technology research. In contrast, it is argued that a more nuanced approach to educational design and research recognizes that the selection of a technology, or an educational design, does not guarantee results from one context to another. Instead, technology mediated feedback designs are dependent on a range of conditions, including variations across student cohorts, disciplinary cultures, and, importantly, the careful orchestration by those involved. Although most papers have not set out to engage in this kind of analysis, it is telling that most include concluding statements, suggesting that educators need to: • Support students and staff in their technical skills • Guide staff on how feedback can be best produced or engaged with • Increase motivation and engagement (often with assumptions that this can be done via awarding marks for student participation) • Be cautious of technology failure, costs, access, and Internet dependency All of these conclusions are implicit acknowledgments of the fact that educational technologies are just one component in a complex and interdependent system.

28

Technology and Feedback Design

731

Moreover, the implementation of a technology practice causes ripples within that system. For example, an educator may choose to use a Student Response System as a way of increasing the frequency of in class feedback loops, however, in doing so new issues arise such as technical proficiencies of both students and teachers, but also the impact on the rhythm and sequencing of classroom activity, and the need for preparation time as well as deeper understanding of how to create effective and useful in-class questions. Third, the papers more often than not report on isolated single interventions, that we argue could be more usefully framed within a design approach in which the design need, iterative development, and measures of success were explicated from the outset. The technology enabled feedback practices reviewed in this chapter are potentially valuable approaches for educators and educational designers to investigate and iteratively build upon. We argue that iterative design is a useful perspective to adopt. Inherent in the concept of design is that it is a response to a human need and that it needs to be iteratively improved through a variety of feedback loops. As a consequence, there needs to be a clear idea of the criteria or measures of success which can guide focused iterative design improvements.

The Future of Feedback and Digital Technology This chapter has shown some recent advances in feedback and technology. In this somewhat speculative section, the authors conclude by considering what the future holds for feedback and technology. The feedback approaches discussed in this chapter have largely been micro-level: they have focused on individual feedback interactions around a single student performance. Comparatively, less research focused on technology enabling highlevel feedback designs. Over the next few years we anticipate a focus on technology that enables feedback designs at the module or program level. In addition to feedback about student performance against standards, this may also include ipsative feedback (Hughes, 2011), which is feedback based on students’ previous performance. One approach to enable longer-term feedback designs could be adapting portfolio tools so they become repositories of not just student work, but also the feedback information related to that work. In such a portfolio, whenever feedback is provided to a student on their work by a teacher, peer, or even themselves, it would be stored in the portfolio. Then when students undertake a task that is similar, perhaps because it addresses similar learning outcomes or because it is a similar genre of task, relevant feedback would be re-presented back to the learner when they commence the new task. Such a design would assist in closing feedback loops that may have otherwise been left open, particularly when feedback is given on major summative tasks at the end of a course unit without immediate action required of the learner. Storing feedback within a portfolio in this way would require appropriate metadata, which would include, at a minimum, the particular learning outcomes the feedback addresses.

732

P. Dawson et al.

When educators know their students, they are able to give different types of feedback, such as the ipsative feedback described earlier. However, when educators know the students they are marking they tend to give biased grades (Malouff & Thorsteinsson, 2016). This has historically led to an either-or decision: blind marking for more accurate marks or non-blind marking for better feedback. However, marking and feedback need not be considered as the same process. It would be relatively simple to implement a system that split the marking process and the feedback process, such that marking could be done anonymously to reduce bias, and then once marks were determined the student’s identity could be revealed and comments made with the knowledge of who the student is. This would provide the best of both worlds: robust anonymous marking and feedback from somebody who knows who you are, where you have come from, and the sort of information that helps (or does not help) in the production of your work. With the growth in technology tools for feedback, it is likely in the coming years that feedback may become less staged and more continuous: rather than completing a piece of work and waiting days or weeks for feedback comments, feedback will be a continuous real-time part of undertaking the work. Just as automated writing evaluation tools allow real-time feedback on writing tasks, other types of work may become targets for real-time feedback tools. These may be incorporated into sophisticated feedback designs such that students have access to staged feedback from experts, which tends to be expensive, as well as cheap feedback from technology tools whenever the student desires it. Meta-analyses of feedback suggest that feedback which is focused on selfregulation has the greatest effect on student learning (Hattie & Timperley, 2007). A challenge historically with providing this sort of feedback is that self-regulation is much more difficult to observe than student task performance. However, recent developments within the field of learning analytics have focused on observing self-regulation (Gasevic, Kovanovic, Joksimovic, & Siemens, 2014) and on providing feedback about self-regulation. The relationship between the fields of learning analytics and feedback are yet to be fully established; however, it could be possible that in years to come student-facing analytics dashboards are commonly used in feedback designs. In addition to automatic feedback from technology systems, future feedback approaches are likely to include semi-intelligent recommender and aggregation technologies that may connect students with people or systems that can support their ability to judge performance and discover strategies for improvement. For instance, work is currently under way for the development of instant messaging systems that will divert student feedback requests to peers within a class who are deemed likely to be able to provide the correct and most useful comments based on profiles built from online performance data. However, such recommendations need not be limited to their immediate peers and class educators. There are a range of potential human feedback sources beyond the education context that can be engaged through online communities, review sites, collaborative projects, and social media. As an example, when students contribute to Wikipedia as part of their studies they can engage in a structured feedback conversation as they edit a page with other

28

Technology and Feedback Design

733

Wikipedia editors (Di Lauro & Johinke, 2017). However, these feedback conversations are currently dispersed across the web; future technological approaches may seek to aggregate them into the feedback portfolios proposed earlier. Returning to the conceptualization of feedback raised at the start of this chapter, feedback is only feedback where it leads to change. This chapter has demonstrated that emerging tools – from bug in ear technology, to automated writing evaluation systems – are having effects on student learning. However, they remain largely isolated within individual tasks. The next frontiers for feedback with technology involve the marriage of sophisticated feedback technologies with sophisticated longterm feedback designs.

References Aguaded Gómez, J. I., López Meneses, E., & Jaén Martínez, A. (2013). University e-portfolios as a new higher education teaching method. The development of a multimedia educational material (MEM). RUSC, 10(1), 188–209. Alvarez, I., Espasa, A., & Guasch, T. (2012). The value of feedback in improving collaborative writing assignments in an online learning environment. Studies in Higher Education, 37(4), 387–400. https://doi.org/10.1080/03075079.2010.510182. Andrichuk, G. (2016). Perceptions of peer review using cloud-based software. Journal of Educational Multimedia and Hypermedia, 25(2), 109. Anson, I. G. (2015). Assessment feedback using Screencapture Technology in Political Science. Journal of Political Science Education, 11(4), 375–390. https://doi.org/10.1080/ 15512169.2015.1063433. Bai, L., & Hu, G. (2017). In the face of fallible AWE feedback: How do students respond? Educational Psychology, 37(1), 67–81. https://doi.org/10.1080/01443410.2016.1223275. Bälter, O., Enström, E., & Klingenberg, B. (2013). The effect of short formative diagnostic web quizzes with minimal feedback. Computers & Education, 60(1), 234–242. Barry, S. (2012). A video recording and viewing protocol for student group presentations: Assisting self-assessment through a wiki environment. Computers & Education, 59(3), 855–860. Beach, R. (2012). Uses of digital tools and literacies in the English language arts classroom. Research in the Schools, 19(1), 45–59. Beckers, J., Dolmans, D., & van Merriënboer, J. (2016). e-pPortfolios enhancing Students’ selfdirected learning: A systematic review of influencing factors. Australasian Journal of Educational Technology, 32(2), 32–46. Bennett, S., Dawson, P., Bearman, M., Boud, D., & Molloy, E. K. (2017). How technology shapes assessment design: Findings from a study of university teachers. British Journal of Educational Technology, 48, 672–682. https://doi.org/10.1111/bjet.12439. Boldrini, E., & Cattaneo, A. (2014). Scaffolding collaborative reflective writing in a VET curriculum. Vocations and Learning, 7(2), 145–165. Borup, J., West, R. E., & Thomas, R. (2015). The impact of text versus video communication on instructor feedback in blended courses. Educational Technology Research and Development, 63(2), 161–184. Boud, D. (1995). Enhancing learning through self assessment. Philadelphia, PA: Kogan Page. Boud, D., & Molloy, E. K. (2013). Feedback in higher and professional education. Oxon, UK: Routledge. Bourgault, A. M., Mundy, C., & Joshua, T. (2013). Comparison of audio vs. written feedback on clinical assignments of nursing students. Nursing Education Perspectives, 34(1), 43–46. https://doi.org/10.5480/1536-5026-34.1.43.

734

P. Dawson et al.

Carruthers, C., McCarron, B., Bolan, P., Devine, A., McMahon-Beattie, U., & Burns, A. (2015). ‘I like the sound of that’ – An evaluation of providing audio feedback via the virtual learning environment for summative assessment. Assessment & Evaluation in Higher Education, 40(3), 352–370. https://doi.org/10.1080/02602938.2014.917145. Cavanaugh, A. J., & Song, L. (2014). Audio feedback versus written feedback: Instructors’ and Students’ perspectives. Journal of Online Learning and Teaching, 10(1), 122. Chang, C.-C., Liang, C., & Chen, Y.-H. (2013). Is learner self-assessment reliable and valid in a web-based portfolio environment for high school students? Computers & Education, 60(1), 325–334. Chang, N., Watson, A. B., Bakerson, M. A., Williams, E. E., McGoron, F. X., & Spitzer, B. (2012). Electronic feedback or handwritten feedback: What do undergraduate students prefer and why. Journal of Teaching and Learning with Technology, 1(1), 1–23. Chapelle, C. A., Cotos, E., & Lee, J. (2015). Validity arguments for diagnostic assessment using automated writing evaluation. Language Testing, 32(3), 385–405. https://doi.org/10.1177/ 0265532214565386. Chew, E. (2014). “To listen or to read?” audio or written assessment feedback for international students in the UK. On the Horizon, 22(2), 127–135. https://doi.org/10.1108/oth-07-2013-0026. Choi, I.-C. (2016). Efficacy of an ICALL tutoring system and process-oriented corrective feedback. Computer Assisted Language Learning, 29(2), 334–364. Chu, Y.-S., Yang, H.-C., Tseng, S.-S., & Yang, C.-C. (2014). Implementation of a model-tracingbased learning diagnosis system to promote elementary students’ learning in mathematics. Educational Technology & Society, 17(2), 347–357. Chui, L., Martin, K., & Pike, B. (2013). A quasi-experimental assessment of interactive student response systems on student confidence, effort, and course performance. Journal of Accounting Education, 31(1), 17. Ciftci, H., & Kocoglu, Z. (2012). Effects of peer E-feedback on Turkish EFL students’ writing performance. Journal of Educational Computing Research, 46(1), 61–84. Clarke, J. L., & Boud, D. (2016). Refocusing portfolio assessment: Curating for feedback and portrayal. Innovations in Education and Teaching International, 1–8. https://doi.org/10.1080/ 14703297.2016.1250664. Demirbilek, M. (2015). Social media and peer feedback: What do students really think about using wiki and Facebook as platforms for peer feedback? Active Learning in Higher Education, 16(3), 211–224. Denton, D. W. (2014). Using screen capture feedback to improve academic performance. TechTrends, 58(6), 51–56. https://doi.org/10.1007/s11528-014-0803-0. Denton, P., & Rowe, P. (2015). Using statement banks to return online feedback: Limitations of the transmission approach in a credit-bearing assessment. Assessment & Evaluation in Higher Education, 40(8), 1095–1103. https://doi.org/10.1080/02602938.2014.970124. DePaolo, C. A., & Wilkinson, K. (2014). Recurrent online quizzes: Ubiquitous tools for promoting student presence, participation and performance. Interdisciplinary Journal of E-Learning and Learning Objects, 10, 75–91. Di Lauro, F., & Johinke, R. (2017). Employing Wikipedia for good not evil: Innovative approaches to collaborative writing assessment. Assessment & Evaluation in Higher Education, 42(3), 478–491. https://doi.org/10.1080/02602938.2015.1127322. Eddy, P. L., & Lawrence, A. (2012). Wikis as platforms for authentic assessment. Innovative Higher Education, 38(4), 253–265. https://doi.org/10.1007/s10755-012-9239-7. Ekahitanond, V. (2013). Promoting university students’ critical thinking skills through peer feedback activity in an online discussion forum. Alberta Journal of Educational Research, 59(2), 247–265. Elola, I., & Oskoz, A. (2016). Supporting second language writing using multimodal feedback. Foreign Language Annals, 49(1), 58–74. https://doi.org/10.1111/flan.12183.

28

Technology and Feedback Design

735

Ertmer, P. A., Richardson, J. C., Belland, B., Camin, D., Connolly, P., Coulthard, G., . . . Mong, C. (2007). Using peer feedback to enhance the quality of student online postings: An exploratory study. Journal of Computer-Mediated Communication, 12(2), 412–433. Fawcett, H., & Oldfield, J. (2016). Investigating expectations and experiences of audio and written assignment feedback in first-year undergraduate students. Teaching in Higher Education, 21(1), 79–93. Gabaudan, O. (2013). E-xperience erasmus: Online Journaling as a tool to enhance students’ learning experience of their study visit abroad (p. 5): Research-publishing.net. La Grange des Noyes, 25110 Voillans, France. Gasevic, D., Kovanovic, V., Joksimovic, S., & Siemens, G. (2014). Where is research on massive open online courses headed? A data analysis of the MOOC research initiative. The International Review of Research in Open and Distributed Learning, 15(5). https://doi.org/10.19173/irrodl. v15i5.1954. Ghahri, F., Hashamdar, M., & Mohamadi, Z. (2015). Technology: A better teacher in writing skill. Theory and Practice in Language Studies, 5(7), 1495–1500. Gibson, L., & Musti-Rao, S. (2015). Using technology to enhance feedback to student teachers. Intervention in School and Clinic, 51(5), 307–311. https://doi.org/10.1177/1053451215606694. Gould, J., & Day, P. (2013). Hearing you loud and clear: Student perspectives of audio feedback in higher education. Assessment & Evaluation in Higher Education, 38(5), 554–566. https://doi. org/10.1080/02602938.2012.660131. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487. Hawkins, S. C., Osborne, A., Schofield, S. J., Pournaras, D. J., & Chester, J. F. (2012). Improving the accuracy of self-assessment of practical clinical skills using video feedback – The importance of including benchmarks. Medical Teacher, 34(4), 279–284. https://doi.org/10.3109/ 0142159X.2012.658897. Henderson, M., & Phillips, M. (2014). Technology enhanced feedback on assessment. Paper presented at the Australian Computers in Eductional Conference 2013, Adelaide, SA. http:// acec2014.acce.edu.au Henderson, M., & Phillips, M. (2015). Video-based feedback on student assessment: Scarily personal. Australasian Journal of Educational Technology, 31(1), 51–66. https://doi.org/ 10.14742/ajet.1878. Henderson, M., Selwyn, N., & Aston, R. (2015). What works and why? Student perceptions of ‘useful’ digital technology in university teaching and learning. Studies in Higher Education, 42, 1–13. https://doi.org/10.1080/03075079.2015.1007946. Ho, M.-c. (2015). The effects of face-to-face and computer-mediated peer review on EFL writers’ comments and revisions. Australasian Journal of Educational Technology, 31(1), 1–15. Huang, K., Chen, C.-H., Wu, W.-S., & Chen, W.-Y. (2015). Interactivity of question prompts and feedback on secondary students’ science knowledge acquisition and cognitive load. Educational Technology & Society, 18(4), 159–171. Huang, S.-C. (2016). Understanding learners’ self-assessment and self-feedback on their foreign language speaking performance. Assessment & Evaluation in Higher Education, 41(6), 803–820. Hughes, G. (2011). Towards a personal best: A case for introducing ipsative assessment in higher education. Studies in Higher Education, 36(3), 353–367. https://doi.org/10.1080/ 03075079.2010.486859. Hung, S.-T. A. (2016). Enhancing feedback provision through multimodal video technology. Computers & Education, 98, 90–101. Hung, W.-C., Smith, T. J., & Smith, C. M. (2015). Design and usability assessment of a dialoguebased cognitive tutoring system to model expert problem solving in research design. British Journal of Educational Technology, 46(1), 82–97.

736

P. Dawson et al.

Israel, M., & Moshirnia, A. (2012). Interacting and learning together: Factors influencing preservice teachers’ perceptions of academic wiki use. Journal of Technology and Teacher Education, 20(2), 151. Jiang, J., & Yu, Y. (2014). The effectiveness of internet-based peer feedback training on Chinese EFL college students’ writing proficiency. International Journal of Information and Communication Technology Education, 10(3), 34. Johnson, G. M., & Cooke, A. (2016). Self-regulation of learning and preference for written versus audio-recorded feedback by distance education students. Distance Education, 37(1), 107–120. https://doi.org/10.1080/01587919.2015.1081737. Jones, N., Georghiades, P., & Gunson, J. (2012). Student feedback via screen capture digital video: Stimulating student’s modified action. Higher Education, 64(5), 593–607. https://doi.org/ 10.1007/s10734-012-9514-7. Jonsson, A. (2013). Facilitating productive use of feedback in higher education. Active Learning in Higher Education, 14(1), 63–76. Kabilan, M. K., & Khan, M. A. (2012). Assessing pre-service English language teachers’ learning using E-portfolios: Benefits, challenges and competencies gained. Computers & Education, 58(4), 1007–1020. Kachingwe, A. F., Phillips, B., & Beling, J. (2015). Videotaping practical examinations in physical therapist education: Does it Foster student performance, self-assessment, professionalism, and improve instructor grading? Journal of Physical Therapy Education, 29(1), 25–33. Kelly, L., O’Neil, K., & Kwon, E. H. (2014). Comparative analysis: On-site versus remote supervision for APE preservice teachers. Research Quarterly for Exercise and Sport, 85(S1), 140–141. Klein, K., & Kientz, M. (2013). A model for successful use of student response systems. Nursing Education Perspectives, 34(5), 334–338. Knauf, H. (2016). Reading, listening and feeling: Audio feedback as a component of an inclusive learning culture at universities. Assessment & Evaluation in Higher Education, 41(3), 442–449. https://doi.org/10.1080/02602938.2015.1021664. Lee, C., Cheung, W. K. W., Wong, K. C. K., & Lee, F. S. L. (2013). Immediate web-based essay critiquing system feedback and teacher follow-up feedback on young second language learners’ writings: An experimental study in a Hong Kong secondary school. Computer Assisted Language Learning, 26(1), 39–60. https://doi.org/10.1080/09588221.2011.630672. Lee, L., & Markey, A. (2014). A study of learners’ perceptions of online intercultural exchange through web 2.0 technologies. ReCALL, 26(3), 281–297. LeFebvre, L., LeFebvre, L., Blackburn, K., & Boyd, R. (2015). Student estimates of public speaking competency: The meaning extraction helper and video self-evaluation. Communication Education, 64(3), 261–279. Leibold, N., & Schwarz, L. M. (2015). The art of giving online feedback. Journal of Effective Teaching, 15(1), 34–46. Li, J., & De Luca, R. (2014). Review of assessment feedback. Studies in Higher Education, 39(2), 378–393. https://doi.org/10.1080/03075079.2012.709494. Link, S., Dursun, A., Karakaya, K., & Hegelheimer, V. (2014). Towards better ESL practices for implementing automated writing evaluation. CALICO Journal, 31(3), 323. Malouff, J. M., & Thorsteinsson, E. B. (2016). Bias in grading: A meta-analysis of experimental research findings. Australian Journal of Education, 60(3), 245–256. https://doi.org/10.1177/ 0004944116664618. Marden, N. Y., Ulman, L. G., Wilson, F. S., & Velan, G. M. (2013). Online feedback assessments in physiology: Effects on students’ learning experiences and outcomes. Advances in Physiology Education, 37(2), 192–200. https://doi.org/10.1152/advan.00092.2012. Mathieson, K. (2012). Exploring student perceptions of audiovisual feedback via screencasting in online courses. American Journal of Distance Education, 26(3), 143–156. https://doi.org/ 10.1080/08923647.2012.689166.

28

Technology and Feedback Design

737

Mauri, T., Ginesta, A., & Rochera, M.-J. (2014). The use of feedback systems to improve collaborative text writing: A proposal for the higher education context. Innovations in Education and Teaching International, 53(4), 411–423. https://doi.org/10.1080/14703297.2014.961503. McCarthy, J. (2015). Evaluating written, audio and video feedback in higher education summative assessment tasks. Issues in Educational Research, 25(2), 153–169. McCarthy, J. (2016). Global learning partnerships in the Café: Peer feedback as a formative assessment tool for animation students. Interactive Learning Environments, 24(6), 1298–1318. https://doi.org/10.1080/10494820.2014.994532. Moore, C., & Wallace, I. P. H. (2012). Personalizing feedback for feed-forward opportunities utilizing audio feedback technologies for online students. International Journal of e-Education, e-Business, e-Management and e-Learning, 2(1), 6. https://doi.org/10.7763/ IJEEEE.2012.V2.72. Morris, C., & Chikwa, G. (2016). Audio versus written feedback: Exploring learners’ preference and the impact of feedback format on students’ academic performance. Active Learning in Higher Education, 17, 125–137. Munro, W., & Hollingworth, L. (2014). Audio feedback to physiotherapy students for viva voce: How effective is ‘the living voice’? Assessment & Evaluation in Higher Education, 39(7), 865–878. https://doi.org/10.1080/02602938.2013.873387. Novakovich, J. (2016). Fostering critical thinking and reflection through blog-mediated peer feedback. Journal of Computer Assisted Learning, 32(1), 16–30. O’Loughlin, J., Ní Chróinín, D., & O’Grady, D. (2013). Digital video: The impact on Children’s learning experiences in primary physical education. European Physical Education Review, 19(2), 165–182. Orlando, J. (2016). A comparison of text, voice, and screencasting feedback to online students. American Journal of Distance Education, 30(3), 156–166. https://doi.org/10.1080/ 08923647.2016.1187472. Parkin, H. J., Hepplestone, S., Holden, G., Irwin, B., & Thorpe, L. (2012). A role for technology in enhancing students’ engagement with feedback. Assessment & Evaluation in Higher Education, 37(8), 963–973. https://doi.org/10.1080/02602938.2011.592934. Penning de Vries, B., Cucchiarini, C., Bodnar, S., Strik, H., & van Hout, R. (2015). Spoken grammar practice and feedback in an ASR-based CALL system. Computer Assisted Language Learning, 28(6), 550–576. Plant, J. L., Corden, M., Mourad, M., O’Brien, B. C., & van Schaik, S. M. (2013). Understanding self-assessment as an informed process: Residents’ use of external information for selfassessment of performance in simulated resuscitations. Advances in Health Sciences Education, 18(2), 181–192. Portolese Dias, L., & Trumpy, R. (2014). Online Instructor’s use of audio feedback to increase social presence and student satisfaction. Journal of Educators Online, 11(2), 19. Ranalli, J., Link, S., & Chukharev-Hudilainen, E. (2017). Automated writing evaluation for formative assessment of second language writing: Investigating the accuracy and usefulness of feedback as part of argument-based validation. Educational Psychology, 37(1), 8–25. https:// doi.org/10.1080/01443410.2015.1136407. Rock, M., Gregg, M., Gable, R., Zigmond, N., Blanks, B., Howard, P., & Bullock, L. (2012). Time after time online: An extended study of virtual coaching during distant clinical practice. Journal of Technology and Teacher Education, 20(3), 277. Rott, S., & Weber, E. D. (2013). Preparing students to use wiki software as a collaborative learning tool. CALICO Journal, 30(2), 179–203. Sancho-Vinuesa, T., Escudero-Viladoms, N., & Masià, R. (2013). Continuous activity with immediate feedback: A good strategy to guarantee student engagement with the course. Open Learning, 28(1), 51–66. Sancho-Vinuesa, T., & Viladoms, N. E. (2012). A proposal for formative assessment with automatic feedback on an online mathematics subject. RUSC, 9(2), 240–259.

738

P. Dawson et al.

Sopina, E., & McNeill, R. (2015). Investigating the relationship between quality, format and delivery of feedback for written assignments in higher education. Assessment & Evaluation in Higher Education, 40(5), 666–680. https://doi.org/10.1080/02602938.2014.945072. Steif, P. S., Fu, L., & Kara, L. B. (2016). Providing formative assessment to students solving multipath engineering problems with complex arrangements of interacting parts: An intelligent tutor approach. Interactive Learning Environments, 24(8), 1864–1880. https://doi.org/10.1080/ 10494820.2015.1057745. Strobl, C. (2014). Affordances of web 2.0 Technologies for Collaborative Advanced Writing in a foreign language. CALICO Journal, 31(1), 1–18. Turner, W., & West, J. (2013). Assessment for “digital first language” speakers: Online video assessment and feedback in higher education. International Journal of Teaching and Learning in Higher Education, 25(3), 288–296. van Popta, E., Kral, M., Camp, G., Martens, R. L., & Simons, P. R.-J. (2017). Exploring the value of peer feedback in online learning for the provider. Educational Research Review, 20, 24–34. https://doi.org/10.1016/j.edurev.2016.10.003. Voelkel, S., & Bennett, D. (2014). New uses for a familiar technology: Introducing mobile phone polling in large classes. Innovations in Education and Teaching International, 51(1), 46. Wang, Y. H., & Young, S. C. S. (2015). Effectiveness of feedback for enhancing English pronunciation in an ASR-based CALL system. Journal of Computer Assisted Learning, 31(6), 493–504. Watkins, D., Dummer, P., Hawthorne, K., Cousins, J., Emmett, C., & Johnson, M. (2014). Healthcare students’ perceptions of electronic feedback through GradeMark ®. Journal of Information Technology Education: Research, 13, 27–47. West, J., & Turner, W. (2016). Enhancing the assessment experience: Improving student perceptions, engagement and understanding using online video feedback. Innovations in Education and Teaching International, 53(4), 400–410. Wittler, M., Hartman, N., Manthey, D., Hiestand, B., & Askew, K. (2016). Video-augmented feedback for procedural performance. Medical Teacher, 38(6), 607. Woo, M. M., Chu, S. K. W., & Li, X. (2013). Peer-feedback and revision process in a wiki mediated collaborative writing. Educational Technology Research and Development, 61(2), 279–309. Wu, W.-C. V., Petit, E., & Chen, C.-H. (2015). EFL writing revision with blind expert and peer review using a CMC open forum. Computer Assisted Language Learning, 28(1), 58–80. Xianwei, G., Samuel, M., & Asmawi, A. (2016). Qzone weblog for critical peer feedback to improve business English writing: A case of Chinese undergraduates. Turkish Online Journal of Educational Technology – TOJET, 15(3), 131–140. Yoo, H. (2016). A web-based environment for facilitating reflective self assessment of choral conducting students. Contributions to Music Education, 41, 113–130. Zheng, B., Lawrence, J., Warschauer, M., & Lin, C.-H. (2014). Middle school students’ writing and feedback in a cloud-based classroom environment. Technology, Knowledge and Learning, 20 (2), 201–229. https://doi.org/10.1007/s10758-014-9239-z.

Phillip Dawson is an Associate Professor and Associate Director of the Centre for Research in Assessment and Digital Learning, Deakin University. His research focuses on assessment design, feedback, educational technology, and academic integrity. Michael Henderson is an Associate Professor and Director of graduate studies in the Faculty of Education, Monash University. His research focuses on feedback design, educational technology, and creativity. Tracii Ryan is a research fellow in the Faculty of Education at Monash University. Her research focuses on assessment feedback, educational technology, cyberpsychology, and individual differences.

28

Technology and Feedback Design

739

Paige Mahoney is an associate research fellow at the Centre for Research in Assessment and Digital Learning, Deakin University. She has worked on a range of research projects examining pedagogical and professional issues in higher education. David Boud is Foundation Director of the Centre for Research in Assessment and Digital Learning at Deakin University, Professor in the Work and Learning Research Centre, Middlesex University, London, and Emeritus Professor at the University of Technology Sydney. His research interests are in assessment for learning and learning in work contexts. Michael Phillips is a Senior Lecturer in the Faculty of Education at Monash University. His research explores the way educators develop knowledge and decision-making capabilities in a variety of contexts. Elizabeth Molloy is a Professor of Work Integrated Learning in the Department of Medical Education, Melbourne Medical School, at the University of Melbourne.

Linking Assessment and Learning Analytics to Support Learning Processes in Higher Education

29

Clara Schumacher

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assessment in Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Functions of Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assessment Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Feedback in Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning Analytics in Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Informative Assessment Using Learning Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Feedback Based on Learning Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developing an Integrative Assessment Analytics Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning Analytics Features for Informative Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications and Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

742 743 743 746 748 750 752 756 759 761 765 768 771 772

Abstract

In higher education assessments are mostly used for summative purposes such as grading and certifying. Albeit, assessments are also considered to support learning processes by offering formative feedback to learners about their current performance and how to improve. Even though such feedback might enhance learners’ self-regulated learning processes, it is used infrequently due to resource constraints. In addition, the competences, skills, and knowledge that should be assessed are evermore complex. To derive valid inferences about learners’ current performance, ongoing assessments across contexts are desirable. With the advancing use of digital learning environments, learning analytics are also coming in for increasing discussion in higher education. However, learning analytics are still not sufficiently linked to learning theory and are lacking empirical C. Schumacher (*) University of Mannheim, Mannheim, Germany e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_166

741

742

C. Schumacher

evidence. Hence, the purpose of this chapter is to propose how theory on assessment and related feedback can be linked to learning analytics with regard to supporting self-regulated learning. Therefore, relevant concepts of assessment, assessment design, and feedback plus current perspectives on learning analytics are introduced. Based on this theoretical foundation, a conceptual integrative framework and potential learning analytics features were derived. The framework and its implications plus further research needs are discussed and concluded. Keywords

Assessment · Learning analytics · Feedback · Higher education · Self-regulated learning

Introduction In higher education, summative assessments are still predominant, and formative assessments for supporting learning are associated with additional workload for the facilitators (Broadbent, Panadero, & Boud, 2017). Nevertheless, assessments (in higher education) are increasingly considered as a means to support learning processes (Cartney, 2010) and can be enhanced by applying educational technologies (Carless, 2017). As Shute and Becker (2010) stated, a shift from collecting numbers to insights into processes of learning and instruction is preferable. Therefore, assessment needs to be an ongoing process of collecting data from different contexts feeding back the inferences for adjusting instruction and learning (▶ Chap. 83, “The Future of Assessment in Technology-Rich Environments: Psychometric Considerations”). However, in order to elicit valid evidence, assessments need to be designed carefully by following a principle-based design approach (Mislevy, Almond, & Lukas, 2003; Shute, Leighton, Jang, & Chu, 2016). In higher education self-regulated learning is the key to successful learning (Cassidy, 2011; Draper, 2009). Self-regulated learning can be defined as “an active, constructive process whereby learners set goals for their learning and then attempt to monitor, regulate, and control their cognition, motivation, and behavior, guided and constrained by their goals and the contextual features in the environment” (Pintrich, 2000, p. 453). During this process, which is mostly assumed to be cyclical (e.g., Zimmerman, 2000) learners set goals, analyze the task, and incorporate the criteria of success. During learning, learners monitor their progression and adjust learning strategies accordingly. They create feedback internally, which can then be enhanced with external feedback (Butler & Winne, 1995). Formative as well as selfassessments are considered to be related to self-regulated learning (Hattie & Timperley, 2007; Panadero, Jonsson, & Botella, 2017; Paris & Paris, 2001). As learning in higher education is increasingly facilitated through digital learning environments, learners’ behavior can be tracked using learning analytics. Learning analytics aim at optimizing and modeling learning processes, learning environments, and educational decision-making by assessing, eliciting, and analyzing dynamic

29

Linking Assessment and Learning Analytics to Support Learning. . .

743

information about learners and learning environments (Ifenthaler, 2015). Therefore, learning analytics collect a variety of data, such as how timely learners access resources, their performance in self-assessments, their digital interactions with peers, access to library resources, and their geolocation, but also individual demographic information, and can be enriched with self-reported data from surveys on learning strategies or motivational dispositions (Ellis, Han, & Pardo, 2017; Gašević, Jovanovic, Pardo, & Dawson, 2017). Based on data collected and the resulting analyses, adaptive and personalized feedback can be offered to learners whenever they need it. As feedback is more effective when it provides learners with information on how to improve (Hattie & Timperley, 2007), solely giving learners an overview about how they performed in a test is not sufficient. Hence, feedback should be enhanced with additional recommendations. For example, learners testing their current knowledge with self-assessments could receive feedback on how they performed both overall and in more detail related to each learning objective. Furthermore, they could receive additional recommendations on content or topics that need revision, as well as on any additional related learning resources. Based on learners’ online learning behavior, learning analytics might also recommend changing learning strategies, such as a timely recapitulation of the lectures. In summary, learning analytics might be capable of supporting the proposed ongoing assessments of learners’ knowledge and learning behavior across different contexts and of giving informative feedback for improvement. In addition, learning analytics can also enhance and support teachers in their assessment practice, by allowing an ongoing collection of evidence and enabling them to adjust their teaching to learners’ needs. Due to the availability of large data sets on student performance, the data could be also used for informing institutional or governmental decision-making (Ifenthaler, 2015). However, even though applying learning analytics for enhancing assessment seems to be fruitful, related research and theory contribution are still at an initial level (Martin & Ndoye, 2016). Hence, the purpose of this chapter is to provide an overview on how assessment can be combined with learning analytics with the aim of supporting self-regulated learning processes of students in higher education. Therefore, relevant aspects of assessment and assessment design, the role of feedback related to assessment and self-regulated learning, plus current perspectives on learning analytics will be introduced. A conceptual framework will be derived, based on the theory, and suggestions will be made as to how learning analytics features could assist this framework. This chapter concludes with a discussion of the model and an outline of upcoming research needs.

Assessment Assessment in Higher Education This section gives an overview about assessment frameworks, as well as practices, requirements, and constraints of assessments in the context of higher education. Furthermore, links between assessment and self-regulated learning are introduced.

744

C. Schumacher

Definitions of assessment differ and may be distinguished by focusing either on the process or the product of assessment (Webb, Gibson, & Forkosh-Baruch, 2013). In this regard, the process refers to the assessment activities, and the product refers to the results of the assessment (e.g., a label of judgment, score) (Black & Wiliam, 2018; Webb et al., 2013), or alternatively, the assessment focuses either on a learning process or on a learning product (e.g., essay) (Falchikov, 2005). Furthermore, depending on the functions of the assessments, their definitions differ (see section “Functions of Assessment”). The basic cyclical assessment process consists of three phases: (1) eliciting evidence, (2) interpreting evidence, and (3) taking action (Wiliam & Black, 1996). To further strengthen the connection to pedagogy and contextualize assessment, Black and Wiliam (2018) propose an enhanced framework of assessment (Fig. 1). They further emphasize an integrative view on formative and summative assessment which are distinguished based on the “kinds of inferences being drawn from assessment outcomes” (Black & Wiliam, 2018, p. 553, emphasis in original) either related to the current status or related to actions for improvement. Their assessment model is considered to be cyclic and entails six components. It starts with (1) pedagogical and instructional approaches and (2) underlying theories of learning, going on to include (3) contextual characteristics, such as discipline, institutional policies, the learning environment, and outcomes. These are culturally valued and promoted (Bennett, 2011) and influence (4) the planning and design of assessments (Bearman et al., 2016),

Fig. 1 Model of assessment integrating pedagogy (Black & Wiliam, 2018, p. 556)

29

Linking Assessment and Learning Analytics to Support Learning. . .

745

which should be guided by design principles (Mislevy et al., 2003). The (5) assessment itself is then implemented and provided to the learners, having either a formative or summative function, depending on how the evidence is used. The evidence may inform (6) external summative testing. This is mostly associated with high-stakes tests, which, as well as being determined by contextual factors, may also impact them (e.g., adjustments of curricula or policies). Assessment in higher education has several functions, such as certificating students’ performance by assigning grades, evaluating learners’ progress and giving support on how to make improvements, ascertaining quality of teaching courses and curriculum, and providing information for the institution or accreditation (Sadler, 2010a). In higher education assessment is most commonly associated with grading or certification (Boud, 2007). Due to larger study cohorts with heterogenous prerequisites, however, the aspect of giving individual support becomes increasingly relevant (Bosse, 2015; Tolstrup Holmegaard, Møller Madsen, & Ulriksen, 2017). Assessment methods used in higher education include portfolio assessments, written and oral examinations, and group assessments and diaries, and assessment modes focus on self- and peer assessment, and formative, continuous, and summative assessments, as identified by a literature review on relevant assessment practices in higher education related to the Bologna Process (Pereira, Assunção Flores, & Niklasson, 2016). Self-regulated learning is both a goal of and a necessity for successful learning in higher education (Cassidy, 2011; Nicol, 2009), and assessment practices (Panadero et al., 2017) plus related feedback (Nicol & Macfarlane-Dick, 2006) are considered to support learners’ self-regulation. Self-assessments in particular are discussed within the context of higher education as they are related to all components of self-regulated learning (Paris & Paris, 2001) and foster learners’ responsibility toward learning (Bennett, 2011). For example, Panadero et al. (2017) used a meta-analytic approach to find that self-assessments produced a positive impact on students’ self-regulated learning strategies, yielding small to medium effect sizes (d = 0.23 to d = 0.65). Furthermore, formative assessment in higher education focuses on peer assessment as a means of supporting learning (Cartney, 2010). Peer assessment is thought to increase learners’ responsibility and autonomy for their learning and help gain a better understanding of what is relevant for achieving high-quality learning products (Cassidy, 2006; Webb et al., 2018). However, to be able to perform in self- and peer assessments but also to react to feedback, students are considered to require capabilities of feedback literacy (Carless & Boud, 2018) including evaluative judgment (Panadero, Broadbent, Boud, & Lodge, 2018). Evaluative judgment is described as “being able to judge the quality of one’s own and others’ work” (Tai, Ajjawi, Boud, Dawson, & Panadero, 2018, p. 468) and is relevant in estimating achievements of standards and criteria related to produced artifacts, such as an essay or programming task. Evaluative judgment can be fostered through engaging students in self- and peer-assessment practices, by emphasizing their justifications for their judgments (Boud & Molloy, 2013; Tai et al., 2018). Conversely, it supports learners in self-regulating their learning (Panadero et al., 2018). In summary, processes related to self- and peer assessment as well as evaluative judgment and feedback literacy are closely related to learners’ self-monitoring and self-evaluating

746

C. Schumacher

activities described in models of self-regulated learning (e.g., Winne, 2011, 2017a; Zimmerman, 2000). As summative assessments in particular, but also formative assessments, can have vital consequences for learners (Shute et al., 2016), assessments need to be reliable and valid (Shute & Becker, 2010). Particularly, (complex) skills cannot be measured directly, as it is only possible to infer learners’ skills, knowledge, competences, and learning processes based on their observable behavior (Bennett, 2011; Mislevy et al., 2003). Hargreaves (2007, p. 186) emphasizes that “validity of [formative] assessment for learning depends on how far the interpretation and use of the assessment actually leads to further learning.” However, it is furthermore crucial to design the assessment based on principles to enhance the validity of the evidence generated (▶ Chap. 83, “The Future of Assessment in Technology-Rich Environments: Psychometric Considerations”). As validity is an interplay between “evidence and theory support[ing] the interpretations” (AERA, APA, & NCME, 2014, p. 11), scores cannot be interpreted without theoretical underpinnings. As it is not obvious why learners do not respond correctly, Bennett (2011) suggests offering a sufficient number of tasks focusing on the same aspect from multiple sources or investigating the reasons for choosing answers in order to recognize patterns of students’ errors, so that interventions are designed appropriately. Due to large cohorts in higher education and limited resources, assessments used for providing feedback are constrained (Boud & Molloy, 2013; Nicol, 2009). Hence, economic assessment practices need to be found both to meet the requirements of having summative assessments resulting in students’ certificates and other accreditation processes and also to support students’ learning with formative assessments and feedback. In meeting these constraints and requirements, Broadbent et al. (2017) divided the required summative assessments into multiple assignments and enhanced them with formative elements by using annotated rubrics, exemplars, and personalized formative audio feedback to support students’ learning and selfassessment. As learning analytics aim at providing adaptive and personalized feedback to individual learners, it might be a meaningful enhancement of current assessment practices in higher education, which will be described in section “Learning Analytics in Higher Education.”

Functions of Assessment In the literature, at least two major functions of assessment are discussed: formative and summative assessment (e.g., Bennett, 2011; Black, 2013; Shute & Becker, 2010; Webb & Ifenthaler, 2018). The summative assessment is often taken at the end of a learning unit or course and mostly related to some kind of judgment where a learner’s performance is related to the predefined objectives with the purpose of grading or certification (Shute & Becker, 2010). Benefits of summative assessment are that “(a) it allows for comparing learner performances across diverse populations on clearly defined educational objectives and standards; (b) it provides reliable data (e.g., scores) that can be used for accountability purposes at various levels

29

Linking Assessment and Learning Analytics to Support Learning. . .

747

(e.g., classroom, school, district, state, and national) and for various stakeholders (e.g., learners, teachers, and administrators); and (c) it can inform educational policy (e.g., curriculum or funding decisions)” (Shute & Becker, 2010, p. 8). The aforementioned functions of summative assessment may be further expanded by an evaluative function, focusing on “Evaluating the quality of educational institutions or programs” (Wiliam & Thompson, 2008, p. 59). Formative assessment as an ongoing cyclical process (Wiliam & Black, 1996) can be defined as “assessment that is specifically intended to provide feedback on performance to improve and accelerate learning” (Sadler, 1998, p. 77). Furthermore, formative assessment should support the adaption of teaching activities to learners’ needs (Black, Harrison, Lee, Marshall, & Wiliam, 2003). Following Wiliam and Thompson (2008, p. 63), formative assessments include defining a shared understanding of standards and criteria of learning outcome (“Where the learner is going”), assessing learning evidence (“Where is the learner right now”), and giving feedback on how to achieve the pre-set outcomes (“How to get there”), by involving the teacher, the peers, and the learners themselves. However, formative and summative assessment are not fully distinct concepts, as, for example, summative test results may be used as feedback for learners, resulting in a change of students’ learning behavior, even though this was not the primary intention (Bennett, 2011; Black & Wiliam, 2018; Smith, 2007). Alternatively, they might result in changes of instructional processes (Bennett, 2011). In addition, when learners prepare for an exam or interact with the assessment tasks (Bennett, 2011), reflective processes might be initiated. Hence, both functions of assessment are considered to be on a continuum, and it is proposed that the same tools (e.g., tests, essays) can be used for both but applied with a different focus (Black & Wiliam, 2018; Wiliam & Black, 1996). Hence, so as not to exclude evidence from assessments used for a primary summative purpose but applied in a formative way (Wiliam, 2011), Black and Wiliam (2009, p. 9) adjusted their definition of formative assessment to the following: “practice in the classroom is formative to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers, to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited.” This definition is more learner-centered as the authors highlight that they associate instruction with teaching and learning (Black & Wiliam, 2009) and emphasize decision-making, based on evidence. In order for assessment activities to support students’ learning processes, Wiliam (2011) highlights two aspects: (a) the assessment needs to be designed in such a way that the generated evidence can be acted upon, by not only showing the gap but by pointing out, how improvements can be made and (b) that learners react to the feedback by initiating activities accordingly. However, learners’ reaction to the feedback provided based on formative assessment will be influenced by their individual characteristics, like their capacity to self-regulate their learning (Butler & Winne, 1995) as well as motivational constructs, such as perceived selfdetermination (Deci, 1992) or attributions of failure and success (Schunk, 2008; Weiner, 1985). Hence, learners should perceive some kind of autonomy and control

748

C. Schumacher

when taking assessments (Bevitt, 2015), and the feedback given should engender learners’ responsibility, autonomy, and competence so that they are motivated for behavioral change. Boud and Molloy (2013, pp. 704–705) emphasized the agency of learners within the process of assessment by stating that “there is an educative purpose of assessment to inform the practice of learners so that not only do they have the capabilities to produce work that meets the standards of others, but also they can make their own informed judgements about the process of production of that work, drawing upon the full range of resources available to them.” To promote such learning-oriented assessments which facilitate both functions – certification and learning – Carless (2007) describes three components: (a) assessment tasks as learning tasks, which refer to the designated learning outcomes and are spread across the learning or course period; (b) involving students by enabling them to understand the learning goals, engage with the criteria and standards, as well as evaluate themselves and their peers; and (c) feedback as feedforward, by providing timely feedback with recommendations for upcoming activities. However, to gain valid inferences about learners’ progression based on their observable behavior as well as using these inferences to deduce appropriate assumptions and interventions, and to achieve a trans-contextual standard which is necessary to combine evidence from different contexts, assessments need to be designed carefully (▶ Chap. 83, “The Future of Assessment in Technology-Rich Environments: Psychometric Considerations”). Therefore, principles of assessment design will be described further.

Assessment Design To be able to infer from assessment data on students’ learning, assessments need to be designed following design principles (Shute et al., 2016). Several frameworks exist, such as the Assessment Triangle (Pellegrino, Chudowsky, & Glaser, 2001), Assessment Engineering (Luecht, 2013), the Four Building Blocks (M. Wilson, 2005), or the Evidence-Centered Assessment Design (e.g., Mislevy et al., 2003; Mislevy & Riconscente, 2005). The Assessment Triangle (Pellegrino et al., 2001) consists of three interdependently connected assessment elements focusing on evidence-based reasoning: (a) cognition includes assumptions about learners’ representations of knowledge and competence development as well as underlying learning theories; (b) observation encompasses the multifaceted tasks or methods used to let learners demonstrate knowledge and skills which need to be designed with a purpose and aligned with the cognitive model for providing the correspondent evidence; and (c) interpretation involves methods used for inferring from the observable assessment data from various sources on learners’ knowledge and skills (as defined in (a) the cognition model). The Evidence-Centered Assessment Design framework prevails in the context of technology-enhanced assessments (Webb & Ifenthaler, 2018). The reason for this is that it considers advances of learning sciences and technology and highlights the need for assessment measures to be aligned with the new complexity (Mislevy et al.,

29

Linking Assessment and Learning Analytics to Support Learning. . .

749

2003), such as the integration of evidence from different contexts and measures as well as over time (Almond, 2010). Hence, for linking learning analytics and assessment, this framework seems promising as it synthesizes different models which is in line with the holistic learning analytics framework developed by Ifenthaler and Widanapathirana (2014). Using a principle-based approach guided by learning theory increases the coherence of the inferences from observations on the interpretations of the intended assessment constructs and thus increases the assessments’ validity (Nichols, Kobrin, Lai, & Koepfler, 2017). In addition, the EvidenceCentered Assessment Design framework is described in reasonable detail making it usable for application. The Evidence-Centered Assessment Design framework consists of five layers (Mislevy & Riconscente, 2005): (1) domain analysis includes knowledge about the domain and about what is relevant in order to perform valued tasks in this domain; (2) domain modeling defines the relevant elements of the assessment (underlying theory, claims of assessment, and defining data to be collected), based on the domain analysis, and states how they can be elicited and observed. Hence, it is about “what an assessment is meant to measure, and how and why it will do so” (Mislevy & Haertel, 2006, p. 8); (3) Conceptual Assessment Framework focuses on designing the elements of an assessment considering different models as a blueprint (as described below in more detail); (4) assessment implementation refers to putting the prior templates into concrete terms by “authoring tasks, fitting measurement models, detailing rubrics and providing examples, programming simulations and automated scoring algorithms” (Mislevy & Riconscente, 2005, p. 24), and determining how the tasks and scaffolds will be presented to the learners; and (5) assessment delivery refers to presenting the assessment to learners by selecting a task or activity, presenting it according to the task model and collecting the work product, processing the response by identifying evidence related to the assessment purpose (evidence model), including giving optional task-level feedback to learners. All evidence collected is accumulated for summative assessment and feedback (based on evidence model), leading to an update of the probabilistic assumptions of the student model. This information again feeds into activity selection, such as further instruction or new assessment tasks. All information required for the processes of assessment delivery is stored in the task/evidence composite library. All related processes and the task/evidence composite library interact and update each other dynamically in each step. For operationalizing and designing an assessment, Mislevy et al. (2003) describe the Conceptual Assessment Framework, containing several intertwined models: • The student model includes the “variables related to the knowledge, skills and abilities we wish to measure” (Mislevy et al., 2003, p. 6). Learners’ knowledge in a certain domain is estimated by inferral from their observable behavior, related to assigned tasks or situations applying probabilistic models, but further influenced by external variables such as environmental or personal conditions (e.g., noise, motivation).

750

C. Schumacher

• The evidence model “provide[s] detailed instructions on how we should update our information about the student model variables given a performance” (Mislevy et al., 2003, p. 8). This contains evaluation standards, how scores are assigned to learners’ work products (e.g., rubrics, automated scoring procedures), and how the collected variables relate to the assumptions in the student model and to the overall performance level of a targeted proficiency (measurement model). • The task model describes the sets of tasks designed based on domain modeling, their key features, and how to present them to the learners to obtain valid data (e.g., considering circumstances influencing performance). The tasks need to be designed to elicit the behavior that is expected from the learners to infer on their knowledge. • The assembly model concerns the representation of a broad variety of tasks to enable a valid inferral of learners’ proficiency, based on their processing of the tasks, as indicated in the student model. Hence, it “orchestrates the interrelations among the Student Models, Evidence Models, and Task Models” (Mislevy & Riconscente, 2005, p. 20) and would serve as the basis for adaptive testing. Nichols et al. (2017) describe three characteristics constituting principle-based assessment designs: (1) construct-centered approach meaning that the constructs which should be assessed are defined and guide the design process; (2) engineering toward intended interpretations and uses, which implies that the assessments are designed to collect evidence and interpret it using measurement and probabilistic models to infer on the assessment targets; and (3) explicit design decisions and rationales supporting an explicit and transparent design process, including detailed definitions of the targets of inference, the stimuli used to elicit them, and of the evidence collected and the way in which it is evaluated and accumulated to infer on the assessment targets. Furthermore, a principle-based approach integrates both formative and summative functions of assessments as they focus on being informative to the learner regardless whether graded or not (Shute et al., 2016). Therefore, however, not only the assessment needs to be delivered to the learner but also feedback needs to be provided to support learning from assessments.

Feedback in Assessment As described before, assessments might be carried out either externally (e.g., by teachers, peers, educational technologies) or in the form of self-assessments internally by the learner. The information gathered will be evaluated against predefined assessment criteria, standards, or learning objectives. In the case of external assessment, the result and optional actions for improvement need to be somehow communicated to the learner (Narciss, 2008, 2017). Hence, the feedback provided can be described as the expression of the assessed, contextualized, and interpreted assessment evidence including information about how to make improvements in terms of reaching the favored outcome (Ramaprasad, 1983; Sadler, 2010b). Feedback therefore can have cognitive (information processing), metacognitive (self-evaluation or

29

Linking Assessment and Learning Analytics to Support Learning. . .

751

reflection), and motivational (encouraging effort and persistence) functions (Narciss, 2008; Narciss et al., 2014). Feedback might be provided on four levels – the task performance, the process of solving the tasks, the level of self-regulation, and the self-level about the learner’s person (Hattie & Timperley, 2007). However, for some authors, in order to be considered as feedback, this information needs to be actually used to close the loop (e.g., Ramaprasad, 1983; Sadler, 1989), thus leading into some kind of adapted learning activities. However, the feedback provided will not necessarily lead to changes in learners’ behavior (Cartney, 2010; Hattie & Timperley, 2007). In line with learning theoretical assumptions of cognitivism (Piaget, 1975) it is not sufficient only to present the feedback information to the learner, as the information needs to be interpreted and processed and is further influenced by individual prerequisites (e.g., prior knowledge, educational background, learning history, attribution patterns, goal setting, learning strategies, beliefs) (Butler & Winne, 1995; Evans, 2013; Nicol, 2009; Nicol & Macfarlane-Dick, 2006; Sadler, 2010b). In particular, if the external feedback is contradictory to the internal feedback or beliefs, the external information needs to be integrated into learners’ cognitive representations to reduce potential perceptions of discrepancies (Butler & Winne, 1995; Piaget, 1975). Thus, as Narciss (2012, 2017) states, besides the type, quality, and the source of the external feedback, individual characteristics of the learners and their learning behavior, as well as contextual factors of the learning setting (e.g., learning objectives and tasks), also determine the effectiveness of the feedback provided. Hence, learners need to be capable and willing to interpret and react upon the feedback provided, as well as actively seeking feedback (Hattie & Timperley, 2007; Narciss, 2008). As Boud and Molloy (2013, p. 709) state, “The challenge for learners is not only to acquire understanding of the appropriate standards and criteria and monitor their performance against these, but also to find new opportunities to put this learning into practice and find ways of judging their own work.” Hence, if learners encounter problems cognitively in understanding the feedback and deriving appropriate actions, or if motivational constraints mean they refuse to react to the feedback, the feedback process might not result in improved learning behavior. Furthermore, feedback should not be a one-way activity from teacher to learner but should consider the learner as an actively involved agent. This process can be enhanced through discussions to achieve a shared understanding and social construction of standards, criteria, and quality of work (Carless, Salter, Yang, & Lam, 2011; Nicol & Macfarlane-Dick, 2006; Sadler, 2010b). These prerequisites make it challenging to provide meaningful feedback that supports individuals in their learning. However, feedback is also considered to increase learners’ motivation to learn (Butler & Winne, 1995; Hattie & Timperley, 2007; Narciss, 2008) and improve learners’ self-regulatory skills (Nicol & Macfarlane-Dick, 2006). Furthermore, their capability to understand, interpret, and make use of feedback can be trained, as discussed in the field of feedback literacy (Carless & Boud, 2018). To promote the social construction of feedback, peer feedback is a practice used in higher education to foster the engagement of students with the assessment criteria and the standards, encouraging them to

752

C. Schumacher

give meaningful feedback to others and learn from others’ work (Cartney, 2010; Cassidy, 2006). Feedback is considered to be a key component of formative assessment (Hattie & Clarke, 2019; Sadler, 1989) and, if provided appropriately, is vital in supporting successful learning processes (Sadler, 2010b; Shute, 2008). Hence, Nicol and Macfarlane-Dick (2006) identified seven principles of good feedback practice to foster self-regulation. These principles include feedback that (1) allows learners to know what is expected from them; (2) supports their self-regulation through selfassessments and reflection; (3) provides high-quality information (e.g., related to criteria, timely, corrective advice, useable limited quantity) about their learning and (4) on how to improve; (5) is constructed in a social interaction between the involved parties; (6) considers and supports motivational concepts; and (7) gives teachers information about how to adjust instruction to meet learners’ needs. Butler and Winne (1995) emphasize that, to foster self-regulated learning, feedback should support learners’ monitoring processes, increasing their awareness about these processes, calibrating internal judgments, choosing appropriate strategies, and adopting suitable goals as the anchor of their monitoring activities. Hattie and Timperley (2007) state that feedback is most efficient when it is not directed on a personal level but rather when it focuses foremost on task performance, then on the process of working on the task, and finally on the self-regulation of these processes. Furthermore, no one level should be overemphasized. However, besides all presumed positive impacts of feedback on students’ achievement, teachers are faced with limited resources and increasing numbers of students with heterogenous prerequisites, constraining their possibilities to provide supportive and personalized feedback to the learners (Pardo, Jovanovic, Dawson, Gašević, & Mirriahi, 2019). With regard to offering individual support to learners, learning analytics are a promising approach.

Learning Analytics in Higher Education Definitions of learning analytics mostly focus on collecting data about learners and learning environments in order to use this information for understanding and optimizing learning processes and environments and educational decision-making (e.g., Ifenthaler, 2015; Siemens, 2010). However, learning analytics are often not defined or distinguished precisely from the related concepts of academic analytics and educational data mining (Ifenthaler, 2015), which are all at the intersection of computer science, education, and statistics (Romero & Ventura, 2013). Educational data mining is more focused on the automatic detection of new patterns, whereas learning analytics are concerned with assessing known assumptions in the data collected, including human judgment (Bienkowski, Feng, & Means, 2012). Academic analytics use aggregated educational data to support institutional decisionmaking, such as resource allocation or retention planning (Ferguson, 2012). Papamitsiou and Economides (2014) identified that studies using learning analytics or educational data mining focus on (a) identifying and modeling learning

29

Linking Assessment and Learning Analytics to Support Learning. . .

753

behavior; (b) using indicators for predicting performance; (c) increasing teachers’ and learners’ reflection and awareness; (d) early prediction of dropout and retention or identification of related learning engagement; (e) improving assessments by making them adaptive and providing feedback; and (f) recommending resources either for the learners or, technically, for analyses. In a recent literature review on educational data mining and learning analytics in higher education, Aldowah, Al-Smarraie, and Fauzy (2019) classified current applications into four main dimensions: (1) computer-supported predictive analytics consider a variety of factors, including learners’ behavior and achievements in assessments to predict dropout and retention and to implement interventions accordingly, plus they focus on evaluating learning material to adjust their quality and fit to learners’ needs as well as determining their impact on performance; (2) computer-supported learning analytics include the identification of collaboration and self-learning processes using data on learners’ interaction with the digital learning environment; (3) computersupported behavioral analytics identify patterns and preferences and detect irregular or successful behavior; and (4) computer-supported visual analytics facilitate the understanding of the complex analyses and enable the derivation of interventions, as well as providing insights into the learning processes to both learners and teachers. Therefore, a variety of data is collected: information on the behavior of learners, from their navigation within the digital learning environment using logfiles to their use of library resources, and data on social interaction. Therefore, a variety of data is collected: behavioral data of learners, from their navigation within the digital learning environment using logfiles (e.g., log-in, time online, access of resources), their use of the library resources; data on social interaction (group work, discussions); external data such as geolocation, access to buildings; socio-demographic information (e.g., age, sponsorship, educational background); self-reported data from surveys (e.g., learning strategies, motivational disposition) or learning artefacts and performance (e.g., assignments, selfassessments, forum discussions) (e.g., Ifenthaler & Widanapathirana, 2014; Sclater, Peasgood, & Mullan, 2016). Based on the data collected, learning analytics allow insights into students’ interactions with the learning resources or learning processes (Vieira, Parsons, & Byrd, 2018; Winne & Baker, 2013). Therefore, the heterogenous data need to be pre-processed to allow the application of data mining methods and algorithms, in order to identify behavioral patterns or relationships within the data, for example (Romero & Ventura, 2013). The analyses allow retrospective as well as real-time insights, and also aim to provide predictive forecasts (Daniel, 2015; Ifenthaler & Widanapathirana, 2014). However, predictive analytics using machine learning require data sets containing a certain amount of historical learning behavior to train the algorithms for valid predictions of unseen test data sets (Brooks & Thompson, 2017; Martin & Sherin, 2013). As learning is always related to the context in which it occurs, inferences and algorithms which were valid for data from one context or previous cohorts might not result in valid analyses in other contexts or cohorts (Greller & Drachsler, 2012; Macfadyen & Dawson, 2012; West, Heath, & Huijser, 2016). In addition, learning and teaching in higher education mostly takes place face-to-face or blended, resulting in small or incomplete data sets,

754

C. Schumacher

due to learning processes outside the digital learning environment (Ifenthaler & Schumacher, 2016; Schumacher & Ifenthaler, under review; Wilson, Watson, Thompson, Drew, & Doyle, 2017). Hence, learning analytics and its related concepts are complex and are intended to provide insights into the multifaceted and complex domain of learning and teaching. Aiming to illustrate the interrelatedness of the various sources and stakeholders of learning analytics, Ifenthaler (2015, 2019) proposes a holistic framework (see Fig. 2). Based on rather static curricular information influenced by governmental and institutional requirements learning objectives are defined, learning settings and assessments are designed. Curricular requirements impact the digital learning environment, the teachers, and their instruction, and vice versa, teachers’ characteristics influence the curriculum and micro-level design of the (digital) learning environment. Within this, learners, with their individual characteristics, social relatedness, and physical prerequisites, interact with each other and the resources, thus generating a huge variety of data. This static and dynamic information needs to be structured and integrated in the learning analytics engine to be analyzed based on pedagogical theoretical assumptions using different data mining methods and algorithms for comparisons, predictions, or identification of patterns. To achieve adaptive and personalized learning environments, this information is provided via the digital learning environment to the learner in form of visualizations, prompts (short hints or questions), recommendations, or other feedback such as digital badges. Besides this, the information gained could also be mediated through the teacher, either to feed it back to the learner or to adjust the learning environment according to the learners’ needs. In line with academic analytics, the information can further be used for reports to different stakeholders for different purposes (e.g., decision-making, resource allocation, curriculum changes). However, learning analytics are still not sufficiently guided through knowledge from learning theory and empirical evidence (Marzouk et al., 2016). Instead of focusing on the relevant theory-driven indicators (Wong et al., 2019) and the prevailing aim of learning analytics supporting learning processes (Clow, 2013; Gašević, Dawson, & Siemens, 2015), technical elaborated systems are created, or algorithms are applied using all data available. In addition, learning analytics still suffer from a lack of evidence that they are actually capable of supporting learning (Ferguson & Clow, 2017; Viberg, Hatakka, Bälter, & Mavroudi, 2018). However, learning analytics offer a huge potential for adaptation and personalization of digital learning environments (Aguilar, 2018; Greller & Drachsler, 2012; Ifenthaler & Widanapathirana, 2014). But, if learning analytics systems should provide feedback to learners on how to improve learning, their current performance and their learning processes need to be validly assessed. Hence, the further theoretical synthesis of current assumptions about learning processes and the possibilities learning analytics can offer is necessary and reasonable. In the next section, this aim will be promoted further by highlighting potential links of assessment, feedback, and learning analytics under consideration of theory on self-regulated learning.

Linking Assessment and Learning Analytics to Support Learning. . .

755

Fig. 2 Holistic learning analytics framework (Ifenthaler, 2019)

29

756

C. Schumacher

Informative Assessment Using Learning Analytics By referring to the Assessment Working Group at EdusummIT 2011, Webb et al. (2013) propose a definition of digitally enhanced assessments “as those that integrate (1) an authentic learning experience involving digital media with (2) embedded continuous unobtrusive measures of performance, learning and knowledge (. . .) which (3) creates a highly detailed (high resolution) data record that can be computationally analysed and displayed so that (4) learners and teachers can immediately utilize the information to improve learning.” Subitems 2, 3, and 4, in particular, are related to the application of learning analytics. However, to identify, capture, and analyze the relevant data using appropriate assessment measures and link them to targeted performance, learning processes, and knowledge, these assessments need to be designed following principle-based approaches. Shute et al. (2016) outlined a vision of “ubiquitous, unobtrusive, engaging, and valid” (p. 53) assessment, including continuous data collection and integration of learners’ interactions with different learning resources in different learning environments to gain increased evidence about learners’ skills and competences across contexts. They further emphasize that assessment should “(a) support, not undermine the learning process for learners; (b) provide ongoing formative information [. . .]; and (c) be responsive to what is known about how people learn” (p. 52). Learning analytics might be capable of assessing cross-contextual learning processes without being intrusive (Vieira et al., 2018; Winne, 2017b) and can be further enriched with multiple sources, such as peer assessment and peer feedback and self-assessments and self-reflections. To meet subitems (b) and (c), learning analytics need to be further aligned with theory on learning, feedback, and assessment (Marzouk et al., 2016; Sedrakyan, Malmberg, Verbert, Järvelä, & Kirschner, 2018). However, only few publications specifically focus on the link of assessment and learning analytics (Ellis, 2013; Ifenthaler, Greiff, & Gibson, 2018; Martin & Ndoye, 2016). Ellis (2013) states that assessment analytics allow learning performance and progress to be measured over time, along with individual, social, and standard-based comparisons. She further suggests that assessment analytics should include aspects such as completed degrees, progression results, module results, individual assessment results, achievements compared to learning objectives and criteria, plus strengths and weaknesses of a student’s work. Martin and Ndoye (2016) assign learning analytics techniques and data measures to four types of assessments used in digital learning environments and investigate their application. Ifenthaler et al. (2018) highlight the potential of enhancing (large-scale) assessments using learning analytics, especially for providing immediate feedback to learners and teachers as well as for processing the immense amount of data. Shute and Becker (2010) adapted the needs for contemporary assessment from the National Research Council (NRC, 1996) to highlight the differences of foci on assessment. The shift from assessing learning outcomes to assessing learning processes to acquire relevant skills and knowledge is of key importance. As Pellegrino et al. (2001, p. 27 f.) state, assessments are static as they only “provide ‘snapshots’ of achievement at particular points in time, but they do not capture the progression of

29

Linking Assessment and Learning Analytics to Support Learning. . .

757

students’ conceptual understanding over time, which is at the heart of learning.” Shute and Becker’s (2010) work was carried over and enhanced with aspects on how learning analytics might support the enhanced focus on assessment (see Table 1). Different assessment types are predominant in certain disciplines, depending on the discipline’s practices and the valued knowledge, skills and competences, or learning goals (Knight, 2006). For example, Neumann, Parry, and Becher (2002) assign assessment types to the different disciplines, distinguishing hard and soft as well as pure and applied: Hard pure disciplines, such as natural sciences and math, focus on frequent knowledge assessments with specific, close, and norm-referenced assessment types, such as calculations. Soft pure disciplines, such as social sciences and arts, put more emphasis on learners’ understanding, judgment, and integration of complex knowledge including continuous assessments, essays, projects, and oral examinations but also on declarative knowledge using multiple-choice tasks. Hardapplied disciplines like engineering focus on factual knowledge using multiplechoice tests in rigorous and ongoing assessments for elimination but also on complex problem-solving and application and integration of knowledge. Soft-applied disciplines like education or management focus on developing professional practice and problem-solving, and assessments include essays and project reports and are enhanced by peer and self-assessments to foster self-reflectory and practical skills. Hence, the practices and assessment needs are different. Martin and Ndoye (2016) list four commonly applied types of online learning assessments, (a) comprehensiontype assessments such as multiple-choice, (b) discussion boards to promote collaboration and interaction, (c) reflection-focused assessments highlighting the solution process (e.g., essays), and (d) project-based assessments integrating different skills to reach an authentic product. While single- and multiple-choice assessments may be easy to analyze, the focus should not be on easily assessable, but on valued learning outcomes (Shute & Becker, 2010). Relevant within the debate on twenty-first century skills are projects engaging learners in behavior that is difficult to measure as the constructs, some of which are on a group level, are latent including behavioral, cognitive, and affective components (Webb et al., 2018). Neither assessments (Bennett, 2011) nor learning analytics (West et al., 2016) can be interpreted without considering their context. Indeed, learning behavior within different contexts is a crucial indicator when it comes to recording holistic skills and competences, especially those occurring cross-contextually (▶ Chap. 83, “The Future of Assessment in Technology-Rich Environments: Psychometric Considerations”). Hence, to obtain valid results, the characteristics of the different contexts (e.g., course format, level of self-directedness) and tasks (e.g., difficulty, complexity, assessed skills) need to be considered in the analyses. Thus, in order to assess these complex constructs involving different types of assessments varying over disciplines, a principle-based assessment design framework is necessary for guiding systematic design and valid integration into learning analytics. Furthermore, by defining how the measured indicators are related to the assessment goals as well as how to proceed with incomplete data would further enhance the validity of learning analytics. In addition, this would support the need for a more theory-driven approach of learning analytics (Bienkowski et al., 2012; Ferguson, 2012; Marzouk et al.,

758

C. Schumacher

Table 1 Changing assessment foci (Shute & Becker, 2010, p. 4) and enhancing the focus on assessment with learning analytics Less focus on assessment Learning outcomes

More focus on assessment Learning processes

What is easily measured

What is most highly valued

Discrete, declarative knowledge

Rich, authentic knowledge and skills

Content knowledge

Understanding and reasoning, within and across content areas

What learners do not know

What learners understand and can do

Supporting assessment through learning analytics Learning analytics enables the tracking of learners’ behavior within digital learning environments and thus adoption of a processual view on learning Learning analytics allow implementation of a great variety of measures, and, if designed and implemented following a principle-based approach, learning analytics can support the measurement of highly valued learning approaches and outcomes by focusing and accumulating relevant indicators based on the underlying evidence model to give an overall indicator for performance, skills, knowledge, or competencies Learning analytics enable evidence from various tasks to be capture and integrate, using multiple data sources within numerous contexts. By tracking learners’ complex problem-solving behavior and their performance and behavior in educational games or collaborative tasks, transfer of knowledge and multiple skills can be assessed Learning analytics might help gain an understanding of whether learners integrate knowledge from different contexts by referring to an intertwined assessment design mapping the various learning objectives and measurable indicators across courses. A holistic representation of learners’ knowledge, skills, and competencies can be derived and might possibly be enhanced with knowledge and skills learned in informal contexts Learning analytics allow feedback concerning both what learners currently know and are capable of (performanceoriented; feedback) and where and how they can further improve their capabilities and skills by providing recommendations (process-oriented; feedforward) (continued)

29

Linking Assessment and Learning Analytics to Support Learning. . .

759

Table 1 (continued) Less focus on assessment By teachers alone

More focus on assessment By learners engaged in ongoing assessment of their work and that of others

Supporting assessment through learning analytics Digital learning environments using learning analytics allow frequent selfassessments with immediate feedback; this feedback can be enhanced with teacher feedback or peer feedback through relevant tools (e.g., different tasks for the same learning objective assessed from different perspectives and combined in joint feedback)

2016) which could be reasonably enhanced with educational data mining techniques for identifying unknown patterns and findings within the trace data. However, to close the feedback loop, the assessed evidence needs to be translated into feedback, either through the system or the teacher, considering the different demands of each individual learner as described in the following section.

Feedback Based on Learning Analytics Feedback in learning analytics systems is mostly provided through dashboards, and research predominantly focuses on investigating satisfaction, design and usability aspects, performance, comparisons with peers, and engagement (e.g., Aljohani et al., 2019; Park & Jo, 2015; Roberts, Howell, & Seaman, 2017; Verbert, Duval, Klerkx, Govaerts, & Santos, 2013; Verbert et al., 2014). Providing feedback using learning analytics is not limited to the use of dashboards as it can be provided by using prompts, by teachers sending feedback messages based on learning analytics results, or by suggesting additional resources when presenting the results of selfassessments. However, only few empirical studies (Howell, Roberts, & Mancini, 2018; Pardo et al., 2019) or conceptual papers (Sedrakyan et al., 2018) focus in particular on how feedback using learning analytics had an impact on learners’ affect and resilience (Howell et al., 2018) or on learners’ satisfaction with feedback and academic achievement (Pardo et al., 2019). However, current learning analytics systems often do not provide information to learners but only to the teachers (Macfadyen & Dawson, 2010; Vieira et al., 2018). But, to close the loop as proposed in the discussion on formative feedback and assessment, learners need to be informed about how they are performing, how they can improve, and where they should go to (Hattie & Timperley, 2007; Wiliam & Thompson, 2008). Feedback provided through learning analytics should incorporate learning theory to not diminish learners’ willingness to react upon it by not considering motivational dispositions (Schumacher & Ifenthaler, 2018b; Lonn, Aguilar, & Teasley, 2015), prior knowledge, and other prerequisites. Hence, especially with a focus on fostering students’ self-regulated learning capacities, learning analytics need to be designed carefully,

760

C. Schumacher

taking into account theory and empirical findings about the interplay of cognition, metacognition, motivation, and behavior. Feedback based on learning analytics in particular often provides students with information on their current performance, comparisons, or even an estimated final grade (outcome feedback), but most systems do not provide informative processoriented feedback to learners as to how they can improve their learning processes (Sedrakyan et al., 2018). However, to provide the process-oriented feedback, learning analytics need to understand why learners did not meet the requirements. Therefore, a huge variety of tasks assessing the same skill or competency need to be offered (Bennett, 2011). The tasks should be developed and accumulated based on the Evidence-Centered Assessment Design framework. Further information about why learners did not perform well can be collected by asking learners to reflect on their answers, judge their learning, by integrating a task on assessing their understanding of the underlying concept, or by identifying behavioral patterns which might be related to misunderstandings, gaming the system, or other nonproductive behavior (Chen, Breslow, & DeBoer, 2018; Hsu, Wang, & Zhang, 2017; Liu et al., 2017; Verbert, Manouselis, Drachsler, & Duval, 2012). Assessments are considered to determine what learners are learning (Boud & Falchikov, 2007; Gibbs & Simpson, 2005). Learners might also feel incited to learn for the data – i.e., to achieve good learning analytics results, instead of focusing on what is actually relevant to learn (Nistor & Hernández-Garcíac, 2018). Furthermore, learners might increasingly rely on data and feedback provided through learning analytics (Corrin & da Barba, 2014). Thus, it needs to be kept in mind that a key focus of higher education is to engender students with high capabilities of selfregulated learning. Hence, the feedback provided through learning analytics needs to be based on the assumption that the learners, as agents, are responsible for their own learning (Boud & Molloy, 2013). Therefore, feedback should promote learners’ selfassessment capabilities and their competences in evaluative judgment which are relevant for academic success and lifelong learning. Consequently, learning analytics should be linked to resources promoting the adequate application and improvement of learning strategies (e.g., rehearsal or elaboration) (Black, McCormick, James, & Pedder, 2006; Weinstein & Mayer, 1986) and other academic competencies such as technology proficiency or research skills (Mah & Ifenthaler, 2018). For example, relevant online or university courses could be recommended, based on digital learning behavior or self-reported learning strategies. Prompts could guide learners’ self-regulation with input such as “What are your goals for today?”, “Have you already set up a study plan?”, or “Try to link new concepts from the text to concepts you already know.” Furthermore, a function to set individual learning goals and assign material to those could support planning and monitoring. In addition, learning analytics should engender learners to engage in reflection and critical thinking, which might be supported through prompting self-regulated learning strategies (Kramarski & Kohen, 2017; Müller & Seufert, 2018; Prieger & Bannert, 2018). Based on learners’ answers to such prompts, teachers can generate evidence on students’ progress (Wiliam & Thompson, 2008).

29

Linking Assessment and Learning Analytics to Support Learning. . .

761

Most research favors immediate feedback over delayed feedback; however findings vary and might depend on the feedback level (task, process, regulation, self) or on the type of task, as well as on learners’ prior knowledge (e.g., Butler & Winne, 1995; Evans, 2013; Hattie & Clarke, 2019; Hattie & Timperley, 2007; Kulik & Kulik, 1988; Shute, 2008). Further, it is assumed that learners are activating their own reasoning capabilities if they do not receive feedback immediately (Schroth, 1992) or if they receive less feedback (Hattie & Clarke, 2019). Hence, considering learners as agents, learning analytics systems might allow them to demand feedback whenever and in the depth they want it. To foster self-regulation, learners should also be prompted to reflect about their answers, solutions, and products, to write down open questions or to reflect on where they are heading and any improvements that can be made. As the interpretation of feedback is also dependent on learners’ prerequisites (Evans, 2013), learning analytics might enable the provision of adaptive and personalized feedback to learners considering their prior knowledge, their current motivational states, or their current goals. Learners’ reactions to feedback might not be as intended, as they might reject the feedback, abandon or change the goal, or change their behavior (Kluger & DeNisi, 1996; Wiliam, 2011). Using trace data enables further investigation of learners’ behavioral reactions to feedback, which could be enhanced with self-report data by prompting them with simple questions on their perceptions of the feedback or their current emotional or motivational states. When learning analytics are used to provide automated feedback, learners need to be made aware of the potential shortcomings of the underlying analyses, for example, by prompts, such as “Based on the data available in our analyses, it seems you have not worked on the study material for a while. Hence, it would be reasonable to catch up. If you have worked on the material offline, you might like to confirm your activities by answering the following questions. Or you can take a self-assessment to test your knowledge of the material.” Furthermore, the feedback provided should be on aspects which are in the control of the learner and are thus malleable. Feedback is a powerful tool for supporting learning but can also have unfavorable results (Hattie & Clarke, 2019). Hence, knowledge about learners’ reactions to feedback is furthermore relevant to design supportive (digital) learning environments (Evans, 2013).

Developing an Integrative Assessment Analytics Framework A processual framework of the interplay between principled-based designed (formative) assessment, feedback, and learning analytics is presented in Fig. 3, which integrates conceptual approaches on assessment (Black & Wiliam, 2018; ▶ Chap. 83, “The Future of Assessment in Technology-Rich Environments: Psychometric Considerations”; Shute & Becker, 2010; Wiliam, 2011), assessment design (Almond, 2010;Mislevy et al., 2003; Mislevy & Riconscente, 2005), assumptions on feedback related to assessment (Gibbs & Simpson, 2005; Hattie & Timperley, 2007; Kluger & DeNisi, 1996; Nicol & Macfarlane-Dick, 2006), and the holistic framework of learning analytics (Ifenthaler, 2015, 2019). The process is assumed to

Fig. 3 Integrative processual assessment framework using learning analytics

762 C. Schumacher

29

Linking Assessment and Learning Analytics to Support Learning. . .

763

be cyclical and includes the assessment design, data collection during learning in the digital learning environment and assessments within multiple contexts, evaluative and interpretative components, a feedback model, learners’ and teachers’ reactions, plus the holistic learning analytics framework. The relationship between the models is reciprocal – together they are adaptive to the learners’ needs and progress, as well as to curricular changes or instructional adjustments. Following the EvidenceCentered Assessment Design (Mislevy et al., 2003; Mislevy & Riconscente, 2005), the assessment is designed considering governmental, institutional, and curricular requirements. This information is embedded in and aligned with the learning analytics engine and the digital learning environment to collect relevant data and to aggregate the evidence collected according to the assessment purpose. To clarify expectations, the overall learning goals, criteria of success, and standards are shared and discussed with the learners and are referred to as feedup (Hattie & Timperley, 2007). Learning analytics could provide this information on different levels of detail for a learning unit, a (self-)assessment, entire courses or programs enhanced with exemplars or rubrics. Following DiCerbo et al. (▶ Chap. 83, “The Future of Assessment in Technology-Rich Environments: Psychometric Considerations”), data collection is an ongoing process within multiple contexts. Therefore, learning products or processes are collected over multiple tasks and learning opportunities and within the digital learning environment. As Bennett (2011) suggests, several assessment tasks should be available and applied to assess the same construct (assembly model and assessment delivery in the Evidence-Centered Assessment Design). In addition, learner characteristics can be collected, based on self-reported data and enhanced with behavioral data, information which is also used to update the student model. Further, contextual factors such as the assessment situation should be considered, as this can influence learners’ behavior. Based on the evaluation standards in the evidence model, the behavior shown or learning products provided are scored and, by relying on the measurement model, are related to the assessed skills, competence, or knowledge. Here task-level feedback can be provided to the learners, as proposed by Mislevy et al. (2003), and should be about misconceptions or misinterpretations of the task, instead of focusing on lacking knowledge (Hattie & Timperley, 2007). The evidence collected from various tasks and over multiple contexts needs to be accumulated based on assumptions in the evidence model (e.g., weighting) and related to the specific assessment purpose (Almond, 2010). This in turn also updates the student model and its underlying probabilistic assumptions. Furthermore, the previous step supports the summative assessment function by assigning grades or certifications and the formative function by deriving suggestions for improvement. In addition, the summative function might also entail suggestions for improvement. In general, these suggestions might be derived either by the teacher or a digital learning environment informed by this integrated framework but can be enhanced with feedback from peers. To connect the assessment process with the other entities (e.g., learner, teacher, institutional), feedback (based on theoretical assumptions of feedback) on how to close the gap between current performance and the designated goals needs to be provided (e.g., suggestions for improvement and/or grades). As suggested by Hattie and Timperley (2007), the

764

C. Schumacher

feedback provided should focus on the task, if this has not yet been done, on the process, on learners’ self-regulation, and ideally not on the personal level. As learning analytics enables the collection of more than only performance indicators, feedback can also be provided on the process and on self-regulation. Based on learners’ proficiency and knowledge, learning analytics could provide as detailed and timely feedback as necessary or allow the learner to choose when to receive feedback. It can also give recommendations on when to ask for feedback and how to use it. By using prompts, additional evidence on motivational and emotional states can be collected, but also regulatory hints can be provided (Bannert, 2009). The learners influenced by their characteristics will interpret the feedback and react to it, by changing either their behavior or the goal, by abandoning the goal, or by rejecting the feedback (Kluger & DeNisi, 1996; Wiliam, 2011). To increase the likeliness that the feedback is used as intended, learning analytics integrate all information on the learners to provide (adaptive) feedback considering learners’ current goals, knowledge, learning activities, motivational states, or needs. However, the feedback and learners’ reaction to it might furthermore influence the teacher’s behavior, the instructional process, and the digital learning environment, which should all be adapted to the learners’ needs, according to Nicol and Macfarlane-Dick (2006). If the assessment has a formative function, another assessment or learning period might follow, in which additional evidence about learners’ skills, competences, and knowledge will be collected. This information could be used to assume learners’ progress and use of feedback. As feedforward is considered the most relevant for learning (Hattie & Timperley, 2007), the learners are further provided with information about where they should go next. Learning analytics can recommend further learning material or learning paths (Ifenthaler & Widanapathirana, 2014) related to the evidence collected and the learning objectives. In both formal and informal learning contexts related to the discussion of lifelong learning, a new assessment cycle, which should be integrated into learning cycles, will begin and will be influenced by the previous. However, it should be kept in mind that both assessment and feedback require either prior knowledge or instruction to be useful (Hattie & Timperley, 2007). Learning analytics are related to this framework as they can, in particular, support the ongoing data collection over tasks and contexts as well as the integration of the data into a unique model of skills, knowledge, and competencies, as suggested by DiCerbo et al. (▶ Chap. 83, “The Future of Assessment in Technology-Rich Environments: Psychometric Considerations”). For example, the learning analytics system needs to be able to draw on the information in the student and evidence model in order to know what data are in the scope of collection and measurement, as well as how the data are related and interpreted (Mislevy & Riconscente, 2005; Shute, Rahimi, & Emihovich, 2017). Furthermore, by using external or sensor data, contextual variables such as noise, the location of the learning environment, or physical resources used can be integrated into the analyses. By collecting enough variables, the probabilistic models in the principle-based assessment design allow students’ performance in a certain domain to be forecast (Mislevy et al., 2003), which is in line with aims of learning analytics focusing on predicting performance, retention, or

29

Linking Assessment and Learning Analytics to Support Learning. . .

765

dropout (Papamitsiou & Economides, 2014; Sønderlund, Hughes, & Smith, 2019). Based on the assembly and student model, the learning analytics system can provide the challenging but not overdemanding tasks to the learners, enabling adaptive (self-) assessments (Shute et al., 2016, as demanded in subitem (c) in Sect. 4). Learning analytics are currently lacking a sound integrative framework, especially with regard to which indicators need to be collected and how, based on these indicators, inferences on learners’ performance, competencies, and skills can be made. In addition, this approach makes it possible to provide tasks for each domain and student model as well as the ability to go beyond assessments using solely singleor multiple-choice tasks (Almond, 2010), as these do not validly assess the valued twenty-first century skills (Shute et al., 2016). As the task model contains a plethora of tasks, not only summative but also formative assessments and additional assessment feedback loops can be provided (Almond, 2010), as demanded by Shute et al. (2016, subitem (b) in Sect. 4). Furthermore, learners might choose a certain amount of formative tasks focusing either on a learning product or process that should be part of their final grade (Webb et al., 2013) and explain why they chose this task (e.g., good performance, learning product, or a perceived learning gain). As no “one-size-fits-all” learning analytics system exists (Gašević, Dawson, Rogers, & Gasevic, 2016) and the context is relevant for each discipline, a unique assessment design mapped to the available learning analytics indicators needs to be developed to satisfy the different assessment needs of the disciplines. This might also facilitate the integration of assessments from different disciplines as, for example, in multidisciplinary study programs. Likewise, with regard to interdisciplinary and non-cognitive skills developed independently of the domain, the snippets of evidence can be integrated into an overarching competency model, and using the learning analytics reporting engine, a competency overview for each learner could be provided at any time.

Learning Analytics Features for Informative Assessment Following the statements on informative assessment of Forster (2009) and Fogarty and Kerns (2009), a definition of informative assessment using learning analytics is outlined. Assessment enriched with learning analytics enhances assessments with ongoing data collection over multiple contexts including (self-)assessment results, learning behavior, and other relevant variables resulting in additional evidence. Furthermore, it is aimed at closing the feedback loop by using the evidence to provide recommendations on improvement or to support learners’ self-monitoring. Additional information is provided to teachers and other stakeholders that can be used to adjust and enrich teaching and assessment practices as well as institutional processes. This approach also aims to narrow down the distinction of formative and summative assessments, as both are capable of providing additional evidence as well as further emphasizing the cooperation of learners and teachers. In addition, the use of learning analytics enables further the investigation of learners’ reaction to feedback. Hence, such assessments are grounded in principle-based designs, include data

766

C. Schumacher

from different contexts to gain insights into how learners learn and understand, use the data to provide feedback on improvement, and analyze how the feedback is used to adjust learning and teaching processes. To highlight these features and emphasize that this type of assessment underpinned with data is going beyond traditional summative and formative assessment, the term informative assessment is used. Exemplar implementations are described to illustrate how learning analytics and a principle-based designed assessment approach can enhance each other and provide the necessary feedback to learners and teachers. To facilitate peer assessments and related feedback, learning analytics could be applied. Learners could upload their assignments, which would then be automatically assigned to their peers on the course. As Cassidy (2006) states, support and training are necessary to foster students’ capability and comfort related to peer assessments which can be enhanced with learning analytics. For example, they can be supported by offering a catalogue of the rubrics and standards related to the assignment, plus a feedback checklist based on principle-based assessment design. To enable the peer reviewers to give more helpful feedback, which can be actively used for improvement, the feedback could be analyzed with regard to the feedback practice (Pinheiro Cavalcanti et al., 2019) or with a focus on the recommendations provided for improvement (Xiong, Litman, & Schunn, 2012) using sentiment analysis and natural language processing. Based on these analyses, the quality of the feedback provided could be evaluated to foster students’ engagement in peer assessments and feedback. In addition, the assignment could be compared to an expert solution for providing recommendations to the learners or the peers. If the assessment is designed to be formative, the learner’s updated version could be compared to the feedback provided to gain insights into how the feedback was used, enhanced with an additional self-report by the learner of how useful they perceived the feedback in improving the assignment. The peer feedback and related analyses, the information on the use of the peer feedback, and its perceived impact on the work, along with the automated analyses comparing the assignment with the expert solution, can be aggregated for the teacher, thus facilitating the identification of learners who might be at risk and enabling the teacher to use the evidence generated to derive appropriate interventions (Cartney, 2010). By providing these additional analyses, the teacher might be able to give additional support to learners at risk, even in larger courses. When it comes to supporting self-assessments, the system could provide learners with the learning objectives, rubrics, and standards. Additionally, Carless (2007) suggests providing learners with feedforward – or feedback in advance – detailing common mistakes or problems faced by earlier cohorts in dealing with the same assignments. Comparably, providing learners with exemplars (Broadbent et al., 2017; Sadler, 1989) from previous cohorts as a “good practice” can be used as feedforward and to clarify how standards and criteria can be applied. According Boud and Molloy’s aim (2013) to increase learners’ agency in assessment and feedback processes when handing in assignments, learners could be prompted by the system to include a statement evaluating their own performance against the requirements and to detail their learning processes (internal monitoring in self-

29

Linking Assessment and Learning Analytics to Support Learning. . .

767

regulated learning), which could be supplemented with external feedback via the tutor. While writing the evaluation statement, learners might reflect on their learning process and products and potentially adjust them before handing them in for final assessment. To include assessment tasks as learning tasks, as proposed by Carless (2007) to support both functions of assessment, the learning analytics system could provide several tasks related to each learning objective (from task model) and prompt the learners throughout the whole course to take the assessments. The feedback provided to the learners should include advice as to the learning objectives, where further learning is required, and should be enhanced with recommendations on relevant materials. Furthermore, a social component could be included as the system might recommend whom to ask for support with particular difficulties (Webb et al., 2018). Academic writing skills are relevant for performance in higher education (Mah & Ifenthaler, 2018) but are challenging, especially for novice learners (Wingate, 2006), who struggle with structuring or referencing. Automatic analyses using natural language processing could provide learners with immediate formative feedback on the syntax, word choice, mechanics, or citations based on rubrics and criteria (J. Wilson & Andrada, 2015). However, not only learners should use the feedback to adjust their learning – teachers should also use the evidence and feedback to adjust their teaching. If learning analytics were based on comprehensive assessments, they could provide teachers with an aggregated overview about learners’ competences and knowledge relevant for the course and either provide learners with additional resources to prepare for the required standard for successful participation or adapt the course material to the cohort’s needs. If the preknowledge is too far away from the requirements, the system could alert the teacher or the student, prompting a counseling or offering a different learning path with further preparatory courses. To guide learners’ expectations of the course in advance, the system could provide them with detailed course objectives, information about how they will be assessed, short introductory materials, and learning strategies suited to the course design plus preparatory learning offers. Furthermore, learning analytics can support teachers during the course period by enabling them to continuously monitor their students’ learning processes, progress, and needs and to use this information to adjust their instruction accordingly (Evans, 2013). A function enabling learners to rate the difficulty of the provided material or to record their need for help could be offered. Based on this evaluation or the information why learners did not successfully perform in tasks (see section “Feedback Based on Learning Analytics”), teachers could either provide additional material (e.g., videos) through the digital learning environment. Likewise, if several students face the same difficulties, teachers could recapitulate related content in the face-to-face session. Working on collaborative projects and tasks are common assessments, and collaborative learning in a supportive culture (Wiliam & Thompson, 2008) or communities of practice (Lave & Wenger, 1991) is considered to be supportive in generating, receiving, and understanding feedback (Evans, 2013). Learning analytics can support the grouping processes by recommending appropriate networks or at

768

C. Schumacher

least enable identification of the network in which learners are engaging in (Clow, 2013). Moreover, if the system allows collaboration, learning analytics enable gaining additional insights to be gained into group working and solution processes. To gain a better understanding of who contributed in which way to the solutions, the collaborative development of products or online group discussions can be analyzed. In addition, group regulation processes and the roles taken on by learners can be analyzed (Volet, Vauras, Salo, & Khosa, 2017). Furthermore, offline data could be added – for example, if students are working together in person, the system could enable all participants to log in and track their presence. If groups are facing difficulties and conflicts or are behind schedule, the teacher could provide additional support or prompt the less active group members to engage more (van Leeuwen, Janssen, Erkens, & Brekelmans, 2014). To analyze collaboration or networks within courses or distributed learning settings, social network analyses or analyses of discussion posts are often used (Ferguson & Buckingham Shum, 2012; Hernández-Garcíac, González-González, Jiménez-Zarco, & Chaparro-Peláez, 2015). The data collected could further be used to infer on learners’ collaborative skills.

Implications and Future Research The integrative framework serves as an initial description of the relationships between learning analytics, principle-based assessment design, and feedback. It needs to be taken into account that such frameworks are always a simplification of the underlying processes and concepts. Furthermore, to investigate its usefulness and evidence, the framework needs to be set into practice. In addition, several conceptual limitations and technical requirements are associated with the proposed approach. Learning analytics only provide very limited insights into learning processes (Ferguson, 2012; Winne, 2017b). To date, most digital learning environments used in higher education scarcely support actual learning; it is difficult to infer on learning processes based on the trace data collected (A. Wilson et al., 2017). Thus, digital learning environments need to offer learning opportunities and holistic systems, for example, by including tools for collaboration, communication, text processing applications, literature management systems interacting with the university’s library resources, and highlighting and annotation tools for reading materials (Schumacher & Ifenthaler, 2018a). Furthermore, cognitive processes where learners are thinking about or integrating new information cannot be tracked by learning analytics as “doing nothing in the system” could also indicate that the learner is grabbing a coffee. As most learning takes place outside the digital learning environments, solutions need to be found to integrate this data. Learners’ activities in other programs, such as text processing, the Internet browser, or communications, could be tracked and integrated, but this would raise severe privacy concerns. If the data were to be collected through learners’ self-reports on their learning behavior and their use of the materials, this would result in reduced accuracy and validity. In addition to the incompleteness, the data sets in educational settings are comparably small with

29

Linking Assessment and Learning Analytics to Support Learning. . .

769

regard to other disciplines. Hence, algorithms might not produce valid analyses calling for the need to apply different algorithmic approaches which can serve both small and large data sets (Baker, Martin, & Rossi, 2017). However, although learning analytics allow additional insights into learning, their incompleteness needs to be considered in the analytical models. To make the analyses more significant and less error-prone, inclusion and exclusion criteria for using or weighting indicators for analyses need to be defined (evidence model). For example, if a learner only pasted a response to the textbox, no information about the actual processing of the learning task can be provided (e.g., checking for mistakes, process of changing the text during writing, estimated time on task). If a learner only downloaded the material but was not online frequently, the system should enhance the data available with selfreported data investigating learners’ perceptions of their learning progress, which can be further enriched with hints to make them aware of the additional learning material. In particular, if learners with “unengaged” online behavior are successful in assessments, they might have available sufficient learning strategies to successfully learn outside the digital learning environment. Using additional inventories to assess their learning strategies might provide further insights. Hence, when teachers receive information about their students’ progress, also information about the aforementioned limitations need to be included (e.g., required data were not available, analyses based on self-reports), to prevent teachers from misinterpretation and initiating inappropriate interventions. In summary, this further highlights the need to assess a huge variety of different snippets of evidence about learning behavior in order to aggregate it to a more fine-grained picture which is, if interpreted correctly, a major benefit of learning analytics. Furthermore, as cognitive learning processes are difficult to measure using learning analytics and data to infer on learning is limited, validly designed and implemented (self-)assessments are becoming even more important especially as predictors for performance or dropout. As learning analytics aim to support learning and instructional processes, they need to provide feedback to both learners and teachers, which is currently still limited (Macfadyen & Dawson, 2012; Vieira et al., 2018). The feedback in some cases includes limited visualizations (Martin & Ndoye, 2016) and should, instead of focusing only on descriptions of performance indicators, provide recommendations for improvement (Sedrakyan et al., 2018). Furthermore, the analyses and visualizations are complex and not easy to understand (Aguilar, 2018; Greller & Drachsler, 2012), much less enabling the receivers to derive actionable knowledge and understanding of their limitedness and biases. To increase the probability of reaction, feedback needs to be presented in a way that the target groups understand (Park & Jo, 2015; Sedrakyan et al., 2018). For example, Corrin and da Barba (2014), using a qualitative approach investigating students’ interpretations of feedback (performance in assessments, frequency of access of the learning management system) provided through learning analytics dashboards, found that students reported using the feedback to adjust their learning, but could not explain how the changed behavior would impact their learning. Van Horne et al. (2018) found that the frequency of checking a learning analytics dashboard more extensively was not associated with positive effects on learning performance and suggested enhancing them by

770

C. Schumacher

prompting learners to use additional timely strategies, instead of only focusing on external feedback after learning had occurred. Thus, the feedback and digital literacy of the target groups of learning analytics need to be fostered, but also additional research is necessary on improving the feedback provided, both in terms of usability and feedback practice. Plus, learners should be encouraged to use self-regulatory strategies to create internal feedback and assess themselves to monitor their learning processes, instead of being dependent on extensive external feedback sources. Further research investigating learners’ understanding of feedback provided, through learning analytics, their knowledge about their limitedness, and their ability to use this feedback proactively to adjust their learning behavior is needed. Investigating their reactions could be enhanced by using trace data supplemented with selfreport data. In particular, teachers need to be developed further in data literacy (Greller & Drachsler, 2012; Ifenthaler, 2017; Vieira et al., 2018) so that they can act as a mediator between learning analytics and the learners as referred to by Ito (2019) as “extended intelligence.” Further research is required on how teachers understand and actually use the additional information to improve their instruction and provide additional support to learners. Research has shown that assessments are related to negative emotions of learners (Carless, 2017). As Shute et al. (2016, p. 55) state, “one risk associated with our vision (of ubiquitous assessment) is that students may come to feel as if they are constantly being evaluated, which could negatively affect their learning and possibly add stress to their lives.” Hence, learners might perceive constraints in their “natural” learning activities or their motivation. Thus, learners in higher education should be aware of and be in control of what data are collected, who has access to the data, and for which inferences which data are used (Ifenthaler & Schumacher, 2016; Pardo & Siemens, 2014; Slade & Prinsloo, 2013). As the focus of formative assessments and learning analytics is on supporting learning, students need to have the possibility to test themselves and receive formative feedback. However, traditional summative assessments could be enhanced with additional evidence from continuous data collection, especially on learning processes. But learners need to be conceded with high autonomy in choosing which assessments or data should be integrated for final grading, so as not to compromise their need for self-determination. Due to the contextualization of disclosure (Nissenbaum, 2010) and the fact that teachers would gain very deep insights into learners’ strengths and weaknesses based on cross-contextual assessments using learning analytics, the information needs, at the least, to be deidentified in order to prevent prejudices or biases which might be promoted further through the underlying algorithms (Bienkowski et al., 2012; Slade & Prinsloo, 2013; Wilson et al., 2017). Hence, further research is needed to investigate learners’ perceptions about the ongoing assessments of their learning processes and their perceived choice so that learning is not impaired or diminished. In the end, the implementation of such holistic approaches faces several challenges with regard to technology, organizational change, curriculum development, and stakeholders’ readiness. In order to realize ongoing and cross-contextual data collection and measure competences across courses, curricular changes are required.

29

Linking Assessment and Learning Analytics to Support Learning. . .

771

To derive potential learning paths, it needs to be defined how the courses relate to a certain study program and to other study programs and which prerequisites are required to study a course. Learning objectives need to be assigned to each course, to each learning resource, and to each assessment task. Following the EvidenceCentered Assessment Design, assessments need to be designed or redesigned. To meet all these requirements, standards across the institution or even across institutions need to be developed. Hence, huge curricular changes are necessary. Furthermore, the curriculum and operationalizations need to be mapped onto the learning analytics system and the underlying algorithms. Therefore, the interfaces of the institutional IT infrastructure need to be defined and implemented (e.g., curriculum profile, student profile, learning profile) (Schumacher, Klasen & Ifenthaler, 2019; Ifenthaler & Widanapathirana, 2014). Thus, to realize such institutional change processes, change management is vital, the institutional culture needs to be open to evidence-based approaches, the stakeholders need to be willing and prepared, and corresponding resources need to be available (e.g., time, infrastructure) (Author, 2019; Macfadyen, Dawson, Pardo, & Gašević, 2014; Tsai & Gašević, 2017).

Conclusion This chapter aims to synthesize assessment, feedback, and learning analytics. Therefore, theory on assessment and how assessment is implemented in higher education was described. Furthermore, assessment serves at least a formative and a summative function, which should both be used to be informative for learners as well as for learning and instructional processes. To design such informative and valid assessments supported through technology, principle-based assessment designs are considered to be a promising approach. However, for assessments to be informative, the results and derived recommendations for improvement need to be provided to learners and teachers as feedback, feedforward, or feedup. To be effective, this feedback needs to consider learners’ prerequisites, such as prior knowledge, motivational and emotional states, or learning goals. Due to the increased use of digital learning environments, the possibilities of learning analytics can provide information about learners’ prerequisites and can be applied to analyze learning processes and environments aiming at supporting and optimizing them. However, learning analytics still suffer from a lack of theoretical foundation and empirical evidence. In increasingly complex learning environments, various cognitive, metacognitive, and non-cognitive skills are required for problem-solving, which makes it difficult to infer from learners’ behavior to the skills in focus (Baker et al., 2017). The underlying constructs plus the associated behaviors of learners are overlapping (Webb et al., 2018). Furthermore, related assessments are still limited and lacking empirical evidence (Shute et al., 2016). Hence, combining the possibilities of learning analytics approaches with a principle-based assessment design can serve as a basis, as it aims to define the interrelatedness of the constructs and consider how the behavioral evidence and performance can be accumulated and related to each assessment target. Furthermore, this approach allows learning analytics to be better

772

C. Schumacher

embedded into theory of learning and cognition which would support its validity and might increase its evidence. By developing an integrative framework, aimed at synthesizing current perspectives on assessment, feedback, and learning analytics, and by providing some exemplary implementations, this aim was promoted further. However, even though learning analytics enable broad support to be offered to learners, it must be borne in mind that the learners are the agents for their learning processes. As such, their digital and feedback literacy needs to be fostered to enable them to evaluate the validity of the results and decide how to react to the feedback provided. Hence, learning analytics should provide additional information to support evidence-based decisions and increase self-reflection and awareness.

References AERA, APA, & NCME. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Aguilar, S. J. (2018). Learning analytics: At the nexus of big data, digital innovations, and social justice in education. TechTrends, 62, 37–45. https://doi.org/10.1007/s11528-017-0226-9 Aldowah, H., Al-Smarraie, H., & Fauzy, W. M. (2019). Educational data mining and learning analyitics for 21st century higher education: A review and synthesis. Telematics and Informatics, 37, 13–49. https://doi.org/10.1016/j.tele.2019.01.007 Aljohani, N. R., Daud, A., Abbasi, R. A., Alowibdi, J. S., Basheri, M., & Aslam, M. A. (2019). An integrated framework for course adapted student learning analytics dashboard. Computers in Human Behavior, 92, 679–690. https://doi.org/10.1016/j.chb.2018.03.035 Almond, R. G. (2010). Using evidence centered design to think about assessments. In V. J. Shute & B. J. Becker (Eds.), Innovative assessment for the 21st century. Supporting educational needs (pp. 75–100). New York, NY: Springer. Baker, R. S., Martin, T., & Rossi, L. M. (2017). Educational data mining and learning analytics. In A. A. Rupp & J. P. Leighton (Eds.), The handbook of cognition and assessment: Frameworks, methodologies, and applications (pp. 379–396). Chichester, WSX: Wiley. Bannert, M. (2009). Promoting self-regulated learning through prompts. Zeitschrift für Pädagogische Psychologie, 23(2), 139–145. Bearman, M., Dawson, P., Boud, D., Bennett, S., Hall, M., & Molloy, E. (2016). Support for assessment practice: Developing the assessment design decisions framework. Teaching in Higher Education, 21(5), 545–556. https://doi.org/10.1080/13562517.2016.1160217 Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18(1), 5–25. https://doi.org/10.1080/0969594X.2010.513678 Bevitt, S. (2015). Assessment innovation and student experience: A new assessment challenge and call for a multi-perspecitve approach to assessment research. Assessment and Evaluation in Higher Education, 40(1), 103–119. https://doi.org/10.1080/02602938.2014.890170 Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analyitcs: An issue brief. Washington, DC: Office of Educational Technology. Black, P. (2013). Formative and summative aspects of assessment: Theoretical and research foundations in the context of pedagogy. In J. H. McMillan (Ed.), SAGE handbook of research on classroom assessment (pp. 167–178). Thousandsand Oaks, CA: SAGE. Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning. putting it into practice. Maidenhead, UK: Open University Press.

29

Linking Assessment and Learning Analytics to Support Learning. . .

773

Black, P., McCormick, R., James, M., & Pedder, D. (2006). Learning how to learn and assessment for learning: A theoretical inquiry. Research Papers in Education, 21(2), 119–132. https://doi. org/10.1080/02671520600615612 Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21, 5–15. https://doi.org/10.1007/s11092-0089068-5 Black, P., & Wiliam, D. (2018). Classroom assessment and pedagogy. Assessment in Education: Principles, Policy & Practice, 25(6), 551–575. https://doi.org/10.1080/0969594X.2018.1441807 Bosse, E. (2015). Exploring the role of student diversity for the first-year experience. Zeitschrift für Hochschulentwicklung, 10(4), 45–66. Boud, D. (2007). Reframing assessment as if learning were important. In D. Boud & N. Falchikov (Eds.), Rethinking assessment in higher education (pp. 14–25). London, UK: Routledge. Boud, D., & Falchikov, N. (2007). Assessment for the longer term. In D. Boud & N. Falchikov (Eds.), Rethinking assessment in higher education (pp. 3–13). New York, NY: Routledge. Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment and Evaluation in Higher Education, 38(6), 698–712. https://doi.org/ 10.1080/02602938.2012.691462 Broadbent, J., Panadero, E., & Boud, D. (2017). Implementing summative assessment with a formative flavour: A case study in a large class. Assessment and Evaluation in Higher Education, 43(2), 307–322. https://doi.org/10.1080/02602938.2017.1343455 Brooks, C., & Thompson, C. (2017). Predictive modelling in teaching and learning. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of learning analytics (pp. 61–68). SOLAR, Society for Learning Analytics and Research. https://www.solaresearch.org/wp-con tent/uploads/2017/05/hla17.pdf Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281. Carless, D. (2007). Learning-oriented assessment: Conceptual bases and practical implications. Innovations in Education and Teaching International, 44(1), 57–66. https://doi.org/10.1080/ 14703290601081332 Carless, D. (2017). Scaling up assessment for learning: Progress and prospects. In D. Carless, S. M. Bridges, C. K. Y. Chan, & R. Glofcheski (Eds.), Scaling up assessment for learning in higher education (pp. 3–17). Singapore, Singapore: Springer. Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment and Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/ 10.1080/02602938.2018.1463354 Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developping sustainable feedback practices. Studies in Higher Education, 36(4), 395–407. https://doi.org/10.1080/03075071003642449 Cartney, P. (2010). Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used. Assessment and Evaluation in Higher Education, 35(5), 551–564. https://doi.org/10.1080/02602931003632381 Cassidy, S. (2006). Developing employability skills: Peer assessment in higher education. Education and Training, 48(7), 508–517. https://doi.org/10.1108/00400910610705890 Cassidy, S. (2011). Self-regulated learning in higher education: Identifying key component processes. Studies in Higher Education, 36(8), 989–1000. Chen, X., Breslow, L., & DeBoer, J. (2018). Analyzing productive learning behaviors for students using immediate corrective feedback in a blended learning environment. Computers & Education, 117, 59–74. https://doi.org/10.1016/j.compedu.2017.09.013 Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6), 683–695. https://doi.org/10.1080/13562517.2013.827653 Corrin, L., & da Barba, P. (2014). Exploring students’ interpretation of feedback delivered through learning analytics dashboards. In B. Hegarty, J. McDonald, & S.-K. Loke (Eds.), Rethoric and reality: Critical perspectives on educational technology. Proceedings ascilite Dunedin 2014 (pp. 629–633). Dunedin, New Zealand.

774

C. Schumacher

Daniel, B. (2015). Big data and analytics in higher education: Opportunities and challenges. British Journal of Educational Technology, 46(5), 904–920. https://doi.org/10.1111/bjet.12230 Deci, E. L. (1992). The relation of interest to the motivation of behavior: A self-determination theory perspective. In K. A. Renninger, S. Hidi, & A. Krapp (Eds.), The role of interest in learning and development (pp. 43–70). Hillsdale, NJ: Lawrence Erlbaum Associates. Draper, S. W. (2009). What are learners actually regulating when given feedback? British Journal of Educational Technology, 40(2), 306–315. https://doi.org/10.1111/j.1467-8535.2008.00930.x Ellis, C. (2013). Broadening the scope and increasing the usefulness of learning analytics: The case for assessment analytics. British Journal of Educational Technology, 44(4), 662–664. https:// doi.org/10.1111/bjet.12028 Ellis, R. A., Han, F., & Pardo, A. (2017). Improving learning analytics – Combining observational and self-report data on student learning. Educational Technology & Society, 20(3), 158–169. Evans, C. (2013). Making sense of assessment feedback in higher education. Review of Educational Research, 83(1), 70–120. https://doi.org/10.3102/0034654312474350 Falchikov, N. (2005). Improving assessment through student involvement. Practical solutions for aiding learning in higher and further education. Abingdon, OX: Routledge. Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304–317. Ferguson, R., & Buckingham Shum, S. (2012). Social learning analytics: Five approaches. In Proceedings of the 2nd international conference on learning analytics and knowledge (LAK) (pp. 22–33). Vancouver, CA: ACM. Ferguson, R., & Clow, D. (2017). Where is the evidence? A call to action for learning analytics. In LAK ‘17 proceedings of the seventh international learning analytics & knowledge conference (pp. 56–65). New York, NY: ACM. Fogarty, R. J., & Kerns, G. M. (2009). inFormative assessment: When It’s not about a grade. Thousand Oaks, CA: Corwin. Forster, M. (2009). Informative assessment: Understanding and guiding learning. Paper presented at the ACER research conference: Assessment and student learning, Perth, WA. Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. Internet and Higher Education, 28, 68–84. Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71. https://doi.org/10.1007/s11528-014-0822-x Gašević, D., Jovanovic, J., Pardo, A., & Dawson, S. (2017). Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics, 4(2), 113–128. https://doi.org/10.18608/jla.2017.42.10 Gibbs, G., & Simpson, C. (2005). Conditions under which assessment supports students’ learning. Learning and Teaching in Higher Education, 1, 3–31. Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Educational Technology & Society, 15(3), 42–57. Hargreaves, E. (2007). The validity of collaborative assessment for learning. Assessment in Education: Principles, Policy & Practice, 14(2), 185–199. https://doi.org/10.1080/09695940701478594 Hattie, J. A. C., & Clarke, S. (2019). Visible learning: Feedback. New York, NY: Routledge. Hattie, J. A. C., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487 Hernández-Garcíac, Á., González-González, I., Jiménez-Zarco, A. I., & Chaparro-Peláez, J. (2015). Applying social learning analytics to message boards in online distance learning: A case study. Computers in Human Behavior, 47, 68–80. https://doi.org/10.1016/j.chb.2014.10.038 Howell, J. A., Roberts, L. D., & Mancini, V. O. (2018). Learning analytics messages: Impact of grade, sender, comparative information and message style on student affect and academic resilience. Computers in Human Behavior, 89, 8–15. https://doi.org/10.1016/j.chb.2018.07.021 Hsu, Y.-S., Wang, C.-Y., & Zhang, W.-X. (2017). Supporting technology-enhanced inquiry through metacognitive and cognitive prompts: Sequential analysis of metacognitive actions in response

29

Linking Assessment and Learning Analytics to Support Learning. . .

775

to mixed prompts. Computers in Human Behavior, 72, 701–712. https://doi.org/10.1016/j. chb.2016.10.004 Ifenthaler, D. (2015). Learning analytics. In J. M. Spector (Ed.), The Sage encyclopedia of educational technology (Vol. 2, pp. 447–451). Los Angeles, CA: SAGE. Ifenthaler, D. (2017). Are higher education institutions prepared for learning analytics? TechTrends, 61(4), 366–371. https://doi.org/10.1007/s11528-016-0154-0 Ifenthaler, D. (2019). Learning analytics and study success. Current landscape of learning analytics research. Paper presented at the innovations in education: Opportunities and challenges of digitization research workshop, Mannheim, BW. Ifenthaler, D., & Schumacher, C. (2016). Student perceptions of privacy principles for learning analytics. Educational Technology Research and Development, 64(5), 923–938. https://doi.org/ 10.1007/s11423-016-9477-y Ifenthaler, D., Greiff, S., & Gibson, D. C. (2018). Making use of data for assessments: Harnessing analytics and data science. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), International handbook of information Technology in Primary and Secondary Education (pp. 649–663). New York, NY: Springer. Ifenthaler, D., & Widanapathirana, C. (2014). Development and validation of a learning analytics framework: Two case studies using support vector machines. Technology, Knowledge and Learning, 19(1–2), 221–240. https://doi.org/10.1007/s10758-014-9226-4 Ito, J. (2019). Forget about artificial intelligence, extended intelligence is the future. Retrieved from https://www.wired.co.uk/article/artificial-intelligence-extended-intelligence Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions of performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254–284. Knight, P. (2006). The local practice of assessment. Assessment and Evaluation in Higher Education, 31(4), 435–452. https://doi.org/10.1080/02602930600679126 Kramarski, B., & Kohen, Z. (2017). Promoting preservice teachers’ dual self-regulation roles as learners and as teachers: Effects of generic vs. specific prompts. Metacognition and Learning, 12, 157–191. https://doi.org/10.1007/s11409-016-9164-8 Kulik, J. A., & Kulik, C.-L. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79. Lave, J., & Wenger, E. (1991). Situated learning. Legitimate peripheral participation. Cambridge, UK: Cambridge University Press. Liu, M., Kang, J., Zou, W., Lee, H., Pan, Z., & Corliss, S. (2017). Using data to understand how to better design adaptive learning. Technology, Knowledge and Learning, 22, 271–298. https://doi. org/10.1007/s10758-017-9326-z Lonn, S., Aguilar, S. J., & Teasley, S. D. (2015). Investigating student motivation in the context of learning analytics intervention during a summer bridge program. Computers in Human Behavior, 47, 90–97. https://doi.org/10.1016/j.chb.2014.07.013 Luecht, R. M. (2013). An introduction to assessment engineering for automatic item generation. In M. J. Gierl & T. M. Haladyna (Eds.), Automatic item generation: Theory and practice (pp. 59–76). New York, NY: Routledge. Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54, 588–599. Macfadyen, L. P., & Dawson, S. (2012). Numbers are not enough. Why e-learning analytics failed to inform an institutional strategic plan. Educational Technolgy & Society, 15(3), 149–163. Macfadyen, L. P., Dawson, S., Pardo, A., & Gašević, D. (2014). Embracing big data in complex educational systems: The learning analytics imperative and the policy challenge. Research and Practice in Assessment, 9, 17–28. Mah, D.-K., & Ifenthaler, D. (2018). Students’ perceptions toward academic competencies: The case of German first-year students. Issues in Educational Research, 28(1), 120–137. Martin, F., & Ndoye, A. (2016). Using learning analytics to assess student learning in online courses. Journal of University Teaching & Learning Practice, 13(3), Art. 7.

776

C. Schumacher

Martin, T., & Sherin, B. (2013). Learning analytics and computational techniques for detecting and evaluating patterns in learning: An introduction to the special issue. Journal of the Learning Sciences, 22(4), 511–520. https://doi.org/10.1080/10508406.2013.840466 Marzouk, Z., Rakovic, M., Liaqat, A., Vytasek, J., Samadi, D., Stewart-Alonso, J., . . . Nesbit, J. C. (2016). What if learning analytics were based on learning science? Australasian Journal of Educational Technology, 32(6), 1–18. https://doi.org/10.14742/ajet.3058. Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design. ETS Report Series, 2003(1), i–29. Mislevy, R. J., & Haertel, G. D. (2006). Implications of evidence-centered design for educational testing. Educational Measurement: Issues and Practice, 25(4), 6–20. https://doi.org/10.1111/ j.1745-3992.2006.00075.x Mislevy, R. J., & Riconscente, M. M. (2005). Evidence-centered assessment design: Layers, structures, and terminology. PADI Technical Report, 2005(9). Müller, N. M., & Seufert, T. (2018). Effects of self-regulation prompts in hypermedia learning on learning performance and self-efficacy. Learning and Instruction, 58, 1–11. https://doi.org/ 10.1016/j.learninstruc.2018.04.011 Narciss, S. (2008). Feedback strategies for interactive learning tasks. In J. M. Spector, M. D. Merrill, J. J. G. van Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (pp. 125–144). New York, NY: Lawrence Erlbaum Associates. Narciss, S. (2012). Feedback in instructional settings. In N. M. Seel (Ed.), Encyclopedia of the sciences of learning (pp. 1285–1289). Berlin, Germany: Springer. Narciss, S. (2017). Conditions and effects of feedback viewed through the lens of the interactive tutoring feedback model. In D. Carless, S. M. Bridges, C. K. Y. Chan, & R. Glofcheski (Eds.), Scaling up assessment for learning in higher education (pp. 173–189). Singapore, Singapore: Springer. Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., & Melis, E. (2014). Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers and Education, 71, 56–76. https://doi.org/10.1016/j.compedu.2013.09.011 Neumann, R., Parry, S., & Becher, T. (2002). Teaching and learning in their disciplinary contexts: A conceptual analysis. Studies in Higher Education, 27(4), 405–417. https://doi.org/10.1080/ 0307507022000011525 Nichols, P. D., Kobrin, J. L., Lai, E., & Koepfler, J. (2017). The role of theories of learning and cognition in assessment design and development. In A. A. Rupp & J. P. Leighton (Eds.), The handbook on cognition and assessment. frameworks, methodologies, and applications (pp. 15–40). Chichester, UK: Wiley. Nicol, D. J. (2009). Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies. Assessment and Evaluation in Higher Education, 34(3), 335–352. https://doi.org/10.1080/02602930802255139 Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles for good feedback practice. Studies in Higher Education, 31(2), 199–218. Nissenbaum, H. (2010). Privacy in context: Technology, policy, and the integrity of social life. Stanford, CA: Stanford University Press. Nistor, N., & Hernández-Garcíac, Á. (2018). What types of data are used in learning analytics? An overview of six cases. Computers in Human Behavior, 89, 335–338. https://doi.org/10.1016/j. chb.2018.07.038 NRC. (1996). National science education standards. Washington, DC: National Academy Press. Panadero, E., Broadbent, J., Boud, D., & Lodge, J. M. (2018). Using formative assessment to influence self- and co-regulated learning: The role of evaluative judgement. European Journal of Psychology of Education, 34, 535. https://doi.org/10.1007/s10212-018-0407-8 Panadero, E., Jonsson, A., & Botella, J. (2017). Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educational Research Review, 22, 74–98. https://doi.org/ 10.1016/j.edurev.2017.08.004

29

Linking Assessment and Learning Analytics to Support Learning. . .

777

Papamitsiou, Z., & Economides, A. A. (2014). Learning analyitics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64. Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalized feedback. British Journal of Educational Technology, 50 (1), 128–138. https://doi.org/10.1111/bjet.12592 Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438–450. Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-regulated laearning. Educational Psychologist, 36(2), 89–101. Park, Y., & Jo, I.-H. (2015). Development of the learning analytics dashboard to support students’ learning performance. Journal of Universal Computer Science, 21(1), 110–133. Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: The National Academies Press. Pereira, D., Assunção Flores, M., & Niklasson, L. (2016). Assessment revisited: A review of research in assessment and evaluation in higher education. Assessment & Evaluation in Higher Education, 41(7), 1008–1032. Piaget, J. (1975). L’Equilibration des Structures Cognitives. Problème Central du Développement. Paris, France: Presses Universitaires de France. Pinheiro Cavalcanti, A., Rolim, V., André, M., Freitas, F., Ferreira, R., & Gašević, D. (2019). An analysis of the use of good feedback practices in online learning courses. Paper presented at the IEEE international conference on advanced learning technologies and technology-enhanced learning (ICALT), Maceió, Brazil. Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). San Diego, CA: Academic Press. Prieger, E., & Bannert, M. (2018). Differential effects of students’ self-directed metacognitive prompts. Computers in Human Behavior, 86, 165–173. https://doi.org/10.1016/j. chb.2018.04.022 Ramaprasad, A. (1983). On the definition of feedback. Behavioral Science, 28, 4–13. Roberts, L. D., Howell, J. A., & Seaman, K. (2017). Give me a customizable dashboard: Personalized learning analytics dashboards in higher education. Technology, Knowledge and Learning, 22, 317–333. https://doi.org/10.1007/s10758-017-9316-1 Romero, C., & Ventura, S. (2013). Data mining in education. WIREs Data Mining and Knowledge Discovery, 3(January/February), 12–27. https://doi.org/10.1002/widm.1075 Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144. Sadler, D. R. (1998). Formative assessment: Revisiting the territory. Assessment in Education: Principles, Policy & Practice, 5(1), 77–84. https://doi.org/10.1080/0969595980050104 Sadler, D. R. (2010a). Assessment in higher education. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (3rd ed., pp. 249–255). Oxford, UK: Academic Press. Sadler, D. R. (2010b). Beyond feedback: Developing student capability in complex appraisal. Assessment and Evaluation in Higher Education, 35(5), 535–550. https://doi.org/10.1080/ 02602930903541015 Schroth, M. L. (1992). The effects of delay of feedback on a delayed concept formation transfer task. Contemporary Educational Psychology, 17, 78–82. Schunk, D. H. (2008). Attributions as motivators of self-regulated learning. In D. H. Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning. Theory, research, and applications (pp. 245–266). New York, NY: Routledge. Schumacher, C., & Ifenthaler, D. (2018a). Features students really expect from learning analytics. Computers in Human Behavior, 78, 397–407. https://doi.org/10.1016/j.chb.2017.06.030

778

C. Schumacher

Schumacher, C., & Ifenthaler, D. (2018b). The importance of students’ motivational dispositions for designing learning analytics. Journal of Computing in Higher Education, 30(3), 599–619. https://doi.org/10.1007/s12528-018-9188-y Schumacher, C., & Ifenthaler, D. (under review). Designing effective means of supporting students’ regulation of learning processes through analytics-based prompts. Schumacher, C., Klasen, D., & Ifenthaler, D. (2019). Implementation of a learning analytics system in a productive higher education environment. In M. S. Khine (Ed.), Emerging Trends in Learning Analytics. Leveraging the Power of Educational Data (pp. 177–199). Leiden: Brill. Sclater, N., Peasgood, A., & Mullan, J. (2016). Learning analytics in higher education. A review of UK and international practice. Retrieved from https://www.jisc.ac.uk/sites/default/files/learn ing-analytics-in-he-v3.pdf Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2018). Linking learning behavior analytics and science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior. https://doi.org/10.1016/j. chb.2018.05.004 Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795 Shute, V. J., & Becker, B. J. (2010). Prelude: Assessment for the 21st century. In V. J. Shute & B. J. Becker (Eds.), Innovative assessment for the 21st century. Supporting educational needs (pp. 1–11). New York, NY: Springer. Shute, V. J., Leighton, J. P., Jang, E. E., & Chu, M.-W. (2016). Advances in the science of assessment. Educational Assessment, 21(1), 34–59. https://doi.org/10.1080/10627197.2015.1127752 Shute, V. J., Rahimi, S., & Emihovich, B. (2017). Assessment for learning in immersive environments. In D. Liu, C. Dede, R. Huang, & J. Richards (Eds.), Virtual, augmented, and mixed realities in education (pp. 71–87). Singapore, Singapore: Springer. Siemens, G. (2010). 1st international conference on learning analytics & knowledge 2011. Retrieved from https://tekri.athabascau.ca/analytics/ Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510–1529. Smith, G. (2007). How does student performance on formative assessments relate to learning assessed by exams? Journal of College Science Teaching, 36(7), 28–34. Sønderlund, A. L., Hughes, E., & Smith, J. (2019). The efficacy of learning analytics interventions in higher education: A systematic review. British Journal of Educational Technology, 50(5), 2594–2618. https://doi.org/10.1111/bjet.12720 Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: Enabling students to make decisions about the quality of work. Higher Education, 76(3), 467–481. https://doi.org/10.1007/s10734-017-0220-3 Tolstrup Holmegaard, H., Møller Madsen, L., & Ulriksen, L. (2017). Why should European higher education care about the retention of non-traditional students? European Educational Research Journal, 16(1), 3–11. https://doi.org/10.1177/1474904116683688 Tsai, Y.-S., & Gašević, D. (2017). Learning analytics in higher education – Challenges and policies: A review of eight learning analytics policies. In Proceedings of the seventh international learning analytics & knowledge conference (pp. 233–242). New York, NY: ACM. Van Horne, S., Curran, M., Smith, A., VanBuren, J., Zahrieh, D., Larsen, R., & Miller, R. (2018). Facilitating student success in introductory chemistry with feedback in an online platform. Technology, Knowledge and Learning, 23, 21–40. https://doi.org/10.1007/s10758-017-9341-0 van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2014). Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL. Computers and Education, 79, 28–39. https://doi.org/10.1016/j.compedu.2014.07.007 Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500–1509.

29

Linking Assessment and Learning Analytics to Support Learning. . .

779

Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., & Klerkx, J. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514. Verbert, K., Manouselis, N., Drachsler, H., & Duval, E. (2012). Dataset-driven research to support learning and knowledge analytics. Educational Technology & Society, 15(3), 133–148. Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110. https://doi.org/ 10.1016/j.chb.2018.07.027 Vieira, C., Parsons, P., & Byrd, V. (2018). Visual learning analytics of educational data: A systematic literature review and research agenda. Computers & Education, 122, 119–135. https://doi.org/10.1016/j.compedu.2018.03.018 Volet, S., Vauras, M., Salo, A.-E., & Khosa, D. (2017). Individual contributions in student-led collaborative learning: Insights from two analytical approaches to explain the quality of group outcome. Learning and Individual Differences, 53, 79–92. https://doi.org/10.1016/j. lindif.2016.11.006 Webb, M., Gibson, D., & Forkosh-Baruch, A. (2013). Challenges for information technology supporting educational assessment. Journal of Computer Assisted Learning, 29, 451–462. https://doi.org/10.1111/jcal.12033 Webb, M., & Ifenthaler, D. (2018). Assessment as, for, and of twenty-first-century learning using information technology: An overview. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), Handbook of information technology in primary and secondary education. Cham, Switzerland: Springer. Webb, M., Prasse, D., Philipps, M., Kadijevich, D., Angeli, C., Strijker, A., . . . Laugesen, H. (2018). Challenges for IT-enabled formative assessment of complex 21st century skills. Technology, Knowledge and Learning, 23, 441–456. https://doi.org/10.1007/s10758-018-9379-7. Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92(4), 548–573. Weinstein, C. E., & Mayer, R. E. (1986). The teaching of learning strategies. In M. C. Wittrock (Ed.), Handbook of research on teaching (pp. 315–327). New York, NY: Macmillan. West, D., Heath, D., & Huijser, H. (2016). Let’s talk learning analytics: A framework for implementation in relation to student retention. Online Learning, 20(2), 1–21. Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37, 3–14. https://doi.org/10.1016/j.stueduc.2011.03.001 Wiliam, D., & Black, P. (1996). Meanings and consequences: A bias for distinguishing formative and summative functions of assessment? British Educational Research Journal, 22(5), 537–548. Wiliam, D., & Thompson, M. (2008). Integrating assessment with learning: What will it take to make it work? In C. A. Dwyer (Ed.), The future of assessment. Shaping teaching and learning. New York, NY: Lawrence Erlbaum Associates. Wilson, A., Watson, C., Thompson, T. L., Drew, V., & Doyle, S. (2017). Learning analytics: Challenges and limitations. Teaching in Higher Education, 22(8), 991–1007. Wilson, J., & Andrada, G. N. (2015). Using automated feedback to improve writing quality: Opportunities and challenges. In Y. Rosen, S. Ferrara, & M. Mosharraf (Eds.), Handbook of research on technology tools for real-world skill development (pp. 678–703). Hershey, PA: IGI Global. Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Lawrence Erlbaum. Wingate, U. (2006). Doing away with ‘study skills’. Teaching in Higher Education, 11(4), 457–469. https://doi.org/10.1080/13562510600874268 Winne, P. H. (2011). A cognitive and metacognitive analysis of self-regulated learning. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 15–32). New York, NY: Routledge.

780

C. Schumacher

Winne, P. H. (2017a). Cognition and metacognition within self-regulated learning. In P. A. Alexander, D. H. Schunk, & J. A. Greene (Eds.), Handbook of self-regulation of learning and performance (pp. 36–48). New York, NY: Routledge. Winne, P. H. (2017b). Learning analytics for self-regulated learning. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of learning analytics (1st ed., pp. 241–249). SOLAR, Society for Learning Analytics and Research. https://www.solaresearch.org/wp-content/ uploads/2017/05/hla17.pdf Winne, P. H., & Baker, R. S. J. D. (2013). The potentials of educational data mining for researching metacognition, motivation and self-regulated learning. Journal of Educational Data Mining, 5(1), 1–8. Wong, J., Baars, M., de Koning, B. B., van der Zee, T., Davis, D., Khalil, M., . . . Paas, F. (2019). Educational theories and learning analytics: From data to knowledge. The whole is greater than the sum of its parts. In D. Ifenthaler, D.-K. Mah, & J. Y.-K. Yau (Eds.), Utilizing learning analytics to support study success (pp. 3–25). Cham, Switzerland: Springer. Xiong, W., Litman, D., & Schunn, C. (2012). Natural language processing techniques for researching and improving peer feedback. Journal of Writing Research, 4(2), 155–176. Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13–39). San Diego, CA: Academic Press.

Clara Schumacher is a research assistant at the chair of Learning, Design, and Technology at the University of Mannheim, Germany. She holds a diploma in educational science, with minor in psychology, and focuses on adult education and work psychology. Clara’s research focuses on educational technology, learning analytics, and self-regulated learning.

Section IV Innovative Design and Development Approaches

Innovative Design and Development Approaches: A Section Introduction

30

Lin Lin-Lipsmeyer and Bernadette Sibuma

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Design Thinking and Design Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bridging Research and Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual Reality Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Professional Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

784 784 785 786 787 787

Abstract

This short chapter is the section introduction for the handbook section “Innovative Design and Development Approaches” in the handbook Learning, Design, and Technology – An International Compendium of Theory, Research, Practice, and Policy. This introduction outlines and summarizes the 15 chapters in this handbook section. Keywords

Innovative design and development · Immersive learning technologies · Learning sciences · Design thinking · Design research · Professional development

L. Lin-Lipsmeyer (*) Simmons School of Education and Human Development, Southern Methodist University, Dallas, TX, USA e-mail: [email protected] B. Sibuma Massachusetts Bay Community College, Wellesley Hills, MA, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_131

783

784

L. Lin-Lipsmeyer and B. Sibuma

Introduction Educational technologists and learning scientists are experiencing new interest in how designers form designs and what they know that makes them experts in their trade. What is on a designer’s mind as the technologies change and designs evolve? What does a designer need to know to function in new entrepreneurial and commercial venues of design and within goal-oriented organizations? What research is needed on designing and how designs emerge? What do design fields outside of instructional design already know about designing that can inform the field? How do those outside the field view instructional design practices? How can we help new designers to form their own understanding of how to design without constraining their future conceptual development as designers? What questions are we not yet asking? Scholarly conversation is beginning to focus on alternative conceptions of the design, the designer, the design team, and design processes that match actual designer behavior. Questions have arisen of how to translate research findings into designs using illustrative case studies. Design-based research has emerged, begging the question of the design-to-theory relationship. This section on “Innovative Design and Development Strategies” focuses on knowledge (i.e., skills, explicit knowledge, concepts, attitudes, dispositions, and tacit knowledge) that a designer calls upon, or might be prepared to call upon, while designing in current contexts. The volume includes disciplinary explorations, innovative approaches, virtual reality considerations, and professional development considerations. Below is a sampling of the content and chapters included in this volume.

Defining Design Thinking and Design Research As a potential starting point, ▶ Chap. 31, “Design Thinking: Towards the Construction of Knowledge,” Hokanson and Nyboer, provides a compelling historical and comparative background on the development of design thinking relative to other disciplines and educational approaches. By walking through similarities and differences in practices and processes, the author communicates the subtleties and nuances of design thinking in an accessible way. Then, in ▶ Chap. 32, “Expanding Design Research: From Researcher EgoSystems to Stakeholder Ecosystems,” Zuiker and colleagues characterize design research approaches that have been established in the learning sciences as compared to educational research, learning sciences, and educational technology more specifically. These approaches are contrasted in relation to answers to three main questions: who forms a design and how do they go about doing it, how are knowledge and expertise of stakeholders mobilized, and what is the ideal impact with respect to both educational change and theoretical refinement?

30

Innovative Design and Development Approaches: A Section Introduction

785

In ▶ Chap. 33, “Design Thinking, Designerly Ways of Knowing, and Engaged Learning,” Donaldson and Smith distinguish between design-thinking approaches that focus on the process and those that focus on the mindset of a designer. The authors propose a framework which integrates aspects of process, mindset, and constructionist principles to create an approach to design that includes framing, idea exploration, and prototyping and iteration.

Bridging Research and Practices Additional innovative approaches embedded in this volume focus on learning science concepts such as embodied learning, personalized learning, collaborative creativity, problem-based learning, critical thinking, and activity theory, and how these theoretical frameworks can be implemented in practices. In ▶ Chap. 34, “Considerations for the Design of Gesture-Augmented Learning Environments,” Wallon and Lindgren introduce embodied learning theory and how using a design approach that incorporates learner’s physical movement – in this case, gestures – as a mode of interaction enhances science learning. Previous research has shown a strong connection between student gestures and depth of learning; however, these gestures occurred normally during discourse. The authors suggest that having students show their scientific understanding through gesturing will foster greater scientific conceptual development. In ▶ Chap. 35, “Personalizing Flipped Instruction to Enhance EFL Learners’ Idiomatic Knowledge and Oral Proficiency,” Wu and colleagues examined the use of idioms and flipped instructional design in students’ learning. The results of their study revealed that the personalized flipped instructional design helped motivate students and enhance their idiomatic knowledge and oral proficiency. In ▶ Chap. 36, “Designing for Collaborative Creativity in STEM Education with Computational Media,” Sullivan and Barbosa present their approach for designing experiences that enhance the development of collaborative creativity. Sharing empirical research findings about the ways in which collaborative learning interactions with computational media (LEGO robotics and Scratch, specifically), they provide support for this approach, arguing that such collaborative skills will be needed as interdisciplinary work teams and convergence continue to define the workforce of the future. In ▶ Chap. 37, “Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort Degree Program,” Ali and colleagues share a case study of a transformational integrated undergraduate cohort-based BS degree program built on the foundation of problem-based learning. The authors identify five pillars of the program, including (1) cohort-based and problem-based learning as the pedagogy, (2) partnership with industry organizations to provide hands-on experience for students, (3) integration of curriculum between all cohort-based classes, (4) internal partnership with student services to get students ready with life skills, and finally

786

L. Lin-Lipsmeyer and B. Sibuma

(5) organizational partnerships to provide three internships for students before graduation. The chapter shares the journey from concept of the program to the completion of the first year, along with what program components worked and did not work. In ▶ Chap. 38, “Toward a Systematic and Model-Based Approach to Design Learning Environments for Critical Thinking,” Tiruneh and colleagues argue that embedding critical thinking instruction in domain-specific courses requires greater clarity about what critical thinking is, what set of critical thinking skills could be targeted, and how specific subject-matter instruction could systematically be designed as an integral part of domain-specific instruction. ▶ Chapter 39, “Design of Innovative Learning Environment: An Activity System Perspective,” by Liu reviews activity theory and how it can be used as an approach to analyzing and designing an innovative learning environment. That is, by examining a learning environment from an activity system perspective, the author makes recommendations for instructional design.

Virtual Reality Considerations As virtual, augmented, and simulated reality environments enter mainstream user interfaces, their use and effectiveness as learning environments remain in the exploratory stage. In ▶ Chap. 40, “A Process Method Approach to Study the Development of Virtual Research Environments: A Theoretical Framework,” Ahmed and Poole propose a conceptual framework for understanding the processes by which virtual research environments (VREs) are developed over time and how these processes contribute to their effectiveness. Models and tracks are reviewed as they pertain to VRE development in the hopes of identifying key elements that might contribute to the success or failure of VREs as they contribute to learning and scientific inquiry. In the context for second language acquisition, the chapter by Wong and Notari reviews the development of virtual, augmented, and mixed reality environments and discusses the affordances of these environments to support immersive language learning. In ▶ Chap. 41, “Exploring Immersive Language Learning Using Virtual Reality,” the authors share two projects that illustrate these affordances in terms of reading, writing, and speaking. As well, a case study from medical education provides an end-to-end account of the development of a virtual patient simulation. In ▶ Chap. 42, “NERVE, InterPLAY, and Design-Based Research: Advancing Experiential Learning and the Design of Virtual Patient Simulation,” Hirumi and colleagues document the design, development, and testing of the virtual patient environment and its impact on medical students’ ability to examine, interview, and diagnose patients. Reflections and considerations from design team members as they developed the system help shed light on the skills, knowledge, and dispositions called upon during the design process and the key lessons learned by team members.

30

Innovative Design and Development Approaches: A Section Introduction

787

Professional Development Several chapters explored approaches toward professional development. In ▶ Chap. 43, “Instructional Design as a Moral Ecology of Practice: Implications for Competency Standards and Professional Identity,” Yanchar examines instructional design from a philosophical perspective and suggests how the development of competency standards has the potential to advance instructional designers’ sense of professional identity. Huang’s ▶ Chap. 44, “Narrative or Expository Video Cases: Exploring the Influence of Video Cases on Junior Staff’s Attitude and Reflection.” The narrative approach involved story telling while the expository approach explicitly presented facts or information, leaving the participants to agree or disagree. The results of Huang’s study indicate that the narrative video type produced greater improvement in the participants’ attitude toward their current careers and in their reflection levels than did the expository video type. The findings offer practical and innovative guidance on the pedagogical use of video in the context of career development training. As an approach to working with subject-matter experts, Figliotti and colleagues describe how they facilitate innovation workshops to help inspire creativity in educators and scientists in their construction of innovative learning environments. ▶ Chapter 45, “Learning Environments for Academics: Reintroducing Scientists to the Power of Creative Environment,” details their approach and shares elements that instructional designers might consider enhancing creative collaboration with disciplinary experts.

Conclusion The section editors hope this volume serves as an informative compilation for design professionals and practitioners worldwide, as well for readers of journals focused on educational technology and the learning sciences. We are grateful to the authors and reviewers for their thoughtful contributions in shaping this volume.

Dr. Lin Lin-Lipsmeyer is Professor and Department Chair of Teaching and Learning in the Southern Methodist University’s Simmons School of Education and Human Development. Lin received her Ed.D. in Instructional Technology and Media from Teachers College, Columbia University. Lin has conducted interdisciplinary research in learning sciences, cognition, and innovative technologies. Her research has resulted in over 110 scholarly publications including journal articles, books, and book chapters. In addition, she has been Principal Investigator, Co-PI, or researcher on National Science Foundation and foundation grants bridging learning sciences, artificial intelligence, and STEAM learning. Lin serves as the Development Editor-in-Chief of one of the top journals in education and educational research, the Educational Technology Research and Development (ETR&D, https://www.springer.com/journal/11423). Dr. Bernadette Sibuma is the Director for Online Learning and Technology Innovation at Massachusetts Bay Community College. Bernadette works with the Provost’s office to provide faculty professional development in the design and development of student-centered online and

788

L. Lin-Lipsmeyer and B. Sibuma

hybrid courses. Bernadette is a member of the Massachusetts Department of Higher Education Open Educational Resources Advisory Council and Course Flagging Committee and co-leads equity initiatives at MassBay. Bernadette is also the project director of an NSF-funded year-long faculty professional development about culturally responsive and inclusive teaching practices. She holds an Ed.D. in instructional technology and media from Teachers College, Columbia University, and a bachelor’s degree from Cornell University.

Design Thinking: Towards the Construction of Knowledge

31

Brad Hokanson and Jody Nyboer

Contents A Different Mode of Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Understanding Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Nature of Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Skills, Attributes, and Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Procedural Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problem Identifying and Refining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Framing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developing Interpretive Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Continuous Re-Briefing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Metacognitive Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Parallel Lines of Thought and Ambiguity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ongoing Evaluation and Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Understanding and Empathy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developing Design Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transferability of Design Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

790 792 793 795 796 797 797 798 799 800 801 801 803 803 804 804 806

Abstract

Viewed as a third way of thinking, design thinking differs from the sciences and the humanities because it involves extensive experimentation and exploration resulting from an iterative process (Cross 1982 Design studies, 3(4), 221–227). This writing presents an explanation of design thinking for instructional designers and describes a variety of skills and traits illustrating how design thinking B. Hokanson (*) University of Minnesota, Minneapolis, MN, USA e-mail: [email protected] J. Nyboer Syracuse University, Syracuse, NY, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_81

789

790

B. Hokanson and J. Nyboer

supports the construction of knowledge. The value of design thinking can be seen from the viewpoint of the instructional or course designer and as a model for student learning processes. Keywords

Design thinking · Framing · Problem finding · Reflection-in-action · Creativity · Abductive thinking

A Different Mode of Thinking Design thinking is viewed as a third way of thinking, different from the sciences and the humanities. Rational, logical, methods are common to the sciences and engineering and generally involve approaching problems by conducting analyses. Reflective methods are common to the humanities (Cross, 1982; Dorst, 2011; Ho, 2001). Cross remarked “These ‘two cultures’ have long been recognized as dominating our social, cultural and educational systems” (Cross, 2001, p. 7). The epistemologies of each larger discipline (science and the humanities) relate to the topics of their domains, their areas of interest. The sciences study the natural world and examine what is “. . .true – the search for knowledge.” (Nelson & Stolterman, 2012, p. 33). The methods, evolved over centuries, primarily focus on an experimental approach with a quantitative orientation. The focus is on analysis and classification (Cross, 1982). Hypotheses, that is, potential explanations, are rigorously accepted or rejected. The decision process is objective and rational. The humanities examine human experiences. They are different in their focus, their values, and by their methods of inquiry. The methods employed include “. . . analogy, metaphor, criticism, evaluation” (Cross, 1982). The focus is on the human experience. It is a more qualitative and subjective understanding. There are other ways of thinking and divergent structures of thought. Design explores and actively creates the “artificial” or “man-made world” (Simon, 1996; Cross, 2007) and has values such as “. . .practicality, ingenuity, empathy, and a concern for ‘appropriateness’. . .” (Cross, 1982). Specifically, design is viewed as “abductive,” a thought pattern that is forward looking and anticipatory. In the terrain of professional practice, applied science and research-based techniques occupy a critically important though limited territory, bounded on several sides by artistry. There are an art of problem framing, an art of implementation, and the art of improvisation – all necessary to mediate the use in practice of applied science and technique. (Schön, 1987, p. 13)

We make designed changes to solve problems, designing rather than inventing the wheel, and other creations. Humans can all be considered involved in design as they seek to consciously create a desired future. Designing has occurred through history in the creation and development of the environment we live in. It can be said that this is the Anthropocene era, “. . .because of the dominant effect human activity has had

31

Design Thinking: Towards the Construction of Knowledge

791

on global systems, making them more unnatural and artificial” (Nelson & Stolterman, 2012, p. 27). Design thinking as an epistemology emerged from a general move in mid-twentieth century for a recognition of design as rational. This was consistent with the modernist approach to design, with a viewpoint of design as a scientific pursuit. The term, “design methodology” was initially used to describe and examine the process of designing, but it was recognized as limiting in an understanding of the rich process of design. Design was viewed as “. . .essentially un-amenable to the techniques of science and engineering, which dealt with ‘tame’ problems” (Rittel & Webber, 1973, per Cross, 2007). Archer stated design has its own distinct “things to know, ways of knowing them, and ways of finding out about them. As a discipline, the process of design does have a distinct mode of thought, its own way of thinking. Design values practicality, ingenuity, creativity, empathy, and appropriateness in contrast with the scientific values where objectivity, rationality, and absolute truth are the focus (1979, p. 17).” “Design wisdom is an integration of reason with observation, reflection, imagination, action and production or making.” (Nelson & Stolterman, 2012, p. 18) Design thinking involves extensive experimentation and exploration, based on an expanding body of knowledge developed as part of an iterative process. Cross suggests designers approach solving problems by synthesizing information, and in this way, design is distinct from the science and humanities (2001). Cross explained the distinction between design and science, thus: “There may indeed be a critical distinction to be made: method may be vital to the practice of science (where it validates the results) but not to the practice of design (where results do not have to be repeatable, and in most cases must not be repeated, or copied.” (2007, p. 43). To some extent, this raises a question regarding the nature of instructional design, whether it is the problem solving application of science or a practice of design. Professions like architecture have long embraced this alternative way of thinking and working. Emerging from the design industries, design thinking has developed as a leading trend among a range of diverse domains such as education or medicine. The concept of design thinking has gained interest in business schools (Julier, 2013) under the premise that “all management decisions are indeed design decisions.” (p. 62) Currently, design thinking has been taught within in the business educational programs. For example, the Rotman School of Management at the University of Toronto currently offers design classes to their business students. Moldoveanu and Martin (2008) contrast design thinking and business thinking as two different means of solving problems. Contrasting these two forms of thinking can highlight the differences creativity plays in constructing knowledge for each and suggest why businesses are readily adopting an alternative. Business thinking seeks predictable outcomes as a result of only incremental change through the use of algorithms, whereas design thinking, based in the use of heuristics and more complex forms of decision making, seeks possible and subjectively better futures. There is a strong interest in “. . .‘Design Thinking’ expressed by the business and management communities, who feel an urgent need to broaden their repertoire of

792

B. Hokanson and J. Nyboer

strategies for addressing the complex and open-ended challenges faced by contemporary organisations [sic]” (Dorst, 2011, p. 522). “Nowadays, ‘Design Thinking’ is identified as an exciting new paradigm for dealing with problems in many professions, most notably Information Technology (IT) and Business” (Dorst, 2011, p. 521). Perhaps the greatest distinction between design thinking and the sciences or humanities, centers on time, as those disciplines focus on what exists or what has happened. Design thinking, in contrast, is future oriented; concerned with “. . .the conception and realization of new things.” Its core is “planning, inventing, making, and doing” (Cross, 1982). Nelson and Stolterman (2012) describe design thinking as forward looking, seeing answers or outcomes in possible futures. It has also been described as “constructive thinking” (Schön, 1987), generative thinking, and “purposeful thinking” (Dorst, 2011, p. 521). The traditional and more well-known forms of thinking and logic, deduction and induction are used in various aspects of design. A third form, termed “abduction,” is a thinking process connecting all design fields. “The basic reasoning pattern in productive thinking is Abduction” (p. 523) as many resulting designs are anticipatory and are the products of the imagination. Designers use this mode of thinking to support the construction of knowledge needed to solve multidisciplinary problems. To better understand design thinking as a concept transferrable to other disciplines, it is valuable to briefly explore some broadly held ideas about the nature of design and the fields generally understood to be design, such as architecture, graphic design, and industrial design. In this writing, we are mapping this strategy to the less commonly included field of education and instructional design.

Understanding Design In popular use, design is often thought of as an aesthetic treatment applied after the solution of challenges or problems. “More often, it is equated with ‘style’; fashionable clothing or handbags, distinctive typefaces, elegant Philippe Starck furniture or Michael Graves teakettles” (Berger, 2009, p. 3). For example, within the field of instructional design, graphic designers are often brought in at the conclusion of programming and technical work to apply their aesthetic skills. Per Kolko, “Because design has historically been equated with aesthetics and craft, designers have been celebrated as artistic savants. But a design centric culture transcends design as a role, imparting a set of principles to all people who help bring ideas to life” (2015, p. 68). This presumption collides with a basic understanding of design thinking, as design is more than aesthetic treatments, and as a thinking process, should be involved throughout the solution of a problem, not as an afterthought. In contrast, others alternatively equate design thinking solely with problem solving, and much design activity is described merely as solving problems. The activity of design does center on problem solving, but is not limited to the resolution of simple problems. Design thinking is a divergent way of developing or constructing knowledge and is particularly valuable in solving complex problems,

31

Design Thinking: Towards the Construction of Knowledge

793

as well as at every phase of solution development. Understanding the nature and scale of problems and problem solving can help explain and present the value of design thinking and our own means of solving problems.

The Nature of Problems The nature of problems can be relatively simply defined: “First, a problem is . . . the difference between a goal state and a current state” (Jonassen, 2000, p. 65). Solving problems is an intentional change from one condition to a more positive state, changing at least some small part of the world. We all solve problems every day, from deciding what to wear to deciding how to teach a given lesson. “Virtually everyone, in their everyday and professional lives, regularly solves problems” (p. 63). We know problems vary in levels of difficulty, and we generally understand our own capability in solving different problem types. Problems vary in complexity from the simple, tame, or well-structured problems which can be solved by simple algorithmic methods to more complex or ill-structured problems which require more substantial experience and understanding. Those problems which are still more difficult or impossible to solve with only contradictory or conflicting information are termed “Wicked” problems. Problem resolution may actually cause subsequent problems (Rittel & Webber, 1973, Jonassen, 2000). Simple problems are those which are well understood and for which the methods of solving the problem are well known. With simpler problems, there may be a defined or implied process, an expected result format, and an anticipated result or value. They may also be described as “tame” or “benign” challenges much like “. . .problems in the natural sciences, which are definable and separable and may have solutions that are findable. . .” (Rittel & Webber, 1973, p. 160). The end goal: the solution is objectively definable, and the process needed to reach the solution is understood. Simple, well-structured problems can be resolved by rules, algorithms or recipes (Nelson & Stolterman, 2012). Dorst (2011) describes solving simple problems as a three-part formula. There is an observable goal or result and has way to reach that goal or objective. A way to proceed is known. “. . .here we know both the value we wish to create, and the ‘how’, ‘a working principle’ that will help achieve the value we aim for; what is missing is a ‘what’ an object, a service, a system that will give definition to both the project and the potential solution space. . .” (pp. 523–524). The results of the simple problem solving process are predictable: “what” is to be designed and the “how” it is to be done are known and will yield a predictable result (see for example, Ho, 2001). As in solving a chess problem or a mathematical equation, completion or resolution can be objectively evident. “There are criteria that tell when the or a solution has been found” (Rittel & Webber, p. 162). With welldefined or simple problems, what is developed, using selected ideas and principles, and shaped by our understanding, leads to the desired result. “This is a form of ‘closed’ problem solving that organizations in many fields do on a daily basis”

794

B. Hokanson and J. Nyboer

(Dorst, 2011, p. 524). Where the result is understood and definite, it is clear how to work toward a goal. “However, not all problems are simple or linear, or fit well within the scope of an existing process or domain. Methods for their solution are not well defined or known. In many cases, more complex problems are broken into segments of simple problems” (Ho, 2001). “Problem solving is not a uniform activity. Problems vary in their nature, in the way they are presented or represented, and in their components and interactions among them” (Jonassen, 2000, pp. 65–66). Many problems are ill-structured and more complex and difficult to solve by simple, linear methods. Ill-structured or complex problems require greater skill and attention to resolve. Design and design thinking address a greater depth and richness of thought than the resolution of these simple problems and are generally engaged in more complexity, where “. . . the problematic situation presents itself as a unique case.. . . Because the unique case falls outside the categories of existing theory and technique, the practitioner cannot treat it as an instrumental problem to be solved by applying one of the rules in her store of professional knowledge. The case is not ‘in the book’. If she is to deal with it competently, she must do so by the kind of improvisation, inventing and testing in the situation strategies of her own devising” (Schön, 1987, p. 5). The cognitive skills needed to resolve complex or ill-defined problems are often in contrast to the normal technical skills displayed in engineering, architecture, or education. “These indeterminate zones of practice – uncertainty, uniqueness, and value conflict – escape the canons of technical rationality. When a problematic situation is uncertain, technical problem-solving depends on the prior construction of a well-formed problem – which is not itself a technical task” (Schön, 1987, p. 6). Additionally, as Dorst notes, the result of solving a complex problem cannot generally be objectively judged, but must be evaluated subjectively. All is known at the outset is the final subjectively valued result, “what” is to be designed is not known, nor are the principles by which is to be accomplished. It has reached a more substantial level of abductive thinking (2011). As problems become more complex, the depth and complexity of abductive thinking must increase and a greater depth of thought must be applied. Schön describes this differentiation metaphorically: “In the very topography of professional practice, there is a high hard ground overlooking the Swamp. On the high ground, manageable problems lend themselves to solution through the application of research-based theory and technique. In the swampy lowland, messy, confusing problems defy technical solution. The irony of the solution is that the problems of the high ground tend to be relatively unimportant to individuals or society at large. . .while in the swamp lie the problems of the greatest human concern” (1987, p. 3). Nelson and Stolterman (2012) also recognize the diversity of possible responses to more complex problems: “There is no one correct approach or methodology for solving these problems, and it is not possible to formulate one comprehensive and accurate description of a problematic situation from the beginning.” (p. 17). This is where design thinking starts to be necessary, with ill-structured, complex, and even “wicked” problems. The specific resolution is often unknown or

31

Design Thinking: Towards the Construction of Knowledge

795

indescribable and can be better understood subjectively as opposed to objectively. In contrast to a well-formed “result,” it may be better described as a “value” (Dorst, 2009, p. 523). While all problem solving is future oriented, the elevated complexity of the challenge marks the necessity of the use of design thinking. It is not just about solving an objective problem, but about subjectively improving the context.

Skills, Attributes, and Practices Design thinking as a topic of academic interest evolved from examinations of design methodology centered on professional design practice. Studies and research of design methods focused primarily of the mechanics of problem solving. In contrast, design thinking is a set of complex skills or attributes involving more than simple “problem solving.” Early research in design methodology focused mainly on models heavily influenced by the theories of technical systems. “The positivist background of research in this area advanced design process being seen as a rational (or rationalizable) process” (Dorst & Dijkhuis, 1995, p. 261). Later studies in the design field have focused more on a reflective model, a thread initiated by Donald Schön and Nigel Cross. Other more practice oriented observations followed. Often described as the design process, applied design thinking is embraced and taught as a broad approach to solving a multidisciplinary spectrum of problems. As a way of working, this mode of thinking is consistently described as “messy,” unique to a problem or individual, and difficult to diagram with absolute certainty. Much of the work in instructional design and educational technology has followed a positivistic orientation. Most educational effort in the field of instructional design has focused on the definition and learning of specific task defined models such as ADDIE. Unfortunately or more likely, fortunately, practice in the field has followed a more complex and context-based model. Visscher-Voerman and Gustafson (2004) investigated the design processes of instructional designers and found no common, defined design process. This argues for a more general understanding of design thinking as well as continued research in the examination of the design process of practitioners. Other models have described effective instructional design processes. For example, Hokanson and Miller (2009) described a working model of different professional roles as a process for instructional design, but not one which proscribed orderly tasks. This is consistent with the broader field of design practice: “Designers do not then have a set of systematic rules that enable them to move from a problem to a solution” (Lawson & Dorst, 2013, p. 124). This is a critical difference in an understanding of design thinking. Rather than seeking a rigid process, i.e., a recipe for the design and development, it is more important to understand the guiding values, the qualities necessary for design thinking. A design process strictly proscribed by educators may be common, but

796

B. Hokanson and J. Nyboer

in practice and in reality, higher order attributes and practices form more responsive and inventive designers and design thinkers. The skills of design thinking can be described in a number of ways; as a reflective and dynamic creative process or as those of an experienced professional. It can be described as: “. . .that which we (could) call design wisdom. . . (it) is a much richer concept than problem solving, because it shifts one’s thoughts from focusing only avoiding undesirable states, to focusing on intentional actions that lead to states of reality which are desirable and appropriate” (Nelson & Stolterman, 2012, p. 17). Design thinking can be linked to a mindful way of solving problems; an inherent seeking beyond the simple production of an end result, or resolving tame problems. Design thinking is a combination of skills to qualitatively improve the results of practice, not an easily defined step-by-step model or algorithm. As with the development of any set of skills or cognitive traits, design thinking is developed over a long period of time through practice in solving progressively more challenging problems. Design thinking is multifaceted and it is more than a simple process that can be learned through a quick set of workshops. As part of the thinking used to address difficult challenges, design-oriented skills or traits are employed by most skilled designers. Design is uniquely connective to unique design events and context and depends on the responsive nature of design thinking. Some of the traits and characteristics involved in design thinking can be described as procedural practices, those regularly employed as part of a design process; representational, the externalizing, sharing, and interacting with the representations of ideas; and metacognitive, those general thinking abilities, and trends which support the broader design process. Each will be briefly examined in this writing.

Procedural Practices Technical tasks are routine actions of practice and occur in various design fields and often make up the explicit actions of a proscribed design process. These may include writing code, cropping an image, or detailing the construction of a wall. They are not, however, directly beneficial to the resolution of complex problems. As Schön states “It is not by technical problem solving that we convert problematic situations to well formed problems. . ..” (1987, p. 5) but rather through a more nuanced set of skills. While most design work can be described as a series of tasks, the nontechnical skills or attributes of the designer can be the most valuable. Included below are descriptions of the procedural skills of design thinking which include problem finding and refining, the setting, and identifying through a close examination of the true nature of the challenge; framing, the development of a focusing idea or vision; the interpretive actions of generating ideas, “moves,” or propositions; and continuous re-briefing, a conscious and ongoing re-examination of the original problem understanding. Included below are descriptions of the procedural skills of problem finding and refining; framing, the development of a focusing idea or vision; continuous re-briefing, a conscious and ongoing re-examination of the original problem

31

Design Thinking: Towards the Construction of Knowledge

797

understanding; and the iterative actions of generating “moves” or interpretive propositions.

Problem Identifying and Refining Much of the early work in a design process deals with understanding, finding and identifying the true nature of a problem. While many design problems are initially presented as a simple program or “brief” by a client, an important area of design thinking is the ability to find, describe, or redefine the real nature of the challenge. This generally happens through extensive research and connection with the eventual users of the work. This may occur through the participation of the stakeholders or what could be described as the development of empathy (Bjögvinsson, Ehn, & Hillgren, 2012). Design thinking is, initially, and at skilled levels, problem setting. Schön (1987) describes it as “. . . the process by which we define the decisions to be made, the ends to be achieved, the means which may be chosen” (p. 40). Complex problems can be difficult to fully understand, seeking a subjectively valuable design resolution as the finished product. By reformulating and restating complex problems, they can be more easily understood and addressed. “Whether we think of it as the reformulation of problems or the identification of elements, making them explicit and developing their characteristics is not a clear-cut thing but very much part of the design project. This is clearly an important and central design skill” (Lawson & Dorst, 2009, p. 50). Re-representing also leads to more creative results (Oxman, 1997, 329). Ho (2001) notes that complex problems often are broken into smaller subproblems, one which are more easily solved. Beyond the identification and defining of problems, most challenges are also tied to existing assumptions. One aspect of design thinking is a challenging of the nature of those assumptions. This is a central aspect of the designing process. It is a questioning of the challenge itself, examining the assumptions of the problem and stretching the “problem space.” (Cross, 2001; Gero, 2002). Questioning assumptions is a habit commonly built into design education and a trait which is valuable in all fields. Working from old assumptions, we can only choose from a limited set of solutions. Challenging a problem delineation or assumption stretches the possible solution space, is part of the nature of design thinking, and increases the variety of possible solutions.

Framing At some point, after researching the challenges and questioning the approach and assumptions, designers begin by defining an organizing aspect or a general direction from which to proceed. This may be implicit or explicit. Schön (1987) as well as Lawson and Dorst (2013) describe this as “framing.” Framing is adopting a particular understanding or driving vision for a project. This new focused understanding

798

B. Hokanson and J. Nyboer

provides a workable conception of both the end result and the ways it will be accomplished. Here, the unique “vision” for the project is developed – the idea, the spark that drives all the work (Löwgren & Stolterman, 2004). Within the design fields, this is also described as a design concept, parti, or précis. It is describing the systemic design effort. “I prefer Dewey’s view of the designer as one who converts indeterminate situations to determinate ones. Beginning with situations that are at least in part uncertain, ill-defined, complex, and incoherent, designers construct and impose a coherence of their own. Subsequently they discover consequences and implications of their constructions–some unintended–which they appreciate and evaluates” (Schön, 1987, p. 42). Framing is one of the greatest skills of the designer. It is the ability to holistically examine and envision the design problem. It is developing an organizing concept for resolution of the challenge, the “setting” of the problem. Dorst describes this most essential aspect of designing as an evolved understanding of the nature of a complex problem providing insight as to a means of solution. It is the synthesis of the complete understanding of the problem (2004). Per Schön, “When a practitioner sets a problem, he chooses and names the things he will notice. Through complementary acts of naming and framing, the practitioner selects things for attention and organizes them, guided by an appreciation of the situation that gives it coherence and sets a direction for action” (1987, p. 6). The framing can take a subjective form, a more lyrical description of the spirit of the design, or one which is more pragmatic and logical, an overall rational organizing of elements. As a form of framing, metaphors are often used to organize design work. This may be in terms of structuring the process or in terms of the end product, or as a way of presenting ideas in a way most people understand. There are current examples within the field of instructional design, such as an educational product framed as a game or as a challenge or as the distribution of information. Each would have different design outcomes and illustrate different epistemologies. Perhaps the most well-established example of re-framing in education is the encouragement to shift “from the sage on the stage to the guide on the side.” This reframing has changed much of classroom practice today.

Developing Interpretive Actions Each design begins with a set of choices based on an understanding of the project. An element is proposed to work in a certain manner, and subsequent choices are in support or response to the initial proposition. Starting with the initial framing of the work, choices can be made, and the implications of each move are made available for examination. It is an ongoing iterative process. “The most obvious set of skills employed by all designers are those to do with making design propositions” (Lawson & Dorst, 2009, p. 48). These “moves” are like those in chess, checkers, or go; there is always a gambit, an initial move, and later moves react and respond to the success or failure of any

31

Design Thinking: Towards the Construction of Knowledge

799

initial move. “Usually, this is a ‘front edge’ process in which partial and rudimentary representations are produced, evaluated, transformed, modified, refined, and replaced by others if need be, until their maker is satisfied with the results” (Goldschmidt, 2003, p. 72). Through the design process, designers try out a variety of different moves. Most won’t work, but as the changes occur in the designing of a solution, they are discarded and better choices are used in their place. Skilled chess players imagine the outcome of individual moves multiple turns in the future (abductive thinking) and make their current moves based on anticipated results. Like skilled chess players, skilled designers have a repertoire of starting ideas, and they apply them in context, discarding those fail. And throughout the experience, skilled designers can anticipate or imagine future challenges. They also are more skilled at setting subproblems for ease of solution (Ho, 2001). In an architectural design, for example, the kitchen could be located at one side of a house, because it would be near the garden or on the other side, as it would be close to the garage. Subsequent choices would react to the initial choice. Similarly, different structural layouts for a web-based learning resource could be considered, with the initial choices examined for effectiveness. The resolution of a complex design problem does not occur with one single change or detail, and with each change or proposition, new possibilities develop to build the quality of the result.

Continuous Re-Briefing As designers begin to make design decisions, they also begin to interact with the original challenge. Inherent in design thinking is an interactive re-stating, re-limiting, and re-examination of the original problem. As the process continues, the nature of the true design problem is often discovered through design. Perhaps the initial problem statement or design brief was incorrect, or limited in scope, or investigations made through design have helped bring to the fore a fuller understanding of what the brief should include. “Designing triggers awareness of new criteria for design: problem solving triggers problem setting” (Schön, 1988, p. 182). Part of design thinking is this constant, iterative reexamination the stated definition of the problem. Through this semantic or graphic re-examination, new insights are developed and new ways of resolution are discovered (Lawson & Dorst, 2013; Oxman, 1997). Understanding the true nature of the design problem changes as ideas are proposed and developed. In any design process, as designers learn about the design challenge, they develop a better understanding of the context of the problem and the information needed for its resolution. This is a normal part of the design process: As ideas are proposed and developed, they are evaluated against the initial problem statement. By testing design solutions, the stated nature of the problem is also tested and evaluated and often re-defined. Questions and details are raised by potential solutions, details which may not have been considered in the most recent problem statement. Similar

800

B. Hokanson and J. Nyboer

to widely used aspects of critical thinking, design thinking involves repeatedly asking if the correct problem is being addressed. Most design efforts begin with a problem statement or design brief. For example, an architect may be asked to design a house. The initial information provided might include size, number of bedrooms, in the space for parking cars. As the design progresses, questions will be asked about other aspects of the clients’ needs and desires, redefining the original brief, perhaps by adding or subtracting space or relocating bedrooms and the garage. More subjective questions may also be asked regarding orientation, natural light, or functions. Within instructional design, it is easy to imagine a design brief to create a mobile app or website that is to present the information for learning. This may have occurred without considering the demographics of the intended learner or teaching methodology. In architecture as well as instructional design alike, questioning the initial understanding of a problem will lead to better results.

Representation Design thinking is often expressed in visual form, through sketches, drawings, diagrams, wire-frames, and more formalized graphic elements. “Designers make sketches because the sketch is an extension of mental imagery, and therefore has the freedom of imagery to retrieve previously stored images and to manipulate them rapidly” (Goldschmidt, 2003, p. 88). This visual representation of an idea makes the abstract more real and more concrete, allowing for editing and revision in the development of any idea. Sketching also helps retain ideas for later reference and development. For example, the initial version of the I Love NY logo was first sketched on a scrap piece of paper by Milton Glaser while riding in the back of a taxi (Lehrer, 2012). History illustrates the ongoing use by designers of sketching and visual representation. The sketchbooks of Leonardo da Vinci are renowned for their representation of design ideas. Frank Gehry’s architectural sketches provide the modern equivalent in arts facilities around the world. Through drawing, relationships can be examined and modified and arrangements can be tested. Interpretation and development of the visual record also helps advance the design ideas. The fuzzy accuracy of sketching, the “unforeseen configurations” allow ideas to be re-interpreted or reinforced as sketches interact with designer’s current thinking. New combinations or ideas are developed: “The resultant displays are open to new interpretations, and if one consciously looks for them, they can be generated with relative ease using additional input from the designer’s memory structures” (Goldschmidt, 2003, p. 88). The externalization of the idea allows an interactive “conversation” with the design, a “back-talk” about the design (Goldschmidt, 2003). There develops an interaction or “reflection-in-action,” per Schön. It is gives “. . .new meanings and directions to the development of the artifact” (Schön, 1987, p. 31). Designers interact

31

Design Thinking: Towards the Construction of Knowledge

801

with their representations, even if the visualization is inexact, allowing for changing, modifying, accepting, and rejecting. Other fields may develop ideas in other media. For example, sound designers develop short pieces and compositions which later are formed into finished pieces. . .designs. Choreographers use visual notation and sketch movement sequences before assembling them into compositions. Stand-up comedians continually sketch their ideas in written form. Representation, whether electronic or physically drawn, also allows sharing with others, for evaluation and critique. “Sketching” as a means to develop instructional design work is a way of representing ideas in a range of media. Examining ideas for a project from both the larger, systemic scale and at the micro- or intimate detail will provide insight. Different forms, media, and scales of representation allow different modes of evaluation. They vary the focus of the designer on large-scale elements as well as small, intimate details. Design thinking involves working at many scales in a design challenge, from the overall systemic approach to the small details of the design. Within education and practice of instructional designers, the use of multiple scales to examine and represented design can be helpful. Reviewing a learning object, for example, in comparison with a larger learning theory will improve the quality of the work. Looking at the small scale of interaction and interface will help ensure clarity of use. This use of multiple scales of representation helps designers understand and coordinate the full scope of their design work.

Metacognitive Attributes There are general aspects of design thinking which are systemic to the design thinker. They are less about the process of a single design effort and more about an ongoing thinking process often in the background of the individual designer. These traits can be described as metacognitive and they are essential to the epistemology of design. These aspects of design thinking such as creativity, tolerance for ambiguity, and ongoing reflection on the work all build on an altruism and empathy for the end user.

Parallel Lines of Thought and Ambiguity Most designers have flexibility in their thinking process. As part of the creativity required of designers, there exists a tolerance for uncertainty and an acceptance of ambiguity. “The ability to think along parallel lines, deliberately maintain a sense of ambiguity and uncertainty and not to get too concerned to get to a single answer too quickly seems to be essential design skills” (Lawson & Dorst, 2013, p. 60). In the design process, it is not unusual for designers to be considering multiple lines of inquiry in solving a given problem or in reaching a design solution.

802

B. Hokanson and J. Nyboer

To some extent, this flexibility of thought may be in response to the complexity of design problems. With ill-formed or wicked design challenges, often the full parameters of the challenge are not known at the onset. The designer must work with limited direction and understanding, following multiple conceptual threads. Also inherent in the design process is a fluidity of thought, focused on the development of new ideas, the generation of new solutions, and the examination of alternative solutions, which could collectively be described as the creativity of the designer (Cross, 2007). This requires a number of traits, one of which the ability for divergent thinking, that is, the capability for generating multiple responses to a given prompt. Also needed is the ability to work with multiple competing ideas at the same time. Design educators usually require multiple project concepts for projects, and practicing professionals routinely develop alternative responses to client programs. Research has proven the generation of a substantial number of ideas that leads to more original ideas (Kudrowitz & Dippo, 2013). In order to pursue multiple, untried ideas in search of a solution, evaluation and judgment of the unique responses must be temporarily suspended or delayed. No truly new answers will be developed unless it is possible for unusual and untried ideas to be proposed and at least considered. Different directions, different design “moves,” and divergent frames all play a role in a design-in-process. Designers need a sense of exploration, as design is curiosity embodied: Each design problem starts without knowing the solution. “The unique thing about such processes is that, since they involve ill-structured problem-solving, it is not clear at the outset where the process is leading to, and what the end result might be” (Goldschmidt, 2003, p. 72). Design thinking also requires that some risk must be accepted, as new, untried ideas may be less successful than old, proven responses. All design thinking needs the ability to accept additional risk, otherwise all solutions would be based solely on existing solutions. Design thinking can embrace risk and failure as subsequent implementation can resolve questionable choices. The general separation between the acts of designing and the eventual implementation of the design work mitigates the risk in the final product, but at the same time, this separation should encourage risk in the design phase of a project. Accepting the risk of untried ideas is an important trait of designers. Not all ideas will be successful, and as with craft, ideas and elements are tried, rejected, and accepted on an ongoing basis it is part of the design and learning process. “[Failure] is instructive. The person who really thinks learns quite as much from his failures as from his successes” (Dewey, 1933, pp. 114–115). Repeated experimentation with various elements exists within the design process and often results in the minor failures of iterative work. This is an important aspect of design thinking and is addressed through constant checking and evaluation. Developing these traits comes through practice – habits which must be developed through education, such as with instructors requiring multiple alternative ideas and supporting selection and development of more unusual or divergent ideas. This internal checking helps the designer develop the ability for what is called “reflection-in-action” (Schön, 1987). It is a habit of constantly reviewing and

31

Design Thinking: Towards the Construction of Knowledge

803

checking one’s own work and evaluating design ideas. The process may be integrated into regular activities, such as a brief check of a design against a larger ideal or concept. Learners can be taught to develop these reflective skills as part of designer or problem-based learning with their own summative or formative reflections.

Ongoing Evaluation and Reflection Design thinking and general design processes include regular evaluation of newly developed ideas. This ranges from the evaluation of initial ideas, evaluated through external representation, up to public presentations and testing. Design ideas do not reach their full potential without being explained and presented to others. Within the design fields, there is a history of critique, where designers receive criticism from other designers and discuss ideas casually or in more formal presentations. Much of the instructional activity of a design education involves a series of critiques, individual to individual, or in small groups, and it is a constant review and interaction with the student work (Schön, 1988; Lawson & Dorst, 2013). Critiques are known for helping designers improve and develop their ideas (Hokanson, 2012). Formal design critiques in educational situations often involve outside experts. Developing evaluative skills in leaners can involve discussion focused in-person reviews or even online peer evaluation of others’ work. Important too is a larger, conscious self-review of the design effort as a whole, called “reflection-on-action” by Schön (1987). It is examining the larger scale of the project from micro- to macroscale and is a form of critical thinking and cognitive awareness of the effort. While Schön says “reflection in action” occurs as a matter of course during a design project, reflection “on” action is a postprocess event. Ideas are mulled over and discussed.

Understanding and Empathy Design thinking is inherently altruistic. Designers create for the benefit of others, for the world, to improve the general condition, and to resolve an existing problem. Within this, there is a need to develop a clear sense of understanding of any given problem and to emphasize with the end users of the work. A designer must be empathic and understand the real needs of the end-users, whether they are students, residents, occupants, or clients as well as an understanding of all stakeholders. To do this, designers need to develop an understanding of how things work, how things are used, how things are worn, and how things fit into real life outside the studio. Many training programs in design thinking emphasize the development of empathy in response to this need. Designers construct understanding through the process of creating and solving problems, it is a process raises a wide range of functional or affective questions which must be answered.

804

B. Hokanson and J. Nyboer

Information about each topic related to a project must be broadly explored to arrive to an appropriate solution. It is an arduous yet essential process consistent with problem-based learning or the case study method. Developing values, empathy, and understanding can be done through the educational settings of studio and problem-based learning. As with problem-based learning, the process itself pulls learners to discover more about each design challenge. Working with clients and end users is very effective both in terms of adding information and developing deeper connections and empathy with others. Participation by end-users in the design process can be most informative and can result in better design results (Bjögvinsson et al., 2012).

Developing Design Thinking Most students begin design education with an extensive background of informationbased and didactic education. The process of design is often foreign to them. Learners developing design thinking skills after long experience in other fields may find a design approach to be unusual or uncontrollable. Becoming a designer is not about remembering facts, but about developing a mindset toward solving complex problems as a way of learning, most often learning by doing and making. “The student cannot be taught what he needs to know. . .” (Schön, 1988, p. 17). “He has to see on his own behalf and in his own way the relations between the means and methods employed and results achieved. Nobody else can see for him, and he can’t just see by being ‘told’, although the right kind of telling may guide his seeing . . .” (Dewey, 1974, p. 151). Designers generally develop their cognitive orientation through a studio-based curriculum where they exercise applying knowledge to devise original solutions. The studio education of a designer is constructivist, with each new designer building their own design capability, through making or active means. As Schön described, it is “Freedom to learn by doing in the a setting that is relatively low risk, with access to coaches who initiate students into the ‘traditions of the calling’. . .” (1974, p. 17). It is important to note that design thinking is not something which can be learned through a single workshop-based approach, but involves the long term development of a design mindset, a calling, per Schön’s (1987). Similar to the development of other complex epistemologies, such as a scientific or medical mindset, or broader cognitive traits, time and experience are required.

Transferability of Design Thinking Becoming a designer involves the development of good qualities of thought, of solving problems, and of building complex capabilities of judgment and nuance. Each of the aspects described in our description of design thinking is applicable to

31

Design Thinking: Towards the Construction of Knowledge

805

other fields and disciplines. The skills of design thinking are transferable to other fields, whether in whole or in part. Design thinking can be mapped to educational theory as well as specific learning models. Under other descriptions and terms, the elements of design thinking are consistent with many current developments in educational theory. For example, problem-based learning is close parallel the actions of design thinking. Design thinking also employs the ideas of active learning, a concept well rooted and accepted in education. The general processes and ongoing cognitive traits of design thinking are complex and subtle; they are not defined by specific technical tasks such as drawing, but they are more in the realm of an epistemology, a way of thinking and acting. The procedural aspects of finding and identifying problems, reframing, positional moves, and continually re-examining problem briefs are evident in the design process and are built into the learning process of design. The metacognitive elements, including fluidity of thought, continuous evaluation of the work, reflection, and an empathic value-based practice, are less evident and develop through practice can all be developed and effectively used in any area of learning. Design thinking seeks an awareness of the thinking process and can be seen as similar to critical thinking as described by Paul and Elder (2013). In fact, many of the skills of design thinking closely parallel critical thinking methods, such as questioning assumptions. There are some differences. Design thinking has an openness to new ideas, with a greater emphasis on divergent thinking (the development of multiple ideas), while critical thinking is more focused on convergent thinking (the selection and improvement of a single idea). Design thinking is also more focused on visual representation and modeling than is critical thinking. But there is a strong emphasis on examining and questioning assumptions in both design and critical thinking in re-examining questions, problems, and knowledge. The value of design thinking can be developed through studio-based learning, whether described as active learning or problem-based learning. The skills developed through the solving of complex problems are most appropriate instructional design and educational technology. Research on the aspects of design thinking both within instructional design and in education would be most appropriate. The reflection and other metacognitive aspects of design thinking should be integrated into both the training and the work of instructional designers. The value of design thinking has much to do with the transferability of good thinking. Schön used the term “artistry” in describing the skills of design thinking and recognized their value to professionals of any field: “Inherent in the practice of the professionals we recognize as unusually competent is a core of artistry. . .Artistry is an exercise of intelligence, a kind of knowing, through difficult different and crucial aspects from our standard model of professional knowledge. It is not inherently mysterious; it is rigorous in its own terms; and we can learn a great deal about it...” (Schön, 1987, p. 13).

806

B. Hokanson and J. Nyboer

References Archer, B. (1979). Design as a discipline. Design Studies, 1(1), 17–20. Berger, W. (2009). Glimmer: How design can transform your life, your business, and maybe even the world. Toronto, ON: Random House Canada. Bjögvinsson, E., Ehn, P., & Hillgren, P. A. (2012). Design things and design thinking: Contemporary participatory design challenges. Design Issues, 28(3), 101–116. Cross, N. (1982). Designerly ways of knowing. Design Studies, 3(4), 221–227. Cross, N. (2007). From a design science to a design discipline: Understanding designerly ways of knowing and thinking. K. T. Edelmann, M. Erlhoff, & S. Grand (Eds.), Design Research Now. Essays and Selected Projects. Basel: Birkhäuser Basel (Springer). Dewey, J. (1933). How we think. Lexington, MA: D.C. Heath and Company. Dewey, J. (1974). On education (Vol. 431). Chicago, IL: University of Chicago Press. Dorst, K. (2011). The core of design thinking and its application. Design Studies, 32(6), 521–532. Dorst, K., & Dijkhuis, J. (1995). Design Studies, 16(2), 261–274. Gero, J. S. (2002). Computational models of creative designing based on situated cognition. In Proceedings of the 4th conference on creativity & cognition (pp. 3–10). ACM. Goldschmidt, G. (2003). The backtalk of self-generated sketches. Design Issues, 19(1), 72–88. Ho, C. H. (2001). Some phenomena of problem decomposition strategy for design thinking: Differences between novices and experts. Design Studies, 22(1), 27–45. Hokanson, B. (2012). The design critique as a model for distributed learning. In L. Moller & J. Huett (Eds.), The next generation of distance education: Unconstrained learning. New York, NY: Springer. Hokanson, B., & Miller, C. (2009). Role-based design. Educational Technology, 49(2), 21–28. Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational technology research and development, 48(4), 63–85. Julier, G. (2013). The culture of design. Thousand Oaks, CA: Sage. Kolko, J. (2015). Design thinking comes of age. Harvard Business Review, 93(9), 66–71. Kudrowitz, B., & Dippo, C. (2013). When does a paper clip become a sundial? Exploring the progression of novelty in the alternative uses test. Journal of Integrated Design and Process Science: Special Issue on Applications and Theory of Computational Creativity., 17(4), 3–18. Lawson, B., & Dorst, K. (2013). Design expertise. New York, NY: Routledge. Lehrer, J. (2012). Imagine: How creativity works. Houghton Mifflin Harcourt. Löwgren, J., & Stolterman, E. (2004). Thoughtful interaction design: A design perspective on information technology. Cambridge, UK: MIT Press. Moldoveanu, M. C., & Martin, R. L. (2008). The future of the MBA: Designing the thinker of the future. Oxford, UK: Oxford University Press. Nelson, H. G., & Stolterman, E. (2012). The design way: Intentional change in an unpredictable world: Foundations and fundamentals of design competence. Cambridge: MIT Press. Oxman, R. (1997). Design by re-representation: A model of visual reasoning in design. Design Studies, 18, 329–347. Paul, R., & Elder, L. (2013). Critical thinking: Tools for taking charge of your professional and personal life. New York, NY: Pearson Education. Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 2(4), 155–169. Schön, D. A. (1987). The reflective practitioner: How professionals think in action (Vol. 5126). Basic Books. Schön, D. A. (1988). Designing: Rules, types and words. Design Studies, 9(3), 181–190. Simon, H. A. (1996). The sciences of the artificial. MIT press. Visscher-Voerman, I., & Gustafson, K. L. (2004). Paradigms in the theory and practice of education and training design. Educational Technology Research & Development, 52(2), 69–89.

31

Design Thinking: Towards the Construction of Knowledge

807

Brad Hokanson is a Professor in Graphic Design at the University of Minnesota. He has a diverse academic record, including degrees in art, architecture, urban design, and received his Ph.D. in Instructional Technology. He teaches in the area of creative problem solving and has published research in the fields of creativity and educational technology. He is currently the Buckman Professor of Design Education. He won the College of Design’s award for Outstanding Teaching in 2008. Jody Nyboer is an Assistant Professor in Environmental Design at Syracuse University. She has a wide ranging academic record, including degrees in education and architecture, and received her Ph.D. in Design at the University of Minnesota. She teaches in the area of creative problem solving and interior design and has published research in educational technology.

Expanding Design Research: From Researcher Ego-Systems to Stakeholder Ecosystems

32

Steven J. Zuiker, Niels Piepgrass, and Mathew D. Evans

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Established Approaches to Design Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Complementary Approaches in Other Disciplines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Emerging Approaches to Design Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design-Based Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design-Based Implementation Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Social Design Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Participatory Design Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Infrastructuring Publics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Common Challenges Among Emerging Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Epistemological Comparison of Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Expanding Design from Researcher Ego-Systems to Stakeholder Ecosystems . . . . . . . . . . . . Tensions in Ecosystemic Design Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

810 811 814 815 816 818 820 821 823 824 825 829 829 831 832 833

Abstract

Although design research in wide-ranging fields organizes user-centered and use-inspired design processes, established approaches to design research in the learning sciences and educational technology have typically developed insights and innovations through efforts led by researchers. However, several emerging approaches to design research in these fields organize increasingly participatory forms that leverage human diversity as a resource. Approaches to design research among many complementary disciplines underscore evolving processes not only to couple thought and action but also to foster more inclusive visions and more distributed forms of agency among stakeholders in design projects. Building on S. J. Zuiker (*) · N. Piepgrass · M. D. Evans Mary Lou Fulton Teachers College, Arizona State University, Tempe, AZ, USA e-mail: [email protected]; [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_74

809

810

S. J. Zuiker et al.

this existing literature, the chapter characterizes and compares four emerging approaches with particular attention to processes of designing and their implications for designs. By considering the expanding repertoire of participatory approaches to design research, the chapter explores three interrelated questions about design research. First, we will consider who forms a design and how do they go about doing it. Second, we will consider how answers to these preliminary questions, in turn, frame expertise as design processes unfold. Third, we will consider the influence of design processes on the reach and impact of design research with respect to both educational change and theoretical refinement. In answering these questions, we seek to better resolve the ecological affordances of design research that not only mobilizes stakeholder perspectives in order to inform design processes but also sustains stakeholder networks in order to improve and evolve designs. Keywords

Design-based research · Design-based implementation research · Infrastructuring · Participatory design research · Social design experiments

Introduction Over the last quarter century, learning scientists and educational technologists have established design research as a signature approach. Design research organizes one way to produce theoretical insights into processes of learning and teaching and transformative designs for systems of learning and teaching. In contrast to experimental approaches, design research begins with the belief that research must carefully attend to the contexts of learning in order to intervene productively. Thus, design research establishes quality and validity in terms of real-world consequences and therefore embraces an interventionist approach. Furthermore, understanding learning and teaching as situated practices reflects sociocultural and ecological views of learning environments. From an ecological perspective, the actors, objects, events, and ideas that transact in social ecologies remain entangled. As a result, social and technological innovations are not additive or subtractive but potentially transformative in their effects (Barab, 2014). Given this view, learning scientists and educational technologists enlist design research in relation to diverse theoretical perspectives and methods in order to enrich understanding of learning and realize meaningful change. More recently, new approaches to design research have emerged. Insofar as established approaches organize design processes in terms of scholarly agendas and therein researcher “ego-systems,” each of the emerging approaches featured in this chapter seeks to organize design processes in terms of scholarly and public agendas with organizations and communities and therein stakeholder ecosystems. The chapter considers this expansion in three ways. First, it characterizes established approaches in the learning sciences and complementary approaches established in

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

811

other disciplines. Second, in relation to these characterizations, it then compares existing approaches with four emerging approaches to design research in order to consider how each attempts to expand the scope of design research. This comparison is organized around three guiding questions: who forms a design and how do they go about doing it, how are knowledge and expertise of stakeholders mobilized, and what is the ideal impact with respect to both educational change and theoretical refinement? Third, the chapter enlists a philosophical and methodological framework in order to relate these emerging approaches to broader notions of educational research operating within but also beyond learning sciences and educational technology. Finally, the chapter concludes by suggesting two themes that cut across these characterizations and comparisons then discusses considerations for future research.

Established Approaches to Design Research The editorial introduction to the first issue of The Journal of the Learning Sciences notes that “we hope, in this journal, to foster new ways of thinking about learning and teaching that will allow the cognitive sciences to have an impact on the practice of education” (Kolodner, 1991, p. 1). While the field remains broadly defined and considers an equally broad range of topics and approaches (Nathan, Rummel & Hay, 2014), its emphasis on impact resolves a common focus on getting ideas “through to teachers already in the schools who might use them in their classrooms, to principals and curriculum coordinators who will pass the ideas on to teachers in their schools, to computer hackers and software designers who will build appropriate software, and to administrators and lawmakers who can encourage and make it possible for new ideas in education to be put into place” (Kolodner, 1991, p. 6). The emphasis on extending the reach and expanding the impact of research has advanced interventionist research that enlists design to restructure educational environments in order to realize new opportunities to learn. In this section, we briefly review design-based research as the established approaches to design research in the learning sciences. As an interdisciplinary community, learning scientists study learning and teaching processes in authentic settings in order to understand and theorize these processes in terms of cognitive, socio-cognitive, or sociocultural perspectives. These perspectives recognize individuals as social beings who learn through participation, all participants as potential contributors to research, and technologies as potentially catalytic tools (Kolodner, 2004). They also inform the kinds of methods enlisted and adapted in order to describe learning and teaching and, over time, to explain and promote them. There are several examples of methodological innovation in the learning sciences. These include seminal papers by Jordan and Henderson (1995) on interactional analysis and by Chi (1997) on quantifying analyses of verbal data. Both forms of analysis contribute to methodic investigations of complex social transactions such as collaboration. Other methodological innovations relate to design-based research (DBR) as an approach to methodic inquiry that stems from seminal papers by Collins (1992) on a science of design and Brown (1992) on design

812

S. J. Zuiker et al.

experiments. The remainder of this chapter concentrates on design-based approaches that trace back to the work of Collins and Brown, if not earlier (i.e., Penuel, Cole, & O’Neill, 2016), for three reasons. First, DBR remains central to inquiry in the learning sciences; second, DBR produces use-inspired innovations with the potential to directly impact education; and third, DBR is a foundation on which multiple emerging approaches to design research build. DBR underscores that context matters (Barab & Squire, 2004). In turn, assuming that context matters underscores that learning and teaching unfold in complex social systems and cannot be readily understood in terms of discrete experimental studies of discrete factors. If context matters, then these laboratory-like studies may only illuminate “strange behaviors in strange places” (Cole, 1979, p. ix) and thus have little potential to impact education. Meanwhile, if learning and teaching processes remain bound up in complex social systems, then research must consider how these processes operate in authentic settings. DBR enlists design for this purpose. Design couples thought and action in order to develop or engineer learning and teaching processes, to systematically study these processes, and to iteratively illuminate and refine how design influences the systems in which learning and teaching operate (Cobb, Confrey, DiSessa, Lehrer, & Schauble, 2003). DBR has been the focus of special issues of journals (e.g., Educational Researcher, The Journal of the Learning Sciences, and Educational Psychologist) and the topic of edited books (e.g., Kelly, Lesh, & Baek, 2008; McKenney & Reeves, 2012). Authors of these articles and chapters represent the learning sciences and, increasingly, adjacent fields such as mathematics education, science education, and educational technology. In relation to educational technology, design-based research provides a complementary approach to instructional systems design. Some educational technologists have proposed design-based research as an alternative to instructional systems design when the complexity of teaching and learning preclude straightforward delivery of instruction (Oh & Reeves, 2010). Meanwhile, Amiel and Reeves (2008, p. 37) envision design-based research as a core approach to the future of educational technology. [I]f we persist in believing in education and technology as value-free, we should not attempt to engage in design-based research and should instead resign ourselves to perpetuating research that effects no systematic change. We may hide our lack of concern for impact behind the veil of academic freedom. But if the case for the new design-based methodologies is sound, then research and practice can become intertwined, and as a result, it becomes impractical and indeed ungrounded to promote the kinds of impartial, unengaged research that dominates the published literature.

Reflecting this effort, learning scientists and many educational technologists report using DBR in wide-ranging scholarly publications. Despite growing interest, a critical review of DBR literature suggests that it “seems [to] have been used to make a difference – but mostly at the level of small-scale interventions and in the lives of individual teachers and schools” (Anderson & Shattuck, 2012, p. 24). Barab (2014) suggests that some designs may be too sophisticated and specialized and cannot generalize to other classrooms

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

813

or schools, though generalization, as a goal of educational research, presents many challenges itself (Berliner, 2002). However, these and other enduring challenges in education motivate continued efforts to enlist design in transforming complex social systems like classrooms and schools. In illuminating local impact with small numbers of teachers and schools, DBR sheds light on the challenges of extending the reach of these design innovations and theoretical insights. Barab (2014) suggests that design-based research runs the risk of yielding discrete, fixed products, akin to “packages of strategies with readily measurable outcomes” (Gutiérrez & Penuel, 2014). DBR can lead to linear approaches that unfold top-down from a researcher to the participants who enact the design (e.g., Tyack & Cuban, 1995). Rather than products or packages to be mobilized, treating designs (at least, designs that have matured over the course of multiple iterations) as something like a continuous service entails ongoing consideration of both settings and underlying social and historical contexts (Barab, 2014). In other words, the idea of a service assumes that DBR cannot optimize a design innovation but rather must continue to attune it to the varied social systems with which it must ultimately resonate. Fidelity, in this sense, is the capacity of designs and contexts to mutually condition one another. In one sense, Barab captures this expansion of design-based approaches when he observes “effectiveness is always integrated with how well the design engenders the ecosystem to optimize its success” (p. 161). A design works simultaneously within ecosystems (i.e., supporting and sustaining learning and teaching processes) and on ecosystems (i.e., transforming learning and teaching processes). The characterization of design as a service rather than a product appears in the DBR chapter of the second edition of the Handbook of the Learning Sciences but not the first edition and highlights a shift that is reflected in expanding approaches to design research considered below. It may also echo a larger challenge facing design agendas. While DBR has developed innovations that improve learning and teaching processes in classrooms and schools, these design innovations struggle to scale across educational systems (Penuel, Fishman, Haugan Cheng, & Sabelli, 2011). Obviously, local impact in particular contexts is necessary, but achieving equity and excellence in education entails broader impact across variable contexts. Therefore, shifting from a view of design as products to one of design as services complements broader arguments about “relevance to practice” as a criterion of rigorous research (Gutiérrez & Penuel, 2014) and the limitations of “producerpush” approaches to scaling knowledge (Nutley, Walter, & Davies, 2007). Together, they underscore that scaling innovations in educational technology, learning sciences, and other scholarly agendas must position designs to scale learning, making a shift “towards more open-ended social or socially embedded experiments that involve ongoing mutual engagement” (Gutiérrez & Penuel, 2014, p. 20). These shifts also resonate with trends unfolding in design disciplines, particularly participatory design. Therefore, in order to frame established and emerging approaches to design research in the learning sciences, the next section considers longer-standing, complementary approaches in other disciplines that precede and parallel the development of DBR.

814

S. J. Zuiker et al.

Complementary Approaches in Other Disciplines Design is a discipline unto itself and one that informs a wide range of disciplines including educational technology and the learning sciences. Central to the consideration of expanding approaches to design research in education, participatory design (PD) characterizes a range of approaches that expand involvement in, and contributions to, design processes. PD is one significant approach that clouds the differences between designer and non-designer, product and service, and even between design disciplines (Sanders, 2006; Sanders, Brandt, & Binder, 2010). Stakeholder participation often traditionally resides in the “fuzzy front end” of a design process where multiple perspectives prove vital for generating and exploring ideas (Sanders, 2006, p. 1). However, an evolution in thinking about PD, design, and research has developed in order to organize stakeholder involvement in varied ways before, during, and after design processes. This section therefore reviews trends in the field as a backdrop against which to compare and understand the role of design in educational technology and the learning sciences. PD reorganizes relationships among researchers, users, and other stakeholders in order to engineer and evolve solutions while positioning individuals with greater agency and legitimacy regardless of their role (Sanders, 2006). This trend emerges against the backdrop of a longer-term evolution of participatory approaches that began in the 1970s. Ehn (1993) traces the origins of formal PD to factory workers in Northern Europe who contributed to design processes along with designers and researchers, which enabled those who built a product to also cocreate it. From these factory projects, PD has expanded into a diverse range of approaches. In order to characterize trends across these approaches, Sanders (2008) compares them in terms of two continua defined by mindset and approach. The first continuum simply characterizes whether the approach follows from a research-led perspective or from a design-led perspective. Research-led PD positions the researcher as translator between users and designers while design-led PD positions the researcher as a facilitator in one or more ways. The second continuum characterizes whether the design process proceeds with an expert mindset or a participatory mindset. With an expert mindset, the user is positioned as a subject or informant who contributes responses or reactions. With a participatory mindset, the user is positioned as a partner who contribute actively as a cocreator. Of special note, Sanders (2008, p. 13) observes that “it is difficult for many people to move from [one mindset to the other], as this shift entails a significant cultural change.” These continua establish two axes on which to map or locate approaches to PD, either within or across four different quadrants (e.g., design-led PD with a participatory mindset). As an example, human factors design maps onto research-led PD with an expert mindset. Human factors designers traditionally assume roles as experts who enlist research to drive design, positioning users as subjects who inform their efforts. Taken together, approaches to PD position users to contribute to design processes, most notably during idea generation but increasingly during decision-making as well (Sanders & Stappers, 2008).

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

815

Other approaches organize design processes in order to integrate stakeholder contributions and often expand the scope of design processes in order to consider “future experiences of people, communities and cultures” (Sanders & Stappers, 2008, p. 10). Meanwhile, in other fields such as business, the broader idea of cocreation organizes design processes with customers to support a broad, interrelated “service ecosystem” (Vargo & Lusch, 2004, p. 240). With citizens rather than customers, public-sector projects similarly organize cocreation around social initiatives (Voorberg, Bekkers, & Tummers, 2015) while also distinguishing cocreation from the co-implementation of such initiatives. This distinction reflects the fact that public-sector projects represent an ongoing series of interactions between citizens and public services or what Weick (1995, p. 6) characterizes as “constructing meaning [and] interacting in pursuit of mutual understanding.” Nevertheless, cocreation in public sector projects typically struggle to represent or include all relevant stakeholder groups, raising the issue of how can participatory processes include marginalized groups or individuals with less agency into the collaboration (Voorberg et al., 2015). Mapping these broader efforts onto Sanders’ continua (2008), co-creation aligns with research-led design processes organized with an expert mindset. This is likely due to public sector leaders typically being “risk adverse” and unwilling to rely on citizen participation which can appear to be “uncontrollable and unreliable” (Voorberg, Bekkers, & Tummers, 2015, p. 1347). The emerging approaches to design research compared in the next section reflect several of these trends. Many of the authors explicitly recognize design disciplines among several others as influences on their work. As example of more direct exchange between design disciplines and the learning sciences, learning scientists have developed a series of architectural and urban planning case studies in order to draw insights about designing for learning (O’Neill, 2016) while designers have begun to review learning and teaching systems in relation to their approaches and methods (DiSalvo & DiSalvo, 2014). One common insight intersecting these disciplinary exchanges is “engag[ing] practitioners, parents, and other stakeholders in identifying [tradeoff relationships with an innovation] through a consultation process” (O’Neill, 2016, p. 149) using tools and activities from design disciplines like PD that “help participants feel empowered to share their perspective no matter their level of expertise” (DiSalvo & DiSalvo, 2014, p. 796). Given these trends and exchanges, it is increasingly important to consider and compare parallel trends in emerging approaches to design research.

Emerging Approaches to Design Research This section compares design-based research (DBR) with four emerging approaches that expand design research in the learning sciences. All four emerging approaches share a common foundation in design-based research and commonalities with other design disciplines, particularly approaches to participatory design. Each has been selected specifically because it seeks to leverage human diversity as a resource for

816

S. J. Zuiker et al.

design. Whereas DBR and complementary approaches are client focused and use inspired, each of these approaches seeks to sustain engagement and essential contributions of multiple stakeholder groups. For the same reason, each bears a family resemblance to the others, and, therefore, a common set of guiding questions serves to organize comparisons. These comparisons do not aim to ascribe relative value nor to suggest relative strengths but simply to juxtapose each approach. Guiding questions place each approach side-by-side in order to compare and contrast features and principles. They also provide a basis for discussing of how these emerging approaches expand design research. Revisiting design-based research (DBR) as an established but shifting approach to design research serves to introduce and establish the intellectual merit of the guiding questions. In a critique of DBR, Engeström (2011, p. 600) observes “scholars do not usually ask: who does the design and why?” That is, the literature on DBR “tacitly assume[s] that researchers make the grand design, teachers implement it (and contribute to its modification), and students learn better as a result.” These assumptions resonate with the idea of designs as ready-made products or packages (Barab, 2014; Gutiérrez & Penuel, 2014). Further, this tacit assumption raises questions about who exercises agency over a design, when they do so, how they do so, and to what end. Drawing on this interpretation of tacit assumptions underlying DBR, the remainder of this section considers established and emerging approaches to design research with respect to the following three guiding questions: 1. Who forms a design and how do they go about doing it? 2. How are knowledge and expertise of stakeholders mobilized? 3. What is the impact with respect to both educational change and theoretical refinement? As a caveat to these comparisons, the idea of stakeholders serves as a general term to reference to the various individuals, groups, organizations, and communities involved in design approaches. Examples of stakeholder groups include students, parents, teachers, school or district administrators, families, vulnerable or non-dominant groups, museum staff, and citizens among many others. Collapsing these diverse contributors provides general coherence and continuity across the approaches, while also recognizing that the particular stakeholder groups in any design efforts will not be well represented by any of these terms.

Design-Based Research Design-based research (DBR) is a signature method in the learning sciences. DBR begins with a vision for learning, typically led by researchers, which establishes the focus of design processes (Cobb et al., 2003). Design concentrates on resources and structures for improving teaching and learning in specific subject matters and in classrooms settings. Five crosscutting features characterize DBR (pp. 9–10):

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

817

1. The purpose is to develop theories about learning processes and designs that support that learning. 2. The methodology is highly interventionist. 3. The theoretical orientation of research aims to create conditions for developing theories while also putting them in harm’s way. 4. The design orientation of research is iterative, involving cycles of invention and revision for enhancing conditions for developing and explaining theory. 5. The research pursues pragmatic goals in order to produce humble theories about domain-specific learning in relation to the activity of design.

Who Forms a Design and How Do They Go About Doing It? Researchers predominantly form designs, drawing on both the use-inspired needs and demands of educational stakeholders and theoretical insights into how people learn in particular domains. Researchers develop designs through careful analysis of existing education systems and the strategic enlistment of theoretical understanding; both inform designs with the potential to bring about novel processes and outcomes for learning and teaching. Researchers then intervene through design in order to refine or enhance these educational systems. In this way, designs remain open to extra-researcher influences whenever a researcher recognizes and integrates those influences. For example, the researcher might adapt the design in real-time during an enactment or between enactments in response to the ways that participants enact the design. How Are Knowledge and Expertise Mobilized? Design-based research seeks to recognize rather than control the influence of participants, institutions, and culture on the processes and outcomes of research. By recognizing that context matters in complex ways, design-based research attempts to understand multiple interactions and often subtle relationships. Doing so can illuminate the underlying conditions in which a process or mechanism operates and, in turn, can inform the iterative refinement of a design. In this way, design adaptations reflect methodic consideration of the knowledge and expertise of the stakeholders directly involved in the enactment of a design, reflecting the fact that participants’ decisions co-determine how enactments of a design occur. However, the expanding approaches to design research to which DBR is compared in the remainder of this section suggest that it is the researcher’s knowledge and expertise that typically determine when and how other stakeholder contributions co-construct design. What Is the Impact with Respect to Both Educational Change and Theoretical Refinement? DBR works in authentic educational settings in order to directly impact teaching and learning. Therefore, one direct form of impact is a design that can be useful and used. At the same time, DBR develops these designs in order to articulate and refine theoretical insights into learning in specific domains as well as design principles that extend the reach of these insights into other design projects. The

818

S. J. Zuiker et al.

impact of DBR therefore also resides in the ways design is positioned to develop humble theories about the processes or mechanisms underlying a design. DBR thereby accounts for whether or not a design works as well as how (i.e., mechanisms reflected in design) and why (i.e., real-world conditions reflected in authentic settings). In this way, impact is tightly coupled with a researcher’s ability to identify, characterize, and understand how design intervenes in messiness of authentic educational settings in which participants engage the design, one another, and researchers.

Design-Based Implementation Research Design-Based Implementation Research (DBIR) is an approach that expands the focus of design research to multiple levels of education systems, from classrooms or schools to one or multiple districts (Penuel et al., 2011). By focusing on education systems, DBIR enlists design to improve the ways programs operate as they scale. DBIR seeks to develop and refine tools and practices in order to solve practical problems. Equally, it aims to align and coordinate implementation supports across multiple levels in order to build systemic capacity to sustain changes associated with new programs. In this way, DBIR seeks to develop practical theory and tools to support local stakeholders as they adopt and adapt programs within the boundaries of the education system they share. DBIR draws on DBR, as described above, as well as other traditions such as evaluation research, community-based participatory research, and implementation research (Fishman, Penuel, Allen, Cheng, & Sabelli, 2013). DBIR draws on these traditions and others to organize design processes around the following four principles (Penuel et al., 2011, p. 332): 1. A focus on persistent problems of practice from multiple stakeholders’ perspectives 2. A commitment to iterative, collaborative design 3. A concern with developing theory related to both classroom learning and implementation through systematic inquiry 4. A concern with developing capacity for sustaining change in systems By integrating these principles, DBIR organizes a participatory design process that can involve key individuals and whole stakeholder groups (e.g., teachers, administrators, students, parents) in order to understand and adapt programmatic features and institutional infrastructure. In this view, DBIR interventions consider the design of the program, the setting, and the context together as a kind of distributed institutional ecosystem (Barab, 2014, p. 161). DBIR projects attempt to transform what is possible within classrooms and the broader education system at school and district levels.

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

819

Who Forms a Design and How Do They Go About Doing It? DBIR focuses on designing for the implementation of relatively mature programs in one or multiple districts. It assumes the design of programs underdetermine implementation in particular education systems. Designing for the implementation of programs concentrates on social innovations among stakeholders in one particular system. Mindful of the relative authority, status, and cultural norms among researcher and practitioners, DBIR positions all stakeholders as co-contributors who, together, organize collaborative efforts to support and sustain program adoption. This includes jointly resolving the focus and the organization of the work. Collaborations concentrate on social innovations, which may involve coordination and program adaptations change at multiple levels of an education system. How Are Knowledge and Expertise Mobilized? DBIR seeks mutually transformative agendas for all stakeholders, including researchers. Collaborations therefore seek to mobilize knowledge and expertise among all stakeholders. Such mutualism can be characterized as a multi-way and recursive relationship among research, policy, and practice (Coburn & Stein, 2010). DBIR assumes that mutualism challenges stakeholders to navigate and coordinate among different organizations and their norms. Crossing these organizational boundaries enables stakeholders to produce solutions that leverage different forms of knowledge and expertise. In particular, the emphasis on joint activity underscores that mobilizing knowledge and expertise depends on framing problems in ways that resonate with stakeholders. To foster mutualism with and across organizations, DBIR seeks to involve individuals at multiple levels of an education system and to construct multiple frames for characterizing the project in different settings. DBIR seeks to develop organizational routines and processes that enable innovations to travel through a system and to different education systems shaped by different settings and contexts. What Is the Impact with Respect to Both Educational Change and Theoretical Refinement? The impact of DBIR includes aligning and coordinating new classroom teaching and learning programs with policies and systems that support them at scale, involving perspectives from across settings and sectors in improving teaching and learning, developing methods to negotiate the focus of multi-stakeholder agendas and to sustain engagement among wide-ranging stakeholders in design processes, and developing policies and infrastructures to sustain program changes and grow capacity for continuous improvement. DBIR seeks to impact education systems by mediating change within the system and across the layers of infrastructure operating therein (e.g., organizational routines and processes). Like DBR, DBIR currently concentrates on suitable contexts, which limits its reach to systems that are ready for change. As research on the scaling of other research, its impact also extends to theorizing about system-wide conditions that can inform emerging and maturing

820

S. J. Zuiker et al.

designs alike. It can also contribute to policy research in education, particularly by considering how new tools such as curricula or technologies can illuminate new needs for alignment and coordination across levels of an education system and for building capacity.

Social Design Experiments Social design experiments (SDE) “are cultural historical formations concerned with academic and social consequences, transformative potential, and new trajectories for historically vulnerable people, especially people from nondominant communities” (Gutiérrez, 2016, p. 191). SDEs draw on DBR but expand it in several ways by drawing on formative experiments in workplace settings (Engeström, 2008), resilience theory, cultural-historical activity theory, as well as equity-oriented inquiry. SDE design principles include: 1. 2. 3. 4. 5. 6.

Attention to history and historicity Focus on remediating activity and systems, not individuals Employing a dynamic model of culture Persistent emphasis on equity Emphasis on resilience and change End goal of sustainable transformations

Who Forms a Design and How Do They Go About Doing It? SDEs concentrate on codesigning models of future social systems in the present. A future-oriented focus seeks to imagine new possibilities rather than representing existing, often limiting perceptions of current social systems. Through codesign, SDEs seek to democratize inquiry into the social systems in which vulnerable or non-dominant communities participate. The transformative potential of SDEs, however, entails participation from among the diversity of stakeholders in a social system. Codesign therefore revolves around individuals, the stakeholder group with which an individual affiliates, and the ecology of stakeholder groups that constitute a social system, or ecology. Gutiérrez and Vossughi (2010) observe that researchers often contribute more to the design process because they assume roles as facilitator in several ways. Researchers facilitate the design process through intentional efforts to reconcile opposing principles among multiple stakeholders. For example, stakeholder groups might recognize differences between everyday and formal practices in science education in order to resolve them as complementary rather than hierarchical practices. In order to engineer a design process to these ends, researchers focus explicit attention on equity and historicity, and they seek to characterize and monitor the broader structures and dynamics of inequity in order to resolve which principles or practices should be the focus of social design (Gutiérrez & Jurow, 2016).

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

821

How Are Knowledge and Expertise Mobilized? Focusing on social systems attunes each SDE to the value of revision, disruptions, and contradictions because each mobilizes knowledge, expertise, and other aspects of stakeholder experiences in order to illuminate and advance an SDE’s transformative potential. By intentionally designing in relation to principles and practices at the intersections of non-dominant groups and relevant institutional stakeholders, SDEs seek to democratize inquiry and co-construct new principles and practices. In particular, SDEs position individuals, especially members of non-dominant communities, in relation to the past and present circumstances of their community as well as its possibilities for the future. What Is the Impact with Respect to Both Educational Change and Theoretical Refinement? SDEs seek to expand design-based approaches in the learning sciences by connecting design-based agendas directly with the broader social purposes of education. The goal of doing so is to change the social systems in which vulnerable or non-dominant groups operate. SDEs intentionally disrupt educational, structural, and historical inequities. By investigating and selecting principles and practices, syncretism seeks to strategically reorganize sociohistorical practices in order to expand learning opportunities. Social designs realize impact by transforming educational and social circumstances as well as the systems of activity in which both operate, particularly for members of non-dominant communities. At the same time, social design experiments position participants as conscious, intentional, historical actors who codesign community practices. The combined focus on community practices and individual actions can achieve greater sustainability, meaning, and impact.

Participatory Design Research Participatory design research (PDR) also resonates with existing forms of participatory design through its commitment to collaborative design with a particular focus on equity and diversity in partnerships. PDR emphasizes equity and diversity during the process of partnering because they fundamentally influence the possible forms of learning that emerge in and through partnerships. That is, PDR assumes that partnering for codesign precedes and informs the process of codesign in terms of “the conceptual lenses, forms of relationality and professional vision developed in these processes that allow participants to see and move in new ways” (Bang & Vossoughi, 2016, p. 182). Attending not only to who partnerships represent but also to how they do so can challenge presumptions of neutrality and can illuminate what a stakeholder values as being inherently perspectival, that is, how a stakeholder’s “values, sensibilities, affects, and desires shape what are ‘right,’ ‘good,’ ‘important,’ or ‘worthwhile’” (p. 181).

822

S. J. Zuiker et al.

Who Forms a Design and How Do They Go About Doing It? PDR concentrates on representational diversity in organizing who forms the design. In particular, it attends to how the political and theoretical history of a project relates to the personal histories of stakeholders, including researchers themselves. PDR therefore seeks to engage stakeholders directly implicated by the setting of design work but also others implicated by the broader context of the same design work. Considering both setting and context draws attention to all that literally surrounds design work (i.e., setting) as well as to that which weaves design work together such as institutional and cultural factors (i.e., context). Understanding context completely is elusive if not impossible, but PDR identifies critical historicity, power, and relational dynamics as key concepts through which to understand and inform the process of partnering for codesign. How Are Knowledge and Expertise Mobilized? PDR seeks to reorganize typical roles associated with design approaches (e.g., researcher, theorist, and designer). In order to remediate the relative value and scale of influence associated with these traditional roles, PDR seeks to organize opportunities for all stakeholders to learn and exercise agency. Reorganizing design processes around equitable stakeholder engagement enables multiple perspectives and values to contribute questions, concerns, ways of knowing, and ultimately aspects of design. In particular, PDR concentrates on the knowledge and expertise historically present within communities. In this way, multiple perspectives substantively inform systematic inquiry as stakeholders change social or ecological relations, again with particular attention to the history and relationality of stakeholders and desired objects or products of design processes. The idea of role remediation is central to fostering critical reflexivity and enabling knowledge to manifest independently from researcher’s discovery or recognition. What Is the Impact with Respect to Both Educational Change and Theoretical Refinement? Role remediations seek to assert and develop the knowledge and expertise historically present within communities in design. Increasing visibility is necessary, but expanding forms of agency underlie individual and collective shifts, particularly for vulnerable or non-dominant communities. In this way, PDR expands learning by repositioning stakeholders. As multiple stakeholder exercise agency, PDR illuminates how design co-constitutes both subject–subject and subject–object relations. Like SDEs, PDR considers how innovations contribute to cultural change, which can inform theoretical refinements about how stakeholders exercise agency to intervene and impact new spaces and sets of relations. Indeed, PDR speculates that a blind spot in many design approaches is whether or how researchers identify and engage with endogenous (sometimes also routine or everyday) processes of design and intervention that stakeholders may already enlist. Gutiérrez, Engeström, and Sannino (2016) suggest that PDR is at a relatively early stage and will continue to sharpen and enrich

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

823

the core concepts that provide analytical and interpretive lenses through which to organize and advance PDR.

Infrastructuring Publics Infrastructuring draws on approaches from design disciplines, namely, participatory design as reviewed above. A technical or practical solution, however, constitutes the beginning rather than the end of infrastructuring. Whereas many forms of participatory design focus on innovations designed for usefulness in the present, infrastructuring seeks to enable “adoption and appropriation beyond the initial scope of the design, a process that might include participants not present during the initial design” (LeDantec & DiSalvo, 2013, p. 247). In this way, infrastructuring is an emerging approach with the potential to expand design research by specifically designing for democratic processes (DiSalvo & DiSalvo, 2014; DiSalvo, 2009). The idea of designing for democracy emphasizes the relationship between practical or technical innovations and ongoing social innovations among stakeholder groups, or publics, which emerge through the adoption and appropriation of practical or technical innovations. In particular, infrastructuring emphasizes that designing infrastructure to capitalize on this relationship can extend the reach, impact, and sustainability of design approaches. In this way, infrastructuring expands the work of design from responding to the relatively discrete framing of known technical or practical issues to continuous (re)framing of evolving issues that entangle technical, practical, and social innovations.

Who Forms a Design and How Do They Go About Doing It? Infrastructuring designs and develops resources to facilitate, support, and ultimately sustain collective processes for collaborative inquiry (DiSalvo, Clement, & Pipek, 2012). Specifically, it emphasizes the development of socio-technical resources, or infrastructure, for sustainable participation in design efforts. These resources can prove critical for involving multiple stakeholders whose commitments to an issue or design challenge reflect varied interests and concerns (DiSalvo, 2009). A goal is an inclusive and sustainable design process. In order to go about such a process, infrastructuring focuses on how design contributes to the construction of publics (DiSalvo, 2009). Issues constitute a loose constellation of stakeholders, or a public, “of those who are affected by the indirect consequences of transactions to such an extent that it is deemed necessary to have those consequences systematically cared for.” Designing infrastructure therefore pays close attention to how issues and publics emerge and evolve by tracing backwards towards the origins of each and forwards as artifacts, events, or ideas continue to shape them. It also attends projections or possible future scenarios that illuminate possible consequences of an issue rather than possible directions for solutions. Tracings and projections therefore seek to organize an active dialectic between the past, future, and present of an issue in order to sustain collective processes and collaborative inquiry.

824

S. J. Zuiker et al.

How Are Knowledge and Expertise Mobilized? In relating technical or practical innovations with social innovations, infrastructuring positions all stakeholder perspectives as sources of knowledge and expertise much like social design experiments and participatory design experiments. In particular, infrastructuring mobilizes knowledge and expertise by concentrating on the dependencies and commitments that link stakeholders to the issue underlying the design process. This emphasis on how stakeholders and issues are linked contrasts with many approaches to participatory design, which initially frame issues for all stakeholders and, therein, run the risk of prioritizing some points of view. Moreover, framing fails to expose how the design process itself organizes multiple emergent and dynamic frames regardless of the ways designers initially frame an issue or design challenge. Thus, rather than framing issues, infrastructuring seeks to position stakeholders to actively and continuously engage with issues directly. Sustaining stakeholder engagement is critical to infrastructuring because their entanglements with an issue and other stakeholders remain the sources of and resources for ongoing social innovation (LeDantec & DiSalvo, 2013). What Is the Impact with Respect to Both Educational Change and Theoretical Refinement? By positioning stakeholders in relation to one another as a public that authors and evolves the issue underlying a design effort, infrastructuring concentrates on design as ongoing socio-technical processes rather than a fixed product. Its reach and impact on educational change is “to create fertile ground to sustain a community of participants” (LeDantec & DiSalvo, 2013, p. 247). Infrastructuring can also extend the reach of design by expanding its focus. Its emphasis on issues and the public’s that are attached to them shifts the focus from partial perspectives that frame known issues to multiple perspectives with the potential to discover unknown issues. The same shift in focus also reframes the work of ownership. The value of a discrete, particular technical, or practical solution at a given moment resides in its shaping influence of an imminent, future defined in terms of stakeholder relationships to that solution and their underlying attachments (i.e., dependencies and commitments) to the issue.

Common Challenges Among Emerging Approaches In addition to the unique features of each emerging approach to design research, they all share practical and scholarly challenges as well. With respect to practical challenges, these approaches are inherently time intensive that create challenges for participants who face competing demands for their time and for full-time researchers as well. In particular, researchers working in university settings must reconcile their contributions relative to college or university expectations for faculty productivity, which is often measured in terms of traditional publication formats like journal articles and, therein, typically fails to recognize or evaluate public and engaged

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

825

forms of scholarship (Fischman, Zuiker, Tefera, & Anderson, in review). A corresponding challenge revolves around grant funding, which often supports projects for brief periods of time relative to the timescales along which these emerging approaches develop, establish, and sustain partnerships with multiple communities and institutions. Moreover, grant funding often requires specific objectives and measurable outcomes that either precede partnerships or preclude ongoing negotiations, thereby positioning researchers with a greater responsibility, if not also agency, over projects. Partnerships among communities and institutions are also challenging because they remain fragile. Partnerships can change, breakdown, and, at one point or another, struggle to function productively as tensions between individual and institutional interests unfold. These practical challenges communicate the complex nature of approaches to design research that focus on stakeholder ecosystems. Several scholarly challenges are also apparent. For example, the dynamic nature of partnerships can obscure who design agendas are ultimately for from one moment or phase to the next as well as what a project partnership ultimately may contribute to theory or practice. Revisions and refinements to designs operate in complex contexts; while they lead to improvements in one setting, there is no guarantee they will extend the reach of the design to other sites as well. These common challenges underscore that in attempting to address some limitations of DBR, each emerging approach continues to wrestle with enduring tensions in educational research, a broader backdrop that also frames the epistemological comparisons considered next.

Epistemological Comparison of Approaches In addition to comparing established and emerging design-based approaches to design research from within the learning sciences, comparing them in relation to broader traditions in educational research can be equally illuminating. While there is significant common ground, approaches to design research articulate different methodological and epistemological perspectives. Like the three guiding questions employed above, these differences organize useful comparisons. This section therefore considers the methodological underpinnings of these five approaches in terms of a framework for general research traditions in education. Building on this framing, this section also considers epistemological commitments in terms of relationships between knowledge and power. To characterize the methodological underpinnings of design-based approaches, Moses and Knutsen (2012) provide a framework for conceptualizing research traditions as falling somewhere in either a naturalistic or constructivist perspective, and flexible in, but inextricable from, their corresponding epistemologies and ontologies. Framing methodological traditions in terms of naturalistic-constructivist dichotomy distinguishes them based on whether knowledge derives primarily from observation of the natural world (i.e., naturalistic) or from the construction of ideas and concepts through social processes (i.e., constructivist). These distinctions also extend to relationships between knowledge and power. That is, whereas the naturalist perspective believes in the possibility of value-free knowledge, the constructivist

826

S. J. Zuiker et al.

perspective assumes that knowledge is never value-free but instead entangled with systems of power. Table 1 below characterizes the five approaches to design research in terms of Moses and Knutsen’s (2012) framework. The remaining rows draw on the answers each approach provides to the three guiding questions in order to characterize the kinds of stakeholder contexts in which the approach is typically employed, the forms of expertise it recognizes, and types of impact it achieves. The epistemological commitments in Table 1 resolve relationships between knowledge and power. Paying particular attention to relationships between knowledge and power is important because each emerging approach advocates for a plurality of stakeholders whose inherently perspectival contributions seek some form of consensus and shared rationale. The remainder of this section develops two points of comparison that characterize these five approaches. First, emerging approaches increasingly attend to the role of the researcher, which is reflected in the constructivist orientations ascribed to SDE, PDR, and IP. This increasing attention to the role of the researcher also relates to increasingly reflexive scholarship involving dual inquiry into a research subject and the research process itself. Second, methodological shift address the relationship between the research process and the power relationships embodied in the construction of knowledge. More specifically, as design research focuses on the researcher’s role in shaping social systems, design researchers attune to how designs and design processes alike interact with existing structures of power and knowledge. By recognizing and attending to relationships between knowledge and power, emerging Table 1 Epistemological comparison of approaches to design research Orientation Knowledge Theory and inference

Stakeholder context

Expertise

Impact

Approach to design research DBR DBIR Naturalist Naturalist Theory Theory building, building, sometimes sometimes towards towards generalizable, generalizable, testable claims testable claims Abductive Abductive and deductive Classrooms Districts, (ideographic) networks (ideographic and nomothetic) Researcher Shared among stakeholders

Functionalist or pragmatic

SDE Constructivist Critical and historical

Interpretive

PDR IP Constructivist Constructivist Critical and intersectional (colonial, racialized, gendered, queered) Interpretive

Communities (ideographic)

Communities (ideographic)

Publics (ideographic)

Communitybased

Communitybased

De-centered and shared among stakeholders Pragmatic

Functionalist or Radical pragmatic

Radical

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

827

approaches seek to realize productive and sustainable transformations of learning, teaching, and schooling. In order to establish these points, a history of approaches to design research in the learning sciences traces relationships between knowledge and power from DBR to DBIR, SDE, PDR, and IP. From the inception of DBR, its practitioners have engaged in debates over the status and relevance of their research. According to its early proponents (i.e., Brown, 1992; Collins, 1992), the impetus for DBR was based in the belief that researchers need intimate knowledge of the context of learning in order to capture how learning and cognition happens. This belief stood in contrast to the prevailing assumptions and practices of traditional positive scientists who adhered to the nomothetic approach of hypothesis testing in large-N and experimental studies. In the view of traditional scientists, the value of knowledge claims was based on the ability to replicate and generalize findings to other settings (e.g., Shavelson, Phillips, Towne, & Feuer, 2003). Design researchers, on the other hand, argued that this paradigm had failed to produce findings of any real practical use. For design researchers, findings in the traditional paradigm may be valid at the general level, but these findings were too vague and general to offer any guidance in an applied setting. However, because traditional scientists refused to recognize the ideographic and interventionist methodology of DBR as scientifically valid, Shavelson et al. (2003) argued that design research was best used for exploratory studies and generating hypotheses that could be tested nomothetically. Thus, in its earliest iterations, the debate over design research centered the relative validity of nomothetic methodology and deductive inference versus ideographic and interventionist methodology. While design researchers were skeptical of the knowledge claims derived from nomothetic methodologies, many held to the naturalistic view of knowledge and sought to ground design research according to the systematic and rigorous practices of natural science. As Brown (1992) explained, “I attempt to engineer interventions that not only work by recognizable standards but are also based on theoretical descriptions that delineate why they work, and thus render them reliable and repeatable” (p. 143). In this way, design research was still concerned with validating theoretical propositions, albeit through inductive and abductive, rather than deductive, inference. Further, Collins (1992) advocated, among other things, “objective evaluation” of designs where designers were not also evaluating their own designs” (p. 5). DBR initially operated squarely within naturalist understandings of knowledge while shifting to ideographic and interventionist methodologies in order to broaden impact by better accounting for local contexts. Although naturalistic views of knowledge have continued to hold for many design researchers, the interventionist nature of design research has led to an acknowledgement of, and scholarly interest in, the influence researchers have on the outcomes of designs. As Barab and Squire (2004) write, “Education is an applied field, and learning scientists bring agendas to their work, seeking to produce specific results” (p. 2). For some, this acknowledgement of the researcher’s role does not necessarily preclude a naturalistic methodology, although it does require a rejection of strict objectivity. Instead, it has prompted some researchers to consider the design research process as part of a broader context of education policy and practice. Barab

828

S. J. Zuiker et al.

and Squire (2004) note, “participating in local educational practices places researchers in the role of curriculum designers, and implicitly, curriculum theorists who are directly positioned in social and political contexts of educational practice (both global and local) and who are accountable for the social and political consequences of their research programs” (p. 2). This has especially been the case for DBIR researchers, who have sought to broaden the scope of design research from individual classrooms to schools, districts, and practitioner networks (e.g., Penuel et al., 2011). In DBIR research, reflexive consideration of the researcher’s role has been driven by a desire to make research findings useful and relevant within the context of existing educational systems. The instrumental approach to research expressed in DBR and DBIR is indicative of a functionalist view of knowledge that sees research as a way to improve the functioning of educational institutions. DBR and DBIR researchers have stressed pragmatism as a guiding value for their work. For some, this has meant eschewing epistemological and theoretical approaches that they perceive as lacking utility. As Cobb et al. (2003) write, “The theory must do real work. General philosophical orientations to educational matters – such as constructivism – are important to educational practice, but they often fail to provide detailed guidance in organizing instruction” (p. 10). In other cases, though, this same pragmatism has led to the embrace of more constructivist perspectives, as when Barab and Squire (2004) argue that “the value of a theory lies in its ability to produce changes in the world. Such a system of inquiry might draw less from traditional positivist science or ethnographic traditions of inquiry, and more from pragmatic lines of inquiry where theories are judged not by their claims to truth, but by their ability to do work in the world (Dewey, 1938)” (p. 6). However, while many DBR and DBIR researchers have recognized the social and political implications of their work, their commitment to pragmatism has led them to focus on improving the existing institutional processes and structures of education. For other design research approaches, however, acknowledging the researcher’s role has meant adopting constructivist perspectives that are critical of the existing power relationships that constitute systems of knowledge. This embrace of constructivist perspectives has been the impetus for SDE, PDR, and Infrastructuring Publics approaches, each of which varies in how they address the role of design researchers and how they conceptualize power structures. In describing SDE, Gutiérrez and Jurow (2016) situate their approach in contrast to DBR and DBIR, which “generally work inside existing institutions,” whereas in the SDE perspective, “working to transform social institutions and their relations is a primary [emphasis added] target of design because only such changes can achieve the equity goals of the research” (p. 565). To that end, SDE sees community participation in the design process as a way to create learners who become “more intentional, historical actors (Espinoza, 2003) who can become designers of their own futures (Gutiérrez, 2008)” (Gutiérrez & Jurow, 2016, p. 566). Like SDE, PDR researchers share an understanding of the role and nature of participation in designing research projects. However, PDR puts a much greater emphasis on the role of researcher within the research context, insisting that the researcher’s role be critically

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

829

interrogated as part of the analysis. As Bang and Vossoughi (2016) explain, “we argue that as claims to and participation in equity work expand, the axiological and ideological underpinnings (Patel, 2016) of equity-seeking research need to become transparently communicated by researchers” (p. 176). For SDE and PDR researchers, then, engaging with the systems of power that shape research knowledge is an essential part of realizing the goals of their work. While researchers involved in Infrastructuring Publics share the view that research processes can and should confront and reshape systems of knowledge and power, they advocate a pragmatic approach that “brings capacity building and associative politics to the fore” of the goals for design interventions (LeDantec & DiSalvo, 2013, p. 248). In this way, the researcher’s role is not to implement a design, as in DBR and DBIR, nor is it to collaborate with participatory design processes in the same way as SDE and PDR approaches. Instead, the researcher’s role in Infrastructuring Publics is to build an infrastructure guided by a pragmatic view of politics that will foster participatory design processes. In other words, these efforts are less concerned with critically interrogating systems of power and more concerned with building systems where power relationships are in accord with pragmatic and participatory views of democracy. Taken together, the three questions and epistemological considerations guiding the comparison of emerging approaches in this chapter provide partial perspective on each approach rather than comprehensive reviews. Developing such a perspective, however, affords a view of how these approaches expand design research, which the remainder of the chapter will discuss.

Discussion The five approaches considered above expand design research by considering design at multiple levels, across multiples settings, and with respect to dynamic notions of context. This section concludes the chapter with a discussion of two themes emerging from this consideration. The first theme considers how the arc of design research in the learning sciences and educational technology is expanding from processes driven by researchers to processes that foster and sustain greater agency for the stakeholders in design projects. The second theme considers attendant tensions associated with shifting design processes from researcher “ego-systems” to stakeholder ecosystems.

Expanding Design from Researcher Ego-Systems to Stakeholder Ecosystems Design-based research organizes design agendas around the articulation and refinement of theory that is put in harm’s way (Cobb et al., 2003). By embodying theory in designs that can be enacted in authentic settings, researchers can interrogate conjectures in the crucible of complex social ecologies. At the same time, the fact that

830

S. J. Zuiker et al.

learning scientists and educational technologists typically author these theories, embody them in design, and refine them in light of stakeholder enactments grounds the design process almost exclusively in researcher perspectives and evolves it in relation to researcher expertise. In this way, design-based research organizes design processes around one stakeholder group and runs the risk of becoming bound up in what might loosely be described as the researcher’s own “ego-system.” That is, design-based research articulates design processes in terms of scholarly agendas in the short term with a view to developing useable insights and innovations in the longer term. In contrast, the four emerging approaches to design research featured in this chapter suggest alternative approaches. Each organizes design process in relation to both scholarly and stakeholder agendas, holds designs accountable to both scholarly and stakeholder communities, and therein expands the scope of design from discretely bounded social systems like classrooms to the messy entanglements of social ecosystems. This expansion provokes further consideration of context and how context matters in design. Considering design approaches in relation to social ecologies begins to reframe how context matters. Gutiérrez (2016) problematizes Bronfenbrenner’s (1979) characterization of a social ecology as neatly nested and concentric spheres of influence upon individual experiences and circumstances. Similarly, in his foreword to Bronfenbrenner’s book, Cole (1979) acknowledges how “the infinite tangles of past experience and present circumstances that make us what we are smother us in particulars, defying explanation or generalization; faced with such complexity, any plausible simplifying procedure can appear to be a lifeline” (p. viii). Rather than being “concentric circles and nested-Russian-dolls” (Gutiérrez, 2016, p. 188), Gutiérrez cites Packer’s (2011) metaphor of tangled roots in order to characterize social ecologies as dynamic, contested, and dispersed. Emphasizing the idea of interconnectedness in this metaphor, the emerging approaches featured above seek to expand design research by methodically interrogating how designs and design processes themselves become entangled in these roots. A challenge that follows from this metaphor is to not only illuminate these entanglements but to transform them. In turn, it also entails understanding together with stakeholders how they do (and can) exercise agency over designs in order to negotiate these entanglements. In this way, social ecologies locate not in the work of scaling educational innovations as design products but rather in the work of scaling learning among stakeholders engaged with continuously evolving design services. Latour (1996, p. 234), for example, emphasizes that local and global scales remain grounded in this practical work: “for humans, an abyss seems to separate individual action from the weight of a transcendent society. But this is not an original separation [. . .] it is an artifact created by the forgetting of all practical activities for localising and globalizing.” Whether innovations are products or services, they remain a unit of concern, but transforming practical activity remains a primary unit of analysis. Accounting for efficacy and variation alike reside in interactions between people and environments, past and present (e.g., Zuiker, 2012). Again drawing on Cole’s (1979) foreword to Bronfenbrenner’s book, these emerging approaches to design research reflect complementary intuitions shared with Latour, Bronfenbrenner, and

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

831

other theorists who can be traced back across a century of research on learning (e.g., Penuel et al., 2016). Importantly, “these are ideas worth having again and again until we are ready to exploit their power” (Cole, 1979, p. x).

Tensions in Ecosystemic Design Processes In expanding design research from ego-systems to ecosystems, the four emerging approaches also raise new opportunities and challenges for organizing ecosystemic design processes. Expanding the scope of design from discretely bounded social systems like classrooms to the messy entanglements of social ecosystems also provokes reconsideration of design processes. The purpose of design is to resolve tensions between what is and what ought to be. To this end, Tatar (2007) develops the idea of design tensions in order to identify and resolve probably relevant criteria and choices from the totality of possibly relevant ones. This idea reasonably extends to design processes as well, and suggests design process tensions, including tensions between thought and action, between vision and agency, and between resonance and resilience.

Design Tension 1: Thought and Action Design processes revolve around the coupling of thought and action. A quote often attributed to Kurt Lewin suggests the important relationship between these two: “if you want to understand something, try to change it.” Thought and action are each necessary in order to transform social activity and educational systems. However, power relations between researchers and stakeholders as well as among different stakeholder groups obviously mediate thought and action. When researchers think alone, design processes remain isolated or sequestered. Expanding the scope of design to focus on ecological systems requires multiple stakeholders to each engage in thought and action. When they share thought and action (i.e., codesigning side by side), accounts of how and why designs operate can move beyond the limitations of simplified or essentialized portraits (Erickson, 2006) or seductive reductions. Each of the approaches to design research featured in this chapter privileges joint activity and mutual relations among stakeholders in a social ecosystem. For example, social design experiments (SDE) and participatory design research alike attend to the social positionality of stakeholders involved in interventions, particularly researchers. Several SDEs described by Jurow and colleagues (Jurow, Teeters, Shea, & Van Steenis, 2016) underscore that making the influence and value of participants’ work visible affords learning across varied positions and from multiple perspectives. It also raises a second design process tension concerning how thoughts cohere into a vision that can be taken as shared and how individual actions cohere into forms of agency. Design Tension 2: Vision and Agency Building on the first design process tension in ecosystemic design, thoughts reflect and inform vision while action exercises agency. As design research organizes many-to-many engagement among stakeholders, the density and quality of social relations and social interactions suggests a second design process tension between

832

S. J. Zuiker et al.

the plurality of stakeholder perspectives informing design processes and the forms of agency that entangle stakeholders with design processes. PDR, for example, underscores that expanding design approaches entails more than attending to partnerships or stakeholder relations; it entails co-organizing mutual responsibility among stakeholders (Bang & Vossoughi, 2016). Together, these design process tensions characterize design in terms of how the shorter-term events that afford thought and action and longer-term processes that afford vision and agency. Similarly, thought and action shape events in particular settings while vision and agency shape the future of the broader context or system. DBIR researchers seek to partner with districts “where they shared a broad vision for improvement” (Penuel et al., 2011, p. 334). This shared vision is coupled with mutualism in order to establish multi-way systems of exchange that enable stakeholder groups to exercise agency over the vision; that is, mutualism positions multiple stakeholders to contribute in ways that influence whether and when to affirm, adapt, or abandon a design in order to realize a shared vision. Regardless of how design processes navigate tensions between vision and agency, design inevitably remains contested and open to ongoing negotiation, evolution, and further innovation. If multiple stakeholders are to sustain engagement in a design challenge, they must perceive it as significant, persistent, and worthy of time, which suggests a third design process tension between resilience and resonance.

Design Tension 3: Resilience and Resonance Design processes must remain resilient in order to embody underlying principles and purposes, and yet they must also establish resonance with the interests and issues of multiple stakeholders. Resilience is critical if designs are to be useful and used within social ecosystems; resonance, meanwhile, is also critical in order to reflect competing thoughts and visions. Drawing on resilience in natural ecologies, Gutiérrez (2016) describes social ecologies in terms of adaptability, their ability to cope with, shape, and adapt to changes. Resilience therefore entails capacity to transform social systems in order to affirm or enhance its resonance with multiple stakeholders engaged in a design agenda. SDEs suggest that design processes are consequential insofar as learning and change express systemic qualities, such as when change is taken up within and across temporal, social, and spatial scales of action (Jurow et al., 2016). Consequential design agendas intervening in stakeholder ecosystems remain dynamic, contested, and dispersed. Tensions between resilience and resonance challenge participants to engage in ongoing negotiations that affirm or evolve insights and innovations.

Conclusions This chapter considered established and emerging approaches to design research employed in the learning sciences and educational technology. It characterized design-based research (DBR) as an established approach, and then in relation to participatory design as a complementary approach to design research employed in other disciplines. Next, it compared DBR to emerging approaches to design research

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

833

in the learning sciences and educational technology: design-based implementation research, social design experiments, participatory design research, and Infrastructuring Publics. Each of the four emerging approaches builds on the foundation of DBR, affirming that context fundamentally matters both for developing theoretical insights that consider what works, how, and why. Each builds on this basic commitment by also recognizing that involving stakeholders in developing and sustaining innovations is also necessary in order to account for what works, how, and why. As such, developing and scaling innovations also entails scaling learning that fosters attendant social innovations and therein accounting for and leveraging a wider range of knowledge and expertise that stakeholders draw upon and produce. In this way, each approach concentrates on design as a situated and ongoing process; each, in turn, positions design processes as joint activity with stakeholders during and sometimes before and after developing designs. Importantly, the chapter is not exhaustive and does not consider all emerging approaches to design research. For example, Gutiérrez et al. (2016) highlight formative interventions, community-based design, and participatory action research. Meanwhile, Gutiérrez and Penuel (2014) identify Cobb and Jackson’s policy analysis using a learning design perspective (2012) as well as implementation science. The characterizations and comparisons above therefore provide a foundation for future research that can expand on this chapter by considering additional approaches and elaborating the scope of analysis.

References Amiel, T., & Reeves, T. (2008). Design-based research and educational technology: Rethinking technology and the research agenda. Journal of Educational Technology & Society, 11(4), 29–40. Anderson, T., & Shattuck, J. (2012). Design-based research a decade of progress in education research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813. Bang, M., & Vossoughi, S. (2016). Participatory design research and educational justice: Studying learning and relations within social change making. Cognition and Instruction, 34(3), 173–193. https://doi.org/10.1080/07370008.2016.1181879. Barab, S. A. (2014). Design-based research: A methodological toolkit for engineering change. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed., pp. 151–170). New York, NY: Cambridge University Press. Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14. Berliner, D. C. (2002). Comment: Educational research. The hardest science of all. Educational Researcher, 31(8), 18–20. Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press. Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141–178. Chi, M. (1997). Quantifying analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6(3), 271–315. Cobb, P., & Jackson, K. (2012). Analyzing educational policies: A learning design perspective. Journal of the Learning Sciences, 21(4), 487–521. https://doi.org/10.1080/10508406.2011.630849.

834

S. J. Zuiker et al.

Cobb, P., Confrey, J., DiSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. Coburn, C. E., & Stein, M. K. (2010). Research and practice in education: Building alliances, bridging the divide. New York, NY: Rowman & Littlefield Publishers. Cole, M. (1979). Foreword. In U. Bronfenbrenner (Ed.), The ecology of human development (pp. vii–vix). Cambridge, MA: Harvard University Press. Collins, A. (1992). Towards a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology (pp. 15–22). New York, NY: Springer. Dewey, J. (1938). Experience and education. New York: Touchstone. DiSalvo, C. (2009). Design and the construction of publics. Design Issues, 25(1), 48–63. https://doi. org/10.1162/desi.2009.25.1.48. DiSalvo, B., & DiSalvo, C. (2014). Designing for democracy in education: Participatory design and the learning sciences. In J. L. Polman, E. A. Kyza, K. O’Neill, I. Tabak, W. R. Penuel, S. Jurow, . . ., L. D’Amico (Eds.), Proceedings of the eleventh international conference of the learning sciences (Vol. 2, pp. 793–799). Boulder, CO: International Society of the Learning Sciences. DiSalvo, C., Clement, A., & Pipek, V. (2012). Participatory design for, with, and by communities. In S. Jesper & T. Robertson (Eds.), International handbook of participatory design (pp. 182–209). Oxford, England: Routledge. Ehn, P. (1993). Scandinavian design: On participation and skill. In D. Schuler & A. Namioka (Eds.), Participatory design: Principles and practices (pp. 41–77). Hillsdale, NJ: Lawrence Erlbaum. Engeström, Y. (2008). From teams to knots: Activity-theoretical studies of collaboration and learning at work. Cambridge, England: Cambridge University Press. Engeström, Y. (2011). From design experiments to formative interventions. Theory & Psychology, 21(5), 598–628. https://doi.org/10.1177/0959354311419252. Erickson, F. (2006). Studying side by side: Collaborative action ethnography in educational research. In G. Spindler & L. Hammond (Eds.), Innovations in educational ethnography: Theory, methods and results (pp. 235–257). Mahwah, NJ: Lawrence Erlbaum. Fishman, B., Penuel, W. R., Allen, A., Cheng, B. H., & Sabelli, N. H. (2013). Design-based implementation research: An emerging model for transforming the relationship of research and practice. National Society for the Study of Education Yearbook, 112(2), 136–156. Gutiérrez, K. D. (2016). Designing resilient ecologies: Social design experiments and a new social imagination. Educational Researcher, 45(3), 187–196. https://doi.org/10.3102/ 0013189X16645430. Gutiérrez, K. D., & Jurow, A. S. (2016). Social design experiments: Toward equity by design. Journal of the Learning Sciences, 25(4), 565–598. https://doi.org/10.1080/10508406.2016.1204548. Gutiérrez, K. D., & Penuel, W. R. (2014). Relevance to practice as a criterion for rigor. Educational Researcher, 43(1), 19–23. https://doi.org/10.3102/0013189X13520289. Gutiérrez, K. D., & Vossoughi, S. (2010). Lifting off the ground to return anew: Mediated praxis, transformative learning, and social design experiments. Journal of Teacher Education, 61(1–2), 100–117. https://doi.org/10.1177/0022487109347877. Gutiérrez, K. D., Engeström, Y., & Sannino, A. (2016). Expanding educational research and interventionist methodologies. Cognition and Instruction, 34(3), 275–284. https://doi.org/ 10.1080/07370008.2016.1183347. Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. Journal of the Learning Sciences, 4(10), 39–104. Jurow, A. S., Teeters, L., Shea, M., & Van Steenis, E. (2016). Extending the consequentiality of “invisible work” in the food justice movement. Cognition and Instruction, 34(3), 210–221. https://doi.org/10.1080/07370008.2016.1172833. Kelly, A. E., Lesh, R. A., & Baek, J. Y. (Eds.). (2008). Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching. New York, NY/London, England: Routledge. Kolodner, J. L. (1991). Editorial: The journal of the learning sciences: Effecting changes in education. Journal of the Learning Sciences, 1(1), 1–6.

32

Expanding Design Research: From Researcher Ego-Systems to Stakeholder. . .

835

Kolodner, J. L. (2004). The learning sciences: Past, present, and future. Educational Technology, 44(3), 37–42. Latour, B. (1996). On interobjectivity. Mind, Culture, and Activity, 3(4), 228–245. https://doi.org/ 10.1207/s15327884mca0304_2. LeDantec, C. A., & DiSalvo, C. (2013). Infrastructuring and the formation of publics in participatory design. Social Studies of Science, 43(2), 241–264. https://doi.org/10.1177/0306312712471581. McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. New York: Routledge. Moses, J., & Knutsen, T. (2012). Ways of knowing: Competing methodologies in social and political research. Basingstoke: Palgrave MacMillan. Nathan, M. J., Rummel, N., & Hay, K. E. (2014). Growing the learning sciences: Brand or big tent? In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed., pp. 151–170). New York, NY: Cambridge University Press. Nutley, S. M., Walter, I., & Davies, H. T. O. (2007). Using evidence: How research can inform public services. Bristol: Policy Press. O’Neill, D. K. (2016). When form follows fantasy: Lessons for learning scientists from modernist architecture and urban planning. Journal of the Learning Sciences, 25(1), 133–152. https://doi. org/10.1080/10508406.2015.1094736. Oh, E., & Reeves, T. (2010). The implications of the differences between design research and instructional systems design for educational technology researchers and practitioners. Educational Media International, 4(47), 263–275. Packer, M. (2011). The science of qualitative research. New York, NY: Cambridge University Press. Penuel, W. R., Fishman, B. J., Haugan Cheng, B., & Sabelli, N. (2011). Organizing research and development at the intersection of learning, implementation, and design. Educational Researcher, 40(7), 331–337. https://doi.org/10.3102/0013189X11421826. Penuel, W. R., Cole, M., & O’Neill, D. K. (2016). Introduction to the special issue. Journal of the Learning Sciences, 25(4), 487–496. https://doi.org/10.1080/10508406.2016.1215753. Sanders, E. B. N. (2006). Design research in 2006. Design Research Society, 1(1), 1–8. Sanders, L. (2008). On modeling: An evolving map of design practice and design research. Interactions, 15(6), 13–17. Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5–18. Sanders, E. B. N., Brandt, E., & Binder, T. (2010). A framework for organizing the tools and techniques of participatory design. In Proceedings of the 11th biennial participatory design conference (pp. 195–198). New York, NY: ACM. Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32(1), 25–28. Tatar, D. (2007). The design tensions framework. Human-Computer Interaction, 22(4), 413–451. https://doi.org/10.1080/07370020701638814. Tyack, D., & Cuban, L. (1995). Tinkering toward utopia: A century of public school reform. Cambridge, MA: Harvard University Press. Vargo, S. L., & Lusch, R. F. (2004). Evolving to a new dominant logic for marketing. Journal of Marketing, 68(1), 1–17. Voorberg, W. H., Bekkers, V. J. J. M., & Tummers, L. G. (2015). A systematic review of co-creation and co-production: Embarking on the social innovation journey. Public Management Review, 17(9), 1333–1357. https://doi.org/10.1080/14719037.2014.930505. Weick, K. E. (1995). Sensemaking in organizations (Vol. 3). Thousands Oaks, CA: Sage. Zuiker, S. J. (2012). Educational virtual environments as a lens for understanding both precise repeatability and specific variation in learning ecologies: EVEs for repeatability and variation. British Journal of Educational Technology, 43(6), 981–992. https://doi.org/10.1111/j.14678535.2011.01266.x.

836

S. J. Zuiker et al.

Steven Zuiker is an Assistant Professor of Educational Technology and Learning Sciences at Mary Lou Fulton Teachers College, Arizona State University. His research combines his interests in the design of interactive learning environments and how these designed environments inform our understanding not only of learning but also the general consequences of learning. In contrast to a conventional views of knowledge transfer, Zuiker enlists the idea of learning transitions in between physical and virtual realities in K-12 education settings and between researcher and practitioner activity systems. Zuiker is designing cyberinfrastructure to sustain and expand project-based gardening to support garden-based learning and outdoor education. He is also a coprincipal investigator on a Spencer Foundation grant examining how social and cyberinfrastructure extend the reach and impact of educational scholarship. Niels Piepgrass is currently a doctoral student in the Educational Policy and Evaluation program at Mary Lou Fulton Teachers College, Arizona State University. His research interests consider democratic theories of education operating in educational policy and teacher-led schools. Mathew D. Evans is currently a doctoral student in the Learning, Literacies, and Technologies program at Mary Lou Fulton Teachers College, Arizona State University. His research interests focus on the intersection of informal learning spaces, technology, and design.

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

33

Jonan Phillip Donaldson and Brian K. Smith

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Designerly Ways of Knowing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Treating All Problems as Wicked Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Framing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Empathetic Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Contextualized Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Abductive Reasoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rapidly Changing Goals and Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Divergent and Convergent Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prototyping from Abstract to Concrete . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Constructing Prototypes According to the Meaning You Construct . . . . . . . . . . . . . . . . . . . . . . . Engaging in Reflection-in-Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reflecting on Relevance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interdependence in Designerly Ways of Knowing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design Thinking Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Frame Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ideation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deploying in Real-World Contexts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Principles of Constructionist Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Situating Learners as Designers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creation of Artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Focused Tinkering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learner Agency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Authentic Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

838 839 840 841 841 841 842 842 842 843 843 843 844 844 845 846 846 847 847 847 848 848 848 849 849 850

J. P. Donaldson (*) School of Education, Drexel University, Philadelphia, PA, USA e-mail: [email protected] B. K. Smith ExCITe Center and School of Education, Drexel University, Philadelphia, PA, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_73

837

838

J. P. Donaldson and B. K. Smith

Intersections, Synergies, and Alignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Design Thinking for Engaged Learning Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion: Learning as Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

850 851 852 856 856

Abstract

Learning is often framed in the learning sciences through a metaphor of construction, a perspective which describes learning as a process in which meaning is individually and socially constructed. Design thinking, designerly ways of knowing, and principles of constructionist learning provide powerful lenses through which to view the process of construction. The design thinking for engaged learning (DTEL) framework brings strands of research regarding design thinking, designerly ways of knowing, and constructionist learning together as a set of principles through which to develop and evaluate designs of constructionist learning environments. Keywords

Design thinking · Designerly ways of knowing · Constructionist learning · Engagement

Introduction The cognitive processes expert designers use, the strategies involved in the design process, and constructionist principles of learning share a number of features. This suggests the possibility of integrating the three domains into a framework for the design of learning environments in which learning is structured around the design process. Designerly ways of knowing are the cognitive approaches and mindsets characteristic of expert designers such as framing, reflection-in-action, and abductive reasoning. Design thinking strategies are the processes involved in design, including frame creation, ideation, prototyping, iteration, and deploying in real-world contexts. Design thinking strategies and designerly ways of knowing align with constructs in the educational and learning sciences literature such as metacognition, situated learning, and critical problem solving. This alignment suggests that design thinking strategies and designerly ways of knowing could be instrumental in facilitating engaged learning. A structuring framework for designs in which learners use design thinking strategies and develop designerly ways of knowing could be crafted around the principles of constructionist learning. In constructionist learning, learners are situated as designers, learning is structured around the creation of artifacts, learners engage in focused tinkering, learners have agency, and artifacts are created for an authentic audience. This chapter will explore the unique features of designerly ways of knowing, design thinking strategies, and constructionist learning. Potential synergies between

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

839

features in each of the three domains will be used as the basis for a proposed design thinking for engaged learning (DTEL) framework. This framework is an attempt to investigate the proposition that design thinking strategies and designerly ways of knowing could inform the design of tools, activities, and environments for engaged learning aligned with the principles of constructionist learning.

Designerly Ways of Knowing There are two distinct lines of design thinking literature, one which focuses on the process of design and another which focuses on what goes on in the mind of the designer (Johansson-Sköldberg, Woodilla, & Çetinkaya, 2013). The literature around designerly ways of knowing describes a set of cognitive skills and approaches unique to design experts. This line of research is interested in describing the cognitive strategies and mindsets typical of experts in design. Designerly ways of knowing developed historically from: (1) attempts to “scientize” design, (2) differentiation between scientific processes and design processes, and (3) exploration of the nature of ill-defined and ill-structured “wicked” problems (Cross, 2006). Cross (2006) suggested that the discipline of design is defined by unique and distinct designerly ways of knowing. These designerly ways of knowing can be best understood by differentiation between design research and research in other domains as summarized in Table 1. The first distinction is that the phenomenon of study in design research is the artificial, while in science it is the natural world and in the humanities it is human experience. Second, in design the appropriate methods of study are modeling, patternformation, and synthesis, while in science they are experimentation, classification, and analysis, and in the humanities they are analogy, metaphor, and evaluation. Finally, the values in design are practicality, ingenuity, empathy, and appropriateness, while in science they are objectivity, rationality, neutrality, and truth, and in the humanities they are subjectivity, imagination, commitment, and justice (Cross, 2006). Researchers in design research have investigated the characteristics of cognitive processes used by expert designers. The literature suggests that designerly ways of knowing include a set of interrelated cognitive processes. We will explore each in turn. Table 1 Differentiation between design, science, and the humanities (as derived from Cross, 2006) Phenomenon of study Methods

Values

Design The artificial

Science The natural world

Humanities Human experience

Modeling, patternformation, synthesis

Experimentation, classification, analysis Objectivity, rationality, neutrality, truth

Analogy, metaphor, evaluation

Practicality, ingenuity, empathy, appropriateness

Subjectivity, imagination, commitment, justice

840

J. P. Donaldson and B. K. Smith

Treating All Problems as Wicked Problems Design problems are by nature “ill-defined, ill-structured, or ‘wicked’” (Cross, 2006, p. 7). Rittel and Webber (1973) introduced the concept of wicked problems as a response to the dominant outcomes-driven approach to problems in academia and society. They described wicked problems in terms of ten defining features: 1. Definitive formulation is impossible. Any formulation, or description, of the problem is dependent on the formulation of a solution. Because every solution formulated will result in different formulations of the problem, it is not possible to achieve a definitive formulation of the problem. 2. No stopping rule. There can be no criteria for determining when the process of developing a solution is complete. The solution is always subject to improvement, or to replacement by another solution. 3. True-or-false solutions are impossible. Solutions can only be judged as better or worse. It is not possible to judge solutions to wicked problems as true or false, correct or incorrect. 4. Immediate and ultimate tests of solutions are impossible. Implementations of solutions to wicked problems produce multiple chains of consequences which continue over time. Because it is impossible to identify and describe all positive and negative consequences, it is impossible to test how good the solution is. 5. Every solution attempted is consequential. Every solution to wicked problems has real-world consequences, and it is impossible to undo the chain of consequences. Every attempt to alter a solution results in new wicked problems. 6. Exhaustive sets of potential solutions are impossible. Because each solution considered results in a reframing of the problem, it is impossible to determine if all possible solutions have been considered. 7. Each problem is essentially unique. Although a wicked problem may share characteristics with other problems, it is possible to find characteristics which are unique and of greater importance because attempts to apply solutions which were appropriate to those other problems would result in unacceptable consequences when applied to the current problem. 8. The problem can be considered a symptom of another problem. The causes of any wicked problem can be traced to other wicked problems, which in turn can be traced to other wicked problems. 9. Multiple explanations are possible. Because the description of a wicked problem is dependent upon potential solutions, it is impossible to develop a definitive explanation of the problem. 10. The designer has no right to be wrong. Because implementation of solutions to wicked problems have real-world impacts, designers are responsible for those impacts. Unlike scientific development and testing of hypotheses in which refutation is expected and proof is impossible, designers must develop solutions which have positive impacts in the world. When faced with a seemingly straightforward or tame problem, the expert designer recognizes that all design problems are wicked problems. If the

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

841

determination is made that it is not a design problem, the designer can solve the problem through nondesign processes (application of a mathematical formula, for instance), hand the problem over to someone else, or reformulate the problem as a wicked problem. Even if a problem appears tame, designers recognize design potential because many wicked problems are mistakenly identified as tame (Coyne, 2005), and tame problems can be transformed into wicked problems (Buenano, 1999). Treating all design problems as wicked problems is a necessary prerequisite to – and integral part of – framing (Cross, 2006).

Framing Schön (1983, 1984, 1995) argued that expert designers engage in framing and reframing as reflective practitioners. Dorst (2011, 2015) built on Schön’s frame theory to develop a theoretical framework in which he identified the core theoretical principle of design thinking as the creation of frames. Framing is the creation of novel perspectives, standpoints, or positions from which a wicked problem can be tackled. It involves creative analysis and conceptualization of wicked problems and the complex nature of contexts within which problems are situated. Designers engage in simultaneous construction of working principles, specific values, and standpoints. Framing is “the act of proposing . . .a hypothetical pattern of relationships” (Dorst, 2015, p. 53). Framing is not an isolated stage of the design process, but rather a constant state of mind and a continual cognitive process: “The designer is always in conversation with the design situation” (Schön & Rein, 1994, p. 172). It is a complex act of parallel construction of a new thing (an object, service, experience, or system), its way of working (principles), a new standpoint, and a new value (Dorst, 2011).

Empathetic Thinking Framing includes various forms of perspective-taking and perspective-making, but empathetic thinking is the widely seen imperative (Köppen & Meinel, 2015). Empathetic thinking situates the designer in an emotional relationship to the problem and potential solutions via the various perspectives of real and imaginary individuals for whom the problem is tangible and solutions would be beneficial. This humancentric approach of expert designers not only informs the framing process but permeates all aspects of design (Razzouk & Shute, 2012).

Contextualized Thinking Whereas empathetic thinking pulls the designer into perspectives of individuals for whom the problem and potential solutions are relevant, contextualized thinking addresses the complexity of the imaginary or real context, including the social dynamics and physical considerations inherent to the environment. Designers “observe the world

842

J. P. Donaldson and B. K. Smith

in minute detail. They notice things that others do not and use their insights to inspire innovation” (Brown, 2008, p. 87). However, observation alone is inadequate to describe the contextualized thinking employed by designers. There is an ongoing interplay between framing, empathetic thinking, observation of environment, and embodiment practices related to ideation and prototyping. Due to this interplay, contextualized thinking is an aspect of design as a situated act (Suwa, Gero, & Purcell, 2000).

Abductive Reasoning Dorst (2011) integrated abductive reasoning theory into frame theory and argued that abductive reasoning is a cognitive process central to the work of the expert designer. Abductive reasoning theory proposes a third type of logical reasoning in addition to inductive reasoning and deductive reasoning. Abductive reasoning as a designerly way of knowing involves simultaneous parallel cognitive processes: (1) creation of an idealized characterization of the value a (yet undefined) solution to the ill-defined problem would create, (2) creation of a new frame which includes both value and working principles, (3) creation of a thing (object, process, experience, etc.) which works only in conjunction with the working principle, and finally (4) evaluation of whether the new thing plus the new principle can reasonably be expected to result in the new value. Abductive reasoning is demonstrated in the coevolution of the problem and the solution. Designers do not first develop an understanding of the problem and then develop solutions. Rather, they simultaneously work on the problem and solution: “solution conjectures should be used as a means of helping to explore and understand the problem formulation” (Cross, 2006, p. 80).

Rapidly Changing Goals and Constraints In contrast to scientific methods of problem solving in which there is rigid delineation of goals and constraints from the onset, designers rapidly change goals and constraints throughout the design process (Razzouk & Shute, 2012). This ability is necessitated by the nature of framing as a continual cognitive process and is mediated by abductive reasoning (Dorst, 2011; Cross, 2006).

Divergent and Convergent Thinking The essence of the work of the designer is creative problem solving. This creativity involves a variety of cognitive strategies which can be interpreted under the conceptual domains described in the creativity literature as divergent thinking and convergent thinking (Runco, 2014). Divergent thinking processes include creation of numerous potential solutions, using analogical thinking, creation of metaphors, and creation of novel perspectives in the framing process; convergent thinking processes include pattern recognition, synthetic thinking, and integrative thinking (Runco, 2014; Dorst, 2015).

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

843

Prototyping from Abstract to Concrete Contrary to popular conceptualizations, prototyping is not about exploring the feasibility of a design. Nor is it the physical act of creating a working model. Prototyping is an extension of the cognitive toolkit of the designer, a means of “thinking with your hands” (Brown, 2009). Prototyping often starts with sketching or diagramming but can take a multitude of forms ranging from physical objects to designed experiences, environments, or processes. As the design project proceeds, the prototypes and mental formulations simultaneously coevolve in a trajectory from abstract to concrete (Razzouk & Shute, 2012). However, it is not a smooth trajectory, but rather an oscillation – and sometimes simultaneous processing – between multiple layers of abstraction (Cross, 2006).

Constructing Prototypes According to the Meaning You Construct Also contrary to popular conceptualizations of prototyping, it is not simply a process of translating ideas into externalized reality. Instead, prototypes are what Seymour Papert called “objects-to-think-with” (Papert, 1980, p. 11). When prototyping becomes a practice of embodied cognition, prototypes are constructed according to the meanings constructed by the designer (Poulsen & Thøgersen, 2011). Thus, prototyping is part of the designer’s conversation with the situation in which meanings regarding the problem and solution take form in the prototype, and meanings emergent in the prototype “talk back” to the designer (Schön, 1983).

Engaging in Reflection-in-Action Reflection-in-action is distinct from other forms of reflective practice in that it arises from – and is situated within – the action of designing (Schön, 1995). During an instance of reflection-in-action, the designer investigates her understanding of the situation – restructuring her framing of the problem, her interpretation of what might be going on, and the strategies she has been employing through her actions. This restructuring suggests new strategies which lead to new actions. This entire reflective process occurs in the midst of the action and is inseparable from the action. Reflection-in-action helps designers make sense of the relationships between the processes and products emergent in divergent thinking and convergent thinking, between the abstract and the concrete, and between the ever-changing problem, solution, goals, constraints, and values. The designer “becomes a researcher in the practice context” and “constructs a new theory of the unique case” (Schön, 1983, p. 68). Each move the designer makes will produce unintended consequences, which pull the designer into reflective conversation with the ever-changing situation. Reflection-in-action and framing are intricately interwoven such that it is impossible to speak of framing without reflection-in-action, nor of reflection-in-action without framing:

844

J. P. Donaldson and B. K. Smith

each practitioner tries to adapt the situation to the frame . . .through a web of moves, discovered consequences, implications, appreciations, and further moves. Within the larger web, individual moves yield phenomena to be understood, problems to be solved, or opportunities to be exploited. . .But the practitioner’s moves also produce unintended changes which give the situations new meanings. The situation talks back, the practitioner listens, and as he appreciates what he hears, he reframes the situation once again. (Schön, 1983, pp. 131–132)

This reflection-in-action is rarely reflected upon by the expert designer – it is often seen as simply the nature of design, the craft of the designer. However, complex and novel problems require a degree of metacognitive awareness and regulation of the reflection-inaction process (Winne & Azevedo, 2014). Therefore, the expert designer not only engages in reflection-in-action, but also reflection upon reflection-in-action.

Reflecting on Relevance Frequently reflecting on relevance ties all the other designerly ways of knowing together. It helps the designer keep the wicked problem, constructed frame, empathetic perspective, and contextual elements within conscious awareness while engaging in other cognitive processes related to the design process. Relevance implies impact on multiple levels, ranging from the individual (the designer and individuals impacted by the design) to larger social, cultural, and political realms (Clark & Smith, 2010; Vogel, 2010), thereby tying the final implementation of the design back to where the whole process began with empathetic thinking (Razzouk & Shute, 2012). Reflecting on relevance is also a crucial aspect of identity exploration and development (Kaplan, Sinai, & Flum, 2014; Hutchinson & Tracey, 2015).

Interdependence in Designerly Ways of Knowing Designerly ways of knowing are an intricately woven set of interdependent cognitive practices. Separating them out into distinct entities in order to label and define them is problematic. Doing so can lead to the erroneous impression that they can be described, practiced, observed, or measured in isolation. Designerly ways of knowing must be considered and practiced holistically as represented in Fig. 1. Although it is possible to conceptualize aspects of designerly ways of knowing – wicked problems, framing, contextualized thinking, abductive reasoning, rapid changing of goals and constraints, divergent and convergent thinking, prototyping from abstract to concrete, constructing prototypes according to the meanings constructed by the designer, reflection-in-action, and reflecting on relevance – as bounded constructs, in practice they are aspects of each-other. No single aspect is possible without all the others, and activity in one aspect spreads like waves with repercussions in all other aspects of designerly ways of knowing. The positioning of the interrelated aspects in Fig. 1 is informed by the degree to which the literature regarding each aspect discusses the relationship to other aspects, as well as the sharing of conceptual features. For instance, framing is often discussed

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

Framing

845

Wicked Problems

Reflection-inAction

Abductive Reasoning

Reflecting on Relevance Divergent and Convergent Thinking

Contextualized Thinking

Constructing Prototypes according to the Meanings You Construct

Rapidly Changing Goals and Constraints

Prototyping from Abstract to Concrete

Fig. 1 Interrelated aspects of designerly ways of knowing

in relation to wicked problems (Dorst, 2011, 2015) and reflection-in-action (Schön, 1995). However, as indicated by the areas of overlap with other aspects of designerly ways of knowing, framing involves reflecting on relevance and abductive reasoning. Although the literature on framing may not directly discuss aspects such as rapidly changing goals and constraints, the conceptual foundations of rapidly changing goals and constraints share features with framing, such as always staying in conversation with the situation (Schön & Rein, 1994). These interrelations between aspects of designerly ways of knowing, and perhaps suggested proximities as well, may be important when operationalizing them in the design of learning contexts.

Design Thinking Strategies Another line of design thinking literature describes design thinking as a set of strategies organized into process models. This approach provides an alternative perspective to the cognitive view of design, and instead embraces a situated view of design as an embodied practice (Kimbell, 2011). This view has led to a conceptualization of design thinking as a set of design strategies, which have been formalized into various design thinking process models involving a series of steps or phases in the design thinking process (Johansson-Sköldberg et al., 2013). A design thinking process model is a structured collaborative problem-solving process which produces innovative product, process, or experience solutions (Welsh

846

J. P. Donaldson and B. K. Smith

& Dehler, 2013). The process involves a series of stages, typically including: (1) empathetic definition of the problem, (2) divergent thinking brainstorming, (3) convergent thinking pattern-recognition, (4) prototyping, and (5) validation (user-testing) of the prototype (Bell, 2008; Cleary, 2015; Mickahail, 2015; Watson, 2015). The most widely used design thinking process model was developed by Tim Brown, Tom Kelley, and others at the design firm IDEO (Collins, 2013). The IDEO process model involves five steps: (1) understand: become familiar with the problem from a user point of view; (2) observe: conduct real-world observations; (3) visualize: brainstorm, “ideate,” and visualize a wide range of possible solutions; (4) evaluate/ refine: select a solution, prototype, evaluate the prototype, and iterate; and (5) implement: deploy the solution (Bell, 2008). The IDEO design thinking process model emphasizes empathy and sustainability (Brown, 2008; Fuge & Agogino, 2015). Other popular process models include the Stanford model (Cleary, 2015; Glen, Suciu, Baughn, & Anson, 2015; Mickahail, 2015) and the Virginia Darden Business School and Design Management Institute model (Liedtka, 2014; Liedtka & Ogilvie, 2011). Design thinking activities are usually conducted in collaborative contexts. Design thinking strategies common across most design thinking process models include frame creation, ideation, prototyping, iteration, and deploying in real-world contexts.

Frame Creation Frame creation is the first stage of the design thinking process in which a problem is defined and situated through the creation of a novel perspective. It includes the procedural formulation of the cognitive process of framing, while treating the problem as a wicked problem and using contextualized and empathetic thinking (Brown, 2009). This stage often involves interviewing people who experience the problem and for whom a solution would be beneficial. It may also involve doing background research and identification of as many different perspectives on the issue as possible. The various perspectives and the lived experience of interviewees form the basis for the construction of a novel perspective from which to view both the problem and potential solutions (Kelley & Kelley, 2014).

Ideation The ideation stage of the design thinking process starts by combining idea generation activities such as brainstorming with the cognitive processes of divergent thinking and abductive reasoning (Carroll et al., 2010). The typical situation involves a group of individuals silently writing or drawing as many ideas as possible on adhesive notes which are then placed randomly on walls and windows. The number of ideas per individual ranges from several dozen to a hundred or more. Once a large number of ideas have been generated they are organized, evaluated, synthesized, and modified until a single solution emerges as the most promising path

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

847

to explore. These activities integrate the cognitive processes of convergent thinking, synthetic thinking, integrative thinking, and a continuation of abductive reasoning (IDEO, 2012). A common approach is that all the individuals in the group silently observe the ideas covering the walls and windows and then move adhesive notes to close proximity to other ideas which seem to bear some aspect of similarity or relationship. This process continues in silence until clusters of ideas have emerged, at which point discussion commences regarding the clusters and relationships between clusters. The discussion continues until consensus begins to emerge regarding the solution which seems to be the most promising avenue to explore.

Prototyping The prototyping stage of the design thinking process involves the creation of a prototype in an appropriate form given the nature of the selected solution, which could be the design of a physical or digital object, a process, or an environment. The prototypes may closely resemble the envisioned solution, or may be symbolic or metaphorical representations. They must, however, always be designed in a form which can be tested with authentic users in authentic situations (Brown, 2008). This stage involves the construction of prototypes according to the meanings constructed by the designers, rather than meanings inherent to the tools, materials, or assumptions regarding similar problems or solutions (Brown, 2009). The prototyping process moves from abstract to concrete, during which the designers rapidly change goals and constraints. This stage often begins with planning and role negotiation among the individuals in the design team. The prototyping stage concludes with the prototype being tested by real-world users in real-world contexts while the designers collect feedback and engage the users in dialogue regarding their experience with the prototype (IDEO, 2012).

Iteration The iteration stage of the design thinking process translates the feedback provided by users into modifications of the design in subsequent prototypes. The number of iteration cycles is determined by the nature of the problem, the nature of the solution, and the expert judgment of the designers regarding readiness of the design for deploying in real-world contexts. Although used throughout the design thinking process, engaging in reflection-in-action and reflecting on relevance is particularly prominent during iteration cycles (Luka, 2014).

Deploying in Real-World Contexts In many academic situations the quality of learner-designed artifacts is measured by the expert judgment of the instructor, and sometimes by peer- or self-evaluation. In

848

J. P. Donaldson and B. K. Smith

contrast, the value of solutions created through the design thinking process is determined by applicability and usefulness in real-world contexts (Köppen & Meinel, 2015). The opinion of experts – and even the opinion of the designers – is relatively inconsequential because the quality of the design can only be evaluated by the degree to which it has impact as a real solution for real people in the real world (Kelley & Kelley, 2014).

Principles of Constructionist Learning Translating design thinking strategies and designerly ways of knowing into practical application in the design of tools, activities, and environments for learning requires grounding in a theoretical framework. The principles of constructionism are particularly well-aligned with the design thinking literature, and therefore are ideally situated as a structuring framework for implementing design thinking strategies and designerly ways of knowing in the design of learning. At the core of constructionism is the assertion that construction of artifacts facilitates the construction of meaning (Papert, 1993, 1999; Papert & Harel, 1991; Kafai & Resnick, 1996). The core principles of constructionist learning are: learners are situated as designers, learning is structured around the creation of artifacts, learners engage in focused tinkering, learners have agency, and artifacts are created for an authentic audience beyond the immediate learning environment.

Situating Learners as Designers Constructionist learning situates learners as designers. In this role, learners collaboratively negotiate purposes, roles, processes, tools, and meanings of the artifacts they will create, and then collaboratively engage in the design, prototyping, iteration, and real-world deployment of their artifacts (Resnick & Rusk, 1996; Kafai, 2006).

Creation of Artifacts The construction of artifacts is at the heart of constructionist learning, where construction of meaning: is supported by construction of a more public sort ‘in the world’ – a sand castle or a cake, a Lego house or a corporation, a computer program, a poem, or a theory of the universe. Part of what I mean by ‘in the world’ is that the product can be shown, discussed, examined, probed, and admired. It is out there. (Papert, 1993, p. 142)

Construction of artifacts in constructionist learning differs from construction of artifacts in other contexts in that learning contexts require that metacognitive strategies be interwoven with construction activities. Metacognition involves thinking

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

849

about one’s own thinking, particularly in reference to three forms of knowledge: declarative knowledge, procedural knowledge, and conditional knowledge (Winne & Azevedo, 2014). It can be seen as operating at three levels: knowledge of cognition, monitoring of cognition, and regulation of cognition (Garrison & Akyol, 2012). Pedagogical practices to facilitate the development of metacognitive strategies include concept mapping, explicit use of problem-solving strategies, connecting learning and relevance to the individual learner, and a variety of individual and group reflective activities (Dimmitt & McCormick, 2012; Zimmerman & Labuhn, 2012).

Focused Tinkering Resnick and Rosenbaum (2013) describe tinkering in constructionist learning contexts as “a playful, experimental, iterative style of engagement, in which makers are continually reassessing their goals, exploring new paths, and imagining new possibilities” (p. 164). When learners are situated as designers, tinkering facilitates development of skills in framing and reflection-in-action: tinkering is closer to the way real scientists, mathematicians, and engineers solve problems. . .[they] follow hunches, iterate, make mistakes, re-think, start over, argue, sleep on it, collaborate, and have a cup of tea. Tinkering encourages making connections. (Martinez & Stager, 2013, p. 45)

Focused tinkering also balances the goal-oriented process of artifact construction with freedom to explore and an atmosphere of joyful engagement – or “hard fun” (Papert, 2002) – which is often absent in contexts driven by learning objectives.

Learner Agency Constructionism places learners in a position of agency, empowering them with the authority and responsibility for directing the goals, processes, and products of the learning process (Blikstein, 2008). Papert (1993) argued that: School has an inherent tendency to infantilize children by placing them in a position of having to do as they are told, to occupy themselves with work dictated by someone else and that, moreover, has no intrinsic value – schoolwork is done only because the designer of the curriculum decided that doing the work would shape the doer into a desirable form. I find this offensive . . .because I am convinced that the best learning takes place when the learner takes charge. (pp. 24–25)

Learners “must be given the freedom to follow their fantasies and the support to make those fantasies come to life” (Resnick 1994, p. 28). The constructionist principle of learner agency aligns with the literature on motivation and engagement. Deci and Ryan (Deci, Koestner, & Ryan, 2001; Ryan

850

J. P. Donaldson and B. K. Smith

& Deci, 1996, 2000) argued that learning is most powerful when intrinsic motivation is high, that extrinsic motivation has negative effects on learning, and that high levels of intrinsic motivation require high levels of learner agency and autonomy. Kaplan et al. (2014) found that learning activities which promote identity exploration, particularly those in which personal relevance is highlighted, facilitates the emergence of higher levels of motivation. Assor, Kaplan, and Roth (2002) found that learner engagement is facilitated not only by autonomy, but also by activities for exploration of relevance.

Authentic Audience Constructionist learning is embedded in projects with a “larger intellectual and social goal” where participants are “in a continuous dialogue with their own ideas and the ideas of intended users” (Kafai, 1996, p. 72). The construction of artifacts alone is insufficient in facilitating powerful learning. The artifacts must be meaningful and relevant, not only to the learners, but also in a larger social context (Kafai & Burke, 2014). When artifacts are constructed for an authentic audience, learners demonstrate greater understanding, are more likely to engage in perspective-taking, and develop intrinsic motivation linked to their perceptions of the utility of their artifacts (Karan, 2016).

Intersections, Synergies, and Alignments Designerly ways of knowing, design thinking strategies, and principles of constructionist learning share certain features. All three domains are concerned with design and describe meaning in terms of real-world impact. They also emphasize the importance of reflective practices. Although designerly ways of knowing and design thinking strategies have shared histories, they are usually treated as distinct domains in the literature. Each of these lines of literature, however, can be interpreted in light of the other and nurture each-other (Johansson-Sköldberg et al., 2013). Design thinking has been introduced into a variety of educational contexts for a variety of purposes (see Carroll, 2014; Abrahamson, 2015; Benson & Dresdow, 2014; Cassim, 2013; Gözen, 2016). However, the relationships and potential synergies between constructionist learning, designerly ways of knowing, and design thinking strategies have yet to be explored. A simple starting point in the exploration of these relationships would be a mapping of alignments between these three domains. Table 2 is a rough description of some possible areas of alignment. Although deeper analyses will be needed to strengthen the arguments for particular areas of alignment, the potential alignment suggests the possibility of designing learning contexts in which the three domains are woven together.

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

851

Table 2 Alignment of principles of constructionist learning, design thinking strategies, and designerly ways of knowing Constructionist principles Situating learners as designers

Design thinking strategies Frame creation, ideation, prototyping, iteration, deploying in real-world contexts

Creation of artifacts

Ideation, prototyping

Focused tinkering

Prototyping, iteration

Learner agency

Frame creation, deploying in realworld contexts

Authentic audience

Deploying in real-world contexts

Designerly ways of knowing Treating all problems as wicked problems, framing, empathetic thinking, contextualized thinking, abductive reasoning, rapidly changing goals and constraints, divergent and convergent thinking, prototyping from abstract to concrete, constructing prototypes according to the meaning you construct, engaging in reflection-in-action, reflecting on relevance Prototyping from abstract to concrete, constructing prototypes according to the meaning you construct Abductive reasoning, rapidly changing goals and constraints, engaging in reflection-in-action Framing, reflecting on relevance, constructing prototypes according to the meaning you construct Empathetic thinking, contextualized thinking, reflecting on relevance

The Design Thinking for Engaged Learning Framework Simon (1996) argues that the study of the science of design – including designerly ways of knowing and design processes – is important beyond the traditional domains of design such as architecture because design is central in the endeavors to which people devote their passions: “Everyone designs who devises courses of action aimed at changing existing situations into preferred ones” (p. 111). When design thinking and designerly ways of knowing are operationalized in the context of learning structured through a theoretical framework of constructionist principles, learners are situated as designers. Through artifact creation informed by design thinking strategies and designerly ways of knowing, learners individually, collaboratively, and collectively construct meaning. The meanings they construct have personal relevance, as well as relevance and impact in the world beyond the immediate learning context. As learners engage in design thinking processes and practice designerly ways of knowing, their conceptualizations of learning, learning strategies, and identities as learners develop towards agentic, purposeful, strategic, and reflective learning. The design thinking for engaged learning (DTEL) framework proposed here is an initial attempt at combining designerly ways of knowing and design thinking strategies to operationalize principles of constructionist learning. The DTEL

852

J. P. Donaldson and B. K. Smith

Table 3 Design thinking for engaged learning – characteristics (DTEL-C) Domain Framing

Idea exploration

Prototyping and iteration

Design thinking characteristics Treat all problems as “wicked” and ill-defined problems Create “frames” – novel perspective, standpoints, or positions Learn by modeling, pattern-formation, and synthesis Value practicality, ingenuity, empathy, and appropriateness Question all “facts,” and critically challenge “reality,” particularly social/ cultural systems Deeply understand problems in context, particularly regarding human aspects of context Use abductive reasoning in simultaneous creation of solution, process, and value Understand through exploring, connecting, and intersecting information and ideas Use both divergent thinking and convergent thinking Use synthetic and integrative thinking Develop your own “toolbox” of creativity-facilitating tools and techniques Prototype according to the meaning you construct (not according to external expectations) Rapidly change both goals and constraints Prototype rapidly from abstract to concrete Give yourself and others the freedom to engage in focused tinkering Engage in frequent reflection and reflection-in-action processes Frequently observe, analyze, and adjust your own thoughts and patterns of thought Often stop and consider how your work is relevant to your life (current and future)

framework describes a set of characteristics of designerly ways of knowing organized into characteristics involved in framing, idea exploration, and prototyping and iteration informed by constructionist principles (DTEL-C – see Table 3). The process model (DTEL-PM – see Fig. 2) integrates features from design thinking with aspects which the education literature suggests are important in the facilitation of engaged learning, as well as principles from design thinking theory. The process model involves five phases: (1) name and frame, (2) diverge and converge, (3) prepare and share, (4) analyze and revise, and (5) deploy. Each phase is broken down into stages with corresponding activities which are described in detail in Table 4.

Design Case This section will briefly describe one design case in which the design of a project used the DTEL framework to operationalize the principles of constructionist learning in a multimedia for teaching and learning course. The participants were students in an

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

Name and Frame

Problem

Perspecves

Diverge and Converge Divergent Thinking

Convergent Thinking

Prepare and Share Deploy Project Prototype Prototype Planning Construcon and Collect Data

853

Analyze and Revise

Data Analysis

Iteration

Deploy

Deploy and Wrap-up

Fig. 2 DTEL process model (DTEL-PM) visualized

education major. The purpose of the project was to facilitate development of participants’ skills in the use and design of multimedia for teaching and learning through development of designerly ways of knowing and the use of design thinking processes. During the first through fifth weeks of the class, participants engaged in group projects in which they used the design thinking for engaged learning (DTEL) process to create designs (multimedia objects, experiences, or environments) which addressed the “wicked problem” of how to help learners reconceptualize learning through an active “construction” conceptualization of learning as opposed to the widespread passive “acquisition/transfer” conceptualization of learning. During the first week participants explored the nature of design. Participants were assigned to one of three project groups. Each group created their own design solution. They went through DTEL phases 1 and 2 during the second week of the project, and phases 3, 4, and 5 during the second and third weeks of the project. Prior to engaging in the intervention, participants recorded short (2–3 min) audio responses to open-ended survey questions. During the project, participants produced short journal entries at the end of each DTEL stage. The researchers conducted observations of the groups as they worked on their projects and recorded observation notes. After the projects, participants recorded short (5–10 min) audio responses to open-ended survey questions. The transcripts of the audio responses to open-ended questions, journal entries, observation notes, and participant-designed artifacts were collected for analysis in a larger ongoing design-based research project. Preliminary analysis suggests that the design achieved the stated purposes. Participants developed skill in designing and using multimedia in learning environments, as well as designerly ways of knowing. This was measured through multiple forms of evidence indicating iterative cycles of internalization and externalization of design thinking characteristics. Participants developed the ability to engage in the design thinking process without the support structures of the DTEL framework as demonstrated in engagement in several rounds of design in subsequent weeks after the initial design project was completed.

854

J. P. Donaldson and B. K. Smith

Table 4 Design thinking for engaged learning – process model (DTEL-PM) Phase Phase 1: Name and frame

Phase 2: Diverge and converge

Stage Stage 1: Problem

Purpose Activate prior knowledge

Stage 2: Perspectives

Inquiry, empathy, and framing

Stage 3: Divergent thinking

Idea generation, divergent thinking, convergent thinking, and reframing

Stage 4: Convergent thinking

Activity The problem is framed as an ill-defined, “wicked” problem Discussion: Participants share what they know about the problem Information gathering to find out about the problem: Talk to at least one person outside the class about the problem; independently conduct informal information searches. Share findings with the group Metacognitive activity – “situating the problem” reflection: Each participant writes a brief reflection piece Framing activity – “new perspective”: Participants collaboratively create a prototypical “user” for whom their solution to the problem will be relevant Divergent thinking activity: Participants think up as many solutions as possible and write them down/draw them on individual sticky notes. Encourage wild and crazy ideas. Discourage selfcensorship. Participants occasionally put their ideas on a wall (random placement). Silence is important Convergent thinking activity: After all the ideas are on the walls, participants silently observe them. Then they silently reorganize them Metacognitive activity – “innovation” reflection: Each participant writes a reflection piece Metacognitive activity – “situating the solution” discussion: Participants discuss the categories that emerged in the convergent thinking stage and possibly label each category. Also, they might collaboratively create a concept map describing the relationship between their various categories (continued)

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

855

Table 4 (continued) Phase Phase 3: Prepare and share

Stage Stage 5: Project planning Stage 6: Construction Stage 7: Deploy Prototype + Collect data

Phase 4: Analyze and revise

Stage 8: Data analysis Stage 9: Iteration

Phase 5: Deploy

Stage 10: Deploy and wrap-up

Purpose Collaborative skill development – role negotiation, project planning skill development, construction of artifacts, real-world application, and data collection skill

Development, data analysis skill development, reframing, and iteration Self-assessment skill development, metacognitive selfregulated learning skill development, consolidation of learning, identity exploration, and knowledge transfer

Activity Project planning: Participants engage in role negotiation and project planning Prototype construction: Participants collaboratively construct a prototype – focused tinkering is encouraged Metacognitive activity – “creation” reflection: Each participant writes a brief reflection piece Deploy in real-world context and collect data Data analysis and iteration planning Reframe, redesign, and redeploy. Repeat as needed Deploy the final design as a public entity Metacognitive activity – “evaluation” rubric: Each participant creates and uses a self-assessment rubric which includes: (1) assessment of the prototype, (2) assessment of the group collaboration, and (3) assessment of their own work throughout the whole process Metacognitive activity – “mastery” learning objectives: Each participant writes retroactive learning objectives for themselves. Mentally putting themselves in time just before starting this project, they write 5–7 learning objectives, each of which completes the phrase: “after successful completion of this project, I will be able to. . .” Metacognitive activity – “transformations” discussion: The whole class discusses the impact this design activity had on their learning, their lives, their identities, and their world

Some aspects of the design indicated need for further development of particular DTEL phases. Participants had the most difficulty with phase 3 (Prepare and Share). In particular, they spent significantly more time in stage 5 of this phase (Project

856

J. P. Donaldson and B. K. Smith

Planning) than anticipated. This may suggest modifications of this stage towards better transition from phase 2 (Diverge and Converge) to phase 3 (Prepare and Share). In terms of operationalizing the principles of constructionist learning, this design achieved high levels of engagement in terms of situating learners as designers, creation of artifacts, focused tinkering, and learner agency. The weakest area was in engaging the participants in designing for an authentic audience. Perhaps due to limited prior experience with authentic audiences, participants had difficulty bridging the gap from hypothetical audience to real audience. This suggests a need for more work connecting phase 1 stage 2 (Perspectives), phase 3 stage 7 (Deploy Prototype + Collect Data), and phase 5 stage 10 (Deploy) – perhaps through explicit “looking forward” activities in phases 1 and 3. This design case supported the viability of the DTEL model, particularly in operationalizing the principles and intersectionality of constructionist learning, designerly ways of knowing, and design thinking processes.

Conclusion: Learning as Design Learning is often framed in the learning sciences through a metaphor of construction, a perspective which describes learning as a process in which meaning is individually and socially constructed. This contrasts with the conceptualization of learning based on the transfer/acquisition metaphor in which learning is seen as the transfer of information from external sources into the minds of learners. When design thinking, designerly ways of knowing, and principles of constructionist learning are applied in the design of learning, an emphasis on the process of construction emerges. The design thinking for engaged learning (DTEL) framework is our first attempt to bring these three-related strands of research together as a single set of principles. We are currently using DTEL in our own work to develop and evaluate constructionist curricula, and example of which is described in the design case. We believe it may be useful for other instructional designers as they create materials that facilitate learning through the process of design.

References Abrahamson, D. (2015). Reinventing learning: A design-research odyssey. ZDM: The International Journal on Mathematics Education, 47(6), 1013–1026. https://doi.org/10.1007/s11858-0140646-3 Assor, A., Kaplan, H., & Roth, G. (2002). Choice is good, but relevance is excellent: Autonomyenhancing and suppressing teacher behaviours predicting students’ engagement in schoolwork. British Journal of Educational Psychology, 72, 261–278. Bell, S. J. (2008). Design thinking. American Libraries, 39(1–2), 44–49. Benson, J., & Dresdow, S. (2014). Design thinking: A fresh approach for transformative assessment practice. Journal of Management Education, 38(3), 436–461. https://doi.org/10.1177/ 1052562913507571

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

857

Blikstein, P. (2008). Travels in Troy with Freire: Technology as an agent for emancipation. In P. Noguera & C. A. Torres (Eds.), Social justice education for teachers: Paulo Freire and the possible dream (pp. 205–244). Rotterdam, Netherlands: Sense. Brown, T. (2008). Design thinking. Harvard Business Review, 86(6), 84–92. Brown, T. (2009). Change by design: How design thinking transforms organizations and inspires innovation. New York, NY: Harper Business. Buenano, G. (1999). The becoming of problems in design: Knowledge in action to frame wicked problems. (Ph.D. Disserataion), University of California, Berkeley, CA. ProQuest Dissertations & Theses Global database. Carroll, M. (2014). Shoot for the moon! The mentors and the middle schoolers explore the intersection of design thinking and STEM. Journal of Pre-College Engineering Education Research (J-PEER), 4(1), 3. https://doi.org/10.7771/2157-9288.1072 Carroll, M., Goldman, S., Britos, L., Koh, J., Royalty, A., & Hornstein, M. (2010). Destination, imagination and the fires within: Design thinking in a middle school classroom. International Journal of Art & Design Education, 29(1), 37–53. https://doi.org/10.1111/j.14768070.2010.01632.x Cassim, F. (2013). Hands on, hearts on, minds on: Design thinking within an education context. International Journal of Art & Design Education, 32(2), 190–202. https://doi.org/10.1111/ j.1476-8070.2013.01752.x Clark, K., & Smith, R. (2010). Unleashing the power of design thinking. In T. Lockwood (Ed.), Design thinking: Integrating innovation, customer experience and brand value (pp. 47–56). New York, NY: Allworth Press. Cleary, B. A. (2015). Design thinking and PDSA: Don’t throw out the baby. The Journal for Quality and Participation, 38(2), 21–23. Collins, H. (2013). Can design thinking still add value? Design Management Review, 24(2), 35–39. https://doi.org/10.1111/drev.10239 Coyne, R. (2005). Wicked problems revisited. Design Studies, 26(1), 5–17. https://doi.org/10.1016/ j.destud.2004.06.005 Cross, N. (2001). Designerly ways of knowing: Design discipline versus design science. Design Issues, 17(3), 49–55. Cross, N. (2006). Designerly ways of knowing. Dordrecht, London: Springer. Deci, E. L., Koestner, R., & Ryan, R. M. (2001). Extrinsic rewards and intrinsic motivation in education: Reconsidered once again. Review of Educational Research, 71(1), 1–27. Dimmitt, C., & McCormick, C. B. (2012). Metacognition in education. In K. R. Harris, S. Graham, T. Urdan, C. B. McCormick, G. M. Sinatra, & J. Sweller (Eds.), APA educational psychology handbook, Vol 1: Theories, constructs, and critical issues (pp. 157–187). Washington, DC: American Psychological Association. Dorst, K. (2011). The core of ‘design thinking’ and its application. Design Studies, 32(6), 521–532. https://doi.org/10.1016/j.destud.2011.07.006 Dorst, K. (2015). Frame innovation: Create new thinking by design. Cambridge, MA: MIT Press. Fuge, M., & Agogino, A. (2015). Pattern analysis of IDEO’s human-centered design methods in developing regions. Journal of Mechanical Design, 137(7), 071405–071405. https://doi.org/ 10.1115/1.4030047 Garrison, D. R., & Akyol, Z. (2012). Toward the development of a metacognition construct for communities of inquiry. The Internet and Higher Education, 17, 84–89. https://doi.org/10.1016/ j.iheduc.2012.11.005 Glen, R., Suciu, C., Baughn, C. C., & Anson, R. (2015). Teaching design thinking in business schools. The International Journal of Management Education, 13(2), 182–192. https://doi.org/ 10.1016/j.ijme.2015.05.001 Gözen, G. (2016). Influence of design thinking performance on children’s creative problem-solving skills: An estimation through regression analysis. British Journal of Education, Society & Behavioural Science, 12(4), 1–13. https://doi.org/10.9734/BJESBS/2016/22153

858

J. P. Donaldson and B. K. Smith

Hutchinson, A., & Tracey, M. (2015). Design ideas, reflection, and professional identity: How graduate students explore the idea generation process. Instructional Science, 43(5), 527–544. https://doi.org/10.1007/s11251-015-9354-9 IDEO. (2012). Design thinking for educators toolkit. Palo Alto, CA: IDEO. Johansson-Sköldberg, U., Woodilla, J., & Çetinkaya, M. (2013). Design thinking: Past, present and possible futures. Creativity and Innovation Management, 22(2), 121–146. https://doi.org/ 10.1111/caim.12023 Kafai, Y. B. (1996). Learning design by making games. In Y. B. Kafai & M. Resnick (Eds.), Constructionism in practice: Designing, thinking and learning in a digital world (pp. 71–96). Mahwah, NJ: Lawrence Erlbaum Associates. Kafai, Y. B. (2006). Constructionism. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 35–46). New York, NY: Cambridge University Press. Kafai, Y. B., & Burke, Q. (2014). Connected code: Why children need to learn programming. Cambridge, MA: The MIT Press. Kafai, Y. B., & Resnick, M. (1996). Constructionism in practice: Designing, thinking, and learning in a digital world. Mahwah, NJ: Lawrence Erlbaum Associates. Kafai, Y. B., Peppler, K. A., & Chapman, R. N. (2009). The computer clubhouse: Constructionism and creativity in youth communities. New York, NY: Teachers College Press. Kaplan, A., Sinai, M., & Flum, H. (2014). Design-based interventions for promoting students’ identity exploration within the school curriculum. In S. Karabenick & T. C. Urdan (Eds.), Motivational interventions (pp. 243–291). Bingley, UK: Emerald Group Publishing Limited. Karan, A. (2016). The role of an authentic audience in computational modeling: Designing models of tides for younger children. Paper presented at the learning sciences graduate student conference, Chicago, IL. Kelley, T., & Kelley, D. (2014). Creative confidence: Unleashing the creative potential within us all. London, UK: William Collins. Kimbell, L. (2011). Rethinking design thinking: Part I. Design and Culture, 3(3), 285–306. https:// doi.org/10.2752/175470811X13071166525216 Köppen, E., & Meinel, C. (2015). Empathy via design thinking: Creation of sense and knowledge. In H. Plattner, C. Meinel, & L. Leifer (Eds.), Design thinking research: Building innovators (pp. 15–28). Dordrecht, London: Springer International Publishing. Liedtka, J. (2014). Innovative ways companies are using design thinking. Strategy & Leadership, 42(2), 40–45. https://doi.org/10.1108/SL-01-2014-0004 Liedtka, J., & Ogilvie, T. (2011). Designing for growth: A design thinking tool kit for managers. New York, NY: Columbia Business School Pub. Luka, I. (2014). Design thinking in pedagogy. Journal of Education Culture and Society, 2014(2), 63–74. https://doi.org/10.15503/jecs20142.63.74 Martinez, S. L., & Stager, G. (2013). Invent to learn: Making, tinkering, and engineering in the classroom. Torrance, CA: Constructing Modern Knowledge Press. Mickahail, B. (2015). Corporate implementation of design thinking for innovation and economic growth. Journal of Strategic Innovation and Sustainability, 10(2), 67–79. Orthel, B. D. (2015). Implications of design thinking for teaching, learning, and inquiry. Journal of Interior Design, 40(3), 1–20. https://doi.org/10.1111/joid.12046 Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. New York, NY: Basic Books, Inc. Papert, S. (1993). The children’s machine: Rethinking school in the age of the computer. New York, NY: BasicBooks. Papert, S. (1999). Eight big ideas behind the constructionist learning lab. In G. S. Stager (Ed.), Constructive technology as the key to entering the Community of Learners (pp. 4–5). Philadelphia, PA: 2005 National Educational Computing Conference (NECC). Papert, S. (2002, June 24). How to make writing ‘hard fun’. Bangor Daily News. Retrieved from http://www.papert.org/articles/HardFun.html

33

Design Thinking, Designerly Ways of Knowing, and Engaged Learning

859

Papert, S., & Harel, I. (1991). Situating constructionism. In S. Papert & I. Harel (Eds.), Constructionism (pp. 1–11). New York, NY: Basic Books. Poulsen, S. B., & Thøgersen, U. (2011). Embodied design thinking: A phenomenological perspective. CoDesign, 7(1), 29–44. https://doi.org/10.1080/15710882.2011.563313 Razzouk, R., & Shute, V. (2012). What is design thinking and why is it important? Review of Educational Research, 82(3), 330–348. https://doi.org/10.3102/0034654312457429 Resnick, M. (1994). Turtles, termites, and traffic jams: Explorations in massively parallel microworlds. Cambridge, Mass.: MIT Press. Resnick, M., & Rosenbaum, E. (2013). Designing for tinkerability. In M. Honey & D. E. Kanter (Eds.), Design, make, play: Growing the next generation of STEM innovators (pp. 163–181). New York, NY: Routledge. Resnick, M., & Rusk, N. (1996). The computer clubhouse: Preparing for life in a digital world. IBM Systems Journal, 35(3 4), 431. Resnick, M., Berg, R., & Eisenberg, M. (2000). Beyond black boxes: Bringing transparency and aesthetics back to scientific investigation. Journal of the Learning Sciences, 9(1), 7–30. https:// doi.org/10.1207/s15327809jls0901_3 Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155–169. Runco, M. A. (2014). Creativity theories and themes: Research, development, and practice (2nd ed.). Burlington, MA: Elsevier Science. Ryan, R. M., & Deci, E. L. (1996). When paradigms clash: Comments on Cameron and Pierce’s claim that rewards do not undermine intrinsic motivation. Review of Educational Research, 66 (1), 33–38. Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. https://doi.org/10.1006/ ceps.1999.1020 Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books. Schön, D. A. (1984). Problems, frames and perspectives on designing. Design Studies, 5(3), 132–136. https://doi.org/10.1016/0142-694X(84)90002-4 Schön, D. A. (1995). Knowing-in-action: The new scholarship requires a new epistemology. Change, 27(6), 26–34. Schön, D. A., & Rein, M. (1994). Frame reflection: Toward the resolution of intractable policy controversies. New York, NY: Basic Books. Simon, H. A. (1996). The sciences of the artificial (3rd ed.). Cambridge, MA: The MIT Press. Suwa, M., Gero, J., & Purcell, T. (2000). Unexpected discoveries and S-invention of design requirements: Important vehicles for a design process. Design Studies, 21(6), 539–567. https://doi.org/10.1016/S0142-694X(99)00034-4 Vogel, C. M. (2010). Notes on the evolution of design thinking: A work in progress. In T. Lockwood (Ed.), Design thinking: Integrating innovation, customer experience and brand value (pp. 3–14). New York, NY: Allworth Press. Watson, A. D. (2015). Design thinking for life. Art Education, 68(3), 12–18. Welsh, M. A., & Dehler, G. E. (2013). Combining critical reflection and design thinking to develop integrative learners. Journal of Management Education, 37(6), 771–802. https://doi.org/ 10.1177/1052562912470107 Winne, P. H., & Azevedo, R. (2014). Metacognition. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed.pp. 63–87). New York, NY: Cambridge University Press. Zimmerman, B. J., & Labuhn, A. S. (2012). Self-regulation of learning: Process approaches to personal development. In K. R. Harris, S. Graham, T. Urdan, C. B. McCormick, G. M. Sinatra, & J. Sweller (Eds.), APA educational psychology handbook, Vol 1: Theories, constructs, and critical issues (pp. 399–425). Washington, DC: American Psychological Association.

860

J. P. Donaldson and B. K. Smith

Jonan Phillip Donaldson is an educational professional. He is currently in the Ph.D. Program in Educational Leadership and Learning Technology at Drexel University. Previously, he worked as an instructional designer and taught online courses in education and technology at Oregon State University and Western Oregon University. His research interests are design thinking, designerly ways of knowing, design-based research, creativity, and critical metaphor analysis. Brian K Smith is a Professor in Education and Computer Science Drexel University. His research involves the use of computation to support and augment human performance and learning, especially in contexts outside of formal education, as well as bringing creativity to the core of educational practices. Prior to Drexel, Smith was Dean of Rhode Island School of Design Continuing Education (RISD|CE) where he oversaw the development of art and design programs for youth and adults and was a co-investigator of RISD’s STEM to STEAM initiative. He has also been a faculty member at the Pennsylvania State University’s College of Information Sciences and Technology and the MIT Media Laboratory teaching and researching in computer, learning, and information sciences.

Considerations for the Design of Gesture-Augmented Learning Environments

34

Robert C. Wallon and Robb Lindgren

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Embodiment and Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Embodiment and Technology Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The GRASP Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Designing Gesture Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . New Interactions for New Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Focusing on Key Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sources of Gestures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Connecting Gestures and Digital Representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Designing Constraints of the Learning Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Custom Gestures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Constraining Nonnormative Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Facilitating Interaction with the Learning Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Precursor Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acclimating Learners to Gesture Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Structuring Embodied Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configure Productive Social Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Final Thoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

862 863 864 865 867 867 869 869 870 871 871 872 872 873 874 876 877 877 878

Abstract

A new genre of learning technologies is emerging that integrates computer simulations with physical or “embodied” interactions such as hand gestures. While this genre presents new opportunities for innovative digital environments that physically engage learners, there is very little guidance on how to design these environments to optimize learning. This chapter presents considerations specifically for the design of gesture-augmented learning environments. Design R. C. Wallon (*) · R. Lindgren College of Education, University of Illinois at Urbana-Champaign, Champaign, IL, USA e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_75

861

862

R. C. Wallon and R. Lindgren

considerations are discussed in three main areas related to (1) what gestural interactions are used, (2) constraints of the learning environment, and (3) what social and contextual supports are offered. The term considerations is used rather than principles or guidelines to highlight the real tradeoffs and legitimate decisions to be made when designing gesture-based technologies for learning. These considerations are illustrated with detailed examples from a project that implements students’ gestures as the primary method of interaction with digital science simulations. Although the examples specifically pertain to learning in science, the considerations are framed such that they can be applied to a broad range of domains. Keywords

Computer simulations · Embodied learning · Explanations · Gesture · Science education

Introduction In this chapter, we argue for an approach to the design of educational technologies that takes seriously the embodied nature of learning. Over a decade ago, computer scientist Paul Dourish (2001) proposed that Human-Computer Interaction (HCI) research adopt embodiment as the foundation of computer interface design and research because, unlike traditional approaches, it acknowledges that human actions have both a physical and social “embedding” in systems of practices and activities. While a framework and understanding of embodied interaction has been embraced by some, learning technology design is still driven largely by information processing models of cognition, and it is typically constrained by available, off-the-shelf devices that can easily be repurposed in educational contexts. We argue that the design of learning applications, even the very hardware used to implement them, needs to begin by examining the physical and social ways that people interact with those applications, and most importantly, what kinds of understanding and meaningmaking are generated by those interactions. This chapter focuses primarily on designs that integrate learner hand gestures, though there is also emerging research on embodied learning and technology design showing the efficacy of employing full-body movement (c.f., Lindgren & JohnsonGlenberg, 2013). Gestures play a significant role in the process of thinking and reasoning (McNeill, 1992), and research has shown that gestures have an impact on authentic educational contexts such as classrooms (Alibali & Nathan, 2012; Roth, 2001). If we combine our growing understanding of how gestures relate to learning with the emerging landscape of affordable motion tracking technologies, there is tremendous potential to create new genres of learning applications that augment the gestures that we perform naturally and connect them to important educational outcomes. Currently, however, there is relatively little in terms of guidelines, principles,

34

Considerations for the Design of Gesture-Augmented Learning Environments

863

Fig. 1 Three categories of considerations for design of gesture-augmented learning environments

or even detailed case studies to nurture the process of incorporating embodiment into the design of learning technologies. The goal of this chapter is to make a contribution that presents considerations for incorporating embodiment into the design of learning technologies based on the authors’ experiences working on a project to develop gesture-augmented science simulations. First, we give a brief review of embodiment theory and how it has been applied to technology-based learning environments. Second, we present an overview of the project to contextualize the examples that will illustrate the design considerations we have extracted. Next, we describe each of the considerations that have arisen out of design experiences on the project. The considerations, addressed in three main sections, are (1) designing gesture interaction, (2) designing constraints of the learning environment, and (3) supporting interaction with the learning environment (Fig. 1). Lastly, the chapter concludes with a discussion of how these considerations speak to embodied learning in additional contexts.

Embodiment and Learning The design of the project described in this chapter and the considerations that emerged are based on theories of cognition that assert a fundamental connection between sensorimotor actions and how people think and reason (Gallagher, 2005; Shapiro, 2010; Wilson, 2002). Hand gestures are a particular form of embodied action, and the effects of performing gestures on cognition and communication have been studied for several decades (McNeill, 1992; Goldin-Meadow, 2005). This research has shown that not only do gestures naturally accompany certain types of

864

R. C. Wallon and R. Lindgren

thinking but gestures can generate new ideas and understandings (Hostetter & Alibali, 2008; Goldin-Meadow, Cook, & Mitchell, 2009). Several studies have demonstrated, particularly in the context of STEM education, that student-produced gestures aid in learning and problem-solving (e.g., Flood, Amar, Nemirovsky, Harrer, Bruce, & Wittmann, 2014; Kim, Roth, & Thom, 2011; Radford, 2009; Singer, Radinsky, & Goldman, 2008; Yoon, Thomas, & Dreyfus, 2011). The gestures produced in these studies, however, emerged naturally from discourse rather than being purposefully designed into an educational intervention. The question of how to approach the design of embodied learning environments has started to be taken up by researchers, and some principles have begun to emerge (Abrahamson & Lindgren, 2014; Black, Segal, Vitale, & Fadjo, 2012; Johnson-Glenberg, Birchfield, Tolentino, & Koziupa, 2014; Lindgren & Johnson-Glenberg, 2013). In this chapter, we especially emphasize the three categories of design principles described by Abrahamson and Lindgren (2014): materials, activities, and facilitation. For each of the areas, the design goal is to forge meaningful connections between elicited actions and the ideas of a learning domain. In this chapter, we describe considerations for forging those connections specifically in the area of gesture-augmented science simulations.

Embodiment and Technology Design Current techniques in HCI strive to go beyond the mouse and the keyboard to more natural and expressive interfaces. Hardware innovations that advance these capabilities, such as Microsoft’s Kinect or the Leap Motion, collect analog data pertaining to users’ movements. These data can be creatively connected to any number of simulation or game environments so that the user experiences a physical interplay with the system. Education researchers have taken note of these technology developments and the potential for new embodied interaction techniques that facilitate student learning. Studies in this area often draw on early work distinguishing between physical and psychological forms of interaction (see Hannafin & Peck, 1988), as well as active versus passive interaction by learners (see Gibson, 1962, 1979; Engelkamp & Zimmer, 1994; Hartman, Miller, Nelson, 2000). Digital technologies can guide students to perform overt physical actions that can act as “conceptual leverage” (Resnick, 2002, p. 33) to help learners recall, retain, and comprehend. Schwartz (2010) showed, for example, that adults who engaged in higher motor activity with a multimedia presentation – e.g., using their hands to “perform” actions (e.g., “walk the dog”) that were described verbally on screen – had higher recall than participants who simply viewed or clicked to instigate the actions. Similarly, a recent study on students using their bodies to learn about centripetal force using a computer simulation showed that participants in the “high embodiment” condition (swinging a trackable object over their heads) showed higher long-term learning gains in physics compared to students in a “low embodiment” condition (initializing a simulation

34

Considerations for the Design of Gesture-Augmented Learning Environments

865

using a mouse) (Johnson-Glenberg, Megowan-Romanowicz, Birchfield, & SavioRamos, 2016). Many different types of interfaces and technologies can be brought to bear on studies of embodied learning, and several of these technologies have been shown to have promise for enhancing educational outcomes. These studies include the use of haptic augmented simulation of simple machines (Han & Black, 2011), touch-screen interfaces for performing arithmetic and estimation tasks (Segal, 2011), and the use of “virtual manipulatives” to learn about grouping and multiplication (Paek, 2012). Here we focus on interfaces that look for students’ gestures as they attempt to represent components of causal explanations in science.

The GRASP Project The focal case used to illustrate the considerations discussed in this chapter is a project named GRASP (GestuRe Augmented Simulations for supporting exPlanations). In this project, existing computer models of scientific phenomena – heat transfer, gas pressure, and the causes of seasons – have been enhanced to enable interaction with hand gestures instead of employing a traditional user interface scheme of mouse-controlled buttons and slider bars. Rather than mimicking the physical manipulation of these traditional controls, the aim of this project is to create environments where middle school students show their understanding through representational gestures that work in concordance with the computer simulations. The main goal of this type of design is to support students in constructing and communicating explanations of the phenomena using causal models (Clement, 2013). Most of the examples in this chapter come from the first 2 years of data collection on the GRASP project. During this time, over 100 diverse students from several middle schools have participated. Students were interviewed in a number of settings, including on site at schools during study hall periods and after school programs, as well as in university research facilities. Research from the project has had multiple foci, including how students gesture while giving explanations (Wallon, Brown, & Lindgren, 2016), effects of prompting students to gesture (Lindgren, Wallon, Brown, Mathayas, & Kimball, 2016), and student sense making while using computer simulations (Mathayas, Brown, & Lindgren, 2016).

Design Process Design of the gesture-based interface has been an iterative process that has cycled through phases of data collection and modification of the learning environments. The first year of the GRASP project primarily involved observations of individual students giving explanations of the three emphasized science topics using extant conventional simulations. The subsequent years of the GRASP project have involved observations of individual students giving explanations in the course of using prototype simulations that attempted to use student gestures as interaction mechanisms (i.e., gesture-augmented simulations). Interview sessions focused on

866

R. C. Wallon and R. Lindgren

one topic and involved three phases. During the first phase, students were asked for their explanations of a phenomenon in order to get a sense of their initial ideas. During the second phase, students used the simulation to engage and develop their ideas. During the third phase, students were asked again to explain the phenomenon presented at the beginning of the interview. Most recently, the project has increased attention to supporting groups of students in using the gesture-augmented simulations. Refining the designs has involved careful examination of how students seem to be making connections with the objects they are representing and the processes they are enacting. For example, our observations of how students appeared to be thinking about light rays hitting the Earth led to specific changes in how students were prompted to show the concentration of light rays in the summer compared to winter.

Hardware The GRASP project has also involved better understanding the affordances of specific motion tracking technologies in order to create a more seamless connection between a student’s gesture and the visualization. For example, the Leap Motion device, which is the hardware used in this project, typically rests on a desk or table below a student’s hands while performing a gesture (Fig. 2), which is somewhat counter to the assumed perspective of a gesturer’s audience – typically a person directly in front of the gesturer, or in some cases the gesturer herself. Other commercially available motion tracking technologies such as the Microsoft Kinect or Intel RealSense are typically positioned differently than the Leap Motion. Given these differences, the hardware component of an embodied learning environment is important for learning technology designers to attend to in order to achieve desired learning outcomes. Now that a brief overview of the GRASP project has been provided, examples from the project will be used to exemplify the ten considerations discussed in next three sections (Table 1).

Fig. 2 The Leap Motion device, hardware used in the GRASP project

34

Considerations for the Design of Gesture-Augmented Learning Environments

867

Table 1 Summary of the ten considerations for the design of gesture-augmented learning environments Design focus Gesture

Learning environment Facilitating interaction

Considerations 1. Are traditional interaction patterns repeated with a gesture interface, or are new interaction patterns conceived? 2. Do the gestures help students focus on key mechanisms? 3. What is the source of gestures that are designed into the system? 4. How should gestures be connected to actions on the screen? 5. Should users be allowed to define their own gestures? 6. To what extent should the learning environment constrain nonnormative actions? 7. What are the precursor activities that most effectively prepare learners to leverage gestures in their thinking and explanations? 8. What are the ways to acclimate learners to the particular gestural interface they are using? 9. How do you structure embodied activities into meaningful tasks that elicit both high performance and understanding? 10. How do you configure a productive social environment that utilizes the target gestures for effective communication?

Designing Gesture Interaction In this section, four considerations for designing gesture interaction in a learning environment are discussed: (1) Are traditional interaction patterns repeated with a gesture interface, or are new interaction patterns conceived? (2) Do the gestures help students focus on key mechanisms? (3) What is the source of gestures that are designed into the system? and (4) How should gestures be connected to actions on the screen? These considerations are first discussed in the context of the designed gesture interaction for the GRASP gas pressure simulation. To use the gas pressure simulation, students use two hands – one hand to represent molecules of a gas and the other hand to represent a movable wall of a closed container such as the plunger in a syringe with the end blocked off (Fig. 3). A student interacts with the simulation by varying the rate of collision between her hands (i.e., the rate of collision between the molecules and the moveable wall). When a student embodies molecular collisions more frequently, she sees that the pressure increases, and the wall moves to show a smaller volume (Fig. 4).

New Interactions for New Perspectives Traditional interactions in learning environments rely on users performing actions such as clicking buttons and dragging slider bars. These familiar ways of interacting with learning environments present a temptation to recreate similar interaction

868

R. C. Wallon and R. Lindgren

Fig. 3 Gesture with one hand representing molecules and the other hand representing a moveable wall

Fig. 4 Visualization of the gas pressure learning environment

patterns with a gesture interface (e.g., pointing a finger in air to “click” a button on the screen). Consequently, designers may miss opportunities to create novel interaction patterns to offer students new perspectives that were not possible with a traditional interface. A primary consideration when designing gesture interaction is, “Are traditional interaction patterns repeated with a gesture interface, or are new interaction patterns conceived?” Recall the gesture for the gas pressure simulation and consider how that varied from the interactions afforded by a conventional simulation. The conventional simulation allowed a user to click a button to increase or decrease the volume of the container, thus the interaction mechanism in the conventional simulation

34

Considerations for the Design of Gesture-Augmented Learning Environments

869

occurred at the macroscopic level. The interaction mechanism in the gestureaugmented simulation is on a microscopic level, allowing learners to directly engage with the activity of molecules and prompting them to think about the phenomenon from an important new perspective. Simply designing gesture interaction into the system does not necessarily mean that opportunities for new perspectives will be afforded. One can imagine, for example, a case where the designed gesture interaction in the gas pressure simulation uses one hand to represent the moveable wall of the container, which would allow the student to change the volume by moving the wall much in the same way she changes the volume of the container in the real world. In that hypothetical example, the gesture serves to replicate the interactions of a conventional simulation rather than offering a new perspective.

Focusing on Key Mechanisms In the context of supporting students with constructing causal explanations, an important consideration is, “Does the gesture interaction help students focus on key mechanisms?” When learning about gas pressure in a closed container, the key mechanism that accounts for increased pressure is more frequent collisions between air molecules and the walls of the container. Therefore, the gesture interaction uses the collisions between a user’s hands to represent a wall of the container and air molecules. This design decision reflects congruencies between the gesture action and the concept to be learned (Lindgren & Johnson-Glenberg, 2013). To contrast a gesture interaction that is conceptually congruent with one that is incongruent, consider an interaction that was part of an early design of the gas pressure simulation. An early prototype allowed a user to add or remove molecules from the container by making a gesture to “drop” additional molecules in or to “pick” molecules out. Ultimately, this mode of gesture interaction was eliminated because it did not reinforce the key concept of molecular collisions. This consideration highlights the need to focus on designing gestures that are congruent with the target concepts, which should be aligned with the intended learning goals.

Sources of Gestures Gestures that are designed into a learning environment should be selected with some indication that they support the types of thinking and reasoning the designer seeks to elicit. But where do these indications come from? How can a designer be confident that their chosen gestures are appropriate? In other words, a third consideration facing designers can be stated as “What sources should be used for gestures that are utilized within a learning environment?” The main source of gestures designed into the GRASP simulations was observation of natural gestures made by middle school students. The first year of the project involved interviewing students and asking them to explain the phenomena after

870

R. C. Wallon and R. Lindgren

using a conventional nongestural simulation. Students experienced a range of success with explaining the gas pressure phenomena, and the gestures of students who were able to give coherent explanations were noted. One particularly salient gesture from students who gave productive explanations of the gas pressure phenomenon was the use of two hands, with one open palm and the fingertips of the other hand tapping the open palm to represent molecular collisions. Seeing this gesture performed naturally by the middle school students became the inspiration for the gesture interaction that was designed into the gas pressure simulation, as previously discussed. While naturally occurring gestures can be a rich source of design ideas, it often becomes necessary to balance these ideas with practical constraints. For example, during the design process it became evident that technical limitations of the hardware and software prohibited using the precise gesture that a student was observed using because the resolution was not fine enough to detect the movement of individual fingertips. Therefore, the gesture interaction was modified so that the hand with fingertips became a closed fist, which was detected much more easily by the simulation. The end result was a gesture that captured the spirit of the studentconceived representation while meeting the technical constraints of available gesture recognition devices. Another potentially fruitful source of gestures is experts. For example, physicists could be asked to explain the gas pressure phenomenon, and their natural gestures could be observed. As another possibility, physicists could be asked explicitly what gestures they think would highlight the key mechanisms of the phenomena. Both of these examples provide additional avenues to explore for sources of gestures. A seductive source of gestures is existing digital environments because certain gestures have become ubiquitous conventions (e.g., pinch to zoom) that may seem intuitive to users. However, designers should use caution when borrowing gestures from other digital environments because these conventions often lack congruency with the target concepts, and they do not necessarily resonate with learners trying to think through specific problems. In the worst case, conventional gestures can constrain thinking by limiting new connections and reinforcing misconceptions. This cautionary note stems from differences between the goals of an operating system and the goals of a learning environment. In the case of the former, the goal is often to require as little effort as possible from the user. In the case of the latter, the goal of a learning environment is often to engage the users in a productive struggle that provides them with the opportunity to re-structure their knowledge and understanding.

Connecting Gestures and Digital Representations After considering which specific gestures will serve as interaction mechanisms for the learning environment, there is the final consideration of how to link a user’s gestures with digital representations. This consideration can be addressed by linking gestures to digital representations either directly or indirectly.

34

Considerations for the Design of Gesture-Augmented Learning Environments

871

A direct connection between a gesture and digital content shows a one-to-one relationship between hand movement and the movement of digital content in the learning environment. To illustrate this type of connection, consider the GRASP seasons simulation that uses a direct approach. In this simulation, the user changes the angle of her hand and that in turn changes the angle of light rays on the screen. Another gesture used in the seasons simulation involves the user changing the distance between two hands facing one another with open palms, and that in turn changes the distance between light rays. Both of these gesture interactions show a direct connection between the user’s gesture and digital content. Another approach to connecting a gesture and digital content is an indirect connection, which shows a relationship between hand movement and aspects of the learning environment that are not mapped in a one-to-one fashion. The GRASP gas pressure simulation uses an indirect approach. In this simulation, the user makes collisions between her fist and palm to represent collisions between molecules and the moveable wall of a container, respectively. However, the fist represents molecules rather a molecule. Therefore the collisions between the fist and palm represent an average rate of molecular collisions rather than a precise number of collisions.

Designing Constraints of the Learning Environment A critical factor in the design of interactive learning environments that often does not get addressed directly is the constraints of the system. In this section, we present two considerations for designing constraints of the learning environment: (1) Should users be allowed to define their own gestures? and (2) To what extent should the learning environment constrain nonnormative actions?

Defining Custom Gestures As previously mentioned, a goal of the first year of the GRASP project was to identify gestures that students naturally used when giving explanations of the scientific phenomena. As could be expected, students gestured in many different ways while explaining the same phenomena. This outcome brought about a question – could the learning environment be designed in such a way that would allow students to define their own gestures to customize how they interact with the simulation? Putting aside considerations of the relatively increased technical complexity of designing such a capability, it is worth exploring whether students should be allowed to define their own gestures in the system. Consider an example from a Year 1 interview when a student was asked what gestures she would use to decrease the temperature in the heat transfer simulation. Her idea was to fold her arms across her body and “shiver.” If the shivering gesture were used to interact with the simulation, then there would not be a focus on the explanatory mechanism – the movement of molecules. What would it take for a

872

R. C. Wallon and R. Lindgren

student to suggest a gesture interaction that focused on the key mechanism? The student will have likely already developed a sophisticated understanding of the system in order to make such a suggestion. This condition diminishes the intended goal of helping students develop a sophisticated understanding of the system. Rather than asking students to define a gesture before they have developed understanding of the intended model, students can be cued to use a gesture that highlights important features of the system and scaffolds a particular way of making sense of the system (Lindgren, 2015). Once cued to use a gesture, students’ understanding of the gesture has the opportunity to codevelop with understanding of the system. In this approach, the performance of prescribed gesture provides the seed from which new learning can grow.

Constraining Nonnormative Activity Another consideration for the design of the learning environment concerns the extent to which user interaction is constrained in a rule-bound system. Consider an example from the heat transfer simulation. When the user stops moving her hand (which is representing molecular motion), the molecules on the screen continue to move slightly. Therefore, a constraint of the system is that it will not allow the molecular motion to completely stop. Consider another example from the seasons simulation. While viewing the angle of sun rays from the Midwest United States, the ray angle is maintained between two extremes such that they are never completely vertical or completely horizontal, even if the user holds her hand that way. Rather than display rays in the system in nonnormative ways, the gesture interaction becomes uncoupled, as indicated by a “graying out” of the light rays on the screen. While affording interaction in nonnormative ways could be beneficial for exploring students’ ideas, doing so may not always be desirable. In simulations designed to put the user “inside” the system, some feedback from the system can help the user develop tacit understandings of the system’s limits. Allowing these limits to be surpassed creates the potential for the formation or reinforcement of noncanonical ideas. If nonnormative interactions are designed as part of the learning environment, then it is recommend that extra attention be directed toward engaging users in reflection on the system with respect to the target concepts.

Facilitating Interaction with the Learning Environment The third area of considerations for gestural interfaces focuses on how the process of embodied interaction can be optimally facilitated. This includes characteristics of the experience that occur before and after using the gestural interface, or pertaining to the context (e.g., the social configuration in which the embodied action occurs). Our four considerations for designing the supports and facilitation are: (1) What are the precursor activities that most effectively prepare learners to leverage gestures in their

34

Considerations for the Design of Gesture-Augmented Learning Environments

873

thinking and explanations? (2) What are the ways to acclimate learners to the particular gestural interface they are using? (3) How do you structure embodied activities into meaningful tasks that elicit both high performance and understanding? and (4) How do you configure a productive social environment that utilizes the target gestures for effective communication?

Precursor Activities It has been noted anecdotally in the GRASP work and in previous research on embodied actions that students often do not view gestures as a consequential component of thinking and learning. Middle and high school students especially can be reticent to use their hands in situations where they are giving explanations, and many act as though it is only the words that they write and speak that matter for demonstrating what they know. Thus, if a learning intervention aims to solicit student gestures as a means of engaging them with new ideas, it may be necessary to prime students with the notion that gestures can play an integral role in thinking and reasoning about challenging concepts. Even delivering simple assurances that it is acceptable to gesture when communicating about topics such as science may be necessary for students who are accustomed to being assessed almost exclusively by formal written statements. However, rather than simply telling students that gestures have benefits and that they should overcome their anxieties about expressing their ideas with their hands, experiences on the GRASP project have demonstrated it to be more effective to start interventions with activities that more naturally elicit gesturing and make salient their efficacy for the current task. First, the adult researchers who interacted with the students would freely use their hands when describing the problem space. Researcher gestures were not intended to model any particular way of representing ideas gesturally and in fact avoided using gestures that contained specific representations of elements or mechanisms. The goal instead was simply to create a safe space where the student was comfortable expressing their thinking physically and informally because the person they were talking to was expressing themselves this way. Secondly, students were encouraged to represent their ideas physically by initially being engaged with props and concrete scenarios that they could interact with using their hands. Before using the gesture-based computer simulations, all three of the science topics started with a casual discussion of a science phenomenon and objects were brought out to ground the discussion. In the case of season’s students were presented with a small globe that they were free to hold, spin, and most commonly, point at to show particular locations and the effects of sunlight on those locations. In some cases students created new representations that merged with the physical objects, such as using a fist to represent the relative position of the Sun or fingers to represent light rays hitting the Earth. At a minimum, these actions encouraged physical engagement that appeared in some cases to set the stage for subsequent gesturing.

874

R. C. Wallon and R. Lindgren

Finally, the researchers working with the students frequently gave prompts to the students that would naturally bring about gesturing. For example, prior to using the computer simulations, the interviewer would often follow up on a student’s attempt at a verbal explanation with a request to “show me.” This simple request often led to gestures that attempted to bring clarity and visibility to their explanation. Elsewhere the researchers have written about how these “show me” prompts led to a student’s increased focus on the causal mechanisms at play in the science phenomena (Lindgren et al., 2016), but for the purposes of this chapter it is notable that these prompts simply led to more gesturing, and in such a way that was seemingly not perceived by the student as coercive or unnatural. While it was not the case that all students involved in the project employed gestures abundantly, the techniques described here certainly encouraged a degree of physicality that likely set expectations and facilitated the eventual interaction with the gesture-based simulations. Other possibilities for eliciting gestures include having students watch other people give explanations, perhaps on different topics, in which gestures are used effectively to convey ideas. A second possibility would be to present students with a more formal introduction to representational gestures and perhaps giving them an activity where they build physical metaphors for nonscience topics (e.g., a prompt to use your hands to explain the conflict in Romeo and Juliet).

Acclimating Learners to Gesture Interfaces Once a student is primed to use gestures generally for a learning activity, the next challenge is to accustom them to the specific gesture scheme that is used in a particular digital platform. This is not a trivial task given that the field of humancomputer interaction is still in the early stages of studying and optimizing gesturebased controls (Isbister & Mueller, 2015), so there are relatively few standards for implementing these controls, and thus it cannot be assumed that a new user will have many instincts on how to use them. And yet, there do seem to be some relatively straightforward things that can be done to acclimate students to a new platform. One is to simply familiarize them with the particular gesture recognition device being used. In the case of GRASP and the Leap Motion device, there are numerous games and other sample applications available for trying out the functionality in a low-stress context. The game that was used during Year 2 of GRASP displayed a realistic set of hands similar to the models used in the GRASP simulations, so students could become familiar with using “virtual hands” to accomplish on-screen tasks that were physical in nature. It was also helpful in some cases to try to explain to the student how the device worked. In the case of the Leap Motion the interviewers pointed out that the device contained cameras that were looking upward at their hands. There was even an option in the Leap control panel to display the camera view so that one could see what their hands look like from the perspective of the device. Familiarity with the device was helpful in clueing users into what kinds of actions were more or less easily detected and potentially gave them information to troubleshoot their operation of the simulation in real time.

34

Considerations for the Design of Gesture-Augmented Learning Environments

875

Fig. 5 “Ghost hand” overlays in the Seasons simulation

A second aspect of acclimating a person to the use of a gesture-augmented interface is to use on-screen cues that guide the user to perform actions that productively engage the simulation and its underlying concepts. The GRASP simulations employed “ghost hands” or overlays that showed the user how to position their hands relative to the Leap Motion to affect the simulation parameters (Fig. 5). For example, after a student has put both their hands over the device in the gas pressure simulation, the student sees a translucent hand rotating so that the thumb is on top, and the onscreen text states, “Rotate the hand to become the plunger.” In this way, the student receives both text and visual cues to participate in the simulation through physical representation. Delivering cues that elicit expert-like behavior that can be elaborated upon with reflection and explanation is one of the facilitation guidelines for embodied design described by Abrahamson and Lindgren (2014), and in this case, the cues are fairly explicit about the productive ways for representing simulation components with one’s hands. These visual cues disappear once the student has fully engaged the simulation components (e.g., when the simulation sees one hand representing the plunger wall and one hand representing the molecules). The disappearing of on-screen cues is consistent with research on scaffolding that argues for “fading” of supports as students find success with a particular task (Collins, Brown, & Newman, 1989). Additional ways that designers can acclimate learners to a novel gestureaugmented environment include using a “Wizard-of-Oz” (WOz) approach to responding to learner gestures through human operation rather than relying on computer-driven gesture recognition (e.g., Nielsen, Störring, Moeslund, & Granum, 2003). In the early stages of the GRASP project the WOz technique was used to try and determine the feasibility of using particular gestures with the simulation interface. The essence of this approach involved the learner attempting to control the simulation by using a suggested or an original hand gesture, and the researcher would operate the simulation using standard keyboard controls such that it responded appropriately. The WOz approach could also be used when a leaner is

876

R. C. Wallon and R. Lindgren

first being exposed to a gesture-augmented simulation so that she could become accustomed to the gesture scheme without being interrupted by a finicky recognition system, overly-sensitive devices, or by noise in the environment.

Structuring Embodied Activities The third, and perhaps the most challenging, way to facilitate gestural interactions with simulations is creating tasks and activities that are meaningful to the learner and can be built upon for more sophisticated understandings. As discussed above, the mappings of gestures to simulation actions can sometimes be done arbitrarily, making it difficult to transform those gestures into significant learning. But it is also the case that introducing congruent gestures to students at the same time that they are asked to perform specific tasks with a simulation has the potential to overload their capacity for constructing meaning from the interaction. For example, for a student with only a primitive understanding of what causes gas pressure, it may not be very productive to expect them to explain the syringe phenomena through a request to bump a fist against a flat hand. Instead, the researchers requested these gestures but gave the student simple challenges such as “try to use the gestures to increase the pressure as high as it will go.” Through experimentation with the interface the student will typically discover that the way to increase the pressure is to hit the fist against the flat hand rapidly. The researchers then encourage the student to reflect upon why this is the case, and to tell what their hands likely represent when they are doing the gesture. This approach builds upon another facilitation principle described by Abrahamson and Lindgren (2014) where it is suggested that for embodied designs students start by performing physical tasks that may not initially be understood, but from which new meanings and new understandings can be constructed through facilitation. Equally important as giving learners simple and performable tasks as a starting point for simulation interactions is to prompt students to reflect upon those actions and what effects they had on the simulation states and parameters. It is through the articulation of their physical strategies for operating the simulation that students will start to make meaningful connections between their actions and the scientific relationships visualized in the simulation. For example, by describing why a steep angle of their hand – which they now know to represent light rays – puts the location of the Earth’s orbit around the sun in winter, the student makes an embodied connection between the path of light rays and the season. Another way to structure physical tasks within the context of a learning simulation is to create games that motivate students to perform particular movements and actions as a means of achieving game objectives and perhaps even competing with others. A study by Johnson-Glenberg, Savio-Ramos, and Henry (2014), for example, showed learning gains for students who used whole-body movements to engage with the Alien Health game to learn about nutrition and making healthy food choices. A well-designed digital game can effectively motivate students to participate

34

Considerations for the Design of Gesture-Augmented Learning Environments

877

enthusiastically with solicited body-actions and provide a meaningful context for interpreting those actions.

Configure Productive Social Environments The final consideration in supporting gesture-based interactions with simulations is to provide learners with a social context that is conducive to effective communication and productive conversations. The authors generally would not advocate for the design of gesture-augmented simulations that were intended to be used by a single individual in isolation because gestures are a communicative act. Therefore, even when gestures are performed for oneself (e.g., gesturing while talking on a cell phone), they typically emerge from an authentic social interaction. So the challenge with simulations is to position their use within a receptive and supportive audience for gesture-driven explanations. For the GRASP simulations this was primarily accomplished by having students attempt to construct their explanations for a knowledgeable other or expert, in this case the researcher. The advantage of this configuration is that the expert can prompt the user to effectively integrate their gestures with their verbal explanations and with the on-screen simulation activity. For example, a student may be accurately describing that the bumping of their hands is making the temperature on the other side of the heat transfer simulation increase, but the researcher can prompt the student to explain why that is happening, and to make reference to the interaction between molecules when doing so. Another, perhaps more natural, way to elicit these explanations is to put students in small groups and to task them with explaining to each other how the target science phenomenon works using the gesture-driven simulation. Having students give explanations to other students can be particularly effective if the explaining student is given some degree of responsibility for what the other students know. There are clever ways that a teacher could instantiate that responsibility, such as assessing the explaining student based on the knowledge of the students who received the explanation, or even having the students who received the explanation then attempt to give an explanation to a third group of students. There is even the potential of building a teachable student into the simulation itself – previous work has looked at the efficacy of teachable agents where students construct explanations (e.g., through causal concept maps) that are delivered to a computer agent whose “knowledge” is assessed and relayed to the student (Chase, Chin, Oppezzo, & Schwartz, 2009). Given current advances in machine learning and data analytics, it is feasible that such a teachable agent could be created that responds to gestural input as well.

Final Thoughts The authors do not want to give the impression that the considerations presented in this chapter need to, or even should be, worked through linearly. It is likely that in most cases the first considerations will come from either the “gesture” circle or the

878

R. C. Wallon and R. Lindgren

“learning environment” circle of Fig. 1, and that a designer will bounce back and forth between these considerations before moving to the support and facilitation space for learning environments. For example, a learning environment designer may start with the notion of creating a simulation that has predefined gestures for interacting with a specific mathematics topic, but it has not yet been worked out what those gestures will be. Likewise, a designer may start with the notion that children seem to really comprehend algebraic functions when they use their hands to show “slope,” but the designer has not yet worked out how slope representations will be captured and visualized in the simulation. Thus the authors’ recommendation for using these considerations for a new design is to strive for breadth at whatever level the designers are at currently. Once one has gotten to the point of considering support and facilitation, for example, ensure that the whole possibility space of interventions – as well as potential interactions between interventions – is contemplated. As far as connecting these considerations with other models of design, the authors feel that they serve as an elaborated subcomponent of existing structures such as Preece, Rogers, and Sharp’s (2002) Interaction Design Model. Most of these considerations fall into the subcomponent of “identify needs and establish requirements,” but they also help guide how the design is iterated or evaluated. For example, our consideration of “connecting digital representations and gestures” begs the question of how they are connected and how well they are connected. An evaluation that determines that the current implementation does not effectively connect gestures to digital representation could spur the next iteration. In this chapter, we sought to bring in some of the unique challenges of designing interfaces that both augmented learning through meaningful embodied connections and also presented users with an intuitive and productive user-interface. Harnessing the power of gestures in interactive learning environments is an exciting new frontier for designers, and when the right considerations are made, the resulting experience can be transformative for learners. Acknowledgments This material is based upon work supported by the National Science Foundation under Grant No. DUE-1432424. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

References Abrahamson, D., & Lindgren, R. (2014). Embodiment and embodied design. In Cambridge handbook of the Learning Sciences (2nd ed., pp. 358–376). Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.022. Alibali, M. W., & Nathan, M. J. (2012). Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures. The Journal of the Learning Sciences, 21, 247–286. https://doi.org/10.1080/10508406.2011.611446.

34

Considerations for the Design of Gesture-Augmented Learning Environments

879

Black, J. B., Segal, A., Vitale, J., & Fadjo, C. (2012). Embodied cognition and learning environment design. In D. Jonassen & S. Lamb (Eds.), Theoretical foundations of student-centered learning environments (2nd ed., pp. 198–223). New York: Routledge. Chase, C. C., Chin, D. B., Oppezzo, M. A., & Schwartz, D. L. (2009). Teachable agents and the protégé effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18, 334–352. Clement, J. (2013). Roles for explanatory models and analogies in conceptual change. In S. Vosniadou (Ed.), International handbook of research on conceptual change (2nd ed., pp. 412–446). New York: Routledge. Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning and instruction: Essays in honour of Robert Glaser (pp. 453–494). Hillsdale, NJ: Lawrence Erlbaum & Associates. Dourish, P. (2001). Where the action is: The foundations of embodied interaction. Cambridge, MA: MIT Press. Engelkamp, J., & Zimmer, H. D. (1994). The human memory: A multi-modal approach. Seattle, WA: Hogrefe & Huber. Flood, V. J., Amar, F. G., Nemirovsky, R., Harrer, B. W., Bruce, M. R. M., & Wittmann, M. C. (2014). Paying attention to gesture when students talk chemistry: Interactional resources for responsive teaching. Journal of Chemical Education, 92, 11–22. https://doi.org/10.1021/ ed400477b. Gallagher, S. (2005). How the body shapes the mind. Oxford: Oxford University Press. Gibson, J. J. (1962). Observations on active touch. Psychological Review, 69, 477–491. Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin. Goldin-Meadow, S. (2005). Hearing gesture: How our hands help us think. Cambridge, MA: Harvard University Press. Goldin-Meadow, S., Cook, S. W., & Mitchell, Z. a. (2009). Gesturing gives children new ideas about math. Psychological Science, 20, 267–272. https://doi.org/10.1111/j.1467-9280.2009.02297.x. Han, I., & Black, J. B. (2011). Incorporating haptic feedback in simulation for learning physics. Computers & Education, 57, 2281–2290. Hannafin, M., & Peck, K. (1988). The design, development, and evaluation of instructional software. New York: Macmillan. Hartman, B. A., Miller, B. K., & Nelson, D. L. (2000). The effects of hands-on occupation versus demonstration on children’s recall memory. American Journal of Occupational Therapy, 54, 477–483. Hostetter, A. B., & Alibali, M. W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15, 495–514. https://doi.org/10.3758/PBR.15.3.495. Isbister, K., & Mueller, F. F. (2015). Guidelines for the design of movement-based games and their relevance to HCI. Human Computer Interaction, 30, 366–399. Johnson-Glenberg, M. C., Birchfield, D. A., Tolentino, L., & Koziupa, T. (2014a). Collaborative embodied learning in mixed reality motion-capture environments: Two science studies. Journal of Educational Psychology, 106, 86–104. https://doi.org/10.1037/a0034008. Johnson-Glenberg, M. C., Savio-Ramos, C., & Henry, H. (2014b). “Alien Health”: A nutrition instruction exergame using the kinect sensor. Games for Health: Research, Development, and Clinical Applications, 3, 241–251. Johnson-Glenberg, M. C., Megowan-Romanowicz, C., Birchfield, D. A., & Savio-Ramos, C. (2016). Effects of embodied learning and digital platform on the retention of physics content: Centripetal force. Frontiers in Psychology, 7, 1–22. https://doi.org/10.3389/fpsyg.2016.01819. Kim, M., Roth, W. M., & Thom, J. (2011). Children’s gestures and the embodied knowledge of geometry. International Journal of Science and Mathematics Education, 9, 207–238. https:// doi.org/10.1007/s10763-010-9240-5.

880

R. C. Wallon and R. Lindgren

Lindgren, R. (2015). Getting into the cue: Embracing technology-facilitated body movements as a starting point for learning. In V. R. Lee (Ed.), Learning technologies and the body: Integration and implementation in formal and informal learning environments (pp. 39–54). New York: Routledge. Lindgren, R., & Johnson-Glenberg, M. C. (2013). Emboldened by embodiment: Six precepts for research on embodied learning and mixed reality. Educational Researcher, 42, 445–452. https:// doi.org/10.3102/0013189X13511661. Lindgren, R., Wallon, R. C., Brown, D. E., Mathayas, N., & Kimball, N. (2016). “Show me” what you mean: Learning and design implications of eliciting gesture in student explanations. In C. Looi, J. Polman, U. Cress, & P. Reimann (Eds.), Proceedings of the Twelfth International Conference of the Learning Sciences (pp. 1014–1017). Singapore: National Institute of Education. Mathayas, N., Brown, D. E., & Lindgren, R. (2016). Exploring middle school students’ sense making of a computer simulation about thermal conduction. In C. Looi, J. Polman, U. Cress, & P. Reimann (Eds.), Proceedings of the Twelfth International Conference of the Learning Sciences (pp. 1267–1268). Singapore: National Institute of Education. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. Nielsen, M., Störring, M., Moeslund, T. B., & Granum, E. (2003). A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In International gesture workshop (pp. 409– 420). Springer: Heidelberg. Paek, S. (2012). The impact of multimodal virtual manipulatives on young children’s mathematics learning (doctoral dissertation). Retrieved from ProQuest dissertations & theses full text (3554708). Ann Arbor, MI. Preece, J., Rogers, Y., & Sharp, H. (2002). Interaction design: Beyond human-computer interaction. New York: Wiley. Radford, L. (2009). Why do gestures matter? Sensuous cognition and the palpability of mathematical meanings. Educational Studies in Mathematics, 70, 111–126. https://doi.org/10.1007/ s10649-008-9127-3. Resnick, M. (2002). Rethinking learning in the digital age. In G. S. Kirkman, P. K. Cornelius, J. D. Sachs, & K. Schwab (Eds.), The global information technology report 2001–2002: Readiness for the networked world. New York: Oxford University Press. Roth, W.-M. (2001). Gestures: Their role in teaching and learning. Review of Educational Research, 71, 365–392. https://doi.org/10.3102/00346543071003365. Schwartz, R. N. (2010). Considering the activity in interactivity: A multimodal perspective (doctoral dissertation). Retrieved from ProQuest dissertations & theses full text (3404551). Ann Arbor, MI. Segal, A. (2011). Do gestural interfaces promote thinking? Embodied interaction: Congruent gestures and direct touch promote performance in math (doctoral dissertation). Retrieved from ProQuest dissertations & theses full text (3453956). Ann Arbor, MI. Shapiro, L. (2010). Embodied cognition. New York: Routledge. Singer, M., Radinsky, J., & Goldman, S. R. (2008). The role of gesture in meaning construction. Discourse Processes, 45, 365–386. https://doi.org/10.1080/01638530802145601. Wallon, R. C., Brown, D. E., & Lindgren, R. (2016). Student gestures during shifts from descriptions to explanations of gas pressure. Paper presented at the annual meeting of the National Association for Research in Science Teaching, Baltimore, MD. Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9, 625–636. https://doi.org/10.3758/BF03196322. Yoon, C., Thomas, M. O., & Dreyfus, T. (2011). Gestures and insight in advanced mathematical thinking. International Journal of Mathematical Education in Science and Technology, 42, 891–901.

34

Considerations for the Design of Gesture-Augmented Learning Environments

881

Robert Wallon is a doctoral student in the Department of Curriculum & Instruction at the College of Education at the University of Illinois at Urbana-Champaign. He is a research assistant on the GRASP project, and his research interests involve educational technologies for science teaching and learning. As a former high school science teacher, he values research that considers implications for practice and that acknowledges the complexities of classroom learning environments. Robb Lindgren is a Learning Scientist and Assistant Professor in the Department of Curriculum & Instruction at the College of Education at the University of Illinois at Urbana-Champaign. He also has affiliate appointments in Educational Psychology, Informatics, and the Beckman Institute. Dr. Lindgren’s research examines theories and designs for learning within emerging media platforms (e.g., simulations, virtual environments, mobile devices, video games, augmented, mixed reality, etc.). He seeks to understand how digital technologies can be used to construct new identities and generate new perspectives that lead to stronger comprehension of complex ideas, particularly in STEM content areas.

Personalizing Flipped Instruction to Enhance EFL Learners’ Idiomatic Knowledge and Oral Proficiency

35

Wen-Chi Vivian Wu, Jun Chen Hsieh, and Jie Chi Yang

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Personalization for the Learner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Flipped Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . English Idioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Significance of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Instructional Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Quantitative Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qualitative Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . RQ1: How Did the Personalized Flipped Instruction Differ from the Conventional Instruction with Regard to the Students’ Idiomatic Knowledge and Oral Proficiency Outcomes? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . RQ2: How Did the Students Perceive the Personalized Flipped Instruction in Comparison with Conventional Lecture-Based Instructions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Suggestions for Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

884 886 886 887 888 888 889 889 889 894 894 896 897

897 899 901 902 903

W.-C. V. Wu Department of Foreign Languages and Literature, Asia University, Taichung, Taiwan e-mail: [email protected] J. Chen Hsieh (*) · J. C. Yang Graduate Institute of Network Learning Technology, National Central University, Taoyuan, Taiwan e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_59

883

884

W.-C. V. Wu et al.

Abstract

Personalized learning transforms learners from passive receivers of information to active learners in the learning process, because it transforms the teaching content and instructional implementation to take advantage of the individual characteristics of the learner. Adapting to individual differences fulfills the ultimate goal of teaching learners in accordance with their aptitude. However, an examination of current EFL instruction reveals that one-size-fits-all instructions without considering personalization and individuality still prevail, that English idioms have not been adequately incorporated in language education, and that mobile-assisted language learning is still in its infancy. Therefore, this study aimed to examine the effects of an idiom-based, personalized, constructivist flipped instruction on EFL learners’ idiomatic knowledge and oral proficiency. The participants were 40 English-major sophomores enrolled in English Oral Training classes in central Taiwan. Multiple sources of data were collected, including idiom-based pre-/posttests, one questionnaire exploring learners’ perception of the flipped experience, focus-group interview, and in-class observation. The results revealed that such a personalized flipped instructional design effectively motivated students, enhanced their learning outcomes (idiomatic knowledge and oral proficiency), and turned them into active learners. Keywords

Personalized learning · Flipped learning · Idiomatic knowledge · Oral proficiency · LINE

Introduction Educators have long recognized that learning is enhanced and optimized when instruction is personalized or individualized, that is, adaptive to individual differences. According to the National Educational Technology Plan developed by the US Department of Education (2017), personalized learning refers to “instruction in which the pace of learning and the instructional approach are optimized for the needs of each learner (p. 9). Learning objectives, instructional approaches, and instructional content (and its sequencing) may all vary based on learner needs. In addition, learning activities are meaningful and relevant to learners, driven by their interests and often self-initiated” (p. 9). The term “personalized learning” could be mainly distinguished into two areas. It can refer to a learning system that features the flexibility to adjust to learners’ needs. Alternatively, it could refer to instructional designs specifically tailored to the needs of individual learners, which is the main focus of this study. With the development of technology, researchers and educational practitioners have directed their attention to how personalized learning and technology could be integrated to enhance learning outcomes (Grant & Basye, 2014).

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

885

Among innovative pedagogies, flipped learning has been a widely practiced approach that integrates technology. More specifically, flipped learning reverses conventional lecture-based instruction by preparing students with content knowledge before class using technology (i.e., previewing online learning materials and watching instructional videos). Such preparation consequently allows for more advanced learning activities during in-class time. In other words, students are given more opportunities to participate in meaningful engaging activities, thus enhancing learning outcomes (Boucher, Robertson, Wainner, & Sanders, 2013; Chen Hsieh, Wu, & Marek, 2017; Hung, 2015). As technology facilitates the practice of flipped learning around the world, the advancement of technology also changes the mutual communication and interaction among people. Synchronous as well as asynchronous communication via diverse platforms or mobile applications has gained increasing popularity around the world. Mobile messaging applications, such as LINE or WeChat, particularly draw the attention of young users because such applications serve as online media for socializing, sharing, and communicating (Sweeny, 2010). While instant and text messaging technologies result in stronger motivation and support (Coniam & Wong, 2004; Wu, Yen, & Marek, 2011), how such technology could be utilized to facilitate language learning is still underexplored. As language learning is not merely for rote memorization of language rules, using language for effective interaction with the international society has become the mainstream. One element that facilitates such effective global interaction is the mastery of English idioms, as the number of idioms an individual commands is positively correlated with the success of communicative abilities (Fotovatnia & Khaki, 2012; Shirazi & Talebinezhad, 2013). Although the mastery of English idiomatic expressions holds the key to successful global interaction, idioms are “one of the most difficult aspects in L2 (i.e., second language) acquisition due to the fact that they are conventionalized expressions peculiar to a language community and they are usually frozen in form and often unpredictable in meaning” (Liu, 2008, p. xiii). Despite evidence supporting personalized learning, flipped learning, mobile messaging technologies, and English idiom mastery in language education, empirical probe into whether flipped instruction with personalized materials and mobileassisted learning tasks could enhance EFL learners’ oral proficiency and idiomatic competence has remained scarce. Accordingly, a mixed-method research design combing quantitative and qualitative data was employed to investigate the effects of the proposed personalized flipped instruction on students’ oral proficiency and idiomatic knowledge. The following research questions guided the study: 1. How did the personalized flipped instruction differ from the conventional instruction with regard to the students’ idiomatic knowledge and oral proficiency outcomes? 2. How did the students perceive the personalized flipped instruction in comparison with conventional lecture-based instructions?

886

W.-C. V. Wu et al.

Literature Review Personalization for the Learner Personalized learning describes a learning context where learning objectives, instructional approaches, and instructional content vary based on learners’ individuality or needs. In other words, a personalized instruction features flexibility and individuality by allowing learners to choose level-appropriate materials for selfdevelopment. As an educational method that highlights the adjustment of the teaching content or instructional implementation in order to adapt to learner characteristics, personalized learning transforms learners from passive receivers of information to active collaborators in the learning process. Parameters involved in personalized learning include learner capabilities, schematic knowledge, learning styles, and preferences. Such considerations that attempt to meet individual differences fulfill the ultimate goal of teaching in accordance with student aptitude. However, the concept of personalized learning is not new. Cronbach (1957) advocated differentiating instructional methodologies in accordance with an individual’s cognitive aptitude. Learning outcomes are based on the interaction between “attributes of person” and instructional treatments, and educators should “find for each individual the treatment to which he can most easily adapt” (Cronbach, 1957, p. 679). Bloom (1971, 1984) suggested that all learners could obtain higher achievement if provided with appropriate learning conditions that adapt instruction to learning rates and learning modalities. MacKinnon (1978) also highlighted the implications of differing student learning preferences by saying that, “The wide range of individual differences surely must mean that there is no single method for nurturing creativity; ideally the experiences we provide should be tailor-made, if not for individual students, at least for different types of students” (p. 171). Exclusive adoption of one single teaching or learning style oftentimes is not conducive to success in educational programs (Barbe & Swassing, 1979), because knowing the learners’ individual preferences benefits the development of more flexible learning environments. Snow and Farr (1987) noted that educational programs could be successful and effective only with appropriate attention to personal cognitive needs of individual learners. Current research exploring the benefits of adapting instruction to individual learner differences has found positive effects of adaptive learning leading to better learning outcomes (Pashler, McDaniel, Rohrer, & Bjork, 2008). A belief among learners that a course has met their learning preferences has been linked to overall satisfaction (Manochehri, 2008), resulting in a positive effect on learning. However, conventional learning employs a one-size-fits-all teaching/learning model – a “static” environment in which all learners are provided with “the same information in the same structure using the same interface” (Wauters, Desmet, & Van den Noortgate, 2010, p. 549). In other words, widely implemented one-size-fits-all instructional methodologies do not take into account pivotal factors such as flexibility and individuality, resulting in the lack of ability to adapt to the idiosyncrasies of learners. It is of vital importance to recognize that learners have individual

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

887

differences that reflect their prior knowledge, cognitive aptitude, and learning preferences (Graf, Liut, & Kinshuk, 2010). It is thus the instructor’s responsibility to take those factors into account and to create a learning environment in which each learner can make best use of personal strengths to enhance learning outcomes (Bishop & Foster, 2011). Such consideration is well embodied in flipped learning where traditional lectures are moved outside the classroom or before class time with the use of technology and the application and practice of concepts is moved inside the classroom via collaborative learning activities.

Flipped Learning As the world witnesses the global explosion of flipped learning, there has been a significant shift from traditional instructor lectures in the classroom to engaging learners actively in “a dynamic and interactive learning environment where the educator guides students as they apply concepts and engage creatively in the subject matter” (Flipped Learning Network, 2014, p. 1). As an alternative teaching approach in which students view instructional videos prior to the class time and perform pre-classroom learning activities, the flipped learning approach enables instructors to devote more class time to designing active, meaningful, and collaborative learning activities with immediate corrective feedback, providing ample opportunities for students to learn (McLaughlin et al., 2014) and thus improving subsequent learning outcomes (Boucher et al., 2013; Sahin, Cavlazoglu, & Zeytuncu, 2015). Because flipped learning reverses the conventional knowledge acquisition paradigm with information introduced to students before class, detailed explanations of microlevel linguistic aspects no longer take up a major proportion of class time. Student understanding can be easily checked by the teacher monitoring the pre-instruction tasks as well as the in-class meaning clarification, allowing for more advanced and meaningful engagement among students during in-class activities. The enhancement of student learning outcomes as a result of flipped learning has been widely supported by numerous studies (Baepler, Walker, & Driessen, 2014; Chen Hsieh et al., 2017; Deslauriers & Wieman, 2011; McLaughlin et al., 2014; Moravec, Williams, Aguilar-Roca, & O’Dowd, 2010; Sahin et al., 2015). Other researchers also mentioned the positive effects of flipped learning on increasing student engagement (Jamaludin & Osman, 2014) and on cultivating student autonomy and awareness (Yang, 2013). Bishop and Verleger (2013) particularly mentioned two vital components of flipped learning, including (1) the use of computer technologies such as video lectures and (2) the involvement of interactive learning activities. With the help of technology, such instruction allows students to “proceed at their own pace, guide themselves to additional content, and assess their own learning gains” (McLaughlin et al., 2013, p. 196). The interactive learning activities, on the other hand, transform the conventional, lecture-based, and unidirectional knowledge delivery instruction into a learner-centered, engaging, and interactive learning context where students apply higher-order critical thinking abilities to diverse learning tasks.

888

W.-C. V. Wu et al.

English Idioms The final major consideration in developing this study was the importance of practicing English idioms in EFL contexts. Most conventional EFL instruction emphasizes on vocabulary, grammar, and sentence structure, yet the mastery of English idioms holds the key to drastically advancing an individual’s English ability. In fact, learning English for effective global communication has been a priority in the field of language education, whether in native language settings or in foreign language contexts. As regards effective global interaction, the overall oral proficiency (fluency and accuracy) of English as a Foreign Language (EFL) learners has been regarded as a major determinant of success. However, even with years of English learning, EFL learners still bumble and have difficulty speaking English. To interact effectively with the international society, one facilitating factor is the mastery of English idiomatic expressions. It has been recognized that idioms, a group of words whose meaning is not deducible from those individual words, are far difficult to be learned and mastered than vocabulary. To EFL learners in non-native English-speaking countries, the learning of idioms poses greater challenge (Motz, 2007). Nevertheless, previous research concerning English idiom learning has been mainly restricted to English-speaking countries. Studies addressing the learning of English idioms in an EFL setting (such as Taiwan) remain scarce (Asl, 2013; Khan & Daşkin, 2014; Mäntylä, 2004; Tărcăoanu, 2012). Despite the needs of acquiring English idioms for communicative purposes, instructional pedagogies, nevertheless, do not always cater to the changing needs of learners of different generations, and English often remains a test-oriented academic subject with microlevel linguistic skills learned in a conventional way. Therefore, educators have tried to innovate, moving away from conventional instructional methodologies, to improve learning and motivate learners to excel (Johnson, Adams Becker, Estrada, & Freeman, 2014).

Significance of the Study The significance of the current study lies in the integration of personalized learning into flipped learning. Despite numerous evidence from previous studies concerning personalized learning and flipped instruction, this study has extended previous research by unraveling the joint effects of the aforementioned aspects on tertiary education in EFL contexts. Specifically, it examined how personalized flipped instruction (featuring level-appropriate learning materials and mobile-assisted learning tasks) in comparison with conventional lecture-based teaching affected idiomatic knowledge and oral proficiency among EFL learners in the tertiary level. Furthermore, this study provides a holistic personalized flipped instructional design, in which passive learning activities such as unidirectional lectures were replaced by active selection of appropriate instructional videos and engagement in pre- as well as in-class learning activities. With the instructional goal of the study aiming to enhance the students’ ability to use English idioms, the study simultaneously enhanced their overall oral proficiency

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

889

and other vital factors in language learning (such as motivation and effectiveness). The flipped instruction provided personalized learning, as it enabled learners to view level-appropriate learning materials prior to physical classroom meetings as they wanted and to have multiple trials of oral recordings via the LINE smartphone app until they felt satisfied.

Method Participants The participants were 40 English-major sophomores at a 4-year university in central Taiwan. They were convenient samples from two intact English Oral Training classes. Mostly female and between the ages of 19 and 20, they had studied English for around 8 years, since junior high school. Based on their English performance on the university admission examination as well as their in-class performance on the oral training course they had taken when they were freshmen, their English proficiency level was considered to be at the upper-intermediate level. On average, they were able to make inquiries and converse on daily topics, express opinions on different subjects, and share personal perspectives in social interactions. The 20 participants in Class A first experienced personalized constructivist flipped instruction (the first 8 weeks) and then shifted to a conventional lecturebased treatment (the subsequent 8 weeks). The 20 students in Class B experienced the opposite instructional sequence, with 8 weeks of conventional lecture-based instruction followed by the 8-week personalized constructivist flipped instructional treatment. Because the researchers desired all of the students to experience both instructional styles rather than depriving students in a control group of the chance to experience the more innovative flipped treatment, the researchers chose the within-subject research methodology (Creswell, 2013). Therefore, the study was not per se a formal experimental design with separate control and experiment groups intended to examine the effects of isolated variables. Prior to the study, an ethical review application was approved by the university, and an informed consent form was signed by all of the participants.

The Instructional Design To meet the instructional goals of the oral training classes, the instructor chose the Good Chats (3rd ed.) textbook as the main instructional material. As an English conversation textbook designed for advanced students of English, Good Chats provides the readers with the most frequently used English idioms, expressions, and turns of phrase for oral communication. The textbook contains 15 chapters, and each chapter features (1) a reading passage that highlights idioms and collocations frequently used by native speakers while chatting about the given topic and

890

W.-C. V. Wu et al.

(2) extensive participatory dialogues that guide students to express their own ideas. Six topics were randomly chosen as the major reading materials and designated to the two instructional treatments: sports, family, and the Internet for the conventional instruction and dating, friends, and beauty for the flipped instruction. Each chapter included a reading passage, 4 comprehension questions, 30–40 idioms, and a chatfor-two-guided dialogue requiring the learners to draft the final dialogue. Instruction of each chapter in the idiom-based personalized flipped treatment lasted for 2–3 weeks, including the students’ before-class preview of reading materials and videos, before-class assignments and interaction in LINE groups, and in-person class meetings. In accordance with the personalized flipped learning model, the instruction thus required students to watch instructional videos before physical classroom meetings. However, the students were able to choose from a range of video materials for each lesson. Since the goal of this study emphasized the adjustment of teaching contents based on learners’ diverse capacities, instructional videos that met the participants’ diverse learning aptitudes were provided online. The instructional videos for each chapter were divided content-wise into three types, including basic illustration, advanced elaboration, and extra online information. Videos of basic illustration (as shown in Fig. 1) were developed by the Taiwanese instructor. The content covered selected important idiomatic expressions (around 20 items) in the form of voiced PowerPoint slides, with each expression containing its definition, collocational usages, and sample sentences. Unlike the basic illustration that detailed definition and sample sentences, videos of advanced elaboration were created for the class by English-speaking college instructors. These videos (see Fig. 2) provided the introduction to frequently used idiomatic expressions with the lecturer’s face seen, real-life objects shown (if necessary), and keyword captions (rather than definition and sample sentence) shown on the screen. For advanced students who were relatively proficient in listening, such content presentation actively engaged them in the process of knowledge acquisition and directed their attention to crucial points covered in a given topic. In order to fulfill the need of students for additional information related to the chosen topics, extra online information (see Fig. 3) was further located and provided. This type of instructional material was collected from YouTube, showcasing supplementary information that might go beyond the contents covered in the textbook for students who wanted to challenge themselves. For example, the YouTube clip entitled “Advanced English Phrases 5 – Love, Romance, Dating and Relationships – Speak English Naturally” (https://www.youtube.com/watch?v=dA8qruho2Ow) served as an advanced supplement to the topic dating. In addition to specific usages related to dating, students were exposed to a much broader scope with various issues integrated, thus strengthening their ideas of dating in a more extensive and organized manner. As regards before-class assignments, LINE smartphone app was employed for group interaction. LINE is a cross-platform application that operates on Android, iOS, and PC systems. With its functions of text messaging, file posting in dual forms (audio and visual), and graphic “stickers” that express personal feelings or emotions,

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

891

Fig. 1 Sample snapshots of basic illustration

LINE has gained overwhelming popularity among millions of users. The students in the study were highly familiar with LINE and did not require training on its functions. The instructor randomly paired students, and each pair was required to establish a LINE group for subsequent learning activities, with the instructor joining each of the groups to provide constructive verbal and written feedback, such as suggestions on idiom usages and pronunciation. In addition to acting as the capable other who provided feedback, the instructor observed the LINE peer interaction and monitored the students’ overall progress. Prior to the physical class meetings, the students reviewed the assigned topics and selected the preferred instructional videos. After that, each student used the idioms covered in the topic and videos to develop a short story (around 400 words), uploaded the oral recording of the story to the LINE group, received feedback from the instructor, revised the story, and uploaded the final text and audio versions. Students in each group also collaboratively completed the guided dialogue of the given topics, took turns recording the related content

892 Fig. 2 Sample snapshots of advanced elaboration

Fig. 3 Sample snapshots of extra online information

W.-C. V. Wu et al.

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

893

(around 5–8 min in length), and uploaded the final audio version to their LINE groups. Physical class meetings were held once a week (around 100 min), during which students were guided through highly interactive discussion, because the pre-classroom learning activities enabled them to have a preliminary knowledge and understanding of how the given idiomatic expressions could be used and applied. Therefore, the instructor did not have to spend valuable in-class time explaining the details of idiomatic expressions and microlevel grammar rules, effectively turning the conventional unidirectional lecture-based teaching into learner-centered instruction where students were given ample opportunities to apply what they had learned to authentic oral communication. Subsequent in-class learning activities for the personalized flipped instruction focused more on engaging collaboration among students to enhance their idiomatic knowledge, to foster critical thinking ability, and to promote active learning. Procedures of the personalized flipped instruction are shown in Fig. 4. The learning activities were diverse in nature (e.g., have a guess, opinion integration, and scenario improvisation) so that students could learn how to apply learned idioms in different situations. Each group was required to prepare a short oral presentation explaining their overall perspectives on the given issues. In “Have a guess,” for example, students were randomly divided into teams of six, and students

Pre-classroom activities

First 100-minute class meeting

Between-class tasks

•Students view levelappropriate instructional videos on idiom introduction and post personal short stories (both text and audio version) on LINE groups •The instructor provides feedback on LINE

•Misconception clarification •Students’ short story oral presentation •Collaborative learning activities

•Students view the reading text •Students collaboratively complete the guided dialogue and post the text as well as audio version on LINE •The instructor provides feedback on LINE

Fig. 4 Procedures of the personalized flipped instruction

Second 100minute class meeting •The instructor briefly goes through the reading text and checks students’ comprehension •The instructor engages students in discussion of comprehension questions •Students express their opinions in groups •Students orally present the guided dialogues

894

W.-C. V. Wu et al.

in each team tried to define the assigned idioms in their own words for their teammates to guess. Teams competed for accuracy and against time. During the conventional lecture-based instructional treatment (also conducted in English), on the other hand, the instructor first detailed on the reading contents for each chapter during physical class meetings. Although the students also experienced collaborative activities like those in the flipped instruction, explaining the idioms and introducing the reading passages by the instructor in class left considerably less time for the students to be engaged in higher-level learning activities.

Research Design Multiple sources of data were collected to examine the effectiveness of the idiombased personalized flipped instructional treatment and to explore student perceptions about the proposed learning experience, including (1) pre- and posttests of idioms, oral reading, and comprehension questions, (2) one questionnaire, (3) semistructured focus-group interviews, and (4) in-class observations by the instructor. As a mixed-method research, this study involves collecting, analyzing, and integrating quantitative and qualitative sources of data. While quantitative data (students’ performance on idioms and oral proficiency and responses to the questionnaire) serve as the primary source, qualitative data (focus-group interview and in-class observation) were utilized as supplementary evidence. With such an approach to research, the integration of quantitative and qualitative sources provides a better understanding of the research problem than either of each alone. Figure 5 shows the alignment among the issues explored, research questions, and data collection. Figure 6 displays the instructional and data collection process.

Quantitative Data Analysis To examine the participants’ overall idiomatic knowledge and oral fluency, in answer to research question one, the participants completed pre- and posttests of idiom, oral reading, and comprehension questions selected from the instructional material. The respective pretests and posttests were identical in content for the conventional instruction and the personalized flipped instruction and were subdivided into three parts: idiom definition/sentence construction (ten English idioms), oral reading aloud (three paragraphs), and comprehension questions (three questions). The students responded orally by defining the chosen idioms and constructing relevant sentences, reading paragraphs aloud, and answering one comprehension question for each of the three units. The pretest and posttest audio recordings were evaluated by the current researchers. To ensure higher inter-rater reliability in evaluating definitions and subsequent sentence construction using the idioms, the researchers adopted the oral idiomatic proficiency rubric (see Table 1) developed by Chen Hsieh et al. (2017). Inter-rater reliability was evaluated via Krippendorff’s alpha.

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

Research Question 1 Idiomatic knowledge and oral learning outcomes

895

Pre-/post-tests of idiom, oral reading,

Quantitative

comprehension questions

Questionnaire: Perception of Flipped Learning Experience 1. Motivation

Quantitative

2. Effectiveness 3. Engagement 4. Overall satisfaction

Research Question 2 Perception of the personalized flipped learning

Focus-group interview: Overall learning expereince 1. Lecture -based instruction vs. personalized flipped learning

Qualitative

2. LINE for English idiom learning

In-class observation

Fig. 5 Issues explored and instruments employed in the current study

EFL oral instruction 1st week 2nd -9th week

Pre-test 1 Class A: Conventional instruction

Class B: Personalized flipped instruction

100 mins 100 mins/week

10th week

Post-test 1

100 mins

11th week

Pre-test 2

100 mins

12th-19th week

Class A: Personalized flipped instruction

Class B: Conventional instruction

100 mins/week

20th week

Post-test 2

100 mins

22nd week

Focus-group interviews

50 mins

Fig. 6 Instructional and data collection procedure of the current study

896

W.-C. V. Wu et al.

Table 1 Oral idiomatic proficiency rubric Points 10 8

6 4 2 0

Criteria Student clearly understood the idiom and quickly and effectively constructed a sentence using the idiom. The sentence was complex and meaningful The student understood the idiom but constructed a sentence using the idiom after a reasonable pause. Or the sentence was not as long and meaningful as if should have been to demonstrate complete understanding The student understood the idiom but had difficulty or took more time than usual to construct a sentence using the idiom. Or, the sentence was very short and simple The student understood the idiom but was unable to construct a sentence using the idiom The student misunderstood the idiom The student was unable to provide any definition of the idiom

The means of the pre- and posttests were analyzed to compare the adaptive flipped instruction versus the conventional teaching design. A paired-samples t-test was also employed to investigate whether significant differences existed in the students’ idiomatic and oral learning outcomes, comparing the adaptive flipped versus conventional methods. With respect to oral reading and comprehension questions, the researchers evaluated the pretest and posttest audio recordings using the IELTS assessment criteria: speaking to quantify the participants’ oral performance, covering (1) fluency and coherence, (2) lexical resource, (3) grammatical range and accuracy, and (4) pronunciation. Inter-rater reliability, measured with Krippendorff’s alpha at 0.80, suggested a good reliability (Hayes & Krippendorff, 2007). The means of the pre- and posttests were calculated to compare differences (i.e., flipped versus conventional). Furthermore, a paired-samples t-test was employed to investigate the students’ oral learning outcomes in two different forms of instruction. In order to examine the students’ perceptions about the adaptive flipped instruction and the use of LINE as the learning platform, the researchers used the “Perception of Flipped Learning Experience” questionnaire (Chen Hsieh et al., 2017), in which students answered a series of questions addressing four constructs, motivation, effectiveness, engagement, and overall satisfaction.

Qualitative Data Analysis Self-developed focus-group interviews with protocols were also employed to probe into research question 2, exploring the students’ perceptions of their overall learning experiences in the course by comparing the learning experiences between the conventional lecture-based treatment and the personalized flipped instruction. In addition to the aforementioned comparison between the two instructions adopted in the current study, the students were also asked whether they liked using LINE as a learning platform for learning English idioms. To elicit the students’ genuine perceptions of the learning experience without being constrained by their English abilities, the interview

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

897

was thus administered in Chinese and later was carefully coded in English by the researchers, without distorting the students’ original intention. Aside from the students’ responses to and reflection of the learning experiences, the researchers also recorded and analyzed the in-class interactions among the students, in order to document the observations of the students’ learning experiences. The data collected from the focus-group interviews was carefully examined and grouped into recurring themes. Researcher notes were also analyzed for further insights into the students’ learning experiences.

Results and Discussion The overall analysis of the differences between the pre- and posttests, one questionnaire (i.e., “Perception of Flipped Learning Experience”), and the focus-group interviews revealed that the idiom-based personalized flipped instruction effectively motivated the students to learn, enhanced their idiomatic knowledge, and sharpened their oral proficiency. The students held positive attitudes toward the flipped instructional design and the use of LINE as the learning platform. The findings are presented in accordance with the research questions.

RQ1: How Did the Personalized Flipped Instruction Differ from the Conventional Instruction with Regard to the Students’ Idiomatic Knowledge and Oral Proficiency Outcomes? Descriptive statistics comparing the pre- and the posttests in the conventional and personalized flipped instructions indicated that the mean scores of the posttests in both instructional treatments were considerably higher than those of the pretests (see Table 2), suggesting their effectiveness in improving the students’ idiomatic knowledge and oral proficiency. A further analysis with paired-samples t-test shown in Tables 3 and 4 revealed that the students in both treatments performed significantly better on the posttests compared to the pretests and that the posttest of the personalized flipped instruction was significantly higher than that of the conventional instruction. The results suggested that although both treatments effectively enhanced the students’ idiomatic knowledge and oral proficiency, the overall design of the personalized flipped Table 2 Descriptive statistics of the pretest and the posttest Test Pretest Posttest

Instruction Flipped Conventional Flipped Conventional

N 40 40 40 40

Idiom Mean 11.52 10.79 70.43 54.69

SD 4.53 4.86 17.43 17.68

Oral proficiency Mean SD 71.35 5.21 63.42 5.17 84.79 5.32 70.86 5.95

898

W.-C. V. Wu et al.

Table 3 Paired-samples t-test of the idiomatic knowledge Paired differences 95% confidence interval of the difference Std. error Mean SD mean 50.91 18.23 2.82

Post (flipped) to Pre (flipped) Post (conventional) to 43.90 18.17 2.82 Pre (conventional) Post (flipped) to 15.74 0.23 0.04 Post (conventional)

Lower 45.76

Upper 56.32

t 18.735**

df 39

Sig. (2-tailed) 0.000

30.81

41.59

15.245**

39

0.000

14.69

14.98

463.531**

39

0.000

**p < 0.001 Table 4 Paired-samples t-test of the oral proficiency Paired differences 95% confidence interval of the difference Std. error Mean SD mean 13.44 1.78 0.23

Post (flipped) to Pre (flipped) Post (conventional) to 7.44 1.52 0.21 Pre (conventional) Post (flipped) to 13.93 1.63 0.20 Post (conventional)

Lower 15.72

Upper 16.82

t 68.792***

df 39

Sig. (2-tailed) 0.000

6.65

7.58

36.063***

39

0.000

19.64

20.39

101.794***

39

0.000

**p < 0.001

treatment contributed to significantly better learning outcomes than the conventional lecture-based method. The affordances resulting from pre-instructional activities (viewing teaching materials, watching instructional videos, interacting in LINEbased tasks) showed how the flipped instruction prepared the students for subsequent in-class activities. Such a result was further validated in the students’ responses to preference for level-appropriate instructional videos, since materials meeting their current capacities enabled the students to be more engaged in learning. To be more specific, the personalized instructional videos provided information appropriate to the students’ levels. Preferred level-appropriate learning materials greatly motivated the students to learn, shown by the students’ observation that fixed and one-size-fitall materials used in conventional instructions might not be suitable for everyone because such fixed materials could be either too challenging or too easy for some learners. Feeling more engaged and motivated, the students demonstrated a better grasp of the idiomatic phrases in meaning.

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

899

The results of the study were in line with the findings of previous research, demonstrating the positive effects of personalized flipped learning on learning outcomes (Chen Hsieh et al., 2017; Fulton, 2012; Manochehri, 2008; Sahin et al., 2015; Strayer, 2012; Zappe, Leicht, Messner, Litzinger, & Lee, 2009). Overall, the students learned more from the personalized flipped instructional design, as a result of the level-appropriate instructional videos and the ample conversational practices in authentic, engaging, and collaborative learning contexts.

RQ2: How Did the Students Perceive the Personalized Flipped Instruction in Comparison with Conventional Lecture-Based Instructions? The students’ responses to the Perception of Flipped Learning Experience questionnaire revealed the students’ acceptance of the personalized flipped instruction practiced in this study. The results of the four constructs (motivation, effectiveness, engagement, and overall satisfaction) shown in Table 5 demonstrated that the students’ responses fell into the upper-intermediate category. The mean scores of motivation, effectiveness, engagement, and overall satisfaction were 3.79, 3.81, 3.93, and 4.24, respectively. In general, the students felt more motivated as well as engaged in the personalized flipped learning treatment, spent more time and effort on the learning activities, recognized the effectiveness of the innovative instruction, and enjoyed the personalized flipped instruction more. Focus-group interviews also addressed research question 2 that explored the students’ perceptions of their overall learning experiences in the course, specifically focusing on comparing the conventional lecture-based learning experience and the personalized flipped learning experience. Using content analysis, the researchers analyzed the interview responses and identified recurring themes in three closely related dimensions: (1) time engagement in the flipped instruction, (2) effectiveness of the personalized flipped instruction on learning outcomes, and (3) active learning.

Time Engagement in the Flipped Treatment Most of the students reported that they were more motivated and engaged in the personalized flipped instructional treatment because it provided far more interactive and collaborative opportunities for them to use learned English idioms and sharpen their oral ability, in sharp contrast with the conventional lecture-based treatment Table 5 Descriptive statistics of the perception of the personalized flipped instruction Constructs Motivation Effectiveness Engagement Overall satisfaction Note: N = 40

Mean 3.79 3.81 3.93 4.24

SD 0.67 0.62 0.69 0.86

N of items 5 4 4 1

900

W.-C. V. Wu et al.

where insufficient material preview and limited time span for in-class activities usually led to shallow learning. One student noted, “I spend time watching the videos of basic illustration. I like the short story creating activity because it allows me to combine idioms into a coherent structure.” Another student stated, “I spend more time in the flipped instruction than in the previous conventional instruction. Although it takes time to watch the videos, those videos help me know more about the learning content before class.” A few students mentioned, “I feel more engaged in the learning activities in the flipped instruction because doing those tasks in the LINE group is fun.”

Effectiveness of the Personalized Flipped Treatment on Learning Outcomes Most students recognized the effectiveness of the personalized flipped instructional design on their idiomatic knowledge and overall oral proficiency, evidenced by their positive attitudes about the adaptive use of instructional video materials and affirmative responses to the flipped instruction. Some students expressed the opinion that the flexibility in choosing materials appropriate to their proficiency levels enabled them to learn more deeply and effectively. Generally speaking, the students’ preference for instructional videos fit the researchers’ expectation that they preferred level-appropriate materials. Most students with lower proficiency in listening and speaking preferred videos of basic illustration, which were more focused, providing detailed content explanation and concrete examples. One student mentioned, “I like videos of basic illustration more because it suits me well. It provides clear definition and sentential usages of the idioms.” For students at a higher proficiency level, most of them preferred videos of advanced elaboration, and some of them challenged themselves by also acquiring related knowledge from the extra online information. One student stated, “Videos of advanced elaboration really help me a lot. I love to challenge myself and the showing of only keywords in the videos enables me to pay more attention to the content.” Another student expressed his personal thought by saying, “I didn’t know much about English idioms before because the teachers were always rushing against time for covering as much information as possible. I have more chances to practice speaking in the LINE tasks and in-class activities.” A few students recognized the researchers’ effort in providing extra online information. One student concluded that “Extra information helps me broaden my knowledge related to the topics. I learn several relevant issues at the same time.” Positive learning transfer was evidenced by the students’ responses, as one students said, “I could use what I have learned from the extra online information in group discussions and presentations.” It was worth noticing that the students’ choosing extra online information echoed what Krashen (1985) called level of input i + 1, where i represents an individual’s existing linguistic competence and + 1 represents new knowledge that learners are ready to acquire. Learners move from their current competence to a higher level by understanding input that contains i + 1. In the current study, videos providing extra information were taken as an extension that

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

901

provided the students with additional information capable of helping them expand their knowledge.

Active Learning The level-appropriate instructional video materials, engaging LINE tasks, and interactive in-class activities greatly motivated the students to learn, gradually guiding them to become active learners. Several students said that after watching the videos providing extra information, they even searched for additional related clips online. Their contribution to the group discussion in providing extra information was highly regarded. One student gladly noted, “I feel pleased when the group members thank me for providing useful information and adding depth to the topic.” Some students mentioned their active roles in the LINE groups, emphasizing the importance of making equal contribution, as reflected in one student’s statement, “I was afraid that I might not contribute enough in the LINE interaction. So I started to take the initiative and do my best to complete the tasks.” The results concerning active learning align with the findings of previous studies revealing student acceptance of the flipped methodology (Chen Hsieh etal., 2017; Lucke, 2014; Mortensen & Nicholson, 2014; Murdock & Williams, 2011), because viewing instructional materials ahead of class time freed valuable classroom time for meaning clarification, problem-solving, and interactive collaboration (Bishop & Verleger, 2013; Boucher et al., 2013; Cole & Kritzer, 2009; Simkins & Maier, 2010; Tucker, 2012) rather than for detailed explanation of microlevel linguistic issues. What is even more important is that the adaptive use of instructional videos provided content-level appropriateness that the students found to be comfortable. Since student accountability was high in the adaptive flipped instruction, both in the LINE tasks or in the in-class learning activities, the students were gradually guided to be self-directed active learners (Boucher et al., 2013; Herried & Freeman, 2013; Overmyer, 2012; Sarawagi, 2014), thus leading to the enhancement of active and spontaneous learning. Researcher Notes The researchers’ observations of the students’ LINE interaction and in-class interaction of both the personalized flipped and conventional lecture-based instructions revealed that compared with the conventional instruction, the students demonstrated greater learning outcomes (accuracy as well as fluency) during the personalized flipped instruction. During classroom activities, the students in the personalized flipped instruction were much more motivated, as observed in their active engagement to a greater extent in related learning activities.

Conclusions The findings revealed that the idiom-based personalized flipped instructional design effectively motivated and engaged the students to learn. The students’ choice of level-appropriate instructional videos, engagement in the LINE tasks, and

902

W.-C. V. Wu et al.

participation in classroom learning activities engendered deep and meaningful learning. The methodology significantly enhanced the students’ idiomatic knowledge and improved their oral proficiency. Furthermore, the instructional design enabled the students to hold themselves accountable for their learning, contributing to the enhancement of active learning. Based on the findings and subsequent discussion of the current study, the researchers offer the following recommendations for practice. First, flipped instruction is an appropriate instructional design for teaching English as a Foreign Language, greatly motivating students to become even more engaged in related learning activities, thus leading to beneficial learning outcomes. Second, offering diverse adaptive instructional materials enables learners to analyze their own abilities, helps them to become aware of their current competence, allows them to choose levelappropriate materials, leads them to optimize the knowledge they have received, and guides them to become self-directed active learners, thus contributing to greater learning outcomes. Last, but not the least, LINE is an appropriate instructional platform for EFL learning. It provides an authentic setting for interaction (whether through voice or text) with which learners are already familiar. Because students might find it inconvenient to type long paragraphs in LINE, instructors could encourage their students to use the function of voice input by having them speak in English and later choose corresponding words to compose personal short stories and collaborative guided dialogues. To do this, students have to talk very clearly so that LINE would not show incorrect word options. Such activity leads students to be more engaged in oral expression, thus contributing to enhanced oral proficiency (both accuracy and fluency).

Suggestions for Future Research The present study not only provided empirical evidence for an idiom-based flipped instruction featuring personalization and constructivist learning among EFL learners of different proficiency levels, but also shed light on the scenario of mobile-assisted flipped learning. The findings of the current research could serve as guidelines for language instructors to develop curricula that meet the diverse needs of EFL learners. In spite of the abovementioned contributions, the current research also has some suggestions for future researchers interested in this topic. First, the sample size in this study was limited to 40 sophomore English majors taking English Oral Training classes in central Taiwan. Generalizability might not be yielded based on findings from a small sample size. To enhance generalizability, more participants could be recruited. It is suggested that future studies could interpret results from the perspective of the “affordances” that a chosen treatment or technology might have for learning outcomes and perceptions. Second, the current study focused on personalization among learners of different proficiency levels. Other factors related to personalized learning such as technology experience or the academic majors of participants could also be examined to obtain a better understanding of personalized flipped instruction in an EFL context.

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

903

The researchers hope that the results of the personalized flipped instructional design provided in the current study pave the way for further research and for integration of innovative instructional designs in an EFL setting.

References Asl, F. M. (2013). The impact of context on learning idioms in EFL classes. TESOL Journal, 37(1), 2. Baepler, P., Walker, J. D., & Driessen, M. (2014). It’s not about seat time: Blending, flipping, and efficiency in active learning classrooms. Computers & Education, 78, 227–236. Barbe, W. B., & Swassing, R. H. (1979). Teaching through modality strengths. New York, NY: Zane-Bloser. Bishop, C., & Foster, C. (2011). Thinking styles: Maximizing online supported learning. Journal of Educational Computing Research, 44(2), 121–139. Bishop, J. L., & Verleger, M. A. (2013). The flipped classroom: A survey of the research. In ASEE National Conference Proceedings, Atlanta, GA. Bloom, B. (1971). Mastery learning. In J. H. Block (Ed.), Mastery learning theory and practice (pp. 47–63). New York: Holt, Rinehart, & Winston. Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6), 4–16. Boucher, B., Robertson, E., Wainner, R., & Sanders, B. (2013). “Flipping” Texas State University’s physical therapist musculoskeletal curriculum: Implementation of a hybrid learning model. Journal of Physical Therapy Education, 27(3), 72–77. Chen Hsieh, J., Wu, W. C., & Marek, M. (2017). Using the flipped classroom to enhance EFL learning. Computer Assisted Language Learning, 30(1–2), 1–21. Cole, J. E., & Kritzer, J. B. (2009). Strategies for success: Teaching an online course. Rural Special Education Quarterly, 28(4), 36–40. Coniam, D., & Wong, R. (2004). Internet relay chat as a tool in the autonomous development of ESL learners’ English language ability: An exploratory study. System, 32(3), 321–335. Creswell, J. W. (2013). Research design: Qualitative, quantitative, and mixed methods approaches. Los Angeles, CA: Sage. Cronbach, L. J. (1957). The two disciplines of scientific psychology. American Psychologist, 12, 671–684. Deslauriers, L., & Wieman, C. (2011). Learning and retention of quantum concepts with different teaching methods. Physical Review Special Topics, Physics Education Research, 7, 1–6. Flipped Learning Network. (2014). The four pillars of F-L-I-P. Retrieved from http:// flippedlearning.org/cms/lib07/VA01923112/Centricity/Domain/46/FLIP_handout_FNL_ Web.pdf Fotovatnia, Z., & Khaki, G. (2012). The effect of three techniques for teaching English idioms to Iranian TEFL undergraduates. Theory and Practice in Language Studies, 2(2), 272–281. Fulton, K. (2012). Upside down and inside out: Flip your classroom to improve student learning. Learning and Leading with Technology, 39(8), 12–17. Graf, S., Liut, T., & Kinshuk, C. (2010). Analysis of learners’ navigational behaviour and their learning styles in an online course. Journal of Computer Assisted Learning, 26, 116–131. Grant, P., & Basye, D. (2014). Personalized learning: A guide for engaging students with technology. Eugene, OR: International Society for Technology in Education. Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1, 77–89. Herried, C. F., & Freeman, N. A. (2013). Case studies and the flipped classroom. Journal of College Science Teaching, 42(5), 62–66. Hung, H. T. (2015). Flipping the classroom for English language learners to foster active learning. Computer Assisted Language Learning, 28(1), 81–96.

904

W.-C. V. Wu et al.

Jamaludin, R., & Osman, S. Z. M. (2014). The use of a flipped classroom to enhance engagement and promote active learning. Journal of Education and Practice, 5(2), 124–131. Johnson, L., Adams Becker, S., Estrada, V., & Freeman, A. (2014). NMC horizon report: 2014 K-12 edition, The New Media Consortium. Austin, TX. Khan, Ö., & Daşkin, N. C. (2014). “You reap what you sow” idioms in materials designed by EFL teacher-trainees. Novitas-ROYAL (Research on Youth and Language), 8(2), 97–118. Krashen, S. D. (1985). The input hypothesis: Issues and implications. London: Addison-Wesley Longman Ltd. Liu, D. (2008). Idioms. Description comprehension, acquisition and pedagogy. New York/London: Routledge. Lucke, T. (2014). Using learning analytics to evaluate the effectiveness of the flipped classroom approach. Paper presented at the Australasian Association for Engineering Education 2014 conference, Wellington, New Zealand. Retrieved from https://www.researchgate.net/profile/ Terry_Lucke/publication/270162113_Using_Learning_Analytics_to_Evaluate_the_Effective ness_of_the_Flipped_Classroom_Approach/links/54a1e9760cf256bf8baf7b15.pdf MacKinnon, D. W. (1978). In search of human effectiveness: Identifying and developing creativity. Buffalo, NY: Creative Education Foundation. Manochehri, N. (2008). Individual learning style effects on student satisfaction in a web-based environment. International Journal of Instructional Media, 35(2), 221–228. Mäntylä, K. (2004). Idioms and language users: The effect of the characteristics of idioms on their recognition and interpretation by native and non-native speakers of English. Retrieved from https://jyx.jyu.fi/dspace/handle/123456789/13453. McLaughlin, J. E., Griffin, L. M., Esserman, D. A., Davidson, C. A., Glatt, D. M., Roth, M. T., . . . Mumper, R. J. (2013). Pharmacy student engagement, performance, and perception in a flipped satellite classroom. American Journal of Pharmaceutical Education, 77(9), 196. McLaughlin, J. E., Roth, M. T., Glatt, D. M., Gharkholonarehe, N., Davidson, C. A., Griffin, L. M., . . . Mumper, R. J. (2014). The flipped classroom: A course redesign to foster learning and engagement in a health professions school. Academic Medicine, 89(2), 236–243. Moravec, M., Williams, A., Aguilar-Roca, N., & O’Dowd, D. K. (2010). Learn before lecture: A strategy that improves learning outcomes in a large introductory biology class. CBE-Life Sciences Education, 9(4), 473–481. Mortensen, C. J., & Nicholson, A. (2014). Improved student achievement through gamification and the flipped classroom. Paper presented at the ADSA-ASAS joint annual meeting, 20–24 July, Kansas City, MO. Motz, S. (2007). Making sense of English: An introduction to American slang, colloquialisms and idioms. Al Jamiat Magazine, Retrieved from http://www.usegtours.com/documents/MakingSen seofAmericanSlang.pdf. Murdock, J. L., & Williams, A. M. (2011). Creating an online learning community: Is it possible? Innovative Higher Education, 36(5), 305–315. Overmyer, J. (2012). Flipped classrooms 101. Principal, 46–47. Retrieved from https://www.naesp. org/sites/default/files/Overmyer_SO12.pdf Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles concepts and evidence. Psychological Science in the Public Interest, 9(3), 105–119. Sahin, A., Cavlazoglu, B., & Zeytuncu, Y. E. (2015). Flipping a college calculus course: A case study. Journal of Educational Technology & Society, 18(3), 142–152. Sarawagi, N. (2014). A flipped CS0 classroom: Applying Bloom’s taxonomy to algorithmic thinking. Journal of Computing Sciences in Colleges, 29(6), 21–28. Shirazi, M., & Talebinezhad, M. (2013). Developing intermediate EFL learners’ metaphorical competence through exposure. Theory and Practice in Language Studies, 3(1), 135–141. Simkins, S. P., & Maier, M. H. (Eds.). (2010). Just-in-time teaching: Across the disciplines, across the academy. Virginia, VA: Scott Stylus Publishing, LLC. Snow, R., & Farr, M. (1987). Cognitive-conative-affective processes in aptitude, learning, and instruction: An introduction. Conative and affective process analysis, 3, l-10. Hillsdale, NJ: L. Erlbaum

35

Personalizing Flipped Instruction to Enhance EFL Learners’. . .

905

Strayer, J. (2012). How learning in an inverted classroom influences cooperation, innovation and task orientation. Learning Environments Research, 15, 171–193. Sweeny, S. M. (2010). Writing for the instant messaging and text messaging generation: Using new literacies to support writing instruction. Journal of Adolescent & Adult Literacy, 54, 121–130. Tărcăoanu, M. C. (2012). Teaching and learning idioms in English (theoretical and practical considerations). Scientific Journal of Humanistic Studies, 4(7), 220–228. The United States National Education Technology Plan. (2017). Reimagining the role of technology in education: 2017 National Education Technology Plan Update. Retrieved from http://www. downes.ca/cgi-bin/page.cgi?post=66298 Tucker, B. (2012). The flipped classroom. Education Next, 12(1), 82–83. US Department of Education. (2017). Reimagining the role of technology in education: 2017 National Education Technology Plan update. Retrieved from https://tech.ed.gov/netp/ Wauters, K., Desmet, P., & Van Den Noortgate, W. (2010). Adaptive item-based learning environments based on the item response theory: Possibilities and challenges. Journal of Computer Assisted Learning, 26(6), 549–562. Wu, W. C., Yen, L. L., & Marek, M. (2011). Using online EFL interaction to increase confidence, motivation, and ability. Educational Technology & Society, 14(3), 118–129. Yang, Y. F. (2013). Exploring students’ language awareness through intercultural communication in computer-supported collaborative learning. Educational Technology & Society, 16(2), 325–342. Zappe, S., Leicht, R., Messner, J., Litzinger, T., & Lee, H. W. (2009). Flipping the classroom to explore active learning in a large undergraduate course. In American Society for Engineering Education. American Society for Engineering Education.

Wen-chi Vivian Wu is a Distinguished Professor in the Department of Foreign Languages at Asia University and a consultant in the Department of Medical Research at China Medical University, Taiwan. She has published extensively on CALL and educational technology related-SSCI journals, with research interests focusing on VR/AR, flipped classrooms, PBL, MALL, cross-cultural communication, and robotics learning. Jun Chen Hsieh is an Assistant Professor in the Graduate Institute of Children’s English at National Changhua University of Education, Taiwan. His research interests include flipped learning, technology-enhanced language learning, intercultural telecollaboration, multimodality, and affective factors. Jie Chi Yang is a Distinguished Professor in the Graduate Institute of Network Learning Technology at National Central University, Taiwan. His research interests include digital game-based learning, computer-assisted language learning, and human factors.

Designing for Collaborative Creativity in STEM Education with Computational Media

36

Florence R. Sullivan and Roberto G. Barbosa

Contents Computational Media and Collaborative Creativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Sociocultural View of Creativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Collaborative Dialogic Inquiry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Constructionist Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Collaborative Creativity Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Robotics and Collaborative Creativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scratch and Collaborative Creativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Curricular Considerations and Pedagogical Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Nature of the Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Nature of Social Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Role of the Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

908 908 909 914 917 917 922 924 924 926 927 929 929

Abstract

In this chapter, we present the findings of our research about how middle school STEM students engage in collaborative creativity while working with computational media, including robotics and Scratch. The development of these media was based on Papert’s constructionist learning theory, and as such, they reify particular constructionist ideals. Our research findings are theoretically rooted in a sociocultural definition of creativity as collaborative dialogic inquiry which accounts for the role of influential voices in the classroom (real and reified) in the F. R. Sullivan (*) University of Massachusetts, Amherst, MA, USA e-mail: [email protected]; fl[email protected] R. G. Barbosa Federal University of Paraná – UFPR (Littoral Sector), Matinhos, PR, Brazil Federal University of Paraná – UFPR (Littoral Sector), Curitiba, PR, Brazil e-mail: [email protected]; betofi[email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_80

907

908

F. R. Sullivan and R. G. Barbosa

collaborative creativity of groups. In this way, we expand the notion of the collaborative group beyond the actual members of the group to include the mediating role of the materials, technologies, and teachers on student creativity. In addition to specific research findings, we report on the overarching factors that bear on students’ ability to enact collaborative creativity while problem solving with computational media. These factors revolve around the nature of the activity, the nature of social interaction, and the role of the tools in the environment. Keywords

Collaborative creativity · STEM · Robotics · Scratch · Constructionism · Dialogism

Computational Media and Collaborative Creativity “As a practice, software development is far more creative than algorithmic.” – J. Bradford Hipps (2016).

Our research has focused on the ways in which collaborative learning interactions with computational media (LEGO robotics and Scratch, specifically) enable the development of collaborative creativity in children and youth. In this chapter, we present the findings of our research and we provide recommendations for pedagogical approaches and curricular designs that create the conditions for student collaborative creative activity with computational media. The chapter is organized thusly: we begin with a theoretical explication of our sociocultural view of creativity as collaborative dialogic inquiry; we then present a discussion of Papert’s (1991) theory of constructionism as it relates to student learning and creativity with computational media – the design of both LEGO robotics and the Scratch software is rooted in constructionist principles of learning. Next, we present our research findings based on these theoretical constructs. We then present both pedagogical and curricular recommendations for using computational media in teaching and learning for collaborative creativity.

The Sociocultural View of Creativity From the sociocultural perspective, creativity overlaps strongly with learning in that both are manifested through the coconstruction of new meanings while engaged in collaborative activities (Eteläpelto & Lahti, 2008; Rojas-Drummond, Albarrán, & Littleton, 2008). Sociocultural researchers focus on the situated nature of the learning environment and they view creativity as a potential outcome of collaborative interactions in open-ended, student-directed, learning environments (Barab & Plucker, 2002; Fernández-Cárdenas, 2008). Sociocultural research related to collaborative creativity focuses on the emergence of local practices and the quality of

36

Designing for Collaborative Creativity in STEM Education with. . .

909

social interactions among members of the collaborative group (Rojas-Drummond et al., 2008; Sawyer & DeZutter, 2009). Following the Vygotskyan emphasis on the role of language and tools in the development of higher-order thinking (Vygotsky, 1978), sociocultural research about collaborative creativity focuses on the nature and quality of student talk (Fernández-Cárdenas, 2008; Rojas-Drummond et al., 2008; Sullivan, 2011; Vass, 2007) as well as student’s collaborative interactions related to available tools in the learning environment (Kangas, 2010; Sarmiento & Stahl, 2008). From the sociocultural viewpoint, collaborative group interactions serve as the basis for the generation and development of creative ideas/solutions/projects. Research findings suggest that productive talk, where children build their knowledge collectively, is the basis of collaborative creativity (Rojas-Drummond et al., 2008; Sarmiento & Stahl, 2008; Sullivan, 2011). Productive talk includes sharing, exploring, and reflecting on one another’s ideas (Vass, 2007) as well as elaborating on each other’s ideas (Sarmiento & Stahl, Sullivan). It also includes negotiating meaning (Fernández-Cárdenas, 2008; Rojas-Drummond et al., 2008). In order to establish a productive working relationship, students need to build trust in the group (Aragon, Poon, Monroy-Hernandez, & Aragon, 2009; Eteläpelto & Lahti, 2008; Vass, 2007). Such trust building may come through highly social talk (Aragon et al.) or play and playful talk within the group (Sullivan & Wilson, 2015). Indeed, both play and playful talk have figured prominently in various groups’ creative interactions in a number of these studies (Fernández-Cárdenas, 2008; Kangas, 2010; Sullivan, 2011; Sullivan & Wilson, 2015; Vass, 2007). Productive talk that results in coconstruction of meaning may lead to creative solutions to problems (Sarmiento & Stahl, 2008; Sullivan, 2011), creative design of environments (Kangas, 2010), or creative expression as manifested in a multimedia work (Fernández-Cárdenas, 2008; Rojas-Drummond et al., 2008; Vass, 2007) or a performative work (Sawyer & DeZutter, 2009). This new perspective on studying creativity as an outcome of collaborative interaction has led research efforts in a new and promising direction. However, missing from some of this current sociocultural research is a theoretically based examination of how local practices (e.g., productive talk) emerge on a moment-to-moment basis in a given environment. Our own research addresses this issue. We have developed a thesis of creativity as collaborative dialogic inquiry (Sullivan, 2011). Our thesis focuses on the theoretical basis of collaborative creativity as an outcome of productive talk, as well as the multivocal influence of real and reified voices in the classroom. Our work builds on Wegerif’s (2007) interpretation of Bakhtin’s (1981) theory of dialogism as it functions in technology-based learning environments, with special attention paid to reified voices. We now present our thesis.

Collaborative Dialogic Inquiry The view of creativity presented here builds on the sociocultural approach to understanding learning and the development of higher-ordered thinking. It does so by establishing a theoretical argument for the discursive basis of creativity. Drawing

910

F. R. Sullivan and R. G. Barbosa

on Bakhtin’s (1981, 1986) theory of dialogism and Pea’s (1993) notion of distributed intelligence, we argue that creativity is an intrinsic element of everyday talk and that the development of a creative idea may be traced through the collective interaction with and interpretation of the discourses present in any given activity context. While a specific individual may voice a creative idea, we argue that the creative idea is formed through the interaction of many voices. These voices may be organically embodied, reified in a tool or artifact, or embedded in the environmental and social structure of the context itself. In this way, the collaborative group is comprised of more than the individual members of the group. Indeed, the group is expanded to include the real and reified voices found in the classroom. Let us now consider, in detail, the theories of dialogism and distributed intelligence as they inform our definition of collaborative creativity.

Dialogism Dialogism is Bakhtin’s (1981) theory of language and communication. Like Vygotsky, Bakhtin was interested in developing a theory that reflected a Marxist viewpoint. Such a viewpoint takes into account the historical, social, and cultural factors that influence human endeavors. While Vygotsky investigated human learning, Bakhtin studied language and communication as manifested in literary works. Bakhtin regarded the spoken word itself to be the most meaningful linguistic unit of analysis (Bakhtin, 1981). In line with the Marxist notion of ascending to the concrete (Lave & Wenger, 1991), it is through historical analysis of language in use, as spoken and written, that one is able to understand how new words come into being and how the meaning of a word shifts and changes over time. The notion of ascending to the concrete refers to the idea that to truly understand what concretely “is” one must attend to the historical development of the given phenomenon. It is only through historical analysis that one may come to an understanding of how social forces shape social realities – such as a national language. Two central ideas in Bakhtin’s (1981) theory of dialogism have special relevance here: heteroglossia and ideological becoming. Heteroglossia refers to the dynamic nature of language that is a result of the tension created by the many social languages that make up a national language. Social language refers to the mode of speaking favored by distinct social groups in society. Social groups may share class status (e.g., upper middle class), ethnic or racial background (e.g., Latinx), or sexual orientation (e.g., heterosexual), they may be comprised of people from the same age group (e.g., youth), or who share a passionate interest (e.g., photography). These social groups will share a social language and their use of a national language (including the meaning of words) will be distinct from other social groups’ use of that same national language. Moreover, the term heteroglossia refers to the idea that the context in which an utterance is made governs the meaning of the utterance. In other words, it is only through understanding the social, historical, and cultural conditions under which a specific utterance is made that one will be able to understand the meaning of that utterance. Meaning is derived from context. As contexts are constantly shifting, so are meanings. In Bakhtin’s formulation the meanings of words are not fixed but are

36

Designing for Collaborative Creativity in STEM Education with. . .

911

dependent on the context, which subsumes the socio-ideological position of the speaker of the words and the social situation in which they are spoken. He describes this phenomenon as the centrifugal forces constantly at work in any language. At the same time, there are centripetal forces that constantly seek to fix the meaning of words in order to create a “unitary language.” It is the tension created by these opposing social forces that account for the dialogic nature of language. In other words, dialogism refers to the constant interplay of social forces on the meanings we make of the words we speak. Dialogism also takes into account the temporal nature of communication. According to Bakhtin (1986), every utterance is spoken as part of a chain of utterances that have come before and those that will come after. All communication is historically situated (words one utters contain the meanings of previous speakers of the words) and are responsive to the anticipated understanding and response of the addressee (we choose our words based on what we know of the person or people to whom we are speaking). In this way, each utterance is multivoiced and temporally dynamic – containing the voices of previous speakers and shaped by the anticipated voice of future speakers. As Wegerif (2007) has noted, dialogism is characterized by an openness to the other “. . .by taking the perspective of the other in a dialogue. . .” (p. 52). This happens as a result of considering the addressee and the possible responses of the addressee to one’s own utterances. It is this openness that creates a space for dialogue to occur and new meanings to emerge. While Bakhtin is not a learning theorist per se, his work sheds light on how individual frameworks for knowing and understanding come into being. Bakhtin (1981) refers to this process of framework development as ideological becoming. Ideological becoming is the method by which people appropriate the discourses of others wherein the discourse goes beyond being “information, directions, rules, models and so forth. . .” (p. 342) to become the basis of one’s own ideological development. These appropriated discourses constitute the lens through which experience is filtered. In his discussion of the concept of ideological becoming, Bakhtin has identified two types of social discourse: authoritative discourse and internally persuasive discourse. According to Bakhtin (1981), authoritative discourse emanates from a hierarchical source and demands to be accepted as it is. Authoritative discourse is monological in nature in that it is not open to the perspective of the other. Authoritative discourse is privileged language that purports to present the “truth” which, from a Bakhtinian perspective, is a suspect concept – particularly given the contextual and shifting nature of the meaning of words. Sources of authoritative discourse may be familial, political, educational, or religious. Authoritative discourse is influential in people’s lives. However, when individuals cease to respond to the power of an authoritative discourse, it becomes a “dead” discourse. An example of this may be the dogma of a particular religion that has much influence over an individual’s life; however, if that individual decides she or he no longer believes in that religion, the authoritative discourse of the religion ceases to have power. In Bakhtin’s view, one does not engage an authoritative discourse; one either accepts it or rejects it.

912

F. R. Sullivan and R. G. Barbosa

Internally persuasive discourses, on the other hand, are open discourses. They may be altered, extended, or framed in new contexts. They are discourses that can be creatively developed to take on new meanings. According to Bakhtin (1981), internally persuasive discourse is unfinished and inexhaustible – always open to further dialogic interaction. One assimilates an internally persuasive discourse into one’s existing framework, comparing it, perhaps, with other internally persuasive discourses, and in this way, further develops one’s own framework. Bakhtin (1981) argues that internally persuasive discourses are significant elements in the development of individual consciousness. As we come to understand language, we do so in specific social and cultural contexts. Bakhtin argues of a word uttered in an internally persuasive discourse that: Its creativity and productiveness consist precisely in the fact that such a word awakens new and independent words, that it organizes masses of our words from within, and does not remain in an isolated and static condition. It is not so much interpreted by us as it is further, that is, freely, developed, applied to new material, new conditions; it enters into interanimating relationships with new contexts. More than that, it enters into an intense interaction, a struggle with other internally persuasive discourses. Our ideological development is just such an intense struggle within us for hegemony among various available verbal and ideological points of view, approaches, directions and values. The semantic structure of an internally persuasive discourse is not finite, it is open; in each of the new contexts that dialogize it, this discourse is able to reveal ever newer ways to mean (p. 345–346) emphasis added.

Assimilating and then transforming various internally persuasive discourses that surround one in a given culture is a creative process – giving rise to a specific and unique consciousness: the self. Viewed in this way, creativity is an inherent part of ideological becoming, or the development of consciousness. The conceptualization of creativity as dialogic inquiry emphasizes the dynamic nature of meaning in a living language and its impact on the development of consciousness – conceived of here as learning in one’s culture. Creativity then, is an act of learning, and this act of learning occurs through the active engagement with, and transformation of, internally persuasive discourses.

Voice and Distributed Intelligence A key element in the dialogic theory of language and communication is voice. Dialogism conceptualizes voice as being present in both spoken words and written texts. Kristeva (1986) has coined the term “intertextuality” to describe Bakhtin’s “. . .conception of the ‘literary word’ as an intersection of textual surfaces rather than a point (a fixed meaning), as a dialogue among several writings: that of the writer, the addressee (or the character) and the contemporary or earlier cultural context.” (p. 35–36: emphasis in the original). In other words, as speech occurs in a dynamic, historically situated context of shifting meanings that features an openness to the perception of the other, so written texts are constructed in relation to other, historically situated written texts and responsive to the perceptions of the readers, and

36

Designing for Collaborative Creativity in STEM Education with. . .

913

those who are the subject of the text (fictional or otherwise). This same notion of dialogic voices may be found in the concept of distributed intelligence. The theory of distributed intelligence (Pea, 1993) extends the idea of intertextuality and voice to include material culture. As D’Andrade, (1986) noted “Material culture – tables, chairs, buildings and cities – is the reification of human ideas in solid medium” (p. 22). Cole (1996) further argues that the historical modifications to a material object as it is put to use in human endeavors provides the object with not only a conceptual but also an ideal aspect. It is this historical modification that is analogous to the idea of intertextuality. Humans encounter tools developed in previous eras, these tools carry with them the ideas of previous generations – the tool may be modified based on current needs, but it remains in dialogue with the previous makers, inasmuch as elements of the previous design are retained. In this way, we, living today, are connected culturally to distant prehistoric ancestors who developed “the flint knife and, later. . . the wheel” (p. 136) (Wells, 1999). Therefore, it is possible to come into contact with an internally persuasive discourse not only through oral or textual means but also through ideas reified in a designed artifact itself, such as a new technology. Indeed, it may be argued that the mediating power of a material object is derived in part from the accumulation of knowledge of prior generations inherent in the design of the artifact itself (Cole & Engeström, 1993). As Pea (1993) argues: These tools literally carry intelligence in them, in that they represent some individual’s or some community’s decision that the means thus offered should be reified, made stable, as a quasi-permanent form, for use by others. In terms of cultural history, these tools and the practices of the user community that accompany them are major carriers of patterns of previous reasoning. (p. 53: emphasis in the original).

It is not only designed artifacts that may convey patterns of previous reasoning, but also designed environments. The design of a classroom environment includes both the physical and spatial aspects of material artifacts found in the room – tables, chairs, desks, chalkboards, computers, etc., as well as the design, or structure, of the lived experience – for example, timed periods, the discreet disciplinary treatment of objects of study, the curriculum, the nature of participation structures, specific discourse norms, classroom routines, and school policy. These lived experiences are what Varenne (1998) refers to as “School,” the always already there social and cultural “facts” that meet students when they enter the school building. Some of these School structures are outside the purview of the teacher to regulate, for example, timed periods, the prescribed curriculum, the disciplinary nature of middle and high school STEM classrooms in USA and school policies that affect students. Teachers, generally speaking, do have control over the enacted curriculum including material resource usage, participation structures, specific discourse norms, and classroom routines. While many of the social and cultural facts of School in a public setting may be similar across sites and classrooms, others are specific to the teacher herself, and it is

914

F. R. Sullivan and R. G. Barbosa

this specificity that allows for the teacher’s voice to emerge in the environment. From a dialogic perspective, the teacher’s voice speaks in relation to other teachers, the voices of the principal, the superintendent, the school board, the voices encountered in their own teacher education programs, and historical discourses related to public education in USA. The teacher’s voice in a designed classroom may be thought of as the dialogized ideas the teacher is trying to convey to the students when he or she sets up the classroom. In this way, the designed environment may overtly convey epistemological and pedagogical beliefs, expectations about the nature of teacher/student relations and student/student relations. It may relate ideas about the importance of specific topics or about the types of lived experiences the students may have as members of the class. The design may intentionally or inadvertently convey ideas about the importance of particular curricular objectives or about expectations related to a student’s membership in a particular group (including racial-, gender-, ethnic-, and ability-based perceptions). In other words, the designer of the classroom in terms of both the material culture and the lived experience is speaking directly to the students about his or her priorities as he or she is able to instantiate them within the constraining tones of other, more powerful, voices present in the system. Viewed through the lenses of dialogism and distributed intelligence, all of the voices in the classroom carry a discourse with which students are either directly or indirectly engaged. The oral and written discourses are the most obvious to identify, yet, as has been argued, the ideas reified in objects and conveyed in structured environments are also active agents in a given activity system; agents that carry the intelligence or voice of the designer, steeped as it is in its own socio-ideological perspective and conveying its own internally persuasive discourse to be engaged, altered and/or extended. It is through the engagement with these internally persuasive discourses found in the classroom that creative ideas emerge and new meanings are made. Let us now consider the voices reified in the design of the two computational media we are discussing in this chapter, LEGO Robotics and Scratch. Both of these computer science based-learning technologies have been developed in the Media Lab at the Massachusetts Institute of Technology. Mitchell Resnick and Fred Martin developed the programmable brick used in the LEGO Robotics kit and Mitchell Resnick headed up a team that developed the Scratch program at MIT. The learning theory that underlies the development of these technologies is Papert’s constructionism.

Constructionist Learning Theory Papert (1991), a student of Jean Piaget and an intellectual leader of the MIT Media lab, has stated that the theory of constructionism shares Piaget’s (1981) basic view that human cognitive development and learning consists of “building knowledge structures” (p. 1) through interaction with the natural and designed environment. However, constructionist theory diverges from Piaget’s constructivism in relation to the hierarchy of stages: most notably the third and fourth stages. Constructionists do

36

Designing for Collaborative Creativity in STEM Education with. . .

915

not privilege abstract thinking as the pinnacle of cognitive development. Rather, they argue that high levels of understanding may also be reached through proximal interactions with concrete objects (Resnick, 2004; Turkle & Papert, 1991). It is this belief that underlies the design of computational media (such as Robotics and Scratch) as learning objects that support the development of creativity. While Papert’s theory builds on the work of Piaget, a large portion of the theory is derived from empirical research on student interactions with computational media directed by Papert, his colleagues, and students at the Media Lab. Interestingly, Papert (1991) has discussed how his anecdotal observations of students in an art class at a local high school influenced his understanding of learning and creativity. Each time he passed by the art class, Papert noticed the intense focus and engagement of the students in the class. In speaking with the teacher, Papert found that the students were allowed to pursue their own artistic ideas, while still being guided by the art teacher. Papert set out to create a similarly engaging math learning situation for students. He aimed to create a software program that allowed students to play, explore, and create, all while working with mathematical ideas. Hence, at the very heart of the constructionist approach to learning is creativity. As is widely known, Papert and colleagues at MIT developed the LOGO programming language and the mechanical (and virtual) LOGO Turtle that enacts the LOGO program for children. The LOGO Turtle, a perceptual object, provides immediate feedback to the student on the elements of their LOGO program. As we shall see, the role of the perceptual object (whether physical or virtual) and immediate feedback in the learning environment are central to the theory of constructionism. A perceptual object may be an artifact that children themselves create or it could be an existing objects. Such objects become, in Papert’s term, objects-to-think-with; as such, they facilitate children’s engagement with ideas that they otherwise would not be able to grasp. According to Papert, objects-to-think-with allow children to materially participate in a meaningful way in their own cultural setting, which then provides them with a connection to others in that setting. Furthermore, these perceptual objects allow children to use their own body knowledge in reasoning about concepts. In this way, Papert’s theory of constructionism introduces three important ideas related to creativity and learning: personally meaningful activity, the central role of the social in learning, and the role of one’s own body in sense making. Personally meaningful activity refers to allowing children to follow their own interests and questions in a particular domain of learning. This is a very important aspect of constructionism that directly supports student’s creativity. Amabile (1983) has demonstrated that choice is an important element in creating the conditions that support creativity. When people are allowed to follow their intrinsic interests, through choosing which activity they will undertake, people develop more creative ideas and/or products. One of the reasons for this may be because choice emphasizes the affective aspect of learning and doing. As Kafai and Resnick (1996) have noted, “In constructionist learning, forming new relationships with knowledge is as important as forming new representations of knowledge” (p. 2). In order to become involved in personally meaningful activity, the learning environment must be materially and conceptually rich enough to allow for student exploration and discovery.

916

F. R. Sullivan and R. G. Barbosa

Papert (1993) argues that a computer-based environment provides access to such material and conceptual richness. Social aspects of learning refer to two notions for Papert. The first is that, through using objects in the environment to think with, children materially participate in their own cultural milieu and this helps them connect to adults in their environment who also are in touch with these objects. In a sense, the objects are an aspect of shared material culture that opens paths of communication between children and adults. These paths of communication become paths to learning for students. The second notion is that as children engage in personally meaningful activity, they become excited about this activity and naturally wish to discuss it with others. Hence, in a materially rich classroom (one that has computers, for instance) where open-ended student interaction is encouraged, much topical, social interaction will occur among students and between students and the teacher. This topical, social interaction spurs student learning and creativity. In terms of using one’s body to learn, Papert argues that computational media that use perceptual objects, such as Scratch and LEGO Robotics, offer students the opportunity to consider the movements of the object from their own bodily perspective and, in so doing, are enabled to reflect more deeply about how to program the object. In essence, the child can imagine her body in place of the object and may then reason, from a physical perspective, about how to program it. Indeed, we have observed this behavior in our research with students solving robotics problems (Sullivan, 2008). In our study, students used their hands as proxies for the robotic vehicle as they simulated the proposed movement of the robot and considered how to program that movement. Such simulation allows students to use their own bodies as a creative tool of exploration and reflection. From the constructionist perspective, then, reflection is a key aspect of learning with computational media. The movement of an object-to-think-with, such as a Scratch Sprite or a LEGO Robot, provides immediate feedback to the learner. This feedback functions to stir reflection in the student that leads to the practice of debugging. When students encounter programming results that are inconsistent with their expectations, they are spurred to develop explanations of why this is so, which improves causal reasoning (Legare, Gelman, & Wellman, 2010). In essence, computational devices have the potential to spur metacognitive or reflective thinking, diagnostic reasoning, conditional reasoning, and problem solving as part and parcel of the activity itself. In terms of computational media, then, the reified voices in the robotics and Scratch technologies that work to support and enable student creativity speak to the following ideas: (1) creativity is supported through engagement in personally meaningful activity; (2) creativity is supported through collaborative dialogue spurred by the perceptual nature of the media; (3) creativity is supported through embodied simulation; (4) creativity is supported through the reflection, explanation, and problem-solving activity that is promulgated by the immediate feedback mechanism of computational media. It is this intellectual activity, based on interaction with the perceptual object, that leads to student understanding of higher-level concepts and to creative expression.

36

Designing for Collaborative Creativity in STEM Education with. . .

917

Collaborative Creativity Research Having presented our thesis on creativity as collaborative dialogic inquiry, and discussed the theoretical basis for the development of constructionist computational media, we now discuss the results of two prior research studies we have conducted on the development of collaborative creativity with these computational media (Sullivan, 2011; Sullivan, Hamilton, & Foley, 2012). These prior studies illustrate how collaborative creativity is enabled through curricula that use computational media. We begin with a case study of the collaborative development of a creative solution in a robotics context. We then present a case study focused on collaborative creativity in a Scratch learning environment.

Robotics and Collaborative Creativity As previously noted, the LEGO Mindstorms robotics kit is equipped with Resnick, Martin, Sargent, and Silverman’s (1996) programmable brick, a small microcomputer, encased in plastic featuring a visual display, power and menu selection buttons, and numerous ports for connecting sensors and motors with cables. The microcomputer is about the size of two decks of cards resting horizontally upon one another, so it fits easily in older children’s hands. Generally speaking, the first activity children take part in when working on a robotics project is to build up the microcomputer into some sort of robotic device using the motors, the sensors and the LEGO pieces that come with the kit. Typically, a vehicular robotic device is assembled, though many other types of robotic structures could be created, for example, vending machines, automatic doors, or robotic arms. In my case study on collaborative creativity (Sullivan, 2011), I followed three focal students (Esteban, Janice, and Yolanda (pseudonyms)), as they worked together to develop a creative solution to a programming challenge that required the use of a light sensor to solve. The study took place in a 6th grade science classroom in a small, economically depressed, city in Western Massachusetts. The study was conducted over a 2-week period. Robotics challenges were used to introduce students to physics-based elements of a light and heat energy unit. The curriculum for this study was cocreated by the first author, a physics professor and four middle school science teachers. The three focal students who took part in the study were all of Latina/o descent and each was 12 years old at the time of the study. The school they attended served students who demonstrated need of a high level of support from the school. More than 80% of the students received free lunch (an indicator of low socio-economic status in USA), and 38% of the students in this school were English language learners. Moreover, the school was, at that time, designated a level four school by the state of Massachusetts, indicating that the students performed at a failing level on the state’s standardized tests. My research with this group of students proceeded from my theoretical view of creativity as collaborative dialogic inquiry, which is to say, I collected and analyzed student interactions with the spoken and reified voices in the classroom. Data collection consisted of audio and videotaping all of the focal group’s collaborative

918

F. R. Sullivan and R. G. Barbosa

interactions, their interactions with the teacher and with other students in the classroom. I also interviewed the teacher at the end of the first and the second week of the robotics unit. The challenge given to the students in this case study was to program a robotic vehicle to perform the following tasks: • • • •

Move forward until a black line (that reflects less light) is detected Make a 90-degree turn, then Back up slowly for one foot, and Repeat forever

The goal of this particular unit was for students to develop their knowledge of how the light sensor worked. A light sensor is a computerized device that functions to trigger an event, given certain lighting conditions. The challenge was for the students to program the robot to respond when a black line was detected. The triggering source for this challenge was pieces of black paper provided by the teacher. These pieces of black paper were set on a grey carpet in the front of the classroom. The focal group initially approached the problem by taking light readings of the surface of the black paper only; they did not take a light reading of the grey carpet surrounding the black paper. At this early juncture, the students were using the light sensor not as a computational device, but as a measurement device, akin to a ruler or a measuring cup. Based on their understanding of the relationship of the color spectrum to the reflection and absorption of light, the students expected the black pieces of paper to absorb more light than the grey carpet. However, each of the pieces of black paper provided by the teacher reflected different amounts of light due to the texture of the paper, and in two out of three cases, the black pieces of paper reflected more light than the grey carpet. This was a confounding variable which complicated the problem for the students. Over time, and through the process of collaborative dialogic inquiry, the students discovered that the reason they were having trouble solving the problem was because the grey classroom carpet was reflecting less light than the black construction paper meant to trigger the light sensor, hence their light sensing program was not working. This discovery is what Koschmann and Zemel (2009) call an occasioned production. It is a discovery that the students did not know they needed to make prior to the moment they made it. This occasioned production occurred as a result of their problem-solving effort the feedback they were getting from the various trials they were performing, their own discussions of the problem, and their interactions with the teacher; it represents their changing understanding of what the problem actually was. Indeed, once the students understood the variable nature of reflected light readings and the important role of the grey carpet in their calculations, they enacted a creative solution to the problem. This creative solution was to repurpose the black cables provided in the robotics kit to serve as the black source that would trigger the rest of the light sensor program. The students’ arrival at a creative solution was supported by their deepening understanding of the problem represented by three key, interrelated realizations. First, they came to understand that they needed a light reading of the approach surface (grey

36

Designing for Collaborative Creativity in STEM Education with. . .

919

carpet) as well as the target surface (black paper), and that the function of the light sensor was not only to take a reading of the amount of light reflected off of various surfaces (sensor as measurement tool), but also to discern between the two light readings (sensor as computational device). Second they realized that not all similarly hued entities reflect the same amount of light. For example, the laminated black paper reflected more light than the black construction paper, due to the texture of the laminating plastic. And third, they reframed the given problem as one in which the light sensor is programmed to simply react to a black (or darker) surface, to an understanding that any number of environmental variables may serve to confound the process (Sullivan, 2008); and, therefore, these variables need to be identified and taken into account. This reframing of the problem is part and parcel of the creative activity engaged in by the students, it reflects the students deepening understanding of the complex nature of the problem and, as the students were better able to identify the variables bearing on the problem, so were they able to develop a creative solution. Dorst and Cross (2001) refer to this as the coevolution of the problem and the solution, which is a typical creative activity in design situations. The better one understands the problem, the more likely one is to develop a creative solution to the problem. Let us now consider how the voices (real and reified) in the classroom contributed to the students’ creative solution.

Classroom-Based Internally Persuasive Discourse As previously discussed, the sociocultural approach to understanding creativity focuses, in part, on the discursive interactions among people. (Rojas-Drummond et al., 2008) in their research on collaborative creativity in the language classroom have developed a good explanation of how idea generation works in a group: Among the common acts present in all the [discourse] data were: joint planning; taking turns; asking for and providing opinions; sharing, chaining and integrating of ideas; arguing their points of view; negotiating and coordinating perspectives; adding, revising, reformulating and elaborating on the information under discussion and seeking of agreements. These data, taken together, suggest that the children engaged in diverse processes of ‘co-construction’ of meaning and knowledge to achieve their goals. . . (p. 186).

The evolution of a creative idea, then, rests in part on an interactive process in which students draw on their own prior knowledge (internally persuasive discourses) and experience to generate ideas and contribute them to the group; these ideas may then be acted upon by others in a number of ways including evaluation, elaboration, clarification, or refutation. In keeping with our theory of creativity as collaborative dialogic inquiry, we argue that in addition to idea generation in collaborative group interactions, ideas are also generated from interaction with other voices that are present in the environment itself. For example, in this case study, we found that in developing a creative solution to the robotics challenge, our focal group of students relied on ideas generated from three different sets of voices in addition to their own, including: (a) the teacher’s voice, (b) the curriculum designers’ voices (as reified in the designed activities), and (c) the technology designers’ voices (as reified in the designed robotics technology).

920

F. R. Sullivan and R. G. Barbosa

The teacher’s voice was present in the classroom not only through the words he spoke but also through the spatial arrangements of the room. Indeed, the classroom environment was designed by the teacher to support a student-centered, projectbased, inquiry approach to learning. The student desks were arranged in clusters of four. Students sat facing one another during class time. In this arrangement, the students were oriented to one another, as opposed to all students facing forward, oriented towards the teacher, as is commonly found in more traditional seating arrangements. The classroom north wall featured student posters reporting on a recently completed inquiry project devoted to making functional model boats from various materials. There were two tables set against this wall that displayed the student completed model boats themselves. The internally persuasive discourse embedded in the classroom environment as designed by the teacher reflects the teacher’s epistemological belief in a collaborative, student-center, project-based, inquiry approach to the teaching and learning of science. This interpretation is validated by the teacher’s own comments. In response to an interview question about his teaching style, he states: [My approach is] more [of an] inquiry-based situation where I challenge the kids with a question or problem, and then they have to attack it by some sort of process that they devise based on what we're learning. I pretty much give them full freedom because I think that they have the ability to move about the classroom and do what they’re supposed to do, like the light scavenger hunt today. They were able to do that. They were then able to answer the questions about what we had in the review. I really think that the kids learn better that way, because they take the control of it for themselves.

Hence, students’ ideas are influenced by the physical and spatial arrangements of the class itself in that they are allowed to pursue their own lines of inquiry. They are free to speak to one another and to move about the classroom. In essence, this teacher provides conditions for “freedom of intelligence,” which Dewey (1938/1997) notes is the most important freedom we have. This freedom sends a message to the children. The message is “I trust you” and “I believe you can learn from interacting with one another.” The teacher’s voice thus reified has a strong impact on student idea generation. Students were free to think, explore, and create. Likewise, the voice of the curriculum designers is reified in the content and the activities of the robotics unit. The curriculum for this study was developed by four middle school science teachers from the local area. The curriculum consisted of eight activities. The first two activities were prescribed, utilizing a hints worksheet for identifying and labeling the pieces and blueprint instructions for building the robotic vehicle. The remaining activities were open-ended; students were given minimal instruction as regards the completion of the remaining activities, they were expected to develop their own approaches to solving the particular robotics challenges utilizing the resources available to them including the written challenge itself, a ruler, the robotics vehicle, the touch, light, and temperature sensors, the Robolab programming software, members of the small group, other classmates, and the teacher. The curriculum designers’ voices are reflected in the choice of openended, inquiry-based activities to be undertaken by the small groups. In this

36

Designing for Collaborative Creativity in STEM Education with. . .

921

formulation, students direct their own group activity in solving the robotics challenges. They are free to move about the room to test their programs, free to discuss their programs with other students beyond their small groups and free to call on the teacher if they reach an impasse. Here, the curriculum designers’ voices reinforce the voice of the teacher, which sends a message of openness and trust to the students in a spirit of collaborative inquiry. Meanwhile, the technology designers’ voices were reified in the robotics technology devices themselves. As noted earlier in this chapter, the Mindstorms kit was created based on constructionist learning principles. At the heart of the Mindstorms kit is the Programmable Brick. In discussing the design of the Brick, Resnick et al. (1996) have this to say: The Programmable Brick makes possible a wide range of new design activities for children, encouraging children to see themselves as designers and inventors. At the same time, we believe that these activities could fundamentally change how children think about (and relate to) computers and computational ideas. . .The Programmable Brick gives users the power to create and control. . .The Programmable Brick is explicitly programmable so that users can continually modify and customize its behavior. In this way, the Programmable Brick fits clearly within a constructionist approach to learning (p. 443–444).

In this case study, I argue there is an alignment among the teacher, the curriculum designers, and the technology designers’ voices. The technology designers have created a technology that gives children “power to create and to control,” it encourages children to see themselves as “designers and inventors.” The technology design itself positions children affirmatively as regards their role in their own learning and creativity, just as the teacher’s arrangement of the classroom and the curriculum designers’ selection of open-ended assignments. Taken together these “voices” create an environment where students are empowered to generate ideas and pursue creative solutions to the robotics challenges. Indeed, I found the interaction of these voices contributed to the emergence of three critical aspects of the enacted curriculum, which influenced the development of the key understandings and the creative solution pursued by the focal group, including: (1) an open-ended, goal-oriented task, (2) teacher modeling of inquiry techniques, and (3) provision of tools and an environment that allowed students to move between dual modes of interaction: seriousness and play.

Curricular and Pedagogical Design Implications In this case study, students were working with a robotics challenge in an open-ended, goal-oriented way. Their activity was constrained by the parameters of the challenge, and therefore structured, but they were given much freedom in pursuing their solution. Furthermore, the teacher modeled modes of inquiry, which included investigation and reasoning (close examination of the functioning of the robot in a neutral setting), as well as playfulness and bricolage (demonstrated in an episode where the teacher used the tip of his black shoe to trigger the students’ light sensor program). Levi-Strauss (1966) defined bricolage as the repurposing of items that are “ready-tohand” in the environment. The students’ creative idea of repurposing the black

922

F. R. Sullivan and R. G. Barbosa

cables was an act of bricolage that reflected the teacher’s use of the tip of his black shoe to trigger the program. While playfulness and bricolage are not generally considered modes of inquiry in science, they may well be important modes of inquiry as regards the development of creative ideas. If this is so, it points to the importance of providing tools and an environment that allows students to move between dual modes of interaction: seriousness and play. In this class, the students had a serious purpose, which was to solve the light sensor challenge, but they were allowed to move between modes of seriousness and modes of play by the teacher. Indeed, in 15 years of research into student learning with robotics, I have consistently observed that middle school students approach robotics as they would approach a toy; they always begin by playing with the robot. Play is also at the heart of the collaborative creativity displayed by two boys featured in the second case study, to which we now turn.

Scratch and Collaborative Creativity Scratch is computational media that allows students to create interactive animations, music videos, video games, and other media genres. Scratch is a constructionist technology that promotes an iterative design cycle that includes imagining, playing, creating, sharing, and reflecting (Resnick et al., 2009). Scratch creations can be shared in the online Scratch community space and others can view the program or run it online. Scratch is an excellent environment for teaching young learners how to program (Lee, 2011). Recent research on student engagement with Scratch emphasizes the motivating role of youth popular culture as the content base of activity (Peppler & Kafai, 2007). We developed an after school club that featured Scratch, engagement with the Scratch web site, and student-directed animation development, thereby providing students the opportunity to pursue their own popular culture genre interests while learning how to program with Scratch (Sullivan et al., 2012). As will be demonstrated below, we found that students’ collaborative creativity was stimulated by the shared genre interest of video gaming and jointly playing a video game that someone else had developed with Scratch. The middle school students in our study were enrolled in an inner city K – 8 school serving predominantly Latino families with low socio-economic power. A high proportion (38%) of the students enrolled in the school where we conducted our study identified as limited English proficient students. Fifteen students participated in our Scratch Club; however, our data analysis focused only on the core eight students who maintained consistent attendance patterns across most sessions. Of these eight students, three were female and all but one identified themselves as Hispanic or Latino. The Scratch Club met for approximately 90 min twice a week over a 16-week term. A similar schedule was followed across all sessions: 15 min of community sharing, 20–30 min of a mini-lesson on a particular computer concept, 30–40 min of independent (or collaborative) work, and 10–15 min of project sharing and reflection. The community sharing time was designed to foster social collaboration

36

Designing for Collaborative Creativity in STEM Education with. . .

923

between students. The mini-lessons were based on ideas generated from the students’ project work. During independent work time, students utilized the online Scratch community website to view, play, and remix downloaded animations created by others, they worked with in-class resource files such as the MIT-provided Scratch cards, engaged in discussions with instructors and classmates, and designed and programmed their independent projects. Project share time included “gallery walks” in which students viewed and discussed each others’ projects. While a number of measures were used to collect data in this study, the most salient for our discussion here are the participant observer field notes we collected. We collected notes for 23 of the 31 sessions (notes were not collected for four special events and the four sessions where the primary activity of the students was completion of the other measures). Data collection and analysis utilized methods described by Bogdan and Biklen (2007). Notes were compiled independently at the end of each session. These notes include recollection of specific utterances made by participants. Our data analysis indicated that student social interactions as related to the technology manifested itself in two forms: (1) as interaction revolving around shared genre interests and (2) as a desire to share one’s accomplishments with others. Furthermore, the first type of interaction appeared to have been influenced by the classroom seating arrangement while the latter did not. As previously noted, the Scratch program allows individuals to create interactive animations, music videos, video games, and other genres. We found that each student showed a preference for a particular genre (Sullivan et al., 2012). For example, four students were interested in downloading, playing, remixing, and creating video games; two students were primarily interested in downloading, remixing, and creating music videos; and two other students were interested in cartoons and/or animated, interactive stories. At times, these shared genre interests became a focal point for student interaction, learning and creative activity. For example, two boys, Mauricio and Raul (pseudonyms) whose primary interests were video games engaged in much discussion and collaborative problem solving related to remixing video games that they found on the Scratch website. These boys who were seated next to one another in the computer lab, downloaded, played, and remixed a chase and shoot video game. Over the 32 sessions of the after school session, Raul and Mauricio devoted approximately 10 sessions to just playing video games. While, initially, this seemed like a waste of time to us, we eventually realized that through repeatedly playing the chase and shoot video game, the boys were informally studying how the game functioned. Later in the term, when one of the boys wished to create a classic arcade video game in which the characters chase and eat yellow dots, he received help from his friend in thinking about how to program the functionality of the game by reference to their prior shared experience with the chase and shoot game. Below is an excerpt from our field notes in relation this exchange: While I was working with Raul, Mauricio began asking me how he could make his maze game into a chase and eat video game. He could make one of his characters move around the maze with the arrow keys, but he couldn’t figure out how to make the other character chase

924

F. R. Sullivan and R. G. Barbosa

the first one. I suggested that he think about how one would track someone in real life, I said “You would have visual cues to where they were, right? There would be some way to trace the person.” I then asked him what happens in the well-known chase and eat video game, he said the character ate up the yellow dots. I asked him to think about what kind of a trace the first character could leave that would allow the second character to chase it. Raul then suggested that Mauricio look at the chase and shoot video game they had played in prior weeks because the characters chase each other in that game. Mauricio opened that game and selected a zombie character and Raul said “No, pick a soldier ‘cause they move.” So, Mauricio clicked on a soldier sprite (Sullivan, field notes 3-17-11).

In this example, Raul scaffolded Mauricio’s development of a chase and eat type video game he was creating by suggesting that Mauricio study an example provided in the chase and shoot video game. Here, we can see there are several voices coming together to influence the creative development of Mauricio’s chase and eat type game. First, and perhaps, foremost, there is the voice of Raul, pointing out the resource that Mauricio could use, second, there is the voice of the designer of the chase and shoot video game, reified in the design of the game itself. Third, there is the voice of the Scratch technology designers who not only created the interactive program but also created an easy way for Scratch developers to share their work and learn from one another. In many ways, the online Scratch community resembles an open source software community, where people work collectively to create the tools they want and need. Finally, there is the voice of the designers of the after school club who provided students the freedom to study and make what they wanted, and the freedom to move about the room and work with any other student in the room. This type of collaborative interaction was also facilitated by the students’ shared interest in the video games genre and their shared history of playing the chase and shoot video game.

Curricular Considerations and Pedagogical Approaches In considering the design characteristics of curricula and pedagogy that support the development of collaborative creativity, our research, and the research of colleagues, has led us to consider three main aspects: the nature of the activity as it unfolds in the classroom (enacted curriculum), the nature of social interactions as enabled through the pedagogical approach (including interactions between the teacher and the students, and among the students themselves), and the role of the tools in the activity.

The Nature of the Activity To support collaborative creativity in students in STEM learning environments, teachers must attend to the nature of the activity. Three types of activities lend themselves well to the enablement of creativity including design activities, problemsolving activities, and project-based, inquiry learning types of activities. From a pedagogical standpoint, these three activities overlap in various ways. As far as

36

Designing for Collaborative Creativity in STEM Education with. . .

925

creativity is concerned, the structural overlap that is most significant is the openended nature of each of these activities. Open-endedness refers to the notion that there is more than one correct approach to engaging in the activity and there is more than one acceptable outcome for the activity. Let us briefly consider each type of activity.

Design Activities Design activities are those in which the students are actively designing a functioning device (in the case of robotics) or a functioning program (in the case of Scratch). One of the main creative elements of design is problem setting (Schön, 1983). Problem setting is the act of naming and framing a problem to be solved through the design of a device or a program. While engaged with this activity of problem setting, students develop a deeper understanding of a problem through research, and in so doing, they begin to develop creative solutions to the problem. As noted earlier, Dorst and Cross (2001) refer to this as the coevolution of the problem and the solution. Developing a design solution to a problem is aided by specific creative techniques such as combination, mutation, analogy, and first principles (Cross, 2004). Combination refers to creating a new design through combining elements of existing designs, while mutation refers to changing one or more aspects of an existing design. Analogy refers to creating a design by making an analogy between an existing design that solves a particular problem to a new design that solves a structurally similar problem. Finally, first principle regards designing from the characteristics of the activity for which one is designing. For example, if one is designing a table, then one characteristic of the design will be a flat surface upon which things may be lain. These techniques, used in combination with a deepening understanding of the problem, will lead to creative activity on the part of student designers. Problem-Solving Activity Problem-solving activity can be approached creatively, especially in terms of idea generation and the invention of strategies. Indeed, as noted above in our research on robotics, students first developed a new understanding of the problem through creating a deeper understanding of the functioning of the light sensor (problem reframing), they then generated various ideas for solving the problem, one of which was to re-purpose the black cables in the Mindstorms box to serve as the source of black in their challenge solution. Moreover, in a prior study of problem solving with robotics, we found students developed their own heuristic methods for understanding and solving the problem. One such method included using ones’ body to simulate the movement of the robot (Sullivan, 2008). Such a strategy, while hypothesized by constructionist theorists, is still an instance of the development of an invented strategy when students develop the idea through interaction with the device. To enable creativity in a problem-solving activity, it is important the problem be ill-structured (Jonassen, 2000). Ill-structured problems have specific characteristics that allow for student creativity to unfold. These characteristics include the partial nature of the initial problem state – not all of the pertinent information that bears on a

926

F. R. Sullivan and R. G. Barbosa

problem is known; also, in ill-structured problems, there are many routes to solutions, and there are many acceptable “correct” answers. When working with ill-structured problems, students must work to define the problem (problem setting) and they must adopt a discovery orientation to the work (Dilon, 2003). Indeed, Sawyer and DeZutter (2009) have argued that collaborative creativity is supported when the outcome of a groups’ work is unpredictable, rather than scripted. In other words, the results of a groups’ work may be new to the teacher, as well as to the students themselves.

Project-Based, Inquiry Learning A third pedagogical approach that features open-endedness and supports creativity is project-based, inquiry learning. The roots of this approach lie in the progressive education movement of the early twentieth-century championed by William Heard Kilpatrick (1918) and pragmatist philosopher, John Dewey (1938/1997). The focus in a project-based, inquiry learning approach is on a driving question, as it is developed by the students themselves (Krajcik & Blumenfeld, 2006). Pedagogically speaking, allowing students to develop and pursue their own interests improves student motivation to learn in the activity. It supports “freedom of intelligence” which Dewey defines as consisting of “. . .freedom of observation and of judgment exercised in behalf of purposes that are intrinsically worthwhile” (p. 61). The driving question grounds the learning activity in the lived experience of the children; this grounding allows children to bring their prior experience and knowledge to bear on the project. Often, a project-based, inquiry learning activity will result in the creation of an artifact. Developing such an artifact requires generative thinking on the part of the students and aids in the development of their creativity, particularly as it is related to both idea generation and problem framing. These three activities, designing, problem solving, and project-based, inquiry learning are very well suited to learning in the STEM disciplines. And, through the open-ended nature of the activity, students are able to engage in a number of practices that enable the development of creativity including: problem setting, using design techniques such as combination, mutation, analogy, and first principles, idea generation, invented strategies, and utilizing their freedom of intelligence to create artifacts that are personally meaningfully. All of these elements support students’ free expression of ideas, autonomy, and initiative.

The Nature of Social Interactions Another element a teacher must attend to when creating the conditions for student collaborative creativity is the nature of social interactions. Here there are two main aspects, first, the role of play and playful talk in a collaborative group context and, second, there is the importance of creating a less hierarchical atmosphere in the classroom, which includes devolving authority from the teacher to the students and supporting student-student interactions. Let us now turn our attention to consideration of the role of social interactions in the enablement of creativity.

36

Designing for Collaborative Creativity in STEM Education with. . .

927

Play and Playful Talk Play and playful talk within the group are integral aspects of the enablement of collaborative creativity. For example, prior research has shown that creativity in collaborative groups is supported when the group itself grounds their interactions in playfulness (Kangas, 2010; Vass, 2007). Play and playful talk are features of the affective climate of the group that have an effect on learning and creativity within the group. Indeed, Vass (2007) has argued that it is the emotional content of play that can serve as a generator of ideas within a collaborative group. Meanwhile, Wegerif (2007) has shown that the notion of playful talk includes verbally “playing” with ideas that may then become the basis of a design, solution, or project element. Moreover, Aragon, et al., (2009) found that the online socio-emotional discussions engaged in by collaborators was a key aspect of two different groups’ abilities to work creatively together. Finally, a collaborative group that engages in a high amount of playful talk may well be using that genre to accomplish important affective work within the group. For example, in our own research (Sullivan & Wilson, 2015), we found that a group of students solving robotics challenges used the playful talk genre as a means of regulating the functioning of the group vis-à-vis creating opportunities to learn for all of the members of the group. Non-hierarchical Teacher-Student Relations A second, vital, aspect of the role of social interactions in supporting collaborative creativity is the establishment of a nonevaluative environment for the exploration of ideas. As such, the role of the teacher must, necessarily, become less authoritative. Social cognitive research has clearly established the negative impact of evaluation on student creativity (Amabile, 1983; Hennessey, 1995). While this proposes a problem for teachers who must assess student learning, it is not an insurmountable one. For example, it is possible to work collaboratively with students, discussing their ideas with them, providing suggestions where appropriate, and generally encouraging students to develop their ideas. Evaluation and assessment may then occur at the end of the unit. In this way, the teacher enables freedom for students to try out new ideas and to make mistakes while learning. We may think of these actions as being very close to an authentic mode of learning, coming to know and transforming reality through interaction and experience; a practice that is more spontaneous and intuitive and less reproductive or reactionary. Beyond that, the teacher should provide minimal instruction regarding the tools and how they are appropriated to the creative action of the students.

Role of the Tools Bruner (1973) has argued that there are three representational modes that support the development of higher order thinking in individuals, the enactive (physical interaction), the iconic (perceptual interaction), and the symbolic (linguistic or other sign system interaction). Moreover, while the developing child passes, over time, through a progressive and sequential process of learning to reason from the enactive, to the iconic, and finally the symbolic mode, an individual continues to use these three representational modes for learning interchangeably throughout a life time.

928

F. R. Sullivan and R. G. Barbosa

When children work with robotics they are working at all three levels of representation, which increases the pathways to understanding and meaning making available to the child. As we have discussed at length elsewhere (Sullivan & Heffernan, 2016) robotics devices are computational manipulatives. As such, they provide students the opportunity to both physically and virtually interact with the tools. Traditional manipulatives are concrete referents to abstract concepts, for example, using Cuisenaire rods to develop knowledge of mathematical concepts and relationships (Manches & O’Malley, 2012). According to Fishkin (2004), interaction with manipulatives engenders analogical reasoning and embodied cognition. When children use traditional manipulatives, they are often working at the enactive and iconic representational levels. Computational manipulatives add the symbolic mode to the representational level students are working at; in robotics, students work with a physical device that they can see and manipulate, and they work with one of several programming languages to program a robotics device. While the physical robotics tools serve the traditional purpose of enabling analogical reasoning and embodied cognition, the computational aspect results in a feedback loop that engenders a number of cognitive processes associated with the habits of mind of scientifically literate people including but not limited to reflection, evaluation, hypothesizing, experimenting, measurement, collaborative discussion, and the development of specific, context-based heuristics for problem solving (Papert, 1993; Sullivan, 2008, 2011). In terms of collaborative creativity, the enactive representational mode has an important role to play. As we found in our prior research, playing with the physical robot serves two important functions. First, it allows students to imagine the robotic device in terms of their own life world. For example, in much of our work with students and robotics, the first thing that happens is students make analogies to objects or entities in their own life, “it looks like a race car,” “it looks like a dog,” “look, I can make it into a purse.” These comments help the students identify with the materials and the activity of robotics. We have found that identification with the activity is an important aspect of creating opportunities to learn with the materials, and that is especially true for girls (Sullivan & Wilson, 2015). In addition to the identification function, the device and the robotics materials, in general, lend themselves to bricolage. As noted above, bricolage is a means of repurposing materials in one’s environment to fit one’s specific needs. Students play with the Lego pieces and the other elements of the robotics kits to help them solve the robotics problems in creative ways (Sullivan, 2011). The bricolage function is very much related to the notion of tinkering and experimenting with materials. While computational media have an important role to play in STEM education, especially when the goal is to promote collaborative creativity, it is important to bear in mind that the tools may also be used in noncreative ways. In other words, just because the tool lends itself to creativity, does not mean everyone will pick up the tool in a creative way. For example, in many cases, new computational technologies may simply be used to reinforce the didactic, transmission-oriented ways of teaching and learning. In order to inspire collaborative creativity, it is important to not only provide the tools but also the social context that supports creativity.

36

Designing for Collaborative Creativity in STEM Education with. . .

929

In our view, it is necessary to adopt tools that afford more opportunities for students to collaboratively create in the STEM teaching context. In our research, we utilized two distinct computational tools: LEGO Mindstorms robotics kits and the Scratch program; both tools have the same goal, to give students the power to create and control things in the physical and virtual world, respectively. Resnick (2008), the creator of these computational media, has designed them to be, above all, easily handled by children and young students. He highlights that the main characteristics of the tools are to support creative thinking, arguing that these tools should open space for exploration, with low thresholds for entry, yet, high ceilings for participation (Resnick & Silverman, 2005). Moreover, these tools allow many paths for many styles, supporting collaboration and the open interchange of ideas (Resnick & Silverman). The role of the tool, then, is twofold, both cognitive and affective. It is both a powerful object-to-think-with operating on all three levels of representation, and it is a playful device inspiring both the identification and bricolage functions in children who are allowed to play and learn with them.

Conclusion From our perspective, collaborative creativity is the wave of the future. Interdisciplinary teams, working over networks to creatively solve the urgent social and scientific problems of today are now and will continue to be the norm. For example, teams that work on environmental issues where many factors need to be taken into consideration may include biologists, geologists, atmospheric scientists, computer scientists, economists, and statisticians. Whereas teams that work on social problems may include doctors, nurses, public health officials, law enforcement, and elected officials. These interdisciplinary teams may use the internet, shared documents in the cloud, mobile phones, and other networked communications to plan and manage their collective work. In our research, we seek to understand the pedagogical and curricular factors that best support students’ ability to engage in collaborative creativity with one another, aided by robust computational media. For, it is not only the robust tools that matter but the conditions of their use. To truly support collaborative creativity, teachers and curriculum designers must create classroom conditions that support freedom of intelligence, choice, play, dialogic inquiry, identification, and bricolage. Here we have provided some insight, gained from our research studies, as to how such conditions may be established in the classroom.

References Amabile, T. M. (1983). The social psychology of creativity: A componential conceptualization. Journal of Personality and Social Psychology, 45(2), 357–376. Aragon, C., Poon, S., Monroy-Hernandez, A., & Aragon, D. (2009). A tale of two online communities: Fostering collaboration and creativity in scientists and children. In Proceedings of the creativity and cognition conference, Berkeley, CA. New York, NY: ACM Press.

930

F. R. Sullivan and R. G. Barbosa

Bakhtin, M. M. (1981/1930’s). The dialogic imagination (C. Emerson & M. Holquist, Trans.). Austin, TX: University of Texas Press. Bakhtin, M. M. (1986). The problem of speech genres (V. W. McGee, Trans.). In C. Emerson & M. Holquist (Eds.), Speech genres and other late essays (pp. 60–102). Austin, TX: University of Texas Press. Barab, S. A., & Plucker, J. A. (2002). Smart people or smart contexts? Cognition, ability, and talent development in an age of situated approaches to knowing and learning. Educational Psychologist, 37(3), 165–182. Bogdan, R., & Biklen, S. (2007). Qualitative research for education: An introduction to theory and practice. Boston, MA: Allyn and Bacon. Bruner, J. S. (1973). The growth of representational processes in childhood. In J. M. Anglin (Ed.), Jerome S. Bruner: Beyond the information given: Studies in the psychology of knowing (pp. 311–323). New York, NY: Norton & Co. Cole, M. (1996). Cultural psychology: A once and future discipline. Cambridge, MA: Harvard University Press. Cole, M., & Engeström, Y. (1993). A cultural-historical approach to distributed cognition. In G. Salomon (Ed.), Distributed cognitions: Psychological and educational considerations (pp. 1–46). New York, NY: Cambridge University Press. Cross, N. (2004). Expertise in design: An overview. Design Studies, 25(5), 427–441. D’Andrade, R. (1986). Three scientific world views and the covering law model. In D. W. Fiske & R. A. Shweder (Eds.), Metatheory in social science (pp. 19–41). London, UK: University of Chicago Press. Dewey, J. (1938/1997). Experience and education. New York, NY: Collier Press. Dilon, T. (2003). Collaborating and creating on music technologies. International Journal of Educational Research, 39, 893–897. Dorst, K., & Cross, N. (2001). Creativity in the design process: Co-evolution of problem–solution. Design Studies, 22(5), 425–437. Eteläpelto, A., & Lahti, J. (2008). The resources and obstacles of creative collaboration in a longterm learning community. Thinking Skills and Creativity, 3(3), 226–240. Fernández-Cárdenas, J. M. (2008). The situated aspect of creativity in communicative events: How do children design web pages together? Thinking Skills and Creativity, 3(3), 203–216. Fishkin, K. P. (2004). A taxonomy for and analysis of tangible interfaces. Personal and Ubiquitous Computing, 8, 347–358. Hennessey, B. A. (1995). Social, environmental, and developmental issues and creativity. Educational Psychology Review, 7, 163–183. Hipps, J. B. (2016, May 21). To write better code, read Virginia Woolf. New York Times. Retrieved from http://www.nytimes.com/2016/05/22/opinion/sunday/to-write-software-read-novels.html?_r=0. Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 48(4), 63–85. Kafai, Y., & Resnick, M. (1996). Introduction. In Y. Kafai & M. Resnick (Eds.), Constructionism in practice (pp. 1–8). Mahwah, NJ: Lawrence Erlbaum Associates. Kangas, M. (2010). Creative and playful learning: Learning through game co-creation and games in a playful learning environment. Thinking Skills and Creativity, 5(1), 1–15. Kilpatrick, W. H. (1918). The project method. The Teachers College Record, 19(4), 319–335. Koschmann, T., & Zemel, A. (2009). Optical pulsars and black arrows: Discoveries as occasioned productions. Journal of the Learning Sciences, 18(2), 200–246. Krajcik, J. S., & Blumenfeld, P. (2006). Project based learning. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 317–333). New York, NY: Cambridge University Press. Kristeva, J. (1986). Word, dialogue and novel. In T. Moi (Ed.), The Kristeva reader. New York, NY: Columbia University Press. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York, NY: Cambridge University Press. Lee, Y. J. (2011). Scratch: Multimedia programming environment for young gifted learners. Gifted Child Today, 34(2), 26–31.

36

Designing for Collaborative Creativity in STEM Education with. . .

931

Legare, C. H., Gelman, S. A., & Wellman, H. M. (2010). Inconsistency with prior knowledge triggers children’s causal explanatory reasoning. Child Development, 81(3), 929–944. Levi-Strauss, C. (1966). The savage mind. Chicago, IL: University of Chicago Press. Manches, A., & O’Malley, C. (2012). Tangibles for learning: A representational analysis of physical manipulation. Personal and Ubiquitous Computing, 16, 405–419. Papert, S. (1991). Situating constructionism. In I. Harel & S. Papert (Eds.), Constructionism (pp. 1–12). Norwood, NJ: Ablex. Papert, S. (1993). Mindstorms: Children, computers and powerful ideas (2nd ed.). New York, NY: Basic Books. Pea, R. (1993). Practices of distributed intelligence and designs for education. In G. Salomon (Ed.), Distributed cognitions: Psychological and educational considerations (pp. 47–87). New York, NY: Cambridge University Press. Peppler, K. A., & Kafai, Y. B. (2007). From SuperGoo to scratch: Exploring creative digital media production in informal learning. Learning, Media and Technology, 32(2), 149. Resnick, M. (2004). Edutainment? no thanks, I prefer playful learning. Associazione Civita Report on Edutainment, 14. Retrieved from http://www.roboludens.net/Edut_Articoli/Playful_Learning.pdf. Resnick, M. (2008). Sowing the seeds for a more creative society. Learning & Leading with Technology, 35(4), 18–22. Resnick, M., & Silverman, B. (2005). Some reflections on designing construction kits for kids. In Proceeding of the 2005 conference on interaction design and children, ACM Press, pp. 117–122. Resnick, M., Martin, F., Sargent, R., & Silverman, B. (1996). Programmable bricks: Toys to think with. IBM Systems Journal, 35(3/4), 443–452. Resnick, M., Maloney, J., Monroy-Hernandez, A., Rusk, N., Eastmond, E., Brennan, K., et al. (2009). Scratch: Programming for all [Electronic version]. Communications of the ACM, 52(11), 60–67. Rojas-Drummond, S. M., Albarrán, C. D., & Littleton, K. S. (2008). Collaboration, creativity and the co-construction of oral and written texts. Thinking Skills and Creativity, 3(3), 177–191. Sarmiento, J. W., & Stahl, G. (2008). Group creativity in interaction: Collaborative referencing, remembering, and bridging. International Journal of Human Computer Interaction, 24(5), 492–504. Sawyer, R. K., & DeZutter, S. (2009). Distributed creativity: How collective creations emerge from collaboration. Psychology of Aesthetics, Creativity, and the Arts, 3(2), 81–92. Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic books. Sullivan, F. R. (2008). Robotics and science literacy: Thinking skills, science process skills, and systems understanding. Journal of Research in Science Teaching, 45(3), 373–394. Sullivan, F. R. (2011). Serious and playful inquiry: Epistemological aspects of collaborative creativity. Journal of Educational Technology and Society, 14(1), 55–65. Sullivan, F. R., & Heffernan, J. (2016). Robotic construction kits as computational manipulatives for learning in the STEM disciplines. Journal of Research in Technology Education, 49(2), 105–128. Sullivan, F. R., & Wilson, N. C. (2015). Playful talk: Negotiating opportunities to learn in collaborative groups. Journal of the Learning Sciences, 24(1), 5–52. Sullivan, F. R., Hamilton, C. E., & Foley, A. (2012). Shared genre interests: How students learn together with Scratch. Paper presentation at the 33rd Annual Ethnography in Education Research Forum, University of Pennsylvania, Philadelphia, PA. Turkle, S., & Papert, S. (1991). Epistemological pluralism and the revaluation of the concrete. In I. Harel & S. Papert (Eds.), Constructionism (pp. 161–192). Norwood, NJ: Ablex. Varenne, H. (1998). Local constructions of educational fact. In H. Varenne & R. McDermott (Eds.), Successful failure: The school America builds (pp. 183–206). Boulder, CO: Westview Press. Vass, E. (2007). Exploring processes of collaborative creativity – The role of emotions in children’s joint creative writing. Thinking Skills and Creativity, 2(1), 107–117. Vygotsky, L. S. (1978). Mind and society: The development of higher mental processes. New York, NY: Cambridge University Press. Wegerif, R. (2007). Dialogic education and technology. New York, NY: Springer. Wells, G. (1999). Dialogic inquiry: Towards a sociocultural practice and theory of education. New York, NY: Cambridge University Press.

932

F. R. Sullivan and R. G. Barbosa

Dr. Sullivan is an Associate Professor of learning technology in the department of Teacher Education and Curriculum Studies in the College of Education at the University of Massachusetts, Amherst. She is the author of Creativity, Technology, and Learning: Theory for Classroom Practice, published in 2017 by Routledge Press. Her research focuses on student collaborative learning with computational media including robotics and Scratch. Dr. Sullivan is particularly interested in the development of computational thinking as it relates to creativity, problem-solving, and students’ interests in computer science. Her work also focuses on issues of gender in computer science and in STEM education. Dr. Sullivan has received support from the National Science Foundation for a project aimed at developing computational means to assist in the microgenetic analysis of discourse data drawn from student collaborative group interactions. The goal of this project is to improve the field’s ability to perform microgenetic learning analysis with large data sets. Dr. Barbosa received his Ph.D. in Teaching Science and Math Education from the State University of Londrina-Brazil in 2015. In 2012–2013, he was a fellow in the Science without Borders doctoral study abroad program. The competitive award was made by the Brazilian government and allowed Dr. Barbosa to study for 1 year with Dr. Sullivan at the College of Education, University of Massachusetts, Amherst, USA. Dr. Barbosa received his master’s degree in Teaching Science and Math Education and his bachelor’s degree in Physics from the State University of Maringá-Brazil. Dr. Barbosa’s research focuses on creativity and Paulo Freire’s Pedagogy for the development of critical teaching methodologies for Scientific and Technological Education at school and in the university context. His work is also informed by a critical epistemology of science, especially regarding a counter-history of Science, where he considers the contribution of African, Afro-descendants, and indigenous peoples to universal scientific knowledge. Dr. Barbosa is currently an Assistant Professor at the Federal University of Paraná, Brazil. He is a teacher educator offering courses on rural education and on the natural sciences; these courses focus on agro-ecology, food justice, and agrarian questions.

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort Degree Program

37

Zain Ali, Nanxi Meng, Scott Warren, and Lin Lin-Lipsmeyer

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Theoretical Foundation of Problem-Based Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PBL in Undergraduate Business Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cohort-Based Learning Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PBL and Cohort-Based Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overview of the PD&A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Five Pillars of PBL Cohort Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pillar 1: Problem-Based Learning Cohort (PBLC) Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pillar 2: Integrated Academic Curriculum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pillar 3: Organizational Partnership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pillar 4: Student Services Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pillar 5: Internships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Further Studies and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

934 936 936 936 937 938 938 938 939 940 941 944 946 947 948 951 951

Abstract

The tuition fee for education in the United States is skyrocketing, and 30% of students across US universities drop out after the first year. Faculty have reported having limited impact on students’ college careers, yet many universities continue delivering classes in a traditional manner with no major changes on the horizon. This chapter shares a case study of a transformational integrated undergraduate Z. Ali (*) · N. Meng · S. Warren Learning Technologies Department, University of North Texas, Denton, TX, USA e-mail: [email protected]; [email protected]; [email protected] L. Lin-Lipsmeyer Department of Learning Technologies, University of North Texas, Denton, TX, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_170

933

934

Z. Ali et al.

cohort-based BS degree program built on the foundation of problem-based learning at the Frisco Campus of the University of North Texas. The chapter identifies five major pillars of the degree program including (1) cohort-based and problem-based learning as the pedagogy, (2) partnership with industry organizations to provide hands-on experience for students, (3) integration of curriculum between all cohort-based classes, (4) internal partnership with student services to get students ready with life skills, and finally (5) organizational partnerships to provide three internships for students before graduation. This chapter shares the journey from concept of the program to the completion of the first year, along with what program components worked and did not work. The study concludes with the achievements and changes that were made for the academic year (2020– 2021), and further research directions are suggested in addition to key lessons learned. Keywords

Problem-based learning · Cohort-based degree model · Industrial partnership · Undergraduate business education

Introduction According to College Atlas (2018), it is projected that 70% of Americans will study at a four-year college, but less than 62% among them will graduate with a college degree in six years, and 30% of students will drop out of college at the end of their first year across all universities in the United States. Another survey by LendEDU (2018) found that 55% of students in the United States struggled to find the money to pay for college, and 51% dropped out of college because of financial issues. An increasing drop-out rate of undergraduate students in the United States has reminded higher education faculty, researchers, and stakeholders that increasing the credit hours and prolonging the dominant 4-year schedule of undergraduate studies are not viable solutions to the issue. Facing the increasingly competitive job market, many post-secondary institutes have prioritized preparing students to acquire the most desired soft skills by employers to increase the chances of their graduates getting employment. The National Association of Colleges and Employers (NACE, Peck, 2017) ranked communication, problem-solving, and collaboration as the top three desired skills for employers. Krause (2009) pointed out that employers want to hire graduates that already have these above-mentioned skills; the consensus was that the essential skills should have developed in college and should only require refining onboard the job. Given the acknowledgment, Bauer-Wolf’s study (2019) still found employers reported having difficulty finding such candidates for their openings. These three major issues prevent undergraduate education in the United States from fulfilling the needs of students. An undergraduate degree that serves the needs of students and is financially affordable is highly desired.

37

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort. . .

935

To address these three issues, a reform of curriculum instruction and program design is desired in the interest of current and prospective students of higher education, faculty, stakeholders, and policy makers. In order to better prepare students facing the job market upon their graduation, one of the suggested reforms, among many others, is to expose students to real-world scenarios to develop problem-solving skills through hands-on experiences that require communicating and collaborating with others. Problem-based learning (PBL) is an instructional method that originally comes from medical education; now it has been adopted and implemented as an instructional pedagogy in diversified fields and domains. Significantly different from traditional teacher-centered and lecture-based teaching methods, PBL is rooted in constructivist learning theory (Savery & Duffy, 1995). It relies largely on student autonomy, which requires students to work autonomously in a goal setting, take on responsibilities, collaborate, and communicate in their learning (Wijnia et al., 2011). While instructors serve the role of learning facilitators, PBL emphasizes on student self-directed discovery and learning through solving real-life issues. PBL instruction, hence, aids students to build on their soft skills and enhance them during their undergraduate studies. To better support the implementation of PBL at the level of undergraduate degree design, cohort-based learning is introduced along with the PBL as the core to facilitate the cultivation of the soft skills. Additionally, four other pillars are added to achieve the overall success of the undergraduate degree education reform: integrated academic curriculum, student service integration, internship, and organizational partnership. The current study shares the exploratory case study of the integration of the above five components at an undergraduate degree program level. The University of North Texas (UNT) Frisco campus launched UNT’s first PBL cohort program for FirstTime-in-College (FTIC) students in fall 2019. Twenty-three freshmen students were admitted to the undergraduate degree program Project Design and Analysis (PD&A). So far students have worked on a transportation project and a business improvement project with a focus on improving efficiency in how restaurants were opened with the City of Frisco (COF) as an industry/organization partner. The PD&A program is an innovative exploration for instruction and program design in higher education. The goal of UNT initiating such a program is to tackle two challenges faced by post-secondary institutes in the United States and worldwide: student retention and career readiness. In the current study, we propose to conduct an exploratory case study to showcase the ongoing PD&A program and seek to answer the following research questions: 1. What are the key processes of integrating five pillars into the PBL cohort PD&A program? 2. How does PD&A program address the typical three issues (employer desired skills, student drop-out rate, and financial burden) faced by US higher education institutes?

936

Z. Ali et al.

3. What key learnings were applied to the PD&A program after the first year in this design-based research approach of teaching?

Literature Review Theoretical Foundation of Problem-Based Learning Problem-based learning (PBL) is an instructional method that originally comes from medical education in the early 1990s, but now it has been adopted and implemented as an instructional pedagogy in many diverse fields and domains. PBL roots are found in constructivist learning theory (Savery & Duffy, 1995), relying largely on student autonomy, which requires students to work autonomously in the learning goal setting, taking responsibilities for their learning, collaboration, and communications (Wijnia et al., 2011). PBL emphasizes on student self-directed discovery while instructors serve the role of learning facilitator and help students learn by solving real-life problems. Barrows (2002) identified four key components of PBL: ill-structured problems that trigger not only students’ thoughts on the cause, but also the solution; studentcentered approach that allows students to decide what they need to learn; teachers serve as facilitators and tutors, instead of traditional knowledge provider, and guide students to ask meta-cognitive questions and gradually release learning responsibilities to students; and authenticity of problems presented to students to learn, keeping alignment to professional or “real world” practice. With the four key components of PBL set forth, Savery (2006) defined PBL as an instructional approach that is learner centered and empowers learners to conduct research, integrate theory and practice, and apply knowledge and skills to develop a viable solution to a defined problem. To implement PBL successfully, Savery and Duffy (1995) provided eight principles and commented on the critical features of PBL on learning goal setting, problem generation, problem presentation, and facilitator role. PBL model addresses collaboration and communication skills, and educators have achieved the common understanding that these skills cannot consistently improve through traditional curriculum and academic program setting. Consistent instillment is the pursued pedagogy of fostering long-lasting soft skills.

PBL in Undergraduate Business Education An increasing number of instructors have recognized the importance of PBL and the implementation of PBL into various courses and curricula, through which evidence has been observed regarding the benefit of PBL in various domains and subjects in undergraduate studies, including medical education, biomedical engineering, chemical engineering, software engineering, and thermal physics. PBL has shown its effectiveness in the above fields, but studies also found that the connection between

37

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort. . .

937

PBL and business and economic studies are tenuous, although these domains are highly practical. Jones and McMaster (2004) applied PBL in a 3-year undergraduate Information Science degree program, where students across all years formed groups to solve real-life business problems, suggesting the possibility of successfully reconciling academic learning objectives with real-life project demands in business studies. However, Bigelow (2004) pointed out that simply implementing PBL in improving students’ problem-solving skills may not be sufficient after students graduate and face organizational problem-solving. More steps are in demand to further prepare students to solve real-world problems and collaborate with others. Brundiers, Wiek, and Redman (2010) proposed that collaborating with community and industrial partners provides students real-world learning opportunities, which emphasized the importance of connecting the learning experience closely with the real world. Martínez León (2019) reported teaching an engineering course with the PBL model through the execution of Lean Six Sigma (LSS) projects implemented through university–industry partnerships. This study reported that such model facilitated the integration and application of theoretical knowledge through the development of professional skills in undergraduate students, as demanded by business partners and organizations. However, the feasibility and the validity of such pedagogical exploration applied at the academic program level remain unclear. Miller, Hill, and Miller’s (2016) study provided a good example of applying PBL in an undergraduate supply classroom to introducing Lean Six Sigma by providing students an ill-structured business problem in a setting students are familiar with, which focused their learning efforts on technical mastery of concepts and tools. It helped cultivating critical thinking, teamwork, and project management beyond the content of the course. Miller, Hill, and Miller (2019) further investigate PBL as in-class simulation to teach operations and process improvement concepts and emphasized problem-solving, teamwork, and intra-firm cooperation for large, semester-long process improvement projects for multiple student groups, which expand the application and instructional design of PBL. In undergraduate business education, research on the application of PBL in curriculum and instructional design beyond one course and one semester is highly desired in order to explore the skills improvement, curriculum integration, and the best practices of adopting PBL as the main pedagogical strategy in undergraduate education.

Cohort-Based Learning Model As educators become more aware of the importance of collaboration, an increasing number of educators and educational researchers have begun to appreciate the educational benefit of cohort-based learning. Cohort-based learning provides students opportunities to foster their creativity and innovation, as these skills are not as readily utilized in other instructional contexts. In the cohort-based learning model, students form long-lasting groups that allow them to extend the learning and

938

Z. Ali et al.

collaboration beyond the classroom thus fostering long-term friendships, which indirectly contributes to keeping the drop-out rates to a minimum. Although cohort-based learning poses special requirements for program structure and curriculum design, the benefits of this model is evident (Saltiel et al., 2002).

PBL and Cohort-Based Learning PBL being adopted and implemented as the major pedagogy program-wise is not commonly seen. PBL implementation throughout a cohort-based undergraduate degree program is an innovative design in post-secondary education. Throughout the documented literature, Jin et al. (2019) reported their achievements in adopting PBL in cohort-based classes of an undergraduate science academic program and found that students who underwent this model showed increased awareness and interest in solving problems. The program itself showed increased enrollment, doubled retention rate, and decreased average time of completing the degree by almost 2 years. However, such examples are uncommon, and little is known about the instructional design, student learning transition, and connection to the real world. Based on the research findings listed above, one of the objectives for the current study is to seek to understand how to better prepare students to be job-ready by integrating industry partnerships and curriculum into a cohort-based PBL instruction for a new Bachelor of Science degree in Project Design and Analysis (PD&A) at University of North Texas (UNT) Frisco campus.

Methodology The current chapter proposes to apply an exploratory case study to examine the impact of a PD&A undergraduate degree with PBL and cohort model as the core feature implemented on UNT Frisco campus. Case study is useful for the study of individual case or cases within a current, real-life context or setting. Given the timeliness and ongoing feature of the PD&A program, exploratory case study helps to identify causal relationships among factors and develop explanations for the ongoing scenario (Maxwell, 2004).

Overview of the PD&A The initial motivation when designing the PD&A degree program with PBL and cohort learning design was to provide a feasible undergraduate degree plan that addresses the three major issues faced by many current US higher education institutes: cultivating employer-desired skills, lowering student drop-out rate, and reducing financial burden. Hence the goals of the PD&A program are to 1) empower the students in applying the knowledge gained in classroom into real life in timely fashion, 2) be “robot proof” and job-ready for their upcoming career after

37

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort. . .

939

graduation, and 3) allow students to complete their undergraduate study with reasonable costs in 3 years. The new campus of UNT at Frisco, Texas, was assigned as the incubator for the launching of the program. The journey of PD&A design was initiated in the summer of 2018 with the objective of admitting the first cohort of students in the fall semester of 2019. The key desired outcomes for the degree program are to 1) apply classroom knowledge in solving real-world problems, 2) prepare students to be job-ready for their upcoming career after graduation, 3) have high-demand employable skills, 4) have reduced cost of education (without lowered tuition fees), 5) be better connected with faculty and staff. The method that was chosen to achieve these objectives was to create a problem-based learning program with a cohort learning model that allows students to graduate in 3 years from UNT, a Tier 1 public majority minority university in Texas. Program administrators and faculty consider the journey of the PD&A program, in regard to problem-based cohort learning (PBCL) model, to be an educational design-based research (DBR). A key characteristic of DBR is that the educational ideas for student or teacher learning are formulated in the design, but can be adjusted during the implementation of these ideas. Program administrators will continuously update and refine the framework and the course design based on the feedback received from the cohort students, partners, faculty, and staff. From the experience of Cohort 1’s program in the first year, changes are being made for the upcoming Cohort 2 students for fall 2020. More degree programs are being launched based on the developing PBL framework in UNT Frisco. The PD&A degree designed with block scheduling enables students to complete the program in 3 years. This lays the foundation for students to save money by paying fixed tuition and fee rates for regular semesters, and the reduced tuition fee in the summer based on the existing “save and soar” tuition plan at UNT. Cohort students can also jumpstart their career 1 year sooner while eliminating all expenses surrounding education, which are not tuition and fees such as transportation and meals.

Five Pillars of PBL Cohort Model This degree program is designed with PBL and cohort model as the core element with a foundational belief in “partnership” and four additional pillars supporting the degree program, as shown in Fig. 1. Successful partnerships with academic colleges, student services, industry (organizations), student organizations, support organizations, students, staff, and faculty are integrated in the only way this program will be successful. The approach of this chapter is to introduce each of the elements shown in the diagram below in detail and share the lessons learned throughout the first-year journey, while discussing the changes that will be made to the program going forward.

940

Z. Ali et al.

Fig. 1 Problem-based learning cohort model

Pillar 1: Problem-Based Learning Cohort (PBLC) Model PBLC model is illustrated in Fig. 1. In order to create a culture of collaboration, the concept of cohorts was added to the PD&A program – all the students start and graduate at the same time. Miller, Hill, and Miller (2019) suggest a class size of 20– 30 students to be appropriate for PBL classrooms. The program hence builds cohorts of approximately 24–30 students to help students fully engage in collaboration with other students and allow faculty to provide sufficient attention and assistance to all. In PD&A, a cohort of students are admitted every fall semester; they take core and major classes taught with PBL as major pedagogy (93 credit hours), along with required applied seminars (6 hours), elective courses (15 to 18 credit hours), and three internships (3 to 9 hours) in 3 years for a total of 120 hours. Students take approximately 17 credit hours for six 16-week semesters and 6 credit hours for three 8-week summer semesters. The distribution of credit hours is shown in Fig. 2. One of the potential disadvantages of the cohort model is that once students drop out or transfer to other institutes, it is challenging to rejoin their original cohort. The program was designed to keep the option of receiving transcripts by semester with individual grades for each class in order to allow students to transfer to or from other programs (Fig. 3). The PBCL program marketing was formally launched in February 2019 for Cohort 1 to start classes in fall 2019 with 23 students signed up. The program was launched with three Texas Core classes in History, English, and Psychology plus a Project Connections class on collaborative thinking, an applied seminar, and one elective of 3 hours. In spring 2020, the students had four Texas Core Classes, Connections class on Professional Communications, and an applied seminar.

37

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort. . .

941

Fig. 2 PD&A credit hours distribution

The challenge to delivering this concept arises from the fact that the New College Frisco campus had to bring in a large number of faculty members from different academic units (colleges, departments) and student service units (student services, recruitment, etc.) to collaborate, as the objectives, measurements, and agendas of each of these units could be difficult.

Pillar 2: Integrated Academic Curriculum The degree was built on the framework of an existing undergraduate BS degree in Integrative Studies with three concentrations: project, design, and analysis. Project courses are designed as connection courses; the connection generated from these courses lays solid ground for partnering with industries and conducting learning in projects for students. The design and analysis courses along with the applied seminar serve as the other major classes for this degree. In the course design of the degree, eight competencies that employers want in graduate students and the key methodologies and certifications that employers are using to train their employees were instilled in the course curriculum with methodologies and tools such as StrengthsFinder, SCRUM, Lean Six Sigma, Agile, PMBOX, Social Styles, and DiSC, which are used by employers to improve the skills of their staff in major corporations around the globe. Additionally, by including the technological components into the instruction such as Microsoft Office, Teams, Slack, JIRA, Google Docs, and mini-tab, students are prepared for the digital capacity of future jobs. A connection class in Collaborative Thinking with a focus on Six Sigma given in the first semester is taken by students with the intent to provide the competencies and skills that allow the students to work on a real partner problem from the first semester in a language that most businesses understand. The partner problem then serves as the center of the integrated curriculum from the first semester allowing all the semester classes to revolve around the project with the exception of electives and the applied seminar as shown in the Fig. 4.

Fig. 3 Sample project design and analysis degree program overview for 3-year period

942 Z. Ali et al.

37

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort. . .

943

Fig. 4 Major and core classes revolve around the project

The faculty of the PD&A program work together as a team to connect individual courses in the Core Curriculum to integrate around the partner project, as well as collaboratively developing integrative syllabus and course work. One example of the integrated curriculum from the fall 2019 semester is when the partner project revolved around mobility, including transportation challenges and driverless cars. In the history course, student learnings from prior to 1865 included the migrations of Mayans to associate with population growth in the City of Frisco as well as the introduction of railroad as a new mode of transportation for that time along with the acceptance criteria from the people at that time. In psychology, the discussions included what constitutes the psychological needs of people when they move and what facets one needs to consider when selecting public transportation routes and vehicles, such as disability. In English, the assigned articles were pertaining to mobility, and most of the writing assignments were focused around mobility, smart cities, etc. All the initial reading and class assignments were revolved around using Six Sigma tools on mobility use cases until students were prepared for being in a position to transition to the class mobility project with the City of Frisco (COF). The connection course of fall 2019 titled “Collaborative Thinking” focused on adopting the Six Sigma methodology of the DMAIC (Define, Measure, Analyze, Improve, and Control) process. Additionally, the students built a code of conduct for their team; learned the Bruce Tuckman five-stage development process that most teams follow of norming, forming, storming, performing, and adjourning; and learned how to run effective meetings with clear objectives, action logs, and parking lot. The Six Sigma toolbox also exposes students to tools like scope development, interviewing techniques, process mapping, stakeholder management, data stratification, fishbone diagrams, 5 Whys, Gemba boards, and presenting skills. By doing so, students are expected to establish a good foundation for team building and an understanding of appropriate tools on the partner project. All the faculty in the

944

Z. Ali et al.

cohort program facilitate the students to build their skills of collaboration and communication. Assignments given in one class can be leveraged in other classes. For instance, in English the students learn to write good business memos that are presented to the client along with the client presentation three times in the semester. The business memo is graded in the English class, as well as the Connections class. Another example is that the students learned how to take surveys in the psychology class, and the results of the survey were analyzed and stratified with the help of the psychology professor. Later the results were used in the connections class in designing the routes and presented to the partner in the final presentation.

Pillar 3: Organizational Partnership Working with real-world business partners can engage students with authentic realworld problems. The pillar named “Organizational Partnership” comes from the partnership established in this program with the Industry, Government institution, or a non-profit organization. The program has identified a Time, Treasure, and Talent partnering model for organizational partnership (3TPM) with expectations identified in each of the categories as shown in Fig. 5. (a) Guest Lecture - Industry partners are invited to provide four 45-minute workshops as applied seminar classes each semester on campus or at the organizational partners’ sites to share their experiences with what the students are learning, such as project management, Six Sigma, transportation, and communication. These workshops serve several purposes: it is an opportunity for the students to see how what they have learned in class can be applied in real life Fig. 5 3T Partnering model (3TPM) for PBL organizational partnerships

37

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort. . .

945

with real examples; it exposes students to industry personnel for potential internship opportunities, and it allows the students to learn how to network with industry professionals. (b) Project Sponsorship – Project sponsorship partners are required to commit to sharing their talent with academia to help the students learn by applying what they have learned in class with the project. In return, the partners benefit by receiving free consultation on a business challenge that they might be struggling with. The first-year partner, the City of Frisco (COF), and the second-year partner, nThrive, helped us in 1) providing a real business scenario or business problem for the students to work on, 2) committing and coming into the class three times, and 3) providing access to facilities, people, data, and policies for the students to acquire additional knowledge and providing deep insights on the project. Of the three visits, the first was at the beginning of the semester to provide an overview of the business scenario/problem, a second visit to review student progress, and a third visit to attend the final presentation at the end of the semester. Occasional visits were made during the semester as well. There are five scenarios that faculty could potentially use to work with the students in the PD&A program (shown in Fig. 6): 1. Fictitious Projects: Projects created for the students based on research. The project outcomes are likely to be predetermined and known to the faculty. Fig. 6 5 Methods of incorporating projects into PBL curriculum

946

Z. Ali et al.

2. Case Study: Faculty use case studies from databases like Harvard Business Review to create projects for the students. These are sometimes used to help derive the background for the anchor project and are typically done with results already defined. 3. Case Interview: The partner company recaps an event that has happened in the past and creates a project with the faculty by taking out the name of the company and the parties involved in the execution of the project and modifying the data, in order to create a scenario for the students to work on. 4. Business Scenario: The faculty work with companies on projects that are based on real business needs and without a predefined answer. In business scenario projects, some dependency on the client exists, and student contact is not overwhelming. In this scenario, the project teams in the class are competing to conduct research on competing companies, products, performing surveys, or interviews to provide different perspectives to the partner on the most ideal business scenario. 5. Business Problem: These projects revolve around industry business problems and have all the traits of the business scenario, but has medium to high dependency on the partner. In this scenario, the partner provides a large complex business problem, and students work in small teams to deliver a solution to the client. More than one of the stakeholders have to be involved in the student journey to make the project meaningful for the students as well as the recommendation meaningful for the partner. As shown in Fig. 6, the projects students worked on with COF are categorized as P3 (Mobility project in fall 2019) and P4 (Business Process Improvement project in spring 2020) accordingly, which indicates that the partner involvement levels in these projects are high. (c) Advisory Board – An organizational partnership advisory board is intended to be launched with representatives from Industry and non-profit organizations in fall 2020 or spring 2021. (d) Internships: Internships with organizational partners are so important to the success of the program that they are considered to be one of the five pillars of the program.

Pillar 4: Student Services Integration The fourth pillar, integrated student services, is intended to provide students full support for their success in PD&A program (Fig. 7). • Different from the traditional applied seminars hosted by Student Services with no grades assigned, PD&A applied seminars are designed to facilitate mutual understanding between faculty and students on the course load and assigned 1 credit hour with 3 hours of contact time, which gives students incentive to attend the seminar. The applied seminars in the first semester guide students

37

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort. . .

947

Fig. 7 Organization and academic partnering opportunities in PBL

starting their college journey by providing them life and learning skills. The focus areas of the student services classes include building life skills, such as coping with stress, timely delivery when the stress is high – especially after week 4, when many tests come up at the same time. • Fostering time management, personal finance, and learning strategy skills. • Preparing students to be internship ready by teaching resume writing, LinkedIn profile development, etc. • Preparing students to be job search ready by teaching interviewing techniques, dressing for success, mockup interviews, etc. Student services and academic services work together in this program by focusing on helping students build life skills over 3 years and bringing professionals from the industry so the students can get firsthand feedback and be ready for their future business career. Providing opportunities for the students to start building a network of professionals for their career can equip students with a cutting-edge advantage for job market competition.

Pillar 5: Internships The final pillar in the model is internships. Partnering companies work with academia and student services to provide internships so the students can obtain applied experience. The partners benefit from receiving students with great skills at a low cost while they are still in school, and full-time well-prepared talents upon

948

Z. Ali et al.

graduation. Three different internships with a total 9 credit hours in 3 years are required before graduation for each student. The internship curriculum focuses on industry partners helping the university to provide opportunities for the students to gain experience in the eight NACE job-ready competencies as previously mentioned, as well as leveraging the learnings from the classroom and projects. UNT’s external organizational partners along with UNT’s academic internal partners, such as Finance, Student Services, and other administrations of UNT, collaborated with New College to provide internships for almost all of the students in the program.

Discussions Exploring the PBL pedagogy in cohort model is an innovation in reformation for higher education. As of now, the following experiences and lessons are learned. First, how to manage dual credit in the cohort program. The original plan was that students with dual credit would only audit the class without enrolling and paying, since the learning from the class would apply to the project. However, without incentive to get grades, the students were not motivated to be in class and maintain scholarships without full-time enrollment status. The stance in the program was changed to welcome students to attend core classes without mandatory attendance requirement, even though portions of the class content would apply to the project. Administrators of the program select teams with enough representation from all classes to fill the gap of missing skills on the project. Additionally, most of the students that came in with dual credit had taken classes in English, History, or Political Science, which are mandatory core classes in Texas. In order to address this challenge, these six classes have been split over six semesters to balance the class loads for students. The second learning was the challenge of the Connections class. The Connections class was designed to be a lab with 3CHR of credit and 6 hours of contact time. This was also the class where all faculty attend the class together. This class had created a number of challenges: 1) increased work load for faculty members – a lecturer in Core was using 50% of their load catering to a cohort of 25–30 students; 2) ineffective time use – having four to five faculty members in a class prevent them from working on their other duties; 3) keeping the students in the classroom for too many hours. To address these challenges, the curriculum committee allowed faculty to shift some course content to the lab (example 6 Sigma methodology for collaborative thinking), adjust the class from 3 CHR to 4 CHR, and match the contact time to the credit hours. Another change to be made is to have all the faculty meet for 30 minutes prior to the weekly Connections class. The implementation of this meeting could provide more flexibility to all the faculty teaching in the PD&A program who are dual appointed in two colleges and increase the efficiency in the integrated curriculum design by focusing on the themes of the projects, and entitle faculty to plan their classes around the themes with a couple of touchpoints instead of managing all the details of the integration. Although significant time was spent

37

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort. . .

949

Fig. 8 PBL Integrated curriculum methodology (ICM)

planning the integration, the team came up with a similar course cycle with the traditional courses. The weekly 30-minute faculty meeting can not only help increase the work efficiency for faculty, but also help students to achieve greater success. The faculty and staff are in the process of implementing the second learning by implementing a new class integration partnering model (CIPM). In this model, each faculty participates in the decision of how much integration they will participate in for the semester based on the provided project. The integrated curriculum methodology is shown in Fig. 8. The methodology has four one-time events: project overview, level of integration discussion, weekly syllabus adjustment, and reflections. The methodology also has three recurring events over the semester: 63-minute touchpoint, 16 invites to the lab, and 4 client presentations. The third learning is the time limitation of executing real problem projects. In the sixteen weeks of the semester, students completed project design, measured current trends for analysis and then suggested recommendations for implementation with measurement controls for success for the project at COF. The timeline of sixteen weeks could be too short for executing projects. The students would have a better learning experience if these learning items are spread over two semesters. Built on the empirical evidence, the quantity of projects remained the same, but the design and continuation of the project into two semesters is one of the changes that was incorporated. Taking projects to completion with proper analysis will improve the learning experience of the students in the current and future cohorts. The fourth learning is the requirement of re-thinking the students and faculty evaluation in the PD&A program. The PBL pedagogy differentiates the UNT at Frisco campus from all other programs at UNT main campus that follow the traditional pedagogy of teaching. At Worcester Polytechnic Institute (WPI), an experienced PBL college, there are no prerequisites, and the minimum grade a student gets is a C, which would be more relevant in Frisco, yet we follow the traditional pedagogy policies from Denton. At UNT Frisco’s PBL program, students work with partners on real business problems that do not have a predefined answer, which requires significantly more time and effort for the student as well as faculty. Additionally, student teams work intensively with faculty, especially for three partner presentations that happen over the course of each semester; hence the grades, faculty load, and sequences of classes should all be reviewed from a PBL pedagogy perspective vs. the traditional pedagogy.

950

Z. Ali et al.

The last major learning is that a better model is needed to collaborate with the organizational partners. This is the first time that UNT worked with a PBL model to involve organizational partners, like nThrive, for long-term projects, which entails a steep learning curve for both parties. UNT needs to have a more efficient model for partnering with both organizations to develop long-term partnership for curriculum integration, internships, and guest lectures. Interviews were conducted by media from UNT, Frisco, and the Dallas–Fort Worth metroplex, as well as independent observers and researchers, who collected some empirical evidence from the Cohort 1 students, faculty, staff, and the industrial partners. Some representative quotes from each category of contributors that provide insights from the participants of the program and reveal their experience are listed below. Some quotes from students are: “It has just really been an awesome experience so far like I have really gotten a lot more experiences with the real world and business, I had not imagined myself doing this as I graduated from high school.” “I am really excited to kind of experiences we get with some of the businesses around Frisco and be able to work with them more hands-on and I am really excited to be working with a closer-knit group of people.” “Whenever I interact with my professors, any question I ever have, anytime I ever have a problem, or I need to talk to them, they are always there, super supportive.”

These quotes reflect that students appreciated the length of the program being 3 years, a bachelor’s degree in science, equipped with a lot of real-world hands-on experience for them in a close-knit group of students, faculty, staff, and industry partners. Besides, the partnership design has played an important role in students’ learning experience and vividly elaborated the essence of PBL. These quotes also reflect the close relationship among students and student-faculty, brought by the cohort model. Familiarity and friendship are the foundation of collaboration, which is one skill that employers seek in all graduates. As for the PBL program model, here are some of the quotes from faculty: “I really love the program, it is great to watch the interaction between students and how they work together while they are building life skills in team work, in collaborative thinking, leadership, and project management at a very young age.” “What you learned was in the service of connecting the content that you gained the knowledge and skills that you gained with real-world skills that could actually have an impact around you.”

Faculty participating in this program also clearly see the beneficial features of the program and enjoy the benefit of working closely with students and teaching with full engagement. From implementing this program, the faculty, staff, and administrators are discovering new paths and possibilities of undergraduate program design in US higher education institutes. This lays the foundation for developing a

37

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort. . .

951

transformational program and will provide opportunities for building the empirical evidence for applying PBL and cohort program as the UNT Frisco’s instructional pedagogical strategy.

Further Studies and Conclusion The launch of the PD&A program is in its infancy at the UNT Frisco campus, and the plan is to continue this longitudinal study with design-based research to constantly reflect on its results. Based on the objectives of the program, as mentioned in this chapter, there is a need to measure if the students are ready for jobs on day 1 by conducting a comparative study on the enhancement of soft skills and/or NACE competencies for the students in the PD&A program compared to their peer group in traditional pedagogy. More comparative studies are also needed on job placement between peer groups and the performance the students have after they start their jobs. The development of business cases and the evaluation scale of Cost Benefit Analysis of PD&A based on cohort size of 25–30 students are also desired to see the retention levels and student partnering after graduation, in order to evaluate the impact of PBL and cohort model on the initial objectives. The scaling should include organization models, partnering models, faculty readiness, traditional faculty workload measurements, student services integrations, etc. Two of the pillars in the PBL Cohort Model require Higher Education Institutions to partner with organizations for the organizations to contribute to the areas of Time, Treasure and Talent. Future studies should be conducted on the development of a long-lasting organizational partnering model that allows both the educational institution and the organization to benefit from this partnership. There is a firm belief that when a student from a problem-based learning cohort is able to experience six projects over six semesters and three internships with partners working in a project-based learning environment where the role of the faculty is that of a of a facilitator and a network of industry professionals, they will be the movers and shakers of the twenty-first century.

References Bauer-Wolf, J. (2019). Survey: Employers Want 'Soft Skills' From Graduates. Retrieved from https://www.insidehighered.com/quicktakes/2019/01/17/survey-employers-want-soft-skillsgraduates. Bigelow, J. D. (2004). Using problem-based learning to develop skills in solving unstructured problems. Journal of Management Education, 28(5), 591–609. Brundiers, K., Wiek, A., & Redman, C. L. (2010). Real-world learning opportunities in sustainability: From classroom into the real world. International Journal of Sustainability in Higher Education, 11(4), 308–324. Jin, L., Doser, D., Lougheed, V., Walsh, E. J., Hamdan, L., Zarei, M., & Corral, G. (2019). Experiential learning and close mentoring improve recruitment and retention in the undergraduate environmental science program at an hispanic-serving institution. Journal of Geoscience Education, 67(4), 384–399.

952

Z. Ali et al.

Jones, M. C., & McMaster, T. (2004). Addressing commercial realism and academic issues in group-based IS undergraduate project work. Journal of Information Systems Education, 15(4), 375–381. Martínez León, H. C. (2019). Bridging theory and practice with lean six sigma capstone design projects. Quality Assurance in Education, 28(1), 41–55. Maxwell, J. A. (2004). Using qualitative methods for causal explanation. Field Methods, 16(3), 243–264. Miller, K. E., Hill, C., & Miller, A. R. (2016). Bringing lean six sigma to the supply chain classroom: A problem-based learning case. Decision Sciences Journal of Innovative Education, 14(4), 382–411. Miller, K. E., Hill, C., & Miller, A. R. (2019). Applying performance measures and process improvements in the classroom: The loch dots company simulation. Decision Sciences Journal of Innovative Education, 17(4), 302–323. Peck, A. (2017). Engagement and employability: Integrating career learning through Cocurricular experiences in postsecondary education. NASPA-Student Affairs Administrators in Higher Education. Saltiel, I. M., Russo, C. S., & Dawson, J. (2002). Cohort programming and learning: Improving educational experiences for adult learners. The Canadian Journal for the Study of Adult Education, 16(2). Savery, J. R., & Duffy, T. M. (1995). Problem based learning: An instructional model and its constructivist framework. Educational Technology, 35(5), 31–38. Wijnia, L., Loyens, S. M., & Derous, E. (2011). Investigating effects of problem-based versus lecture-based learning environments on student motivation. Contemporary Educational Psychology, 36(2), 101–113.

Zain Ali is Clinical Professor and Program Director for Problem-Based Learning (PBL) at University of North Texas (UNT), Frisco. He played an instrumental role in establishing the project design and analysis of PBL program at UNT as a consultant before accepting the role of professor. Zain brings 31 years of cross-industry and academia experience by serving over 70 clients in more than 10 countries as a consultant or as an employee including leadership roles in engineering, operations, sales, and technology. His tenure has allowed him to work with a few startup companies as well as several Fortune 100 clients like Bombardier, Emerson Electric, Schlumberger, AT&T, AmerisourceBergen, Mack Trucks, Renault VI, and Raytheon while working for consulting companies like Accenture, the Hackett Group, and Wipro. In 2009, Zain started his first company Sunbonn, a global consulting and technology company, where he is now the Chairman of the Board. Zain also provides strategic consulting and advisory services as part of his second company, Azvantage. Zain is the author of the book Cultivate Your Leader, creator of multiple mobile applications, and currently focused on building a customizable multi-rater leadership development platform that uses AI and ML to enhance mentoring. Zain is also currently pursuing his Ph.D. in Learning Technologies at UNT. Nanxi Meng is both Senior Lecturer of Foreign Language Education and a Ph.D. candidate of Learning Technologies at University of North Texas. Her research interests include language and culture acquisition, instructional design, PBL, and application in higher education and pedagogy and teacher education. Dr. Scott Warren is an educator and researcher, who has focused primarily on the engineering and study of complex, system-driven online educational experiences such as learning games and simulations. After teaching and coaching in public school, his early work focused on the development of immersive learning game worlds such as Anytown and Taiga project. His games taught students advanced science concepts, writing process, and computer literacy using advanced instructional methods. His more recent games supported undergraduate computer literacy using lower-cost

37

Integrated Problem-Based Learning: A Case Study in an Undergraduate Cohort. . .

953

digital tools in a method commonly called transmedia. This approach was used to create immersive, narrative games to support complex problem-solving and self-regulated learning. Students engaged with challenges related to global issues framed in business, science, computer literacy, and history. The research outcomes of the related studies are described in Learning Games: The Science and Art. He also co-founded the Koan School from 2013 to 2016, which employed his Learning and Teaching as Communicative Actions framework. His current development and research focuses on instructional design in online settings, as well as the evaluation of educational games, and the role of educational technology businesses. He also provides evaluation and consulting on business operations and strategy for small businesses across the United States. Dr. Lin Lin-Lipsmeyer is Professor and Department chair of Teaching and Learning in the SMU’s Simmons School of Education & Human Development. Lin received her Ed.D. in Instructional Technology and Media from Teachers College Columbia University. Lin has conducted interdisciplinary research in learning publications including journal articles, books, and book chapters. In addition, she has been PI, Co-PI, or researcher on NSF and Foundation grants bridging learning science, artificial intelligence, and STEAM learning. Lin serves as the Development Editor-in-Chief of one of the top journals in education and educational research, the Educational Technology Research and Development (ETR&D, https://www.springer.com/journal/11423).

Toward a Systematic and Model-Based Approach to Design Learning Environments for Critical Thinking

38

Dawit T. Tiruneh, Mieke De Cock, J. Michael Spector, Xiaoqing Gu, and Jan Elen

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Definition of Critical Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Domain Specificity and Domain Generality of Critical Thinking Skills . . . . . . . . . . . . . . . . . . . Assessment of Critical Thinking Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Instructional Approaches to Teach Critical Thinking Skills in Higher Education . . . . . . . . . Design, Implementation, and Evaluation of an Immersion- and Infusion-Based Critical Thinking Instructional Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The First Principles of Instruction Model in Brief . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Study 1: Systematic Design, Implementation, and Evaluation of an Immersion-Based Learning Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Study 2: Comparison between Immersion- and Infusion-Based Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

956 958 960 961 962 964 967 968 969 976

D. T. Tiruneh (*) University of Cambridge, Cambridge, UK e-mail: [email protected] M. De Cock Department of Physics and Astronomy & LESEC, KU Leuven, Leuven, Belgium e-mail: [email protected] J. M. Spector Department of Learning Technologies, University of North Texas, Denton, TX, USA e-mail: [email protected] X. Gu Department of Educational Information Technology, East China Normal University, Shanghai, China e-mail: [email protected] J. Elen Centre for Instructional Psychology and Technology, KU Leuven, Leuven, Belgium e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_79

955

956

D. T. Tiruneh et al.

General Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Some Considerations for the Design of CT-Supportive Learning Environments . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

983 984 986 987

Abstract

As a vital twenty-first-century skill, the development of critical thinking (CT) is considered as one of the major goals of higher education. Identifying effective instructional approaches that can foster the acquisition and transfer of CT skills needed in the rapidly changing world of work has therefore been the focus of a large body of research. However, despite the widespread emphasis on CT from educators and researchers, there is little evidence to support that the objective of CT development is being realized. In this contribution, we begin by reviewing some of the theoretical and empirical evidence on the conceptualization of CT, the instructional approaches to stimulate the acquisition of CT skills, and the assessment of CT. We describe some of the limitations of current research on the design of effective learning environments for CT, and two empirical studies that aimed to address the identified limitations are illustrated. Our main line of argument in the two studies is that systematic design of learning environments in accordance with theoretically sound and empirically valid instructional design principles can be an effective approach to foster the development of CT. It is argued in this chapter that embedding CT instruction in domain-specific courses requires greater clarity about what CT is, what set of CT skills could be targeted in domain-specific instruction, and how specific subject-matter instruction could systematically be designed considering CT as an integral part of domain-specific instruction. We conclude this chapter by suggesting some considerations for the design of CT-supportive learning environments. Keywords

Domain-general critical thinking · Domain-specific critical thinking · Instructional design · Intervention · Learning environments

Introduction Various stakeholders in education have recently urged renewed emphasis on the development of critical thinking (CT) as one of the major goals of higher education (Association of American Colleges and Universities, 2005; Davies & Barnett, 2015; OECD, 2015; Pascarella & Terenzini, 2005). Identifying effective instructional approaches that could foster the development of CT has therefore been the focus of much research (for reviews, see Abrami et al., 2008, 2015; Davies & Barnett, 2015; Pascarella & Terenzini, 2005). However, despite the widespread emphasis on CT from educators and researchers, there is little evidence to support that higher education institutions are producing graduates who are competent critical thinkers (Arum & Roksa, 2011; OECD, 2015; Pascarella & Terenzini, 2005). In a recent publication,

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

957

OECD reported that too many young people in OECD countries leave higher education without having acquired the relevant CT skills to function effectively in the twenty-first-century workplace (OECD, 2015). In their large-scale longitudinal study that assessed college students’ CT skills, Arum and Roksa (2011) found that 45% of students made no significant improvement in their ability to think critically during the first 2 years of college, and 36% made no significant improvement after an entire 4-year college degree. In their extensive review of more than three decades of CT empirical evidence, Pascarella and Terenzini (2005) concluded that college and university education does not sufficiently prepare students to apply their knowledge to solve CT tasks related to daily life and work settings. In sum, several studies showed that college instruction is generally associated with modest CT development (Arum & Roksa, 2011; Evens, Verburgh, & Elen, 2013; Pascarella & Terenzini, 2005; Verburgh, 2013). There has been increased interest over the past couple of decades to respond to the challenges of limited CT development. Most previous efforts to address the challenge took place in a context in which general CT skills were taught separately from domainspecific courses (e.g., de Bono, 1991; Paul, 1992; Siegel, 1988). Advocates of this instructional approach, which is referred to as the general approach (Ennis, 1989), argued that the ability to think critically is best developed when CT skills are taught in stand-alone courses because the teaching of those skills is not overshadowed by domain-specific content instruction. The main assumption in this approach is that CT is largely independent of domain-specific content knowledge and that learned CT skills in stand-alone courses can be automatically transferred to solve CT tasks across domains (e.g., de Bono, 1991). However, the general approach has become less dominant in recent years. Empirical attempts to develop students’ CT have shifted mainly toward embedding CT instruction within specific domains (e.g., Bensley & Spero, 2014; Davies, 2006; Heijltjes, van Gog, & Paas, 2014; Helsdingen, van Gog, & van Merriënboer, 2011). The main argument against the general approach is that domain-specific content knowledge is very much part of what is involved in thinking critically, and as such the development of CT cannot be separately viewed from the teaching of domain-specific content knowledge (e.g., Bailin, 2002; Smith, 2002). However, there is little consensus to date among educators and researchers on how to best embed CT instruction within domain-specific courses (e.g., Bailin, Case, Coombs, & Daniels, 1999; Davies, 2013; Ennis, 1989; Halpern, 2014; Jones, 2009; Moore, 2011). Efforts to embed CT instruction within specific domains have long been intertwined with controversies over several issues, such as the conceptualization of CT, the instructional approaches to best stimulate the acquisition of CT skills in domain-specific instruction, and the assessment of CT outcomes (e.g., Bailin et al., 1999; Davies, 2013; Ennis, 1989; McPeck, 1990b; Moore, 2004). As a result, there is limited empirical evidence at present to draw any conclusions about the features of effective learning environments for the development of CT. CT requires complex learning because it involves integrating knowledge, skills, and attitudes and transferring what is learned in a particular domain to daily life and work settings (e.g., van Merriënboer & Kirschner, 2013). It is argued in this contribution that the teaching of CT skills within domain-specific courses may therefore benefit from a systematic approach based on theoretically sound and empirically valid

958

D. T. Tiruneh et al.

instructional design models promoting complex learning. We argue that most efforts to embed CT instruction in domain-specific courses appear to lack systematicity in bringing together the teaching of domain-specific content knowledge and CT skills by using instructional design principles known to foster the acquisition and transfer of CT skills. Following recent evidence on the design of learning environments to promote complex learning (e.g., De Corte, Verschaffel, & Masui, 2004; Merrill, 2002; van Merriënboer & Kirschner, 2013), we argue that developments in instructional design research can have rich implications to guide the design and development of learning environments that will optimally support the development of CT. In this chapter, the effects of a systematic and model-based approach to design learning environments on the development of CT are explored. A systematic approach is defined in this chapter as an orderly method to design learning environments that may support the attainment of desired learning outcomes in terms of the subsequent and interactive instructional design phases, namely, analysis, design, development, implementation, and evaluation. A model-based approach is defined as an approach to design learning environments based on comprehensive instructional design models oriented toward the promotion of complex learning. In the subsequent sections of this chapter, we first briefly describe some of the longstanding controversies in the literature involving the definition of CT, domain specificity and domain generality of CT, assessment of CT, and instructional approaches to foster the development of CT. We then illustrate our own research on the design, implementation, and evaluation of CT-supportive learning environments. Finally, we discuss the implications of our research findings to design CT-supportive learning environments.

Definition of Critical Thinking Scholars hold diverse views with regard to the definition of CT and the core processes involved in thinking critically. Ennis (1993) defines CT as “reasonable reflective thinking focused on deciding what to believe or do” (p. 180). In order to decide what to believe or do, Ennis emphasizes that a person needs to be engaged in some mental processes such as drawing conclusions, judging the credibility of sources, planning experiments, reflecting, defending a position on an issue, and asking appropriate clarifying questions. Halpern (2014) also proposes a definition that she claims “captures the main concepts” reflected on various CT definitions: “Critical thinking is the use of those cognitive skills or strategies that increase the probability of a desirable outcome” (p. 8). Halpern further describes CT as purposeful, reasoned, and goaldirected, and she states that critical thinkers use relevant cognitive skills appropriately, without prompting, and usually with conscious intent in a variety of settings (Halpern, 1998, 2014). Similarly, an American Philosophical Association Delphi panel of 46 renowned experts in the field proposed a broad definition of CT:

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

959

We understand critical thinking to be purposeful, self-regulatory judgement which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based. CT is essential as a tool of inquiry. . . . The ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fair-minded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in the selection of criteria, focused in inquiry, and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit. (Facione, 1990a, p. 3)

There are two essential components of CT identified in most CT definitions: CT skills and dispositions. CT skills are defined as “those strategies for finding ways to reach a goal” (Halpern, 2014, p. 51), and dispositions are defined as a person’s inclination or tendency to use CT skills appropriately without prompting and with conscious intent in a variety of settings such as when faced with problems to solve, ideas to evaluate, or decisions to make (Ennis, 1996; Halpern, 1999). It is claimed that a more meaningful and comprehensive understanding of CT must include the disposition or attitude to think critically on the ground that the acquisition of CT skills alone does not suffice to demonstrate reflective and reasonable thinking (e.g., Ennis, 1996; Halpern, 1998, 2014). Definitions of CT consist of a large set of CT skills and dispositions. Halpern, for example, identifies five major categories of CT skills that represent elements of a desirable outcome in any domain: reasoning, hypothesis testing, argument analysis, likelihood and uncertainty analysis, and decision-making and problem-solving (Halpern, 2014). Besides, Halpern (2014) identifies the following dispositions that she claims to be exhibited by a critical thinker: willingness to plan, flexibility, persistence, willingness to self-correct, admit errors, being mindful, and seeking consensus. Similarly, following his definition, Ennis (1993) identifies a large set of CT skills that are useful to deciding what to believe or do, such as identifying conclusions and assumptions, judging the credibility of sources, defending a position on an issue, asking appropriate clarifying questions, planning experiments, and drawing conclusions. Regarding CT dispositions, Ennis (1996) identifies three broad categories of dispositions: represent a position honestly and clearly, care that one’s beliefs be true and that one’s decisions be justified, and care about the dignity and worth of every person. Likewise, the Delphi panel identified core components of CT skills and dispositions in their consensus CT definition (Facione, 1990a). Six core CT skills involving both cognitive and metacognitive processes are identified: interpretation, analysis, evaluation, inference, explanation, and self-regulation. At the same time, the Delphi panel specified some dispositional components to CT, such as inquisitiveness, self-confidence, open-mindedness, flexibility, and fair-mindedness. In general, numerous sets of CT skills and dispositions that are assumed relevant to increase the probability of a desirable outcome have been identified, and there is a great deal of agreement among scholars that critical thinkers can be characterized as exhibiting a set of CT skills and dispositions. There appears to be an agreement among scholars that the development of CT needs to include training students both to execute a set of cognitive skills and to develop the

960

D. T. Tiruneh et al.

habits of engaging in CT (e.g., Ennis, 1996; Halpern, 1999, 2014; Heijltjes, van Gog, Leppink, & Paas, 2014; Paul, 1992). In relation to this, Halpern (1999, p. 72) noted that “critical thinking is more than the successful use of the right skill in an appropriate context. It is also an attitude or disposition to recognize when a skill is needed and the willingness to exert the mental effort needed to apply it.” However, the distinction between mastery of a set of CT skills and dispositions has been less clear (e.g., Ennis, 1996; Facione, 1990a; Halpern, 1998, 1999; Perkins, Jay, & Tishman, 1993). Along with the emphasis that CT development should focus on both CT skills and dispositions, some argue that the assessment of CT needs to focus on both CT skills and dispositions (e.g., Ennis, 1996; Facione, 1990a; Halpern, 2013). However, the view on the separate assessment of CT skills and dispositions has been controversial. McPeck (1981), for example, defines CT as the “propensity and ability to engage in an activity with reflective skepticism” and insists that CT skills essentially integrate the dispositional component as “one must develop the disposition to use those skills.” Similarly, the Delphi experts’ panel depicts the difficulty of viewing CT skills separately from dispositions as follows: A CT skill, like any skill, is the ability to engage in an activity, process or procedure. In general, having a skill includes being able to do the right thing at the right time. So, being skilled at CT involves knowing . . . both a set of procedures and when to apply those procedures. Being skilled also involves having some degree of proficiency in executing those procedures and being willing to do so when appropriate. (Facione, 1990a, p. 28).

It may be inferred from the Delphi panel description that a discussion about someone’s skill to identify unstated assumptions, for example, accompanies the dispositions to do so. That means, a person who is skilled in identifying unstated assumptions can be said to have the disposition to execute the skill of identifying unstated assumptions. If there are situations where someone is not inclined to execute that particular skill in a given moment, then we may not talk about proficiency in CT skills. Overall, it is possible to note from the above discussion that CT remains an elusive construct. In terms of the core list of CT skills and dispositions underlying CT, most of the above definitions reflect the diversity of views held among scholars while at the same time stressing common aspects: deducing and inferring conclusions from available facts, asking appropriate clarifying questions, defending a position on an issue impartially, demanding that claims be backed by evidence, judging the credibility of sources, reasoning objectively, willingness to plan, being mindful, etc. (Ennis, 1989; Facione, 1990a; Halpern, 2014; Norris, 1989; Willingham, 2007). Whether dispositions are regarded as integral components of CT skills or function in isolation from CT skills is also unclear.

Domain Specificity and Domain Generality of Critical Thinking Skills Whether CT skills are a general set of skills that can be productively applied across domains or whether these skills vary from domain to domain has long been

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

961

the subject of much debate in CT research (Bailin et al., 1999; Davies, 2013; de Bono, 1991; Ennis, 1989; Glaser, 1984; Halpern, 1998; Jones, 2009; Kuhn, 1999; McPeck, 1981, 1990b; Perkins & Salomon, 1989). On the one hand, major proponents of the domain-general view (e.g., Davies, 2006, 2013; Ennis, 1989; Halpern, 1998, 2014; Paul & Elder, 2014) claim the existence of a set of CT skills that are general and transferrable across a wide variety of domains such as science, history, literature, and psychology on the ground that CT tasks across domains share significant structural commonalities. These proponents acknowledge that content differs from one domain to another, and domain-specific knowledge is a necessary condition to demonstrate the ability to think critically (see, e.g., Ennis, 1989; Halpern, 2014). They argue at the same time that there is a set of CT skills that can be acquired regardless of context and be usefully applied across domains as CT tasks share commonalities (see, e.g., Davies, 2006; Ennis, 1997; Halpern, 1998). On the other hand, proponents of the domain-specificity view argue that each domain of study has its own particular epistemology, and the ability to think critically is largely associated with specific criteria within a domain (e.g., Barrow, 1991; McPeck, 1981, 1990b; Moore, 2004, 2011). For example, McPeck (1981, 1990b), the most prominent proponent of this view, contends that different domains involve different facts, concepts, and principles. This implies, McPeck argues, different sort of things count as good reasons in different domains, and thus there is not a set of CT skills that can be acquired and applied independent of a particular domain in question. Overall, the debate regarding domain specificity versus domain generality of CT skills has been a question of the extent to which a CT skill learned in one domain is predictive of its applicability in another. Approaches to teach CT in higher education have recently focused on a synthesis of the domain-generality and domain-specificity views of CT (Bailin et al., 1999; Davies, 2013; Jones, 2009). The synthesis view assumes the following two major points: (1) Domain-specific content knowledge is indeed necessary to competently perform a CT task as one cannot engage in CT without having the relevant knowledge base, and (2) a set of CT skills that can be taught and applied across a wide variety of domains do exist, although content and issues differ from one domain to the next.

Assessment of Critical Thinking Skills Alongside the diverse conceptualization of CT and the debates on domain specificity and domain generality of CT skills, one of the main challenges in CT instruction has been the assessment of CT skills. CT has largely been associated with everyday reasoning, and the assessment of the effectiveness of CT-supportive learning environments has mainly focused on content from everyday life, without reference to domain-specific content knowledge. Researchers employed various kinds of standardized CT tests that use a broad range of formats, scope, and psychometric characteristics to measure CT outcomes (for reviews, see Ennis, 1993). Some of the available standardized CT tests include the Cornell Critical Thinking Test (CCTT: Ennis, Millman, & Tomko, 1985), the California Critical Thinking Skills

962

D. T. Tiruneh et al.

Test (CCTST: Facione, 1990b), the Watson-Glaser Critical Thinking Appraisal (Watson & Glaser, 2002), the Ennis-Weir CT Essay Test (Ennis & Wier, 1985), and the Halpern Critical Thinking Assessment (HCTA: Halpern, 2015). The aforementioned CT tests use content from a variety of real-life situations with which test takers are assumed to already be familiar with, and they are mostly labeled as domain-general CT tests. Despite the recent shift toward the synthesis of the domain-specificity and domaingenerality views of CT, the assessment of CT has thus far mainly focused on domaingeneral CT skills. The expectation of embedding CT skills within domain-specific courses has been that it will facilitate the acquisition of CT skills that are applicable to a variety of CT tasks within the specific subject-matter domain in question and to other CT tasks beyond school subjects (e.g., everyday life situations). Successful teaching of CT skills in coherence with the teaching of domain-specific content knowledge is in other words expected to result in the development of both domain-specific and domain-general CT skills that are necessary to perform CT tasks requiring a considerable mental activity such as predicting, analyzing, synthesizing, evaluating, reasoning, etc. However, the experience of measuring domain-specific CT skills has not been well valued. A few researchers developed and validated CT tests based on content from specific subject-matter domains: the psychological CT assessment in the domain of psychology (Lawson, 1999), the biological CT exam (McMurray, 1991), and the critical thinking in electricity and magnetism test in the domain of physics (CTEM: Tiruneh, De Cock, Weldeslassie, Elen, & Janssen, 2016). The empirical evidence on whether performance in a domain-specific CT test relates to performance in one of the abovementioned domain-general CT tests is scant. In sum, the assessment of CT skills is largely viewed as separate from domain-specific content knowledge despite the growing consensus that CT skills are also domain specific (Moore, 2011; Smith, 2002), and the extent to which CT skills that are taught in a particular intervention relates to those that are assessed is unclear (e.g., Dwyer, Hogan, & Stewart, 2014).

Instructional Approaches to Teach Critical Thinking Skills in Higher Education The domain-generality versus domain-specificity views of CT have had major implications in approaches to teach CT skills (e.g., Bensley, Crowe, Bernhardt, Buckner, & Allman, 2010; Davies, 2013; Ennis, 1989; Jones, 2009; Moore, 2011; Resnick, 1987; Salomon & Perkins, 1989; Willingham, 2007). The generalists contend that as long as there is a set of CT skills that can be taught and applied across domains, those CT skills can be taught either separately from domain-specific instruction (e.g., in a stand-alone course) or can be embedded explicitly within domain-specific instruction. However, the notion of embedding CT instruction within specific domains has aroused considerable controversy among researchers over the past couple of decades (e.g., Barrow, 1991; Ennis, 1989; Halpern, 1998; Kuhn, 1999; McPeck, 1990b; Perkins & Salomon, 1989; Willingham, 2007). The main controversy lies on whether CT skills need to be taught explicitly within domain-specific instruction or not. On the

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

963

one hand, some proponents of the domain-specificity view (e.g., Barrow, 1991; McPeck, 1990a, 1990b; Moore, 2011) argue that meaningful instruction in any subject domain inherently involves the development of CT skills. In other words, proficiency in CT skills can be achieved as students construct knowledge of a subject-matter domain without any explicit teaching of selected CT skills during domain-specific instruction. The following examples may illustrate the line of thought of the specifists: to learn a body of content within a domain (e.g., physics) implies learning to think critically in the domain of physics, to learn history implies learning to think critically in the domain of history, and so on. We say students learn well physics if they solve novel physics problems that involve the ability to think critically. This suggests that subjectmatter instruction in every domain involves the acquisition of CT skills without investing additional instructional time on the teaching of CT skills explicitly as such. According to specifists, therefore, CT instruction should take place as an integral part of instruction in specific domains without any explicit emphasis on the so-called domain-general CT skills. McPeck (1990a) particularly assumes that such domainspecific instructional practice can empower students to perform a different kind of CT tasks in another domain as long as they have sufficient domain-specific knowledge of the domain in question. Ennis (1989) refers to such instructional strategies as an immersion CT instructional approach. On the other hand, some scholars (e.g., Beyer, 2008; Davies, 2013; Halpern, 1998, 2014; Kuhn, 1999) argue that explicit emphasis on CT skills in specific subject-matter instruction is essential to facilitate both the acquisition and transfer of CT skills. Following the domain-generality view, those scholars argue that transfer of CT skills from one domain to the next seldom occurs spontaneously, but it needs to be guided and primed explicitly by an instructional agent in domain-specific instruction (e.g., Ennis, 1989; Halpern, 2014; Perkins & Salomon, 1989). Ennis (1989) refers to such explicit instructional strategies on CT skills within specific subject-matter instruction as an infusion CT instructional approach. It seems evident from the above review that the field of CT research has been struggling with fundamental and sometimes controversial issues with regard to the conceptualization, teaching, and assessment of CT. Relating to these issues, we identify some limitations from the available CT empirical evidence on the integration of CT instruction in domain-specific courses. First, there seems to be a lack of clarity and transparency in current CT theoretical and empirical research with respect to identifying desirable CT outcomes. It is hardly possible to find answers from the empirical literature to the following questions: How is CT defined? What are the CT skills desired in a typical CT-targeted instructional intervention? And how are the activities designed in an intervention tailored toward helping students acquire targeted CT skills? It appears CT instruction does not build upon a clear conception of CT. Second, it appears that most previous attempts to embed CT instruction in specific domains do not systematically build on theoretically sound and empirically valid instructional design principles, and thus the features of CT-supportive learning environments have been insufficiently clear. More specifically, the potential benefits of instructional design principles in creating effective immersion- or infusion-based learning environments seem to be largely ignored. The concepts “immersion” and

964

D. T. Tiruneh et al.

“infusion” appear to be underspecified in various CT empirical studies. An immersion approach to CT instruction seems to be operationalized in a number of empirical studies analogous to “traditional” instruction, which is normally characterized as less engaging, highly dominated by the teacher, limited collaboration among students, little practice in answering higher-order thinking questions, etc. (e.g., Bensley et al., 2010; Reed & Kromrey, 2001). An infusion approach seems to be operationalized largely as a mere addition of explicit teaching of general CT skills in a specific subject-matter instruction that is not optimally designed (e.g., Bensley & Spero, 2014; Solon, 2007). Overall, immersion and infusion CT instructional approaches appear to remain underspecified in the CT literature, and hence we do not have yet sufficient and valid empirical evidence to draw any conclusions about the effectiveness of these two CT instructional approaches in stimulating the acquisition of domain-specific and domain-general CT skills. How do we then best implement immersion and infusion CT instructional approaches? This question guided our two empirical studies that are described below.

Design, Implementation, and Evaluation of an Immersionand Infusion-Based Critical Thinking Instructional Interventions We illustrate below the main findings of our two empirical studies that aimed to examine the effects of a systematic and model-based approach to design learning environments in fostering the learning of domain-specific and domain-general CT skills. Particularly, we examined how immersion- and infusion-based learning environments can be designed, implemented, and evaluated in view of the development of both domain-specific and domain-general CT skills. In the following subsections, we describe the procedures involved in designing, implementing, and evaluating the learning environments. Step 1: Operationalize CT Our first task in the design process was operationalizing what we mean by CT. Acknowledging the lack of consensus on the definition of CT, we adopt Halpern’s (2014) definition: “Critical thinking is the use of those cognitive skills or strategies that increase the probability of a desirable outcome” (p. 8). Based on her analysis of the various CT definitions, Halpern (2014) identifies five major categories of CT skills: reasoning, hypothesis testing, argument analysis, likelihood and uncertainty analysis, and decision-making and problem-solving. These categories of CT skills were the focus of our immersion- and infusion-based instructional interventions. See Table 1 for a description of the five categories of CT skills. Moreover, as argued earlier, we view CT from a domain-specific and domaingeneral perspective. From a domain-specific perspective, it is assumed that domainspecific content knowledge is a necessary condition to competently perform a CT task as one cannot engage in CT without having the relevant knowledge base.

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

965

Table 1 Description of the CT skills targeted in the empirical studies CT dimension Reasoning Hypothesis testing Argument analysis Likelihood and uncertainty Problem-solving and decision-making

Description The skills needed to identify and avoid persuasive or misleading information contained in a variety of contexts The skills needed to make observations and to formulate, test, and evaluate the validity of hypotheses The skills needed to elaborate and analyze the soundness and strength of personal and external arguments The skills needed to predict probability of success and failure in everyday decision-making in a wide variety of contexts The skills needed to identify and generate creative and alternative courses of action to make decisions and solve problems

However, some CT tasks may rather require domain-specific expertise in order to solve them competently (e.g., one may need to have knowledge of medicine to analyze arguments specific to medical science). Therefore, we operationalize domain-specific CT skills as those CT skills acquired in domain-specific instruction and utilized to solve CT tasks within the boundaries of the domain in question. Unlike specifists, however, our use of the term “domain-specific CT skills” does not necessarily suggest that a CT skill employed to competently solve a domain-specific CT task just applies to that specific domain only. Rather, we are referring to the fact that a CT task may require domain-specific expertise for it to be competently performed. From a domain-general perspective, we also assume that CT tasks across domains share significant commonalities (e.g., analyzing medical information, analyzing psychological information, analyzing physics experimental graph, etc.). It is therefore assumed that CT skills acquired in domain-specific instruction (e.g., physics) may facilitate one’s competency to solve CT tasks in other domains (e.g., psychology). In this contribution, we view transfer as a situation where a CT skill acquired in a domain-specific instruction informs how a CT task is solved in a different domain. In view of this conception of transfer, CT skills that transcend the domain in which they were initially introduced and make desirable outcomes more likely in daily life settings are referred to as domain-general CT skills. Step 2: Identify Desired Domain-Specific and Domain-General CT Competencies Following our argument that most previous empirical attempts to embed CT instruction in specific domains lack transparency, efforts were made at the initial stage to specify the domain-specific and domain-general CT outcomes learners are expected to demonstrate when they have completed a CT-targeted instruction (see Table 2). We acknowledge that there is not a single set of CT skills that every CT-supportive instruction would target, but it was assumed that specifying desired domain-specific and domain-general CT competencies at the initial stage could serve as an important basis to decide on the learning activities and assessment measures to be implemented at a later stage.

966

D. T. Tiruneh et al.

Table 2 Description of desired domain-specific and domain-general CT competencies CT skill Reasoning

Thinking as hypothesis testing

Domain-specific CT competency In the context of E&M, the student will be able to: Recognize ambiguity of terms Recognize errors of measurement Interpret the results of an experiment Identify important relationships Examine the adequacy of observations/samples/repetitions of an experiment to draw a conclusion Check for adequate sample size and possible bias in sampling when making a generalization

Argument analysis

Identify the key parts of an argument on issues related to E&M Infer a correct statement from a given data set Criticize the validity of generalizations drawn from the results of an experiment

Likelihood and uncertainty analysis

Predict the probability of events (but understand the limits of extrapolation) Identify assumptions (e.g., recognize what assumptions have to be maintained in generalizations drawn from experimental results) Understand the need for additional information in making decisions Make valid predictions Identify the best among a number of alternatives in solving E&M-related problems Examine the relevance of procedures in solving scientific problems Evaluate solutions to an E&Mrelated problem Make sound, evidence-based decisions Use analogies to solve E&M-related problems

Problemsolving and decisionmaking

Domain-general CT competency In the context of everyday situations, the student will be able to: Recognize ambiguity of terms Evaluate/analyze ideas from different perspectives Identify cause and effect relationships Recognize the need for more information in order to make valid conclusions Examine the adequacy of observations/samples/repetitions before drawing a conclusion Identify the key parts of an argument: e.g., given a conclusion, identify the reason(s) that support the conclusion Infer a correct statement from a given data set Criticize the validity of generalizations Understand the probability and likelihood of an event occurrence Identify assumptions Make valid predictions: what-if questions Understand the need for adequate sample sizes

Identify the best option from a number of alternatives in solving everyday problems Decide on the validity of a particular scientific explanation when applied to new situations Examine the relevance of the procedures in solving problems Use analogies to solve problems Develop reasonable, creative solutions to a problem

Step 3: Select a Relevant Instructional Design Model The third important task was identifying a relevant instructional design model that could guide the design and development of the planned instructional interventions. There are several instructional design models that offer explicit guidelines for the design of learning environments toward student’s constructive acquisition of

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

967

knowledge and skills (Jonassen, 1999; Merrill, 2002, 2013; van Merriënboer, 1997). The First Principles of Instruction model was selected to guide the design and development of the immersion- and infusion-based learning environments because of its comprehensiveness, strong theoretical foundation, and suitability to facilitate the development of complex learning outcomes.

The First Principles of Instruction Model in Brief The First Principles of Instruction model comprises five empirically validated instructional design principles which emerged from research on subject-matter teaching and offers concrete guidelines to design learning environments for the acquisition of higher-order learning outcomes: problem-centered, activation, demonstration, application, and integration principles (Merrill, 2002, 2013). A conceptual framework that relates each of the instructional principles is shown in Fig. 1. This model advocates the use of authentic and contextually relevant learning tasks in subject-matter instruction and aims to provide students with a variety of learning activities that facilitate the active and constructive acquisition of knowledge and skills (Merrill, 2002, 2013). Taking a moderate constructivist view on learning, Merrill claims that subject-matter instruction designed systematically in accordance with the First Principles of Instruction model can result in effective, efficient, and engaging learning, which leads to student acquisition of the knowledge and skills necessary to competently perform complex tasks (Merrill, 2013). Below is a concise description of the instructional principles included in the model. (a) Problem-centered principle: This principle states that learning is promoted when learners acquire knowledge in the context of real-world problems. Instead of a topic-by-topic instructional approach, this principle advocates that getting

2. ACTIVATION Learning is promoted when learners activate relevant prior knowledge as a foundation for new knowledge.

5. INTEGRATION Learning is promoted when learners reflect on, discuss and defend their newly acquired knowledge. 1. PROBLEM-CENTERED

Learning is promoted when learners acquire knowledge in the context of real-world problems. 4. APPLICATION Learning is promoted when learners apply their newly acquired knowledge and skills in solving novel problems.

Fig. 1 First Principles of Instruction model (Merrill, 2013, p. 22)

3. DEMONSTRATION Learning is promoted when learners observe a demonstration of the knowledge and skills to be learned.

968

(b)

(c)

(d)

(e)

D. T. Tiruneh et al.

students involved with realistic (whole) problems will help them form appropriate mental models that facilitate application of newly acquired knowledge and skills to novel situations. Activation principle: This principle states that learning is promoted when learners activate prior knowledge as a foundation for new knowledge and skill. If learners have had relevant experience, then the first phase of learning is to be sure that this relevant information is active and ready for use as a foundation for the new knowledge. If learners have not had sufficient relevant experience, then the first phase of learning a new content element should be to provide hands-on experience that they can use as a foundation for the new knowledge. Demonstration principle: This principle states that learning is promoted when learners observe a demonstration of the knowledge and skills to be learned. Merrill emphasized that instruction should not be merely telling or presenting information to learners but should also involve showing learners what to do to apply this information to specific situations. Application principle: This principle states that learning is promoted when learners apply their newly acquired knowledge to solve new problems, and when they receive corrective feedback and are properly coached, with coaching gradually withdrawn for each subsequent problem. Integration principle: Merrill stated that instruction needs to provide learners an opportunity to reflect on how the newly acquired knowledge and skills relate to what they already know, explain the acquired knowledge and skills to others, and defend what they know when it is challenged.

Several studies showed that designing subject-matter instruction based on the First Principles of Instruction model results in significant improvement on course achievement (e.g., Francom, Bybee, Wolfersberger, Mendenhall, & Merrill, 2009; Francom, Bybee, Wolfersberger, & Merrill, 2009; Gardner, 2011; Gardner & Belland, 2012). To our knowledge, the effectiveness of the model in promoting the development of CT is not yet investigated.

Study 1: Systematic Design, Implementation, and Evaluation of an Immersion-Based Learning Environment The development of CT is largely explored through loosely defined instructional interventions that consist of teaching general CT skills within less optimally designed subject-matter instruction (Tiruneh, Verburgh, & Elen, 2014). Research attempts to embed the teaching of CT skills within subject-matter instruction have not systematically built on instructional design research, and the link between the acquisition of domain-specific and domain-general CT skills appears to be vague. The first study investigated the effectiveness of a systematically designed immersion-based learning environment on the acquisition of both domain-specific and domain-general CT skills. The following research question is addressed: What is the effect of systematically

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

969

designed immersion-based learning environment on the acquisition of domain-specific and domain-general CT skills compared to a regular subject-matter instruction? It was hypothesized that a systematically designed immersion-based instructional intervention would significantly promote the learning of both domain-specific and domaingeneral CT skills compared to a regular subject-matter instruction.

Methodology Participants of the Study Study participants were first-year students with physics majors at two universities in Northwest Ethiopia. Students at one of the universities formed the experimental group (n = 45), while those at the other university constituted the control group (n = 44). The experimental group was comprised of 24 women and 21 men between the ages of 19 and 23 years (M = 20.09, SD = 0.93), while the control group consisted of 23 women and 21 men between the ages of 19 and 24 years (M = 20.32, SD = 0.98). Design and Implementation of the Immersion-Based Learning Environment The intervention focused on a freshman introductory physics course, namely, introductory electricity and magnetism (E&M). This course was taught at both universities based on a harmonized national curriculum, with the same content and credit hours. The intervention focused only on the first five chapters of the course: electric field, electric flux, electric potential energy, capacitor and capacitance, and direct current circuits (as specified in the course textbooks of the two universities). In line with the First Principles of Instruction model, instructional activities for the immersion-based E&M lessons included the following: (a) engaging students to solve various types of authentic and contextually relevant E&M tasks; (b) activating students’ prior knowledge related to E&M topics; (c) modeling, coaching, and feedback by the course teacher; (d) providing challenging and sequenced learning tasks that could create several opportunities for the students to engage in applying newly presented information; and (e) encouraging students to interact in small-group discussions and reflect their opinions on various topics. An interdisciplinary team of researchers and two regular E&M teachers collaborated in designing the immersionbased E&M instructional intervention. Efforts were made to embrace the desired CT skills as part of the regular E&M instructional activities during the design process. At the beginning of each chapter, the experimental students were given authentic and contextually relevant E&M problems that required them to collaborate to find solutions. Throughout the intervention, students were made to observe well-scripted instructor demonstrations that modeled the important procedures and reasoning involved in solving various E&M problems. The demonstrations were followed by multiple opportunities for the students to practice solving E&M problems both individually and in small groups. A number of activities that encouraged students to activate their prior knowledge and

970

D. T. Tiruneh et al.

communicate their ideas to both their groups and the entire class were carefully designed and implemented. Both peer and instructor feedbacks were provided as needed. The first author monitored the overall implementation of the intervention, which lasted 8 weeks. Three lessons of 2 h each were taught every week. Students in the control condition followed the E&M instruction designed by the regular instructor. This group was similar to the immersion group in terms of previous course and parallel courses enrollment during the intervention. However, the E&M lessons for this group were not designed according to the First Principles of Instruction model, and we refer to the instructional method in the control class as “regular” E&M instruction. The lesson duration for this group was the same as that for the immersion group: a total of 8 weeks with three lessons of 2 h each per week. See Table 3 for a detailed comparison of the immersion and control learning environments.

Instruments The Halpern Critical Thinking Assessment (HCTA: Halpern, 2015) was administered both as a pretest and a posttest to measure students’ domain-general CT competency. The HCTA focuses on the five categories of CT skills targeted in the intervention and consists of 20 items based on a variety of real-life problems such as health, education, politics, and social policy. Each item is followed by questions that require respondents to provide brief constructed responses (constructed-response items) and to subsequently select answers from a short list of alternatives (forcedchoice items). The internal consistencies for both formats of the HCTA in the present study were acceptable based on the guidelines by Nunnally (1978): Cronbach’s α = 0.74 for the posttest HCTA constructed-response and 0.72 for the posttest HCTA forced-choice formats. The critical thinking in electricity and magnetism test (CTEM: Tiruneh et al., 2016) was administered to measure students’ domain-specific CT competency. The test consists of 20 items: two of which are forced-choice and the remaining are constructed-response format items. The test authors designed the CTEM items to mirror the five categories of CT skills identified in the HCTA, but focus on E&M content (see Fig. 2 for sample HCTA and CTEM items). The CTEM was administered in the present study as posttest only. Because the test requires prior knowledge of E&M, we felt it was reasonable to administer the test only at the end of the intervention. In return, however, the grade 12 university entrance national exam scores for physics were used to control for physics prior knowledge of the study participants. The internal consistency of the CTEM (Cronbach’s α = 0.73) for the present study was found to be acceptable (Nunnally, 1978). Results Domain-Specific CT Competency: CTEM Initial comparison of prior physics knowledge revealed no significant differences between the experimental and control group, t(87) = 0.15, p = 0.88. An independent sample t-test was therefore conducted to compare the performance of the two groups on the domain-specific CT test. The results indicated that the CTEM mean score

Activation principle

Instructional principle Problemcentered principle

Learning is promoted when learners activate existing knowledge and skills as a foundation for new knowledge and skills. Instruction should not begin from abstract representations that learners require more background in

(continued)

Design and implementation of the control learning environment Instruction was primarily “topic-centered.” at the beginning of a new chapter, the instructor presented information related to that chapter (or subtopic). Students were sometimes shown solutions to one or two textbook problems related to the newly presented information. At the end of the lesson, students were given selected textbook problems as homework assignments. Overall, the lessons were not designed to echo real-world problems. Comprehensive problems with real-world significance that might prompt students’ CT skills were not introduced at the beginning of a chapter There were no systematic or adequate attempts to activate learners’ prior knowledge before information on a new topic was presented. When a lesson on a new topic began, the instructor usually started by briefly explaining the topic and subsequently presenting detailed information on the topic. The instructor sometimes began a new lesson by revisiting the content of a previous lesson. In most cases, the instructor himself revisited the previous lesson rather than asking students to do this. Sometimes, the instructor encouraged students to tell him what they remembered of the previous lesson, but no further prompts were offered to help students describe the preceding lessons in detail

Table 3 Comparison between the experimental and control learning environments in relation to the First Principles of Instruction model Design and implementation of the experimental learning environment For each chapter, relatively complex, meaningful, and comprehensive problems were carefully designed by seeing each chapter as a minicourse (based on the suggestion by Merrill, 2013). An attempt was made to keep the tasks relevant to the lives of students and thus make them more motivating. A whole task for a particular chapter was given 1 or 2 days before the instruction began; students were subsequently asked to answer the questions in the whole task by referring to the course textbook or consulting experts (or senior students with physics majors) Various activities that helped learners make meaningful connections between newly acquired and their prior knowledge were carefully prepared in advance and implemented during instruction. For example, learners received questions about a specific topic that aimed to relate the concepts of the new topic to their prior knowledge, and they were required to share their answers with other learners (peer sharing)

Toward a Systematic and Model-Based Approach to Design Learning. . .

Description of the principle Learning is promoted when learners acquire knowledge and skills in the context of real-world problems. Problems need to be comprehensive, challenging, and representative of the problems learners will encounter in real life

38 971

Application principle

Instructional principle Demonstration principle

Description of the principle Learning is promoted when learners observe a demonstration of the knowledge and skills to be learned that is consistent with the type of content being presented. This principle pertains to presenting necessary concepts and facts of the domain in question and modeling how the newly presented concepts and facts could be applied in solving various tasks Learning is promoted when learners apply newly acquired knowledge and skills on tasks that are consistent with the type of content being taught. Learning from an application is effective when learners receive corrective feedback and are properly coached, with the coaching gradually withdrawn with every subsequent problem

Table 3 (continued)

Relevant and challenging E&M tasks were designed that created multiple opportunities for the students to engage in applying newly presented information with an implicit emphasis on the desired CT skills. When students were engaged in solving problems, activities that facilitated instructor coaching and guidance were clearly described and implemented. For example, the instructors provided partial solutions, halted at each group and observed students’ discussions, provided hints as needed, acted as group members and asked thought-provoking questions, encouraged students to formulate questions using specific verbal prompts, and facilitated discussion among group members

Design and implementation of the experimental learning environment We mainly changed the textbook’s standard numerical E&M problems into more qualitative/ conceptual problems. An attempt was made to qualify the tasks so that the desired CT skills could implicitly be modeled by the instructor when presenting the solutions (e.g., what can we conclude from the answer? What other options are there to solve this problem? What other information do we need to solve this problem? Etc.)

Students mostly listened to the instructor and took notes. They were not engaged in applying the newly presented information to solve new and meaningful E&M problems; rather the instructor gave them homework assignments to practice solving the traditional end-ofchapter problems. Moreover, there was no dedicated time for students to practice solving as many practical and comprehensive questions as possible during the lessons. Even when they were asked questions, the questions focused on recalling information and did not invite further elaboration and explanations from the students. Group activities took place during some of the sessions. However, the activities for small-group activities were not adequately and purposely designed. The instructor did not adequately coach the group activities and feedback was limited

Design and implementation of the control learning environment A great deal of information was presented, but was on telling rather than both telling and showing the information. After he introduced a lesson, the instructor presented detailed information on the topic, but he did not adequately show how the presented information might be used to solve a new problem. Tasks that might have facilitated the demonstration of the newly presented information were not systematically designed in advance

972 D. T. Tiruneh et al.

Integration principle

Learning is promoted when learners integrate their new knowledge into their everyday lives by being required to reflect on, discuss, or defend their newly acquired knowledge or skills via peer collaboration and peer critique. Instruction should provide learners an opportunity to reflect on how the newly acquired knowledge and skills relate to what they already know, to explain the learned knowledge and skills to others, and to defend what they know when challenged Activities that encourage students to present their solutions either to group members or full class were designed, and both peer and instructor feedback were offered. At the end of each chapter, a 2-h tutorial session was organized. The sessions mainly focused on revising the main topics of each chapter by asking students to prepare a summary of the facts and concepts discussed in the chapter and solving a few E&M problems. Students were required to attempt to solve all the problems in advance. During these sessions, students were asked to discuss their solutions in their respective groups, and the tutors acted as coaches during the group activities. Representatives from at least two groups were asked to present solutions to a particular question in front of the full class. Students in other groups were encouraged to ask questions, and the student presenters were asked to defend their solutions when challenged by their classmates or the instructors Students usually did not have the opportunity to present and defend their solutions to the full class. Interaction between the students during the lessons was very limited: They did not engage in exchanging ideas and explaining solutions to problems between themselves and to the instructor. At the end of each chapter, a 2-h tutorial session was arranged so that students could solve exercises in groups. The regular instructor and his assistant provided assistance to the students during the tutorial sessions. In most cases, however, the tutorial questions did not encourage students to apply what they had learned to solve new and meaningful problems. The questions usually promoted retention of information

38 Toward a Systematic and Model-Based Approach to Design Learning. . . 973

974

D. T. Tiruneh et al.

Sample HCTA item Four patients were waiting to see a doctor who specializes in treating headaches. Three of the four patients were women, which led the male patient to declare that more women seek medical help for their headaches than men. A. Is this a reasonable conclusion based on the people waiting to see this doctor? B. Please explain your answer ___________________________________________________________ ___________________________________________________________ Sample CTEM item The electrical resistivity of different materials is measured as a function of temperature. The results are given in the table below. Temperature

Resistivity

Resistivity

Resistivity

Resistivity

(K)

Aluminum

Gold

Iron

Copper

1 10 100 200 300 400 500 600 700

(10-8 Ωm) 0.0001 0.000193 0.442 1.587 2.733 3.87 4.99 6.13 7.35

(10-8 Ωm) 0.0220 0.0226 0.65 1.462 2.271 3.107 3.97 4.87 5.82

(10-8 Ωm) 0.0225 0.0238 1.2800 5.2000 9.9800 13.1000 23.7000 32.9000 44.0000

(10-8 Ωm) 0.002 0.00202 0.348 1.046 1.725 2.402 3.09 3.792 4.514

Based on these measurements, can you conclude that ‘resistivity increases with increasing temperature’? Explain your answer. ________________________________________________________________ ________________________________________________________________ Fig. 2 Sample HCTA and CTEM items

for the experimental group was significantly higher than that of the control group, t(87) = 7.15, p < 0.001, d = 1.55. The effect size for this analysis was found to exceed Cohen’s (1988) convention for a large effect (d = 0.80). An analysis of covariance (ANCOVA) was conducted to examine whether the statistically significant mean score differences could be maintained after controlling for physics prior knowledge. The ANCOVA results showed that the CTEM mean score of the experimental group was significantly higher than that of the control group, F(1, 86) = 52.56, p < 0.001, Ƞ2 = 0.379. The results indicated that the intervention accounted for 37.9% of the variance in the acquisition of domainspecific CT skills. See Table 4 for descriptive statistics of the CTEM test.

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

975

Table 4 Descriptive statistics for experimental and control groups: prior knowledge, CTEM, and HCTA scores Prior knowledge Group M (SD) Experimental 40.93 (8.21) Control 41.18 (7.38)

CTEM Pretest HCTA M (SD) Range M (SD) Range 31.56 (5.44) 21 71.49 (4.76) 19 24.57 (3.57) 14 70.36 (4.11) 17

Posttest HCTA M (SD) Range 73.04 (5.21) 24 71.45 (5.12) 18

Domain-General CT Competency: HCTA In order to examine the effect of the instructional intervention on students’ domaingeneral CT performance, a 2 (groups: experimental and control)  2 (testing time: pretest and posttest) mixed design ANOVA was conducted. The results of the mixed design ANOVA revealed that the two groups together demonstrated a statistically significant improvement on the HCTA mean scores across the two time points, F(1, 87) = 4.61, p = 0.035, Ƞ2 = 0.05. The effect size value (Ƞ2 = 0.05) suggested a small practical significance. However, there was no significant interaction between the intervention type (experimental-control) and the testing time (pretest-posttest), F(1, 87) = 0.14, p = 0.71. In other words, the HCTA mean score for the experimental group did not show a significant pretest-posttest improvement compared to the control group. This indicates that the experimental learning environment did not result in a significantly greater pretest-posttest improvement in the acquisition of domain-general CT skills compared to the control learning environment. The descriptive statistics of the HCTA scores are shown in Table 4.

Discussion 1 The findings revealed that an immersion-based instructional intervention that consisted of contextually relevant E&M tasks, activation of prior knowledge, teacher modeling, small-group discussion, coaching, and feedback resulted in a significantly higher acquisition of domain-specific CT competency than the regular instruction (as measured by the CTEM). However, there was no significant difference between the two groups on the acquisition of domain-general CT skills (as measured by the HCTA). The finding with regard to domain-specific CT competency suggests that the systematic design of domain-specific instruction based on empirically valid instructional design principles promotes the learning of domain-specific CT skills. This implies that acquisition of domain-specific CT skills can be improved through systematic design of subject-matter instruction without explicit teaching of general CT skills. However, the immersion-based learning environment did not result in a statistically significant improvement on domain-general CT skills compared to the control learning environment. Gains in domain-specific CT competency found in the experimental condition were not accompanied by gains in domain-general CT competency. The absence of an explicit teaching of desired CT skills during subject-matter instruction was picked as one of the main possible reasons that kept students from mindfully abstracting the learned domain-specific CT skills and

976

D. T. Tiruneh et al.

applying them in solving domain-general CT tasks. Some proponents of an explicit CT instructional approach (e.g., Davies, 2013; Ennis, 1989; Halpern, 1998) argued that teaching CT skills explicitly within subject-matter instruction is the best way to stimulate the development of transferrable CT skills. An explicit approach may include informing students at the outset that they are being taught CT skills, encouraging students to recognize similar patterns across domain-specific tasks, modeling by an instructional agent through thinking aloud the problem-solving strategies involved in solving complex tasks, encouraging students to reflect on learned CT skills at the end of a lesson or chapter, and reminding students that they will be expected to use those learned skills to solve everyday problems or issues they will come across. The findings from our first study instigated Study 2: comparison between an immersion- and infusion-based learning environment on the acquisition of both domain-specific and domain-general CT skills.

Study 2: Comparison between Immersion- and Infusion-Based Learning Environments Building on the findings from Study 1, Study 2 examined the effects of systematically designed immersion- and infusion-based learning environments on domainspecific and domain-general CT competencies compared to a regular learning environment. Particularly, the following research question was addressed: What are the effects of systematically designed immersion- and infusion-based learning environments on domain-specific and domain-general CT competencies, compared to a regular subject-matter instruction? Our first hypothesis was that systematically designed immersion- and infusion-based instructional interventions would result in a significantly higher acquisition on domain-specific and domain-general CT competencies compared to the regular condition. Because CT skills were explicitly taught in the infusion condition, our second hypothesis was that participants in the infusion condition would significantly outperform those in the immersion condition on domain-general CT competency.

Study Participants Similar to Study 1, the study participants (N = 147) were first-year students at two universities in Northwest Ethiopia with majors in physics, chemistry, or geology, and enrolled in an introductory E&M course. The physics majors at University 1 were purposely assigned into an infusion group (infusion-physics: n = 33) and the physics majors at University 2 into a control group (n = 42). Both the chemistry (n = 30) and geology (n = 42) majors were at University 1, and each of them was randomly split into two equal groups. Half of the chemistry and geology majors were combined and constituted one group (chem-geo-1: n = 36), and the remaining half of each major formed another group (chem-geo-2: n = 36). These two groups were randomly assigned to an infusion (infusion-chem-geo: n = 36) and immersion conditions (immersion-chem-geo: n = 36). Three participants from the control group and one from the immersion had to be excluded because of missing posttest data, leaving a final total sample of 143 students.

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

977

Description of the Immersion- and Infusion-Based Learning Environments Similar to Study 1, we initially identified a list of desired domain-specific and domaingeneral CT competencies that our study participants were expected to achieve at the end of the interventions (see Table 2). The First Principles of Instruction model (Merrill, 2013) was used as a framework to guide the design of the immersion- and infusion-based E&M learning environments. Both immersion and infusion instructional approaches equally focused on helping students develop deep understanding of E&M content. The immersion-based E&M instruction engaged students in various domain-specific instructional activities that could result in the achievement of desired domain-specific and domain-general CT competencies, but without any explicit teaching of general CT skills. In the case of the infusion-based E&M instruction, however, an explicit emphasis on the desired CT skills was included as an additional layer to the immersion-based intervention. An interdisciplinary team of researchers and two regular E&M teachers collaborated in designing the immersion and infusion E&M interventions. Description of the Control Learning Environment As mentioned in Study 1, the content and lesson durations for the control condition in Study 2 were the same as that for the immersion and infusion conditions. Efforts were made also to carefully control students’ time on task as far as the E&M course was concerned. It should be noted that the immersion and infusion groups were required to solve comprehensive E&M problems ahead of the first lesson in each chapter and submit brief reports. To counterbalance the time on task, students in the control group were in return given reading assignments of selected topics a few days prior to the beginning of each chapter, and they were required to submit summary reports during the first lesson of each chapter. To obtain an overview of the instructional processes, the first author observed two of the control group’s lessons, and interviews were conducted with the E&M teacher on three separate occasions: at the beginning of the semester, a month after the semester started, and at the end of the intervention. A detailed analysis of the classroom observations and interview data revealed the precise differences of the regular E&M instruction with respect to the First Principles of Instruction model (see Table 5 for a detailed description of the differences between the immersion, infusion, and control learning environments). Similar to Study 1, both the CTEM and HCTA (as pre- and posttests) were administered to measure domain-specific and domain-general competencies, respectively. The internal consistency of the CTEM (Cronbach’s α = 0.73, N = 143) for the present study was found to be acceptable (Nunnally, 1978). The internal consistencies for both formats of the HCTA in the present study were acceptable based on the guidelines by Nunnally (1978): Cronbach’s α = 0.72 for the pretest constructed-response and 0.71 for the pretest forced-choice formats, N = 147; 0.74 for the posttest constructed-response and 0.72 for the posttest forced-choice formats, N = 143.

Activation principle

The design principle Problemcentered principle

Various activities that helped learners make meaningful connections between newly acquired and their prior knowledge were carefully prepared in advance and implemented during instruction. For example, learners received questions about a specific topic that aimed to relate the concepts of the new topic to their prior knowledge, and they were required to share their answers with other learners (peer sharing)

Immersion-based E&M instruction For each chapter, relatively complex, meaningful, and comprehensive problems were carefully designed by seeing each chapter as a minicourse (based on the suggestion by Merrill). An attempt was made to keep the tasks relevant to the lives of students and thus make them more motivating

The instructional activities involving the activation principle were the same in the infusion condition

Add-ons to the infusion E&M instruction For about 20 min, students were introduced explicitly at the beginning of the course regarding the desired CT outcomes. The orientation mainly focused on explaining what it means to think critically and a brief introduction of the five targeted CT skills

Regular E&M instruction Instruction was primarily “topic-centered.” at the beginning of a new chapter, the teacher presented information related to that chapter (or subtopic). Students were sometimes shown solutions to one or two textbook problems related to the newly presented information. At the end of the lesson, students were given selected textbook problems as homework assignments. The E&M problems were not designed to echo real-world problems There were no systematic attempts to activate learners’ prior knowledge before information on a new topic was presented. When a lesson on a new topic began, the instructor usually started by briefly explaining the topic and subsequently presenting detailed information on the topic. Sometimes, the teacher encouraged students to tell what they remembered of the previous lesson, but no further prompts were offered to help students describe the preceding lessons in detail

Table 5 Description of the immersion, infusion, and regular E&M instructions in relation to the First Principles of Instruction model

978 D. T. Tiruneh et al.

Application principle

Demonstration principle

We mainly changed the textbook’s standard numerical E&M problems into more qualitative/conceptual problems. An attempt was made to qualify the tasks so that the desired CT skills could implicitly be modeled by the instructor when presenting the solutions (e.g., what can we conclude from the answer? What other options are there to solve this problem? What other information do we need to solve this problem? Etc.) Relevant and challenging E&M tasks were designed that created several opportunities for the students to engage in applying newly presented information. When students were engaged in solving problems and activities that facilitated teacher coaching and guidance were clearly described and implemented. For instance, the teacher provided partial solutions, halted at each group and observed students’ discussions, provided hints as needed, acted as group members and asked thought-provoking questions, encouraged students to formulate questions using specific verbal prompts, and facilitated discussion among group members Explicit reference was made by the teacher on the targeted CT skills when modeling the solutions to various E&M problems (e.g., do I need to draw a free body diagram? What can I conclude? What can I say about the relationship between these two variables? What kind of reasoning am I making? Inductive or deductive reasoning? This is an example of inductive reasoning; this is an example of argument analysis, etc.) The instructional activities were largely similar to the immersion group, but the teacher kept the students focused on how a particular CT skill can be applied to solve the E&M problems

(continued)

A great deal of information was presented, but was on telling rather than both telling and showing the information. After he introduced a lesson, the teacher presented detailed information on the topic, but he did not adequately show how the presented information might be used to solve a new problem. Tasks that might have facilitated the demonstration of the newly presented information were not systematically designed in advance Students were not engaged in applying the newly presented information to solve new and meaningful E&M problems; rather the instructor gave them traditional end-ofchapter problems as homework assignments. Moreover, there was no dedicated time for students to practice solving as many practical E&M problems as possible during the lessons. Even when they were asked questions, the questions focused on recalling information and did not invite further elaboration and explanations from the students

38 Toward a Systematic and Model-Based Approach to Design Learning. . . 979

The design principle Integration principle

Immersion-based E&M instruction Activities that encourage students to present their solutions either to group members or the whole class were designed, and both peer and instructor feedback were offered. Representatives from groups were sometimes asked to present solutions to a particular question in front of the full class. Students in other groups were encouraged to ask questions, and the student presenters were asked to defend their solutions when challenged by their classmates or the instructors

Table 5 (continued) Add-ons to the infusion E&M instruction The instructional activities were similar to that of the immersion condition, but students in this condition were required to prepare a summary of the learned CT skills and how those skills were applied in solving the E&M problems

Regular E&M instruction Students usually did not have the opportunity to present and defend their solutions to group members or the full class. Interaction between the students during the lessons was very limited: They did not engage in exchanging ideas and explaining solutions to problems between themselves and the instructor Most importantly, the E&M problems did not usually invite students to apply what was learned in solving new and meaningful E&M problems

980 D. T. Tiruneh et al.

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

981

Implementation of the Experimental Interventions The designed interventions for all the conditions were implemented over 8 weeks with three lessons of 2 h each per week. The immersion and infusion teachers had collaborated during the design and development phases of the interventions. In order to control for the teacher effect, teachers who had the same education level and equivalent years of teaching experience were involved in implementing the interventions. The immersion and infusion teachers received all the necessary information regarding the purpose of the interventions and what they were required to do in implementing the lessons as designed, and the first author monitored the execution of the interventions. Results Despite the absence of complete random assignment of participants to the different conditions, the groups were comparable in a number of important features. First, they were all freshmen and there were no marked differences on average age. They had also similar educational backgrounds, and a one-way analysis of variance (ANOVA) revealed no significant difference between the four groups in their physics prior knowledge (as measured by the national university entrance exam for physics), F(3, 139) = 0.064, p = 0.97 and pretest HCTA scores, F(3139) = 0.191, p = 0.90. The two infusion groups, namely, infusion-physics and infusion-chem-geo had participated in exactly the same E&M instructional interventions and were taught by the same teacher. Because initial comparisons of prior physics knowledge and pretest HCTA proficiency revealed no significant differences between the four groups, we merged the infusion-physics and infusion-chem-geo groups into one infusion group on the post-intervention comparisons. The research hypotheses were tested by using Type III sums of squares, which weighs the sample means equally irrespective of differences in sample sizes (Tabachnick & Fidell, 2007). A one-way MANOVA was conducted to test the first hypothesis that there would be one or more significant mean differences between the immersion, infusion, and regular E&M instructional conditions on domain-specific and domain-general CT competencies. Using the Wilks’ statistic, there was a significant effect of the instructional conditions on the two outcome variables, Λ = 0.74, F(6, 276) = 7.31, p < 0.001. The multivariate effect size was estimated at 0.137, which implies that 13.7% of the variance in combination with the outcome variables was accounted for by the instructional interventions. The homogeneity of variance assumption was separately tested for the two outcome variables prior to conducting a series of followup ANOVAs. Based on a series of Levene’s F tests, the homogeneity of variance assumption was considered satisfied for the CTEM (F(2, 140) = 0.452, p = 0.64) and posttest HCTA (F(2, 140) = 0.31, p = 0.74). The one-way ANOVAs on the two outcome variables revealed significant intervention effects only on domain-specific CT competency, F(2140) = 13.54, p < 0.001, ηp2 = 0.162, but not on domaingeneral CT competency, F(2140) = 0.241, p = 0.79. The effect sizes associated with the statistically significant effects are considered large based on Cohen’s (1988) guidelines, with the instructional interventions accounting for 16.2% of the variance on domain-specific CT.

982

D. T. Tiruneh et al.

Table 6 Descriptive statistics for the CTEM, HCTA, and course achievement scores across the instructional conditions Group Immersion (n = 35) Infusion (n = 69) Control (n = 39)

CTEM M 31.34 33.32 27.87

SD 5.31 5.43 4.76

HCTA Pretest M 80.43 80.06 79.46

SD 9.15 7.05 8.11

Posttest M 82.17 82.80 81.85

SD 7.69 6.81 7.27

In order to examine whether there was significant pretest-posttest improvement on domain-general CT competency across the three instructional conditions, a mixed-design ANOVA was conducted. The results revealed nonsignificant interaction between the testing period (pretest-posttest) and the instructional conditions (immersion-infusion-control), F(2140) = 0.162, p = 0.85. This implies the domaingeneral CT scores for either the immersion or infusion condition did not show significant pretest-posttest improvements compared to the control condition. The descriptive statistics associated with all the variables across the three instructional groups are reported in Table 6. Comparison of the Groups on Domain-Specific CT Skills In order to examine the pairwise differences across the means of the three instructional conditions on domain-specific CT competency, the ANOVA was followed up with the Hochberg’s GT2 post hoc test. This test is selected as the sample sizes were different across groups (Field, 2009). For domain-specific CT competency, the results revealed statistically significant differences between the infusion and control groups, p < 0.001, d = 1.07, and the immersion and the control groups, p = 0.015. d = 0.69. However, the result indicated the domain-specific CT scores did not differ significantly between the immersion and infusion groups ( p = 0.196). The effect sizes associated with the statistically significant differences are considered moderate to large based on Cohen’s (1988) guidelines.

Discussion The finding regarding the immersion-based learning environment was consistent with our previous finding (Study 1). The effectiveness of the infusion-based learning environment in fostering the learning of domain-specific CT skills was also consistent with our expectation. Given that the design of our infusion-based E&M intervention was an attempt to reorient the immersion-based E&M intervention (with the inclusion of an explicit emphasis on selected CT skills during domain-specific instruction), it was indeed expected that an infusion-based learning environment would also significantly foster the learning of domain-specific CT skills compared to a regular E&M instruction. This finding is consistent in general with the CT theoretical literature that argues for the effectiveness of well-designed subject-matter instruction in enabling students to solve domain-specific thinking tasks (e.g., Glaser, 1984; Perkins & Salomon, 1989; Resnick, Michaels, & O’Connor, 2010; Smith, 2002).

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

983

Comparison between the immersion and infusion conditions on domain-specific CT competency resulted in nonsignificant differences. Both the immersion and infusion instructional conditions equally focused on students’ in-depth understanding of the E&M content (i.e., lessons were carefully designed based on the First Principles of Instruction model), and CT skills were integral components of the E&M instructional activities in both cases although CT skills were not explicitly taught for the immersion condition. We argued in the Introduction that domain-specific CT skills can and should be inherently targeted in well-designed subject-matter instruction and the fact that the immersion group demonstrated domain-specific CT competency equally to that of the infusion group was consistent with our expectation. Regarding the acquisition of domain-general CT competency, no significant improvement was demonstrated between any of the three instructional conditions. Particularly, contrary to our expectation, the infusion-based learning environment failed to significantly stimulate higher acquisition of domain-general CT competency compared to the regular E&M instruction. Because of the explicit teaching of the desired CT skills in the infusion condition, we particularly anticipated that the infusion-based instructional intervention would produce significantly higher acquisition on domain-general CT competency compared to the regular and immersion conditions. In sum, neither the immersion- nor infusion-based E&M learning environment was effective in fostering the acquisition of domain-general CT competency. A number of reasons may explain why the infusion E&M instructional condition did not result in a significant improvement on domain-general CT competency. One may be related to the design features of our infusion-based E&M intervention. The CT skills were probably not sufficiently explicit in the infusion lessons and were overshadowed by the E&M content. Perhaps the duration of the intervention (8 weeks) was insufficient to produce CT skills that can transfer across domains.

General Discussion It is argued in this chapter that CT requires complex learning. In line with this view, our assertion has been that learning environments that may be successful for simple learning outcomes may not work well for the development of complex learning such as CT. It was particularly argued that the teaching of CT is largely taking place in learning environments that are far from optimal. Accordingly, systematic and modelbased immersion and infusion CT-supportive learning environments were designed and implemented, and their effects were evaluated for domain-specific and domaingeneral CT competencies. The findings revealed that both immersion- and infusionbased learning environments foster the acquisition of domain-specific CT skills. However, neither of the learning environments was helpful in fostering the acquisition of domain-general CT skills. Given the positive outcome of the systematically designed learning environments in stimulating the learning of domain-specific CT skills, our research has brought to light that the teaching of domain-specific CT skills can be approached from an instructional design perspective: (a) identify the desired domain-specific and

984

D. T. Tiruneh et al.

domain-general CT competencies students are expected to demonstrate at the end of a domain-specific instruction; (b) design and develop domain-specific instructional activities in accordance with empirically valid instructional design principles known to promote complex learning; (c) take into account desired CT skills during the design, development, and implementation of domain-specific instruction; and (d) evaluate the effectiveness of CT-targeted learning environments by administering domain-specific and domain-general CT tests that correspond to targeted CT skills. As argued earlier, such a systematic approach to CT instruction is largely missing in the CT literature, and we hope the procedures and findings we were able to demonstrate in the two empirical studies strengthen the view that CT instruction can go hand in hand with domain-specific instruction.

Some Considerations for the Design of CT-Supportive Learning Environments Our review of the literature and the two empirical studies have allowed us to identify some of the potentials of a systematic and model-based approach to design learning environments toward stimulating the development of domain-specific and domaingeneral CT competencies. Based on our findings, the following are some of the points to be considered to design CT-supportive learning environments pertaining to the First Principles of Instruction model. 1. CT requires complex learning, and thus, nurturing the development of domainspecific and domain-general CT competencies within specific subject-matter instruction involves some special requirements with respect to identifying and implementing relevant instructional methods for complex learning outcomes. Some gains in CT can be observed when designers and teachers systematically and deliberately teach for CT in domain-specific course instruction. 2. There is no single set of CT skills that subject-matter domain instruction may always target, but the design of CT-supportive learning environments needs to always begin by clearly describing the domain-specific and domain-general CT competencies learners are expected to demonstrate when they have completed specific subject-matter instruction. 3. Learning environments for CT need to be designed systematically based on theoretically sound and empirically valid instructional design principles that are relevant to foster the learning and transfer of CT skills. This means that designers of any domain-specific instruction, who target student learning of transferrable CT skills, need to make an explicit and systematic emphasis on desired CT skills from design to implementation. In line with the First Principles of Instruction model, the following prescriptions are specified to design either an immersion- or infusion-based learning environment capable of nurturing CT development: 3.1. The problem-centered principle of the model states that learning is promoted when learners acquire knowledge and skills in the context of real-life tasks. We suggest that embedding CT instruction within subject-matter

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

985

instruction in any domain should involve designing authentic and contextually relevant learning tasks that can facilitate the acquisition and transfer of CT skills. Situating subject-matter instruction in such learning tasks is expected to promote the initial acquisition of CT skills and eventually facilitates the transfer of what is learned in the school setting to daily life and work settings. Moreover, learners’ prior knowledge needs to be taken into consideration in designing learning tasks: either activating it if learners have already acquired prior knowledge or providing bridging information if learners lack it. 3.2. The demonstration principle states that learning is promoted when learners observe a demonstration of the knowledge and skills to be learned. Following this principle, we suggest that teacher modeling through thinking aloud of desired CT skills while demonstrating solutions to domain-specific learning tasks is an essential feature of CT-supportive learning environments. Modeling desired CT skills as part of the domain-specific instructional activities by an instructional agent may be implicit (immersion approach) or explicit to the students (infusion approach) as far as the acquisition of domain-specific CT competency is concerned. 3.3. The application principle states that learning is promoted when learners apply newly acquired knowledge and skills in solving novel problems. We suggest the design of CT-supportive learning environments should involve various instructional methods. First, challenging and sequential learning tasks that address the desired CT skills and create several opportunities for the students to engage in applying newly presented information need to be carefully designed. The sequence of the application tasks needs to increase in complexity over time in the instructional process. Second, a variety of social interaction forms needs to be used in encouraging the application of acquired subject-matter knowledge and skills: individual work, working in small groups, and whole-class discussion. Third, students’ attempt to solve learning tasks need to be sufficiently coached by an instructional agent. Students may need to be provided with partial solutions to tasks at the beginning of the instructional process, monitor small-group activities and provide hints as needed, act as a group member and ask thought-provoking questions, encourage students to formulate questions using specific verbal prompts, and facilitate discussion among group members. 3.4. The integration principle states that learning is promoted when learners reflect on, discuss, and defend their newly acquired knowledge and skills. We suggest various instructional activities, which could encourage students to present and defend their solutions to group members and the whole class, which need to be designed and implemented. Both peer and instructor feedback that offers immediate, specific, and corrective information on students’ learning progress should be incorporated into the learning environment.

986

D. T. Tiruneh et al.

Conclusion Attempts are made in this chapter to clarify the potentials of a systematic and modelbased approach to designing subject-matter instruction on the acquisition of domainspecific and domain-general CT skills. Our empirical findings suggest that if one expects subject-matter instruction in any domain to foster the development of (domain-specific) CT, systematic design of learning environments by using theoretically sound and empirically valid instructional design principles appears to be a promising approach. We hope we have particularly addressed three major limitations of current research on CT. First, it appears that current CT theoretical and empirical research lacks clarity with respect to what CT refers to. We argued that clearly describing and articulating the CT competencies learners are expected to demonstrate when they have completed CT-targeted instructions is an essential step in designing CT-supportive learning environments. Second, it was argued that current research on CT does not sufficiently differentiate between domain-specific and domain-general CT competencies. CT is largely detached from disciplinary competence and viewed as an add-on to domain-specific instruction in much of the current empirical evidence. Following the review of the CT literature (e.g., Jones, 2009; McPeck, 1990b; Perkins & Salomon, 1989; Resnick et al., 2010; Willingham, 2007), we argued that CT does not operate in a vacuum but always within the context of a particular domain. This implies that the ability to think critically involves domain-specific competence about issues and procedures specific to the domain in question. At the same time, it was assumed that CT tasks across domains share some commonalities, and thus some CT skills acquired in one domain can usefully be applied to solve a CT task in another domain (e.g., Bailin et al., 1999; Jones, 2009; Smith, 2002; Willingham, 2007). In view of this growing consensus, we argued that the development of CT needs to be considered from both domainspecific and domain-general perspectives. This implies that a major goal of embedding CT instruction in domain-specific courses is to enable students to (1) learn the CT skills that can be utilized to competently solve CT tasks specific to the domain in question and (2) transfer those CT skills in solving other CT tasks across domains. That means embedding CT instruction in specific domains should aim both at the learning and transfer of CT skills. Third, we noted that although the teaching of CT skills has been the focus of a large body of research, there is little consensus to date among educators and researchers on how to best support the development of CT in domain-specific courses. The empirical evidence with respect to effective instructional approaches to foster the development of CT is scant. We particularly argued that systematic and model-based approach to design learning environments is largely missing in the current CT literature, and we hope that research on the development of CT might benefit from a systematic and model-based approach to instruction that involves analysis, design, development, implementation, and evaluation of learning environments. In sum, there is a great deal of urgency among various stakeholders in education to identify suitable instructional approaches to stimulate the learning and transfer of

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

987

CT skills. It is obvious that this urgency will continue to grow with the accelerating changes and complexity of the world. It is evident that the development of CT has been given increasing emphasis in higher education (Davies & Barnett, 2015; Paul & Elder, 2014). At the same time, however, the literature largely depicts CT as an elusive concept with little direction on how to translate the diverse views into concrete CT instructional practices. Acknowledging the longstanding controversies involved in defining, teaching, and assessing CT, we have made an attempt to show how CT can be handled as an integral part of domain-specific instruction. More specifically, we have demonstrated that embedding CT instruction in specific subject-matter instruction requires greater clarity about what CT is, what set of CT skills could be targeted in domain-specific instruction, how specific subject-matter instruction could systematically be designed considering CT as an integral part of domain-specific instruction, and how best CT outcomes be assessed. A continuing focus on the aforementioned issues is expected to shift the excessive theoretical and philosophical debates on CT toward resolving the issues of CT development from an instructional design perspective. Our hope is that the approaches and findings of our empirical studies will act as a drive to strengthen this promising line of research on CT development: linking instructional design research with research on CT.

References Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102–1134. https://doi.org/ 10.3102/0034654308326084 Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, A., & Persson, T. (2015). Strategies for teaching students to think critically: A meta-analysis. Review of Educational Research, 85(2), 275–314. https://doi.org/10.3102/0034654314551063 Arum, R., & Roksa, J. (2011). Academically adrift: Limited learning on college campuses. Chicago: The University of Chicago Press. Association of American Colleges and Universities. (2005). Liberal education outcomes: A preliminary report on student achievement in college. In Liberal education. Washington, DC: AAC&U. Bailin, S. (2002). Critical thinking and science education. Science & Education, 11, 361–375. Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. (1999). Common misconceptions of critical thinking. Journal of Curriculum Studies, 31(3), 269–283. https://doi.org/10.1080/ 002202799183124 Barrow, R. (1991). The generic fallacy. Educational Philosophy and Theory, 23(1), 7–17. https:// doi.org/10.1111/j.1469-5812.1991.tb00172.x Bensley, D., Crowe, D. S., Bernhardt, P., Buckner, C., & Allman, A. L. (2010). Teaching and assessing critical thinking skills for argument analysis in psychology. Teaching of Psychology, 37(2), 91–96. https://doi.org/10.1080/00986281003626656 Bensley, D., & Spero, R. (2014). Improving critical thinking skills and metacognitive monitoring through direct infusion. Thinking Skills and Creativity, 12, 55–68. https://doi.org/10.1016/j. tsc.2014.02.001 Beyer, B. K. (2008). How to teach thinking skills in social studies and history. The Social Studies, 99(5), 196–201. https://doi.org/10.3200/TSSS.99.5.196-201 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

988

D. T. Tiruneh et al.

Davies, M. (2006). An “infusion” approach to critical thinking: Moore on the critical thinking debate. Higher Education Research & Development, 25(2), 179–193. https://doi.org/10.1080/ 07294360600610420 Davies, M. (2013). Critical thinking and the disciplines reconsidered. Higher Education Research & Development, 32(4), 529–544. https://doi.org/10.1080/07294360.2012.697878 Davies, M., & Barnett, R. (2015). Introduction. In M. Davies & R. Barnett (Eds.), The Palgrave handbook of critical thinking in higher education (pp. 1–25). New York: Palgrave Macmillan. https://doi.org/10.1057/9781137378057 de Bono, E. (1991). The direct teaching of thinking in education and the CoRT method. In S. Maclure & P. Davies (Eds.), Learning to think: Thinking to learn (pp. 3–14). Oxford: Pergamon Press. De Corte, E., Verschaffel, L., & Masui, C. (2004). The CLIA-model: A framework for designing powerful learning environments for thinking and problem solving. European Journal of Psychology of Education, 19(4), 365–384. https://doi.org/10.1007/BF03173216 Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12, 43–52. https://doi.org/10.1016/j.tsc.2013.12.004 Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed research. Educational Researcher, 18(3), 4–10. https://doi.org/10.3102/0013189X018003004 Ennis, R. H. (1993). Critical thinking assessment. Theory Into Practice, 32, 179–186. Ennis, R. H. (1996). Critical thinking dispositions: Their nature and assessability. Informal Logic, 18(1996), 165–182. https://doi.org/10.1353/jge.2007.0011 Ennis, R. H. (1997). Incorporating critical thinking in the curriculum. Inquiry: Critical Thinking Across Disciplines, 16(3), 1–9. Ennis, R. H., Millman, J., & Tomko, T. N. (1985). Cornel critical thinking test level Z. Pacific Grove, CA: Midwest Publications. Ennis, R. H., & Wier, E. (1985). The Ennis-Wier critical thinking essay test. Pacific Grove, CA: Midwest Publications. Evens, M., Verburgh, A., & Elen, J. (2013). Critical thinking in college freshmen: The impact of secondary and higher education. International Journal of Higher Education, 2(3), 139–151. https://doi.org/10.5430/ijhe.v2n3p139 Facione, P. A. (1990a). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Research findings and recommendations. Newark, NJ: American Philosophical Association.: ERIC Document Reproduction Service No. ED315 423. Facione, P. A. (1990b). The California critical thinking skills test – College level. In Experimental validation and content validity (pp. 1–14). Millbrae, CA: Academic. Retrieved from http://eric. ed.gov/?id=ED327549 Field, A. (2009). Discovering statistics using SPSS (3rd ed.). London, UK: Sage. Francom, G., Bybee, D., Wolfersberger, M., Mendenhall, A., & Merrill, M. D. (2009). A taskcentered approach to freshman-level general biology. Bioscene, 35(1), 66–73. Francom, G., Bybee, D., Wolfersberger, M., & Merrill, M. D. (2009). Biology 100: A task-centered, peer-interactive redesign. TechTrends, 53(3), 35–42. https://doi.org/10.1007/s11528-009-0287-5 Gardner, J. L. (2011). Testing the efficacy of Merrill’S First Principles of Instruction in improving student performance in introductory biology courses. UtahState University. https://doi.org/ Paper 885 Gardner, J. L., & Belland, B. R. (2012). A conceptual framework for organizing active learning experiences in biology instruction. Journal of Science Education and Technology, 21(4), 465–475. https://doi.org/10.1007/s10956-011-9338-8 Glaser, R. (1984). Education and thinking: The role of knowledge. American Psychologist, 39(2), 93–104. https://doi.org/10.1037//0003-066X.39.2.93 Halpern, D. F. (1998). Teaching critical thinking for transfer across domains. American Psychologist, 53(4), 449–455. https://doi.org/10.1037//0003-066X.53.4.449 Halpern, D. F. (1999). Teaching for critical thinking: Helping college students develop the skills and dispositions of a critical thinker. New Directions for Teaching and Learning, 80, 69–74.

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

989

Halpern, D. F. (2013). The Halpern Critical Thinking Assessment: A response to the reviewers. Inquiry: Critical Thinking Across Disciplines, 28(3), 28–39. https://doi.org/10.5840/ inquiryct201328317 Halpern, D. F. (2014). Thought and knowledge: An introduction to critical thinking (5th ed.). New York: Psychology Press. Halpern, D. F. (2015). Halpern critical thinking assessment. Modling: Schuhfried GmbH. Heijltjes, A., van Gog, T., & Paas, F. (2014). Improving students’ critical thinking: Empirical support for explicit instructions combined with practice. Applied Cognitive Psychology, 28, 518–530. Helsdingen, A. S., van Gog, T., & van Merriënboer, J. J. G. (2011). The effects of practice schedule on learning a complex judgment task. Learning and Instruction, 21(1), 126–136. https://doi.org/ 10.1016/j.learninstruc.2009.12.001 Jonassen, D. (1999). Designing constructivist learning environments. In C. M. Reigeluth (Ed.), Instructional design theories and models: A new paradigm of instructional theory (Vol. II, pp. 215–239). Mahwah: Erlbaum. Jones, A. (2009). Redisciplining generic attributes: The disciplinary context in focus. Studies in Higher Education, 34(1), 85–100. https://doi.org/10.1080/03075070802602018 Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher, 28(2), 16. https://doi.org/10.2307/1177186 Lawson, T. J. (1999). Assessing psychological critical thinking as a learning outcome for psychology majors. Teaching of Psychology, 26(3), 207–209. McMurray, M. A. (1991). Reliability and construct validity of a measure of critical thinking skills in biology. Journal of Research in Science Teaching, 28(2), 183–192. McPeck, J. (1981). Critical thinking and education. New York: St Martin’s Press. McPeck, J. (1990a). Critical thinking and subject specificity: A reply to Ennis. Educational Researcher, 19(4), 10–12. https://doi.org/10.3102/0013189X019004010 McPeck, J. (1990b). Teaching critical thinking: Dialogue and dialectic. New York: Routledge. Merrill, D. M. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43–59. Merrill, D. M. (2013). First principles of instruction: Identifying and designing effective, efficient, and engaging instruction. San Francisco: Wiley. Moore, T. (2004). The critical thinking debate: How general are general thinking skills? Higher Education Research & Development, 23(1), 3–18. https://doi.org/10.1080/0729436032000168469 Moore, T. (2011). Critical thinking and disciplinary thinking: A continuing debate. Higher Education Research & Development, 30(3), 261–274. https://doi.org/10.1080/07294360.2010.501328 Norris, S. P. (1989). Can we test validly for critical thinking? Educational Researcher, 18(9), 21–26. https://doi.org/10.2307/1176715 Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill. OECD. (2015). OECD skills outlook 2015: Youth, skills and employability. Paris: OECD Publishing. Retrieved from https://doi.org/10.1787/9789264234178-en Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research (vol. 2). San Francisco: Jossey-Bass. Paul, R. (1992). Critical thinking: What, why, and how? New Directions for Community Colleges, 77, 3–24. Paul, R., & Elder, L. (2014). Critical thinking: Tools for taking charge of your professional and personal life (2nd ed.). Upper Saddle River, NJ: Pearson Education. Perkins, D., Jay, E., & Tishman, S. (1993). Beyond abilities : A dispositional theory of thinking. Merrill-Palmer Quarterly, 39(1), 1–21. Perkins, D., & Salomon, G. (1989). Are cognitive skills context-bound? Educational Researcher, 18(1), 16–25. https://doi.org/10.3102/0013189X018001016 Reed, J. H., & Kromrey, J. (2001). Teaching critical thinking in a community college history course: Empirical evidence from infusing Paul’s model. College Student Journal, 35(2), 201–215. Resnick, L. (1987). Education and training. Washington, DC: National Academy Press.

990

D. T. Tiruneh et al.

Resnick, L., Michaels, S., & O’Connor, M. (2010). How (well-structured) talk builds the mind. In D. D. Preiss & R. J. Sternberg (Eds.), Innovations in educational psychology: Perspectives on learning, teaching, and human development (pp. 163–194). New York: Springer. Salomon, G., & Perkins, D. (1989). Rocky roads to transfer: Rethinking mechanisms of a neglected phenomenon. Educational Psychologist, 24(2), 113–142. Siegel, H. (1988). Educating reason: Rationality, critical thinking, and education. New York: Routledge. Smith, G. (2002). Are there domain-specific thinking skills? Journal of Philosophy of Education, 36(2), 207–227. https://doi.org/10.1111/1467-9752.00270 Solon, T. (2007). Generic critical thinking infusion and course content learning in introductory psychology. Journal of Instructional Psychology, 34(2), 15. Retrieved from http://eric.ed.gov/ ERICWebPortal/recordDetail?accno=EJ774169 Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Pearson Education. Tiruneh, D. T., De Cock, M., Weldeslassie, A., Elen, J., & Janssen, R. (2016). Measuring critical thinking in physics: Development and validation of a critical thinking test in electricity and magnetism. International Journal of Science and Mathematics Education, 1–20. https://doi.org/ 10.1007/s10763-016-9723-0 Tiruneh, D. T., Verburgh, A., & Elen, J. (2014). Effectiveness of critical thinking instruction in higher education: A systematic review of intervention studies. Higher Education Studies, 4(1), 1–17. https://doi.org/10.5539/hes.v4n1p1 van Merriënboer, J. J. G., & Kirschner, P. (2013). Ten steps to complex learning: A systematic approach to four-component instructional design (2nd ed.). New York: Routledge. van Merriënboer, J. J. G. (1997). Training complex cognitive skills: A Four-Component Instructional Design model for teaching technical training. Englewood Cliffs: Educational Technology. Verburgh, A. (2013). Research integration in higher education: Prevalence and relationship with critical thinking (Doctoral dissertation). KU Leuven, Leuven. Retrieved from https://lirias. kuleuven.be/handle/123456789/415510 Watson, G., & Glaser, E. (2002). Watson-Glaser critical thinking appraisal. London, UK: Pearson Assessment. Willingham, D. T. (2007). Critical thinking: Why is it so hard to teach? American Educator, (Summer), 8–17. https://doi.org/10.3200/AEPR.109.4.21-32

Dawit Tibebu Tiruneh is a Research Associate at the Research for Equitable Access and Learning (REAL) Centre at the University of Cambridge, UK. Dawit completed his Ph.D. in Educational Sciences at the University of Leuven in Belgium in 2016. Before joining the University of Cambridge, he was a postdoctoral fellow for two years at the East China Normal University in Shanghai, China, and a Lecturer in the Faculty of Education at Bahir Dar University, Ethiopia. Dawit’s main research interests include educational access and equity, school effectiveness, and instructional design. Mieke De Cock is an Associate Professor in the Department of Physics and Astronomy of the KU Leuven, Belgium, where she is responsible for the physics teacher training program. Her research focuses on conceptual understanding in physics, student use of mathematics in physics, and integrated STEM education. She is teaching both introductory physics courses and teacher training courses. J. Michael Spector, Regents Professor and former Chair and Doctoral Program Director in the Learning Technologies Department at the University of North Texas, was previously Professor of Educational Psychology and Doctoral Program Coordinator at the University of Georgia, Professor and Associate Director of the Learning Systems Institute at Florida State University, Chair of

38

Toward a Systematic and Model-Based Approach to Design Learning. . .

991

Instructional Design, Development and Evaluation at Syracuse University, and Director of the Educational Information Science and Technology Research Program at the University of Bergen. He earned a Ph.D. from the University of Texas. He is a Visiting Research Professor at Beijing Normal University, East China Normal University, and the Indian Institute of TechnologyKharagpur. His research focuses on assessing learning in complex domains, developing inquiry and critical thinking skills in children, and program evaluation. He was Executive Director and Treasurer of the International Board of Standards for Training, Performance and Instruction and a Past President of the Association for Educational and Communications Technology. He is Editor Emeritus and Featured Papers Editor of Educational Technology Research & Development. He edited two editions of the Handbook of Research on Educational Communications and Technology and the SAGE Encyclopedia of Educational Technology. He is currently lead editor of Learning, Design and Technology: An International Compendium of Theory, Research, Practice and Policy and section editor for educational technology in the Routledge Encyclopedia of Education. He has more than 200 academic publications to his credit and has recently been awarded an NSF-IUSE grant focusing on STEM education in small and minority serving colleges and universities. Xiaoqing Gu is a Professor of Educational Technology in East China Normal University. She is the Head of the Department of Educational Information Technology and Director of Shanghai Engineering Research Center of Digital Education Equipment. Her main research interests are learning science, learning design, and CSCL. Jan Elen is a Professor of Educational Technology and teacher of Education at the Faculty of Psychology and Educational Sciences of the KU Leuven, Belgium. His main research interest is in the field of instructional design. He teaches both introductory and advanced courses in instructional psychology and educational technology. He is the Senior Editor of Instructional Science.

Design of Innovative Learning Environment: An Activity System Perspective

39

Juhong Christie Liu

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Activity Theory and Innovative Learning Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Characteristics of Innovative Learning Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ILE Design with Activity Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Activity Theory as an Analytical Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design with Who and Where of Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design with Object and Outcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design with Tools, Resources, and Mediated Artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design with Community and Division of Labor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Design with Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

994 995 996 997 998 999 1001 1004 1008 1010 1011 1013

Abstract

Effective learning takes place in social, cultural, and historical contexts and consists of activities beyond mere dissemination and acquisition of content knowledge. Teaching and learning practice and research in science, technology, engineering, and mathematics (STEM) and humanities education, inquiries into educational reforms, and exploration in newer instructional strategies have demonstrated the needs of refreshing perspectives in learning environment design (Freeman et al., Proc Natl Acad Sci 111(23):8410–8415, 2014; Jonassen, Educ Technol Res Dev 48(4):63–85, 2000; Wouters et al., J Educ Psychol 105 (2):249–265, 2013). The demand and complexity are accompanied with ever evolving technologies. Related social-cultural changes constantly challenge professionals in instructional design (ID) to take an interdisciplinary and adaptive lens. Experiences in medical education, e-learning design, teacher education, J. C. Liu (*) James Madison University, Harrisonburg, VA, USA e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_85

993

994

J. C. Liu

sports coaching, and human-computer interaction have revealed the value and potential of Activity Theory (AT) in the analysis, design, and deployment of transformed learning environments (Benson, Br J Educ Technol 39(3):456–467, 2008; Engeström, J Educ Work 14(1):133–156, 2001; Jones, Sport Educ Soc 21 (2), 2016; Kuutti K, Activity theory as a potential framework for humancomputer interaction research. In: Nardi BA (ed) Context and consciousness: activity theory and human-computer interaction. MIT Press, Cambridge, MA, pp 17–44, 1996; Lazarou, Journal of Computer Assisted Learning, 27(5):424–439, 2011). This chapter focuses on the design possibilities of analyzing the characteristics of and reconfiguring the key components in an activity system (AS), including Subject and Object, Tools/Resources, Rules, Division of Labor/Roles, and Community to creatively design an innovative learning environment (ILE). Drawing upon the association between AS and ILE, the author will provide ID recommendations based on the analysis of major AS components. Keywords

Activity theory (AT) · Activity system (AS) · Innovative learning environment (ILE) · Instructional design (ID)

Introduction Innovations in education have manifested their appearance and impact in emerging methods, formats, and modalities of teaching and learning, which call for adaptation and transformation of instructional design practice. Recent trends such as flipped learning, maker education, active learning strategies like POGIL (Process-Oriented Guided Inquiry Learning), massive open online course (MOOC), integration of open educational resources (OER), universal design, personalized learning, and learning with virtual reality (VR) and augmented reality (AR) have changed the traditional concepts of teaching and learning location, process, access, accessibility, and assessment. The ubiquity of mobile and wearable technologies, prevalence of synchronous web conferencing, and integration of cognitive and social aspects of learning community have revolutionized lifelong learning at workplace and in people’s daily life. The incorporation of entrepreneurial conceptualization in education, blending classroom and online learning activities, merging spaces, and focusing on collaboration and hands-on learning; reconfigurations of curriculum with crossdiscipline inquiries; and expansion beyond the brick walls of classrooms to local and global communities present potential as well as challenges to instructional design (ID). What does a designer need to know to function effectively in new and existing learning landscape and optimally shape innovations to successfully support learning? This chapter explores answers to this question based on literature and practice from the holistic perspective of activity systems according to Activity Theory (AT) and the characteristics of an innovative learning environment (ILE). Activity

39

Design of Innovative Learning Environment: An Activity System Perspective

995

Theory views human behaviors and motives in an interrelated context with planned or anticipated objects and outcomes, analyzing human and related components as a system and evaluating tools within the context and thinking the context of each activity system as connected rather than isolated (Engeström, 2000; Jonassen & Rohrer-Murphy, 1999). Comparing the ILE design to an activity system will help avoid discrete treatment of elements when designing a learning environment. The discussion and analysis in this chapter are set in the context of affordance provided by technologies, trends of interdisciplinary collaboration, needs and practices of project management in ID, and evaluation aspects of teaching and learning activities.

Activity Theory and Innovative Learning Environment Activity Theory (AT) originated in the early twentieth century in Europe for psychological and philosophical analysis of human behaviors (Jonassen & RohrerMurphy, 1999). Later, scholars led by Yrjö Engeström advanced AT as a prescriptive framework to understand the intricate relationship between human being activities and their surroundings (Engeström, 2001; Yamagata-Lynch & Smaldino, 2007). As illustrated in Fig. 1, a human activity system consists of eight major components, including subject, object, tools and signs, mediated artifacts, rules, community, division of labor, and outcome. “Subjects are participants of the activity and tools are the resources that subjects use to obtain the object or the goal (outcome). Rules can be informal or formal regulations that subjects need to follow while engaging in the activity. The community is the group that subjects belong to and the division of labor is the shared responsibilities determined by the community” (Yamagata-Lynch & Smaldino, 2007, p. 366).

Fig. 1 Structure of an activity system (Engeström, 2001, p. 135)

996

J. C. Liu

A learning environment is “an organizational form that embraces the learning arrangements catering for a group of learners in context and over time. It may be primarily located in a particular institution (for instance, a school) but it is not necessary that it is school-based, whether for some or all of the learning taking place” (Istance & Kools, 2013, p. 49). According to the research by the Organisation for Economic Co-operation and Development (OECD), Activity Theory views human learning in the intersections among individuals, between individuals and their context, and between groups of individuals in an entire learning ecology. A learning environment is usually situated inside and overlaps with and aligns with expectations of larger communities such as higher education in general, and informal learning environments such as museums and libraries, families, and local communities (Istance & Kools, 2013). Building a learning environment from an activity system perspective respects the core value of human interactivity as individuals as well as a community. More importantly, an activity system view contextualizes the activities in a learning environment with the means, that is, tools, artifacts, rules, and roles, and the goals, that is, object and outcome. A learning environment has its default base as an activity system because of its social-cultural lens of human learning and its attention to human as individual learners in reference to the larger ecological system (Engeström, 2000; YamagataLynch, 2007, 2010). The ID literature reveals that three aspects of Activity Theory integration in research and practices are related to learning environments (Jonassen, 2000; Yamagata-Lynch, 2007). These include guidelines for designing a constructivist learning environment, identifying what could be changed so that educational settings could be improved, and leveraging the historical-cultural influence on learning. From the perspective of an activity system, the design of a learning environment needs to take into consideration of the eight key components (Engeström, 2000). Only when designed properly can the appropriate coordination among subject, tools, rules, community, and roles of subject which define division of labor, all work together and lead to the right object and outcome.

Characteristics of Innovative Learning Environment In the recent decade, the exponential growth of technologies and their creative use have enabled the global and interoperable connectivity, ubiquitous access to information, and cutting-edge cognitive interventions. In the meantime, these innovations carry with them the characteristics such as relative advantage, complexity, compatibility, trialability, and observability (Rogers, 2010). The characteristics come along with the tools and signs of a learning activity system are expressed through the mediated artifacts created with the tools or delivered with the signs. These entail possibilities to invite a renewed lens to look at the key components for designing and innovating the learning environment. These include taking informed risks in experimenting with the tools for creating or modifying objects, reevaluating the

39

Design of Innovative Learning Environment: An Activity System Perspective

997

rules that influence how the subjects use tools and how the community are connected, and redefining the division of labor so that the roles of subject can be continually optimized or shaped along with the social-cultural relationship changes. An ILE can be designed with potential enabled by technological affordance that has never been so in human history. Google, Apple, Microsoft, Baidu, WeChat, and Line, just a few examples, have facilitated the extension of learning opportunities to individuals with technologies, possibilities for connectivity, communication, and collaboration, and mobility. The traditional student-teacher relationship has changed with practices such as flipped classrooms, team-based learning, and synchronous online communication potential. Students’ view of day-to-day teachers in the same physical location has also changed through broadened academic communication channels such as those facilitated by MOOCs and open-access educational resources. Globalized connections and scholarly exchanges have modified the traditional relationship between internationally renowned scholars and college students in developing countries. Artificial intelligence (AI) can be utilized to capture and analyze data that are related to learning process, which can then be used to enable personalized adaptive learning paths for individual learners. Augmented reality (AR) and virtual reality (VR) can enable what used to be impossible for observation to learners in a potentially fuller, more complete, and near reality view (Berrett, Mcmurtrie, & Supiano, 2018). These affordances provided by emerging technologies to teaching and learning come with challenges of new angles of observability, expressions of complexity, demand of compatibility, and needs of willingness to take risks in testing trialability (Rogers, 2010). To make informed and effective use of these affordance, ILE design needs to consider the social-cultural adaptation of subject and the modification of context and role changes and to mediate the contradictions in an activity system (Engeström, 1995; Yamagata-Lynch, 2007). Many scholars conducting research on Activity Theory have urged collaborations in designing such transformed systems (Engeström, 1995; Jonassen, 1997; Park, 2015).

ILE Design with Activity Theory Learning activity systems have been shaped by the emergence of possibilities, conflicts, limits, and boundaries in human history. As new methods and mediated artifacts appear and merge with existing learning environments, contradictions arise and require design and redesign. Effective design of a learning environment needs deliberate analysis of the system in which learning activities will take place. Transformational modification and integration of new components need to adopt a risktaking perspective when facing complexity, observability, and triability (Hayes, Eljiz, Dadich, Fitzgerald, & Sloan, 2015). In this process, Activity Theory can serve as an analytical tool to help the designer(s) associate convoluted elements to major components in a system.

998

J. C. Liu

Activity Theory as an Analytical Tool Activity is a collective, systemic formation that has a complex mediational structure. Activities evolve historically, usually over long periods of time, as institutions and structures emerge in the collected, socially distributed meeting of needs. (Rybacki, 2009, p. 291)

The analysis of an activity system can include probing answers to questions in many layers and possibly involve shifts of different contents associated with different components (Engeström, 2000) (Fig. 2). These questions can include: What specific activity am I analyzing now? Why is this activity taking place? Is it linearly interacting with other activities in student learning process? Who should be involved in carrying out this activity? What is the outcome of the activity? Are there standards and/or benchmarks existing as references to the desired outcomes? What are the norms, rules, or regulations that can monitor the performance of this activity? Who is responsible for what? When will this activity be carried out and how are these roles configured? What is the context in which this activity is carried out? How will the context affect the sense or meaning of the mediated artifacts getting across to the interpretation of outcome? What are available as resources, tools, and signs? What factors may influence the transfer of learning, within and extending from the learning activity system? Asking these analysis questions can lead to a clear identification of relationship between micro- or macro-level systems such as a course and the larger system, that is, the curriculum of a degree program. For instance, researchers and teachers took an

Fig. 2 Design with an analysis of an activity system

39

Design of Innovative Learning Environment: An Activity System Perspective

999

activity system approach to reanalyzing the curricular of social work and literacy education (Brayko, 2013; Fire & Casstevens, 2013). After the analyses, the researchers identified foundational competency that could be integrated across courses in the curriculum. The findings facilitated a constructivism design of the new learning environments and provided evidences and guidance for developing new activities to engage students. As the macro-level outcome, the Brayko study (2013) promoted subject-specific literacy education for high-demanding communities and helped build connection among teachers, students, and student families. In the redesign of children’s medical care program, Engeström (2000) explained clearly the shifts of components of an activity system at different procedural steps of analysis. Through the analysis, the researcher identified contradictions which caused the lack of coordination among different medical care providers and therefore inefficiency of operation. The analysis also identified where the innovation occurred when a care agreement as a tool made the innovation possible. The similar analysis of a medical intervention on the basis of Activity Theory provided scaffolding to the successful identification of design components for a patient self-management and educational environment (Schaffer, Reyes, Kim, & Collins, 2010). The results of these analyses can affect the design of a learning environment in terms of subject, object, tools, mediating artifacts, rules, division of labor, community, and outcome.

Design with Who and Where of Subjects Any individual involved in the teaching and learning activity system is a subject (Collins, Shukla, & Redmiles, 2002; Engeström, 2000). The analysis of subject, that is, a student or a teacher in a learning activity system, is of primary importance for an optimal design. The questions to aid the design with subject include Who the subject is? and Where she/he performs a learning activity? The experience of a subject can affect the perception of and approaches to learning process. As importantly, in a technology-infused environment, the analysis of support and service providers and enablers needs to be conducted. Therefore, the interpretation and understanding of who and where of the subject expect an expansion from the traditional needs analysis in instructional design to looking at the motives and goal setting for those who provide and receive instruction and those who support instructional delivery and learning interactivity. The design also needs to intentionally consider the dialectic nature of subject interaction and its relationship with the historical-cultural backgrounds of the subject (Jonassen & Rohrer-Murphy, 1999). Who the subjects are and where they are located can lead to customized support to learners, modification of learning resources, and closer look at historical-cultural influence, as demonstrated with the two quadrants in Fig. 3. The experience of subjects can affect the design of a new learning environment. For instance, in a 2016 study about learning and teaching with synchronous videoconferencing technologies, students and faculty (N = 305) in a comprehensive university located on the East Coast of the United States responded to the question about using synchronous technology for learning/teaching purposes. More students

1000

J. C. Liu

Fig. 3 Design with who and where of subject

When it comes to using synchronous online technologies for learning/teaching purposes 60.0%

50.9%

50.0% 40.0% 30.0%

30.5%

20.0%

18.2%

39.8%

42.1%

18.6%

10.0% 0.0%

I use them very often.

I use them sometimes.

learning

I never use them.

teaching

Fig. 4 Experience of subjects in synchronous online learning environment

responded that they used synchronous technology very often or sometimes than faculty did (Fig. 4). These varied experiences required designers to take into consideration of customizable training and support for faculty when the learning environment became facilitated with synchronous technologies and more videoconferencing activities were integrated in courses and curriculum. The cultural norm of subjects can shape the design of learning environments. For instance, MOOCs and open-access educational resources can bypass the physical boundaries of countries and disseminate knowledge. However, depending on the

39

Design of Innovative Learning Environment: An Activity System Perspective

1001

cultural backgrounds of subjects (instructors and students), the mediated artifact in the environment (a video clip with a pet or certain color background) can convey different meanings even with the mutually understandable language (Bonk, Lee, Reeves, & Reynolds, 2015). In these types of scenarios, the design of a learning environment with cross-culture delivery needs additional cultural consultation or training for designers. The who and where of subjects can also affect the reconfiguration of technological infrastructure and related resource allocation. According to Education Week (June 12, 2017), the recent “Course Choice” initiatives can benefit learners in rural communities in the United States through leveraging the intentional distribution of e-rate funds and opening the gateway to look at course or academic program options offered outside of the United States (Flanigan, 2017). This would leave substantial space for the redesign of learning environments. The subjects’ locations and varied access to learning environment expect designers to consider customizable languages, technological infrastructure, and funding/resource redistribution.

Design with Object and Outcome In a learning environment, the object and outcome are closely related or aligned with each other. The object includes those that are generated through learning activities to reflect the fulfillment of outcome, while the outcome can be paraphrased from standards and/or benchmarks as quality or measurement indicators in the subject area or profession. As Fig. 5 illustrates, the key questions to address when designing with object and outcome include, What is the desired outcome of the activity? Are there standards and/or benchmarks as reference to the desired outcomes? What will be generated in this activity to reflect the fulfillment of outcome?

Fig. 5 Object and outcome in a learning environment

1002

J. C. Liu

Fig. 6 Various learning process and products as object

An object out of a learning activity system can be very diversified (Fig. 6). Designing activities that lead to interdisciplinary and process-oriented learning objects with appropriate measurement anticipates design competencies of assessment strategies and techniques. These can include but are not limited to development and utilization of rubrics; video-taped portfolios; video analysis and photovoice projects; eye-tracking projects, capturing user experience in VR/AR environments; and use of learning analytics. For instance, a POGIL project needs the development of respective suites of critical thinking questions and application scenarios to accompany content delivery, as well as strategies for group work and role play (Moog & Spencer, 2008). In the meantime, these newly developed assessment instruments are subject to reliability and validity tests to enhance the rigor of ILE design. To ensure significant learning experience, an upfront input analysis of object and outcome is of primary importance to start the ID project with a project management (PM) plan (Fig. 7). An ID professional needs interpersonal skills and PM competency to understand what the clients’ goals are, what accreditation standards the learning outcomes need to be aligned with, what resources and funding are available for the development and implementation of the project, and what timeline the clients are looking for (Fig. 7). Very often, the designer in charge of the ID project may also need an informal learning of the technical aspects of assessment mechanism and understanding the organizational culture to figure out what works or not with the expected project. There are certainly parallel and divergence between project management (PM) and ID that both clients and instructional designers can benefit from the knowledge (Layng, 1997; Schiffman, 1986; Van Rooij, 2010).

39

Design of Innovative Learning Environment: An Activity System Perspective

1003

Fig. 7 PM plan for instructional design with object and outcome

For instance, an engineering department expecting an approaching Accreditation and Program Review (APR) in 5 years attempts to align all signature assignments in the program with the outcome and rubric functions in a learning management system (LMS). Such a project may need the ID team to get familiar with the Accreditation Board for Engineering and Technology (ABET) accreditation guidelines, technical details of the LMS assessment mechanism, and assessment practice as part of organizational culture. Guided by the core outcome of the project, another aspect of PM in this project required the ID professional to sit down with subject matter experts and looked at the entire project holistically. The designer also conducted some field studies through class observations and mapped the key learning objectives with measuring rubrics or checklists. These rubrics and checklists were later reviewed to establish reliability and validity prior to large-scale implementation. The analysis also discovered that retrieving student performance data in an anonymous format from a learning management system needed a programmer with the proficiency of certain programming languages. The analysis laid an informed foundation for the PM plan for both the department and ID professional to move forward with appropriate budget and decision making on resource and tool selections. Knowledge about assessment culture in education in general and in the organizational context in particular is also instrumental. For instance, the large-scale study on meaningful assessment of learning with VALUE rubrics started with the outcome, the rubrics (Flaherty, 2017). The objects came from 92 public and private universities and colleges and 21,000 web-based sample student works, which were objects from their learning activities in different discipline areas and institutions. If the ID teams involved in the project did not have the holistic knowledge in the first place, how would the study turn out to be such a successful validation of the VALUE rubrics and become a launching point of assessing critical thinking and significant learning as a large-scale model without standardized test?

1004

J. C. Liu

Design with Tools, Resources, and Mediated Artifacts “A learning environment is a place where people can draw upon resources to make sense out of things and construct meaningful solutions to problems” (Wilson, 1996, p. 3). In an activity system, the design aspects related to tools can start appropriately by probing answers to the questions of What tools and/or resources are available? and What tools and/or resources are needed? (Fig. 8). Availability of tools and resources prepares and enables an effective learning environment. These also directly or indirectly interact with the subject, object, and community in an activity system, through mediating or mediated artifacts. These mediated artifacts come along with tools and resources but take on contextualized social, cultural, and historical interpretative appearance in a learning activity system. These tools and resources can include physical entities, such as schools and museums, and a maker space; these can also include virtual space or entities such as videoconferencing system, open-access information resources or repositories, or digital objects. The tools and resources can enable or challenge the function and interaction of other components in an activity system (Fig. 8). Mediated artifacts can be messages, images, symbols, sounds, videos, objects, etc. that carry meanings and are related to certain tools and resources. From a project management (PM) perspective, their availability and existence can be appraised at the input analysis stage (Fig. 7). When analyzing the availability and needs of tools and resources and mediated artifacts, the answers to the following questions will help an ID professional obtain the understanding about how to use these tools and/or resources in the right context, that is, the techniques. Asking the clients why the tools and/or resources are needed for the project will help connect the goals or core objectives of the project.

Fig. 8 Design with tools and resources

39

Design of Innovative Learning Environment: An Activity System Perspective

1005

Explaining to clients with previous examples, cases, and/or literature will establish mutual understanding. Knowing whether the tools and/or resources are directly available and accessible to the client project will help detect how efficient and effective these can be used. Sometimes, the direct or indirect access makes a big difference in budgeting, support, and results. The third question and the most pragmatic one is about how certain tools can be used for the design and project purposes. Attributes for design may include visualization, communication, collaboration, etc. Attributes for project fulfillment may include usability, customization, accessibility, etc. This question is addressed with some representative attributes and examples below.

Tools Enabling the Reconceptualization of Time and Space Asynchronous and synchronous technologies make possible the explicit articulation of time and space when designing a learning environment. A content management system or LMS facilitates delivery and access to content at different times and spaces, while a web conferencing or video-equipped robot such as Suitable Technologies Beam can connect all subjects in a system at the same time even when each party is located in different space; and simulations with VR and AR can enable experiential learning activities in a simulated space with no limit of location nor time and can enable what used to be impossible, for instance, transforming a cadaver observation to similar experience without the normal concerns (Berrett et al., 2018). Table 1 illustrates how tools with attributes of time, physical space, spontaneity with human factors, access to instructional content, and interaction can affect the design of a learning environment. The conceptual changes of space and time enabled by evolving technologies reveal potential of ILE design from both micro- and macro-levels. At microlevel, changing concepts of time on learning activities have been explored in designing blended courses or flipped classes. Blended classes can be designed with pre-class asynchronous learning activities with companion low-stake assessment, in-class or synchronous concept and theory clarification or problem solving, and post-class asynchronous transfer of learning (Estes, Ingram, & Liu, 2014). Asynchronous and synchronous technologies also make merging physical and virtual spaces possible when designing a learning environment. In a physics laboratory class of assembling GPS-programmed drones, external experts can be brought to the class from hundreds of miles away without the need of commuting. A virtual videoconference system and/or a mobile robot like Suitable Technologies Beam can bring interdisciplinary expertise together by merging the physical and virtual space among various organizations and university campuses (Estes, Liu, Zha, & Reedy, 2014). At a macro-level, blending online learning platforms and resources with residential requirements has been explored to offer degree-pursuing programs in a flexible way. By intentionally blending online and residential curriculum, MIT has had successful experience selecting talents through a first-half online MOOC learning and the second half of residential completion of a degree program (Straumsheim, 2017). Numerous business administration, health science, and nursing programs

1006

J. C. Liu

Table 1 Design with physical space and time (Liu, 2012) Spontaneity with human factors Instant and non-mediated

Physical space Dependent

Time Dependent

Online synchronous

Independent

Dependent

Instant and mediated

Online asynchronous

Independent

Independent

Delayed and mediated

Modality Face-to-face in-class

Instructional content One-time access and location bound One-time access and not location bound Repeated access

Interaction Multidirectional

(Mediated) multidirectional

One direction

have also adopted blended format, to accommodate the career or professional needs of developing virtual professional network or adapting with flexibility to support clinical placements (Butz, Stupnisky, Pekrun, Jensen, & Harsell, 2016; Carter, Beattie, Caswell, & Fitzgerald, 2015; Thompson, Dagenhard, Castor, & BrookinsFisher, 2016). The design of these macro-level learning environments expects designers to collaborate with stakeholders at administrative, instruction, IT support, and sometimes relevant career agencies where students’ professional life resides. For instance, the MIT blended degree program has a transformed admission process and a connection between student degree completion and recommendation to potential hiring agencies. The Butz et al. study (2016) found that the academic achievement of students in synchronous blended business and administration programs was correlated with their positive emotion perception. Just as learning taxonomy and significant learning define, affective domain is also an inseparable part in ILE design (Fink, 2013; Krathwohl, 2002). With emerging mobile robotic technologies, higher education institutions also start to facilitate degree programs with robots so that students can overcome the barriers of space and distance to attend classes (Double Robotics, 2017). These ILE designs will not only involve consideration of resources but also design of orientation to students about technology facility expectations and skill sets (Liu & Adams, 2017). Instructors’ technological competency and institutional rewarding mechanism for faculty professional development to obtain new pedagogical and technological competencies also need to be in the project management plan for an instructional design project (Liu & Alexander, 2017).

Tools and Resources as Agency Affecting Design for Object Tools and resources can affect the design of object in a learning activity system. As illustrated in Fig. 8, subjects achieve objects through tools and/or mediated artifacts, which needs core attention in the ILE design. Tools or signs can be contextualized. Even more so are the mediated artifacts, which may vary between interacting with a robot and interacting with a “Flat Stanley,” and therefore, the conveyed sense and

39

Design of Innovative Learning Environment: An Activity System Perspective

1007

meaning can vary. In these instances, the integration of a PM flowchart (Fig. 7) in the analysis and design of an activity system will help the design fit the social-cultural context and can also help the future evaluation of the design project (Cooksy, Gill, & Kelly, 2001; Foot, 2014; McDavid & Hawthorn, 2006; Postholm, 2015). For instance, to achieve the learning outcome of identifying geographical characteristics of the North African area in a sociology class may generate different learning objects depending whether students have a real study abroad class or a simulation with a VR simulation.

Other Aspects Related to Design with Tools and Resources As inclusiveness and openness have become the major drives in facilitating learning activities in the contemporary society, integrating universal design (UD) principles and intellectual property considerations is intriguing and indeed necessary for ID with tools and resources. Intentionally applying UD principles can include all audience with all variety of abilities, especially in a learning environment with various digital and networked information as tools and mediating artifacts. Accessibility checking in online LMS can ensure that text is friendly to screen readers, images have descriptive alt text, tables contain meaningful headings, and videos have transcripts and captioning. For instance, an instructional designer can have micro-level practice in the procedures of applying closed captioning to video clips when creating a laboratory-based analytical geoscience blended learning textbook. Instructional designers can also participate in decision making at a macro-level by investigating how friendly a learning management system or a WordPress-based content management system is to screen readers and how inclusive the interface design can be to an audience with disabilities. Knowledge about pros and cons of tools with the consideration of UD is an integral part as well. Table 2 is an example of comparison about applying closed captioning (CC) with representative lecture recording programs. Design with tools and resources also needs the leverage of using copyrighted materials vs. open-access content. This requires a close collaboration between ID professionals and librarians. As the professionals knowledgeable of the traditions Table 2 Comparison of CC options in lecture recording programs Software Adobe Captivate Adobe Presenter 11 Camtasia Studio ScreenFlow YouTube

Sync CC with audio Syncing with clicking keyword Speech to text conversion engine Syncing with clicking keyword Postproduction in YouTube Google speech-totext recognition

Creating CC text In audio editing, pasting phrases to match audio clips In video editor, just for video recordings In tools/caption, pasting phrases to match audio clips Copy/paste or postproduction upload with YouTube UTF-8 .txt file or .srt file upload

Editing options Editable publishing skin In script/notes panel for recorded presentation) Depending on preset Using YouTube Video Manager Editing subtitle or CC in Video Manager

1008

J. C. Liu

and current development in scholarship production and publication, data management, and collaborative/open access to copyrighted vs. open access materials, librarians have unique contributions to the ILE design in the digital and connected age (Mahraj, 2012; Rodriguez, Greer, & Shipman, 2014). Sometimes, the foundation of information literacy that is beyond the class subject topics may be best accomplished with a customized librarian instruction session or embedded librarian in an online class (Pritchard, 2010).

Design with Community and Division of Labor Community as an activity system component in a learning environment refers to individual or group of stakeholders who have direct or indirect participation and influence in the learning process. As digital devices and web connection prevail in formal and informal learning, the concept of community can refer to micro-level groups of students and teachers in a class and related support personnel. It can also refer to macro-level networks with alumni, local community members or scholars, and experts who can function as mentors. These newly formed communities can usually affect the predefined division of labor or roles of various stakeholders in the system. The design focus here is to address the following questions (Fig. 9): How do the group members communicate and interact to fulfill the outcome? How does the group interact with other communities if needed? Who is responsible for what in the activity? Are there shifts of responsibilities in the process? Answering these questions and especially taking a between-system interactive perspective usually help substantially an ID project. At the micro-level, the design for community of learners has been explored by methods such as team-based learning (TBL) and peer-led team learning (PLTL), which have modified the division of labor or roles from those in traditional learning environments. The instructors’ role has shifted from “Sages on the Stage” to “Guides

Fig. 9 Design with community and division of labor

39

Design of Innovative Learning Environment: An Activity System Perspective

1009

on the Side.” Peers among students and instructors can play roles such as tutors, mentors, and connectors. Peer-led team learning (PLTL) and other peer learning initiatives trust and train skilled and experienced students as leaders to help their peers learn with languages that students can understand (Michaelsen & Sweet, 2011; Quitadamo, Brahler, & Crouch, 2009; Tien, Roth, & Kampmeier, 2002). The design of this type of ILE needs to take into consideration of training for student peer leaders and grouping strategies. Scaffolding in affective domains, such as empathyoriented mentoring, also needs to be provided so that the peer learning community can be successful. Community also plays a crucial role in the design of community-based service learning (CSL) classes. These CSL courses connect what students learn in classrooms to real-world practice, provide hands-on opportunities for students, and possibly share resources and expertise between higher education institutions and their adjacent communities (Eick & Reed, 2002; Rinaldo, Davis, & Borunda, 2015). These CSL opportunities have proven valuable application and integration of student learning in real-world scenarios. However, successful design and teaching require time commitment from both teachers and community partners. Planning for logistics outside of classrooms can indicate additional complexities (Lewis, Henriques, Liu, & Brantmeier, 2016). Related to the extended community for learning activity systems, the roles or division of labors among subjects of teachers and students can change as well. Therefore, analysis of role change for faculty and students needs partnership with community and considering the respective design questions, as presented in Table 3. At a macro-level, community enriches the ILE design with professional mentors and interdisciplinary expertise which are not easily accessed in a traditional educational environment. Mentoring has been incorporated in advising and professional development programs for teachers who are new to a setting or pedagogy (Dorner & Kárpáti, 2010; Graves, Abbitt, Klett, & Wang, 2009; Kidwell, Freeman, Smith, & Zarcone, 2004). The rising entrepreneurial education invites interdisciplinary expertise from alumni network, industries, and businesses as the mentoring community in ILE. In a collaborative design through the 4-VA project, the instructional designers as researchers investigated the possibilities of bringing an external entrepreneur mentor to a college physics class. The design utilized a suite of web conferencing, mobile robots, and cloud file sharing systems (Estes et al., 2014; Liu, Swayne, & Adams, 2017). External mentorship shortened the distance between abstract learning in college classrooms and real applications. It also enabled closeup demonstrations through sharing resources in labs on sites that were located hundreds of miles apart. Progressively through semesters, the project also witnessed the role changes of the teachers and students and the adaption of languages from different labs with the between-system interaction. Primary concerns of ILE design for a macro-level mentoring component include (1) alignment of mentoring with learning outcome; (2) intentional communication for demonstration, directions of discourse, asynchronous or synchronous exposition, or file sharing; (3) room for ambiguity; (4) flexibility of role adjustment; and (5) proximity between class learning objects and real-

1010

J. C. Liu

Table 3 Design questions for community-based service learning classes Community partner

Students Teachers

Design questions What are previous experiences with the community partner in communitybased service learning (CSL) partnership? What value can the CSL class bring to the community? How can the community help student learning? How can the CSL class support the mission of the community? How do students perceive their previous CSL experience? How is the CSL class relevant to student learning goals? How are the CSL activities aligned to the learning objectives? Where are the resources and support for teachers to plan, develop, and teach CSL classes? What are the institutional expectations and rewarding mechanism for faculty to design and teach CSL classes? What are previous experiences of the teacher with CSL?

world scenarios (Liu et al., 2017; Merriweather & Morgan, 2013; Schuch, Liu, & Bona, 2015).

Design with Rules The reconfiguration of rules is becoming part of innovative pedagogy in various disciplines since they set the regulatory context for subjects’ behaviors and performance in an activity system. “As the productive subject of labor, the person becomes more knowledgeable and skilled and, . . . the person is subject(ed) to the activity and the societal relations that come with it. A person undergoes subjectification in each and every activity where he/she participates in the course of the day, week, or month” (Roth, 2004. p. 179). Both teachers and students in a learning activity system are subject to such rules and changes of rules constantly; so are the rules of using tools and interpreting mediated artifacts and rules of organizational culture and supported technological affordance. The ILE design with rules can take place at micro- and macro-levels. At microlevel, ID professionals need to know the rules of using tools and resources. For instance, using rules can be designed as assessment directions for learning activities and class interventions; knowing the technical rules of using learning analytics systems is becoming the basic rules for ID in a connected learning world (Siemens, 2013). Rules can also be a certain type of teaching and learning behavior. In a foreign language learning (FLL) study, Valentín-Rivera (2016) discovered that pairing FLL speakers with heritage language learning (HLL) tutors and providing indirect corrective feedback substantially improved co-construction of linguistics through storytelling. In a professional development program for math teachers, a group inquiry project required teacher trainees to conduct inquiries in a local community (Takeuchi & Esmonde, 2011). The expectations on the math teachers in their inquiry project turned them to changing agents to the ELL-speaking community and modified

39

Design of Innovative Learning Environment: An Activity System Perspective

1011

teachers’ view and understanding of English language learners (ELLs). The rule change also allowed them to rethink the way that they could modify when teaching math to ELLs in classrooms. In science classes, pedagogies such as Process-Oriented Guided Inquiry Learning (POGIL), problem/project-based learning (PBL), and team-based learning (TBL) have been proved effective (Freeman et al., 2014; Woods, 2014). These innovative pedagogies usually have redesigned assessment techniques and team learning as rules. For instance, in POGIL, the rules of model presentations of content, key questions or critical thinking questions, self-assessment, application scenarios, and group role play in cooperative learning need contextualized design and redesign (Ruder & Hunnicutt, 2008; Simonson & Shadle, 2013). So are the techniques and rules in social sciences and humanities subject areas, such as Question Formulation Techniques (QFT) (Lu, Deen, Rothstein, Santana, & Gold, 2011; Rothstein & Santana, 2011). To implement them, especially in rarely applied disciplinary areas, requires intentional collaboration between subject matter experts and ID professionals to pilot and validate new rules for assessment and learning activities. At macro-level, the rules of ILE design are associated with standards and guidelines for educational program evaluation, literacy in the access to and use of information and technologies, and appraisal of rising trends and principles related to teaching and learning. For instance, when designing with digital and online technologies, knowledge of benchmarks like Quality Matters Rubric and Online Learning Consortium Scorecards with digital classrooms will set the design in alignment with quality and program evaluation from the very beginning. When designing a blended class project for adaptive physical education for learners with disabilities from off campus, knowing accommodation guidelines on campus and regulations of information technology infrastructure will make the designed project accessible, inclusive, and function properly. When selecting multimedia objects for an adaptive personalized learning project for postsecondary science education, proficiency in using Learning Object Review Instrument (LORI) and the subsequent analysis will help provide evidences for the soundness of the design (Leacock & Nesbit, 2007). Being open to changes, rising trends in ID, and related innovations will also make design with rules adaptable to new projects. For instance, the successful design of learning activities with VR may need ID professionals to experiment with the reach and use of sense and space that have not been possible in a physical or monitor-based simulation environment (Merchant, Goetz, Cifuentes, Keeney-Kennicutt, & Davis, 2014). In such innovative environment, the content presentation of a certain subject, subject support, resources and community, and the usability considerations may need substantial redesign as well.

Conclusion and Discussion ID professionals face many opportunities and challenges enkindled by the possibilities offered with emerging technologies, as well as those from the social expectations of accessibility, inclusiveness, and standards for teaching and learning. These

1012

J. C. Liu

shape the production of object, representation of mediated artifacts, rules, and roles, therefore, how subjects interact with each other and in a learning activity system. For instance, the prevalence of using mobile phones and practicing with UD principles has shaped the design with accessibility facility and features in a learning environment. What used to be design considerations of selecting screen-reader technologies for an Assistive Technology Lab now becomes the essential design attribute of a portable and accessible learning environment. This contextual awareness has enabled and calls for a general orientation of all users with various mobile devices in universities, community centers, museums, and schools to use accessibility features to read, listen, and feel. This also triggered the connected technology communities to take risks and provide more flexible applications, such as a browser add-on screen reader like ChromeVox. Another example is the use of messages, methods, or nonmaterial interventions enabled by innovative technologies and their accompanying mediated artifacts, such as VR, AR, and artificial intelligence. Based on Activity Theory, ILE design needs to look at learning activities with foci on each component in an activity system, analyzing subject, object, tools, mediated artifacts, division of labor/roles, rules, community, and outcome as individual entities as well as in a connected holistic context. The leverage between focus and system views in an ILE design enables an ID professional to apply the principles of learning theories and ID models and be adaptive to innovative calls to design learning activities in a changing world with newer technology affordance and social expectations. The understanding of interacted components in a learning activity system and that learning activity systems are connected also opens cross-disciplinary perspectives for ID professionals. As principles of design can be shifted by contemporary technologies and tools (Amory, 2010), designers can build creativity for ILE design based on solid

Fig. 10 Creativity in ILE design

39

Design of Innovative Learning Environment: An Activity System Perspective

1013

foundations of learning theories, instructional design models, and social science research. These are the sources of self-efficacy (Fig. 10). In a human-centered learning activity system, creativity is inseparable from social orientation and empathy (Chell, 2007). Awareness of context in terms of resources, rules, and roles of constituents has made its impact in innovative maker education (Halverson & Sheridan, 2014; Martin, 2015). Creativity of ILE design has also made its initial impact in the social entrepreneurial education through quickness of recognizing opportunities, willingness of risk-taking, being flexible, and exploring with endurance (Chell, 2007; Entwistle & Peterson, 2004; Hokanson, Miller, & Hooper, 2008; Russell & Schneiderheinze, 2005). Innovative learning project examples like Stanford d.school, digital entrepreneurship education, hackathon for environmental stewardship, interdisciplinary learning by design, etc. are just a few. These potential directions and growth for ID professionals and their contributions to human learning need a perspective of viewing learning in an activity system and connected systems.

References Amory, A. (2010). Education technology and hidden ideological contradictions. Educational Technology & Society, 13(1), 69–79. Berrett, D., Mcmurtrie, B., & Supiano, B. (2018, June 21). Not just for video games: Virtual reality joins the classroom. The Chronicle of Higher Education-Teaching Newsletter. Retrieved from https://www.chronicle.com/article/Not-Just-for-Video-Games-/243729 Bonk, C., Lee, M. M., Reeves, T. C., & Reynolds, T. H. (2015). MOOCs and open education around the world. London, England: Routledge. Brayko, K. (2013). Community-based placements as contexts for disciplinary learning: A study of literacy teacher education outside of school. Journal of Teacher Education, 64(1), 47–59. Butz, N. T., Stupnisky, R. H., Pekrun, R., Jensen, J. L., & Harsell, D. M. (2016). The impact of emotions on student achievement in synchronous hybrid business and public administration programs: A longitudinal test of control-value theory. Decision Sciences Journal of Innovative Education, 14(4), 441–474. https://doi.org/10.1111/dsji.12110. Carter, L. M., Beattie, B., Caswell, W., & Fitzgerald, S. (2015). An examination of interprofessional team functioning in a BScN blended learning program: Implications for accessible distancebased nursing education programs. Canadian Journal of University Continuing Education, 41(1), 1–14. Chell, E. (2007). Social enterprise and entrepreneurship: Towards a convergent theory of the entrepreneurial process. International Small Business Journal, 25(1), 5–26. https://doi.org/ 10.1177/0266242607071779. Collins, P., Shukla, S., & Redmiles, D. (2002). Activity theory and system design: A view from the trenches. Computer Supported Cooperative Work, 11(1–2), 55–80. https://doi.org/10.1023/ A:1015219918601. Cooksy, L. J., Gill, P., & Kelly, P. A. (2001). The program logic model as an integrative framework for a multimethod evaluation. Evaluation and Program Planning, 24(2), 119–128. https://doi. org/10.1016/S0149-7189(01)00003-9. Dorner, H., & Kárpáti, A. (2010). Mentoring for innovation: Key factors affecting participant satisfaction in the process of collaborative knowledge construction in teacher training. Journal of Asynchronous Learning Network, 14(4), 63–77. Double Robotics, Inc. (2017). Blended learning and hybrid classrooms. Retrieved from https:// www.doublerobotics.com/education/.

1014

J. C. Liu

Eick, C. J., & Reed, C. J. (2002). What makes an inquiry-oriented science teacher? The influence of learning histories on student teacher role identity and practice. Science Education, 86(3), 401–416. https://doi.org/10.1002/sce.10020. Engeström, Y. (1995). Objects, contradictions and collaboration in medical cognition: An activitytheoretical perspective. Artificial Intelligence in Medicine, 7(5), 395–412. https://doi.org/ 10.1016/0933-3657(95)00012-U. Engeström, Y. (2000). Activity theory as a framework for analyzing and redesigning work. Ergonomics, 43(7), 960–974. https://doi.org/10.1080/001401300409143. Engeström, Y. (2001). Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and Work, 14(1), 133–156. https://doi.org/10.1080/ 13639080020028747. Entwistle, N. J., & Peterson, E. R. (2004). Conceptions of learning and knowledge in higher education: Relationships with study behaviour and influences of learning environments. International journal of educational research, 41(6), 407–428. Estes, M. D., Ingram, R., & Liu, J. C. (2014). A review of flipped classroom research, practice, and technologies. International HETL Review, 4(7), 1–8. Estes, M. D., Liu, J., Zha, S., & Reedy, K. (2014). Designing for problem-based learning in a collaborative STEM lab: A case study. TechTrends, 58(6), 90–98. https://doi.org/10.1007/ s11528-014-0808-8. Fink, L. D. (2013). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco, CA: Wiley. Fire, N., & Casstevens, W. J. (2013). The use of cultural historical activity theory (CHAT) within a constructivist learning environment to develop core competencies in social work. Journal of Teaching in Social Work, 33(1), 41–58. https://doi.org/10.1080/08841233.2012.749828. Flaherty, C. (2017, February 23). Large-scale assessment without standardized tests. Inside Higher Ed. Retrieved from https://www.insidehighered.com/news/2017/02/23/aacu-releases-reportnational-large-scale-look-student-learning Flanigan, R. L. (2017, June 12). “Course-choice” efforts grow to give students more options. Education Week. Retrieved from https://www.edweek.org/ew/articles/2017/06/14/coursechoice-efforts-grow-in-rural-schools.html Foot, K. A. (2014). Cultural-historical activity theory: Exploring a theory to inform practice and research. Journal of Human Behavior in the Social Environment, 24(3), 329–347. https://doi. org/10.1080/10911359.2013.831011. Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415. https:// doi.org/10.1073/pnas.1319030111. Graves, S. M., Abbitt, J., Klett, M. D., & Wang, C. (2009). A mentoring model for interactive online learning in support of a technology innovation challenge grant. Journal of Computing in Teacher Education, 26(1), 5–16. https://doi.org/10.1080/10402454.2009.10784626. Halverson, E. R., & Sheridan, K. (2014). The maker movement in education. Harvard Educational Review, 84(4), 495–504. https://doi.org/10.17763/haer.84.4.34j1g68140382063. Hayes, K. J., Eljiz, K., Dadich, A., Fitzgerald, J. A., & Sloan, T. (2015). Trialability, observability and risk reduction accelerating individual innovation adoption decisions. Journal of health organization and management, 29(2), 271–294. Hokanson, B., Miller, C., & Hooper, S. (2008). A Contemporary Perspective for Innovation in Instructional Design. TechTrends, 52(6), 36–43. Istance, D., & Kools, M. (2013). OECD work on technology and education: Innovative learning environments as an integrating framework. European Journal of Education, 48(1), 43–57. https://doi.org/10.1111/ejed.12017. Jonassen, D. H. (1997). Instructional design models for well-structured and III-structured problemsolving learning outcomes. Educational Technology Research and Development, 45(1), 65–94. https://doi.org/10.1007/BF02299613.

39

Design of Innovative Learning Environment: An Activity System Perspective

1015

Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 48(4), 63–85. https://doi.org/10.1007/BF02300500. Jonassen, D. H., & Rohrer-Murphy, L. (1999). Activity theory as a framework for designing constructivist learning environments. Educational Technology Research and Development, 47(1), 61–79. Kidwell, P. K., Freeman, R., Smith, C., & Zarcone, J. (2004). Integrating online instruction with active mentoring to support professionals in applied settings. Internet and Higher Education, 7(2), 141–150. https://doi.org/10.1016/j.iheduc.2004.03.003. Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory Into Practice, 41(4), 212–218. Layng, J. (1997). Parallels between project management and instructional design. Performance Improvement, 36(6), 16–20. https://doi.org/10.1002/pfi.4140360605. Leacock, T. L., & Nesbit, J. C. (2007). A framework for evaluating the quality of multimedia learning resources. Educational Technology & Society, 10, 44–59. https://doi.org/10.1017/ CBO9781107415324.004. Lewis, K., Henriques, J., Liu, J. C., & Brantmeier, E. J. (2016). Opportunities and barriers for community-engaged scholarship: An exploratory study at a comprehensive university. National Social Science Journal, 47(1), 50–59. Liu, J., & Alexander, R. (2017). Factors affecting faculty use of video conferencing in teaching: A mixed-method study. Journal of Educational Technology Development and Exchange (JETDE), 10(2), 37–54. Liu, J. C. (2012). Customized consultation to support design and development of blended courses. Presentation at the 18th annual Sloan consortium international conference on online learning, Orlando, FL. Liu, J. C., & Adams, A. (2017). Design of online student orientation with conceptual and procedural scaffolding. In F. Q. Lai & J. D. Lehman (Eds.), Learning and knowledge analytics in open education (pp. 41–68). Cham, Switzerland: Springer International Publishing. https://doi.org/ 10.1007/978-3-319-38956-1_5. Liu, J. C., Swayne, D., & Adams, A. (2017). Design for deep learning through facilitating an entrepreneurial mindset. Presentation at HKAECT2017 international research symposium, Hong Kong. Lu, W.-H., Deen, D., Rothstein, D., Santana, L., & Gold, M. R. (2011). Activating community health center patients in developing question-formulation skills. Health Education & Behavior, 38(6), 637–645. https://doi.org/10.1177/1090198110393337. Mahraj, K. (2012). Using information expertise to enhance massive open online courses. Public Services Quarterly, 8(4), 359–368. https://doi.org/10.1080/15228959.2012.730415. Martin, L. (2015).The promise of the maker movement for education. Journal of Pre-College Engineering Education Research (J-PEER), 5(1), 30–39. https://doi.org/10.7771/21579288.1099. McDavid, J. C., & Hawthorn, L. R. L. (2006). Program evaluation & performance measurement: An introduction to practice. Thousand Oaks, CA: Sage. Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70, 29–40. https://doi.org/10.1016/j. compedu.2013.07.033. Merriweather, L. R., & Morgan, A. J. (2013). Two cultures collide: Bridging the generation gap in a non-traditional mentorship. Qualitative Report, 18(6), 1–16. Michaelsen, L. K., & Sweet, M. (2011). Team-based learning. New Directions for Teaching and Learning, 2011, 41–51. https://doi.org/10.1002/tl.467. Moog, R. S., & Spencer, J. N. (2008). POGIL: An overview. ACS Symposium Series, 994, 1–13. Washington, DC: American Chemical Society. https://doi.org/10.1021/bk-2008-0994.ch001. Park, K. (2015). Instructional design models for blended learning in engineering education. International Journal of Engineering Education, 31(2), 476–485.

1016

J. C. Liu

Postholm, M. B. (2015). Methodologies in cultural–historical activity theory: The example of school-based development. Educational Research, 57(1), 43–58. https://doi.org/10.1080/ 00131881.2014.983723. Pritchard, P. A. (2010). The embedded science librarian: Partner in curriculum design and delivery. Journal of Library Administration, 50(4), 373–396. https://doi.org/10.1080/ 01930821003667054. Quitadamo, I. J., Brahler, C. J., & Crouch, G. J. (2009). Peer-led team learning: A prospective method for increasing critical thinking in undergraduate science courses. Science Educator, 18(1), 29–39. Rinaldo, S. B., Davis, D. F., & Borunda, J. (2015). Delivering value to community partners in service-learning projects. Journal of Community Engagement and Scholarship, 8(1), 115–125. https://doi.org/10.5193/JEE33.3.208. Rodriguez, J. E., Greer, K., & Shipman, B. (2014). Copyright and you: Copyright instruction for college students in the digital age. The Journal of Academic Librarianship, 40(5), 486–491. https://doi.org/10.1016/J.ACALIB.2014.06.001. Rogers, E. M. (2010). Diffusion of innovations (4th ed.). New York, NY: Simon and Schuster. Roth, W.-M. (2004). Activity theory and education: An introduction. Mind, Culture, and Activity, 11(1), 1–8. https://doi.org/10.1207/s15327884mca1101_1. Rothstein, D., & Santana, L. (2011). Teaching students to ask their own questions. Harvard Education Letter, 27(5), 1–2. Ruder, S. M., & Hunnicutt, S. S. (2008). POGIL in chemistry courses at a large urban university: A case study. ACS Symposium Series, 994, 133–147. https://doi.org/10.1021/bk-2008-0994. ch012. Russell, D. L., & Schneiderheinze, A. (2005). Understanding innovation in education using activity theory. Educational Technology & Society, 8(1), 38–53. Rybacki, K. (2009). Cultural historical activity theory as a tool for informing and evaluating technology in education. Children, Youth & Environments, 19(1), 279–305. Schaffer, S. P., Reyes, L., Kim, H., & Collins, B. (2010). Using activity theory to understand learning design requirements of patient self-management environments. Educational Media International, 47(4), 329–342. Schiffman, S. S. (1986). Instructional systems design: Five views of the field. Journal of Instructional Development, 9, 14–21. https://doi.org/10.2307/30220829. Schuch, D., Liu, J. C., & Bona, S. (2015). Mentoring graduate students in instructional technology: What we learned from PacifiCorp D&D mentoring. Presentation at the 2015 Association for Educational Communications and Technology (AECT) International Convention, Indianapolis, IN. Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400. Simonson, S. R., & Shadle, S. E. (2013). Implementing process oriented guided inquiry learning (POGIL) in undergraduate biomechanics : Lessons learned by a novice. Journal of STEM Education, 14(1), 56–64. Straumsheim, C. (2017, July 26). MIT deems half online, half in-person master’s program a success. Inside Higher Ed. Retrieved from https://www.insidehighered.com/news/2017/07/26/ mit-deems-half-online-half-person-masters-program-success Takeuchi, M., & Esmonde, I. (2011). Professional development as discourse change: Teaching mathematics to English learners. Pedagogies, 6(4), 331–346. Thompson, A., Dagenhard, P., Castor, T., & Brookins-Fisher, J. (2016). Health education doctoral degree programs: A review of admission and graduation requirements. Health Educator, 48(2), 16–22. Tien, L. T., Roth, V., & Kampmeier, J. A. (2002). Implementation of a peer-led team learning instructional approach in an undergraduate organic chemistry course. Journal of Research in Science Teaching, 39(7), 606–632. https://doi.org/10.1002/tea.10038.

39

Design of Innovative Learning Environment: An Activity System Perspective

1017

Valentín-Rivera, L. (2016). Activity theory in Spanish mixed classrooms: Exploring corrective feedback as an artifact. Foreign Language Annals, 49(3), 615–634. Van Rooij, S. W. (2010). Project management in instructional design: ADDIE is not enough. British Journal of Educational Technology, 41(5), 852–864. https://doi.org/10.1111/j.14678535.2009.00982.x. Wilson, B. G. (1996). Constructivist learning environments: Case studies in instructional design. Englewood Cliffs, NJ: Educational Technology Publications. Woods, D. R. (2014). Problem-oriented learning, problem-based learning, problem-based synthesis, process oriented guided inquiry learning, peer-led team learning, model-eliciting activities, and project-based learning: What is best for you? Industrial and Engineering Chemistry Research, 53(13), 5337–5354. https://doi.org/10.1021/ie401202k. Yamagata-Lynch, L. C. (2007). Confronting analytical dilemmas for understanding complex human interactions in design-based research from a cultural – Historical activity theory (CHAT) framework. Journal of the Learning Sciences, 16(4), 451–484. https://doi.org/10.1080/ 10508400701524777. Yamagata-Lynch, L. C. (2010). Understanding cultural historical activity theory. In L. C. YamagataLynch (Ed.), Activity systems analysis methods (pp. 13–26). Boston, MA: Springer US. https:// doi.org/10.1007/978-1-4419-6321-5_2. Yamagata-Lynch, L. C., & Smaldino, S. (2007). Using activity theory to evaluate and improve K-12 school and university partnerships. Evaluation and Program Planning, 30(4), 364–380. https:// doi.org/10.1016/j.evalprogplan.2007.08.003.

Dr. Juhong Christie Liu is an Assistant Professor/Senior Instructional Designer in James Madison University. She teaches undergraduate, graduate, and organizational learning classes in face-toface, online asynchronous, and synchronous formats. Her research focuses on discipline-based education research (DBER) related to instructional design for science education and communitybased learning, teaching and learning with emerging technologies, and evaluation and assessment for technology-mediated learning. Juhong has a Ph.D. degree in Curriculum and Instruction/ Instructional Design and Technology from Virginia Tech. She has published peer-reviewed journal articles and book chapters. Regularly she presents at international, national, and regional conferences. She also serves on the NSF/DRL review panels and Editorial Board of Journal of Educational Technology Development and Exchange (JETDE) and reviews for more than six journals including Journal of Computer Assisted Learning (JCAL).

A Process Method Approach to Study the Development of Virtual Research Environments: A Theoretical Framework

40

Iftekhar Ahmed and Marshall Scott Poole

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Process Research and VRE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Activity Tracks of the Proposed Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The VRE Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Management and Organizational Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Critical Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developmental Models for the Tracks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developmental Model for Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developmental Model for Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developmental Model for the VRE Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developmental Model for Management and Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developmental Model for Critical Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interactions Among the Developmental Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1020 1023 1024 1024 1026 1028 1029 1030 1030 1031 1033 1033 1034 1035 1035 1036 1037

Abstract

In recent years, there has been a wave of advanced cyberinfrastructure development to support distributed collaborative science. These cyberinfrastructures or “Virtual Research Environments” (VRE) are electronic spaces for inquiry and meeting places for interaction among scientists and scholars created by combining software tools and computer networking. VREs have been hailed as having I. Ahmed (*) Department of Communication Studies, University of North Texas, Denton, TX, USA e-mail: [email protected] M. S. Poole Department of Communication, University of Illinois Urbana-Champaign, Urbana, IL, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_118

1019

1020

I. Ahmed and M. S. Poole

the potential to enhance the quality of science, to speed up the conduct of scientific research, and to foster global scientific communities around key research and learning areas. Multiple approaches have been applied to investigate technological, organizational, managerial, and human factors that influence VREs for good or ill, and these have yielded insights, but there is not yet a “formula” for an effective VRE and therefore all VREs involve experimentation and trial-anderror learning. This chapter suggests a framework for understanding the processes by which VREs are developed over time and how these processes contribute to their effectiveness or lack thereof. Keywords

Process study · Virtual organization · Virtual research environments · Virtual learning environments · Scientific collaboration · Organizational development · Longitudinal analysis

Introduction In recent years, there has been a wave of advanced cyberinfrastructure development to support distributed collaborative science. These cyberinfrastructures, which we will refer to as “Virtual Research Environments” (VRE), are electronic spaces for inquiry and meeting places for interaction among scientists and scholars created by combining software tools and computer networking. VRE is a general term that subsumes other more specialized terms, including collaboratory, cyberenvironment, virtual learning environment, scientific cyberinfrastructure, and virtual laboratory. These virtual research environments (VREs) have been hailed as having the potential to enhance the quality of science, to speed up the conduct of scientific research, and to foster global scientific communities around key research and learning areas. The Joint Laboratory for Extreme Scale Computing (JLESC) is a good example of a VRE that was founded in 2009 by French academic computing organization INRIA and the National Center for Supercomputing Applications (NCSA) of the University of Illinois at Urbana Champaign. The purpose of this initiative was to bring together supercomputing centers to engage in projects that advance supercomputing and ensure that supercomputing serves its application communities effectively. JLESC not only wanted to bring supercomputing centers into contact to engage ideas or to address common problems, but to foster learning. Over the years, JLESC evolved into a multinational and multidisciplinary consortium of large scientific organizations. It includes diverse participants, including management, senior scientists, technologists, postdocs, and graduate students, of different cultural and linguistic background from four different continents. JLESC developed a model of loose structured collaborative practice to work and advance state of the art in supercomputing. Moreover, it developed a system of mentor–mentee relationship based teaching and learning process through dispersed collaboration, evolving set of workshops, and site visits.

40

A Process Method Approach to Study the Development of Virtual. . .

1021

JLESC provides a comprehensive picture of dispersed scientific collaboration, community development, and learning; however, not all VREs are developed for the same purpose. There are seven types of VREs or collaboratories based on their functionality (Bos et al., 2007; Olson & Olson, 2013): 1. Distributed research center (aggregate scientific talent, effort, and resources; unified by a topic area of interest and joint projects in that area) 2. Shared instrument (remote access to expensive scientific instruments supplemented with communication and collaboration tools) 3. Community data systems (data and information repository created and maintained by geographically distributed research community) 4. Open community contribution system (an open project that aggregates efforts of many geographically separate individuals toward a common research problem; contributions come in the form of work rather than data) 5. Virtual community of practice (network of individuals who share a research area and communicate about it online; not focused on actually undertaking joint projects) 6. Virtual learning community (cyberinfrastructure to support knowledge generation but not necessarily to conduct original research) 7. Community infrastructure project (seek to develop common resources that facilitate science, such as software tools, standardized protocols, new types of scientific instruments, and educational methods to further work in a particular domain) There is a move toward standardization for VREs. Scientists and developers are experimenting to work out the appropriate cyberinfrastructure to facilitate research. Sakai, a collaboration among various academic, governmental, and commercial entities to develop collaborative technologies for virtual learning environments (http://sakaiproject.org/portal), and the Joint Information Systems Committee (JISC), a UK-funded project that is experimenting with information and communication technologies to support education and research (http://www.jisc.ac.uk/), exemplify those activities. Cyberinfrastructure-enabled projects have made impressive advances, and the potential of VREs seems evident. However, still at issue is how best to realize that potential. We know little regarding what technologies should be included and what combinations of technologies and features are most effective for various types of VREs. A large literature on online communities and the technologies and practices that make them effective exists (e.g., Preece, 2000), and this provides some guidance for builders and users of VREs. However, due to the unique nature of scientific inquiry and the specialized knowledge and expertise required of scientific team members, scientific teams differ from traditional user or interest-based communities. What we know about the impacts of dispersion and of computer-mediated communication on work processes and communication (e.g., Paré & Dubé, 1999; Rice & Gattiker, 2001; Scott, 2000; Poole & Zhang, 2005) suggests that the scientific process in dispersed groups is likely to be similar in many ways, but different in others than that in collocated scientific teams.

1022

I. Ahmed and M. S. Poole

Virtual learning environments (VLEs) face similar uncertainty, especially regarding factors that contribute to their design and sustainability (Teo, Chan, Wei, & Zhang, 2003). Moreover, the concept of VLE is also not recognized by scholars and practitioners in its full potential. VLEs are predominantly understood as technology mediated environments fostering virtual education, almost similar to an online classroom. From this viewpoint, we can define a virtual learning environment as an information space that integrates a variety of tools and technologies to support learning and learning management, provide information, and foster communication and collaboration (Dillenbourg, Schneider, & Synteta, 2002). Although several scholars argue that the major interest of VLEs is to develop an educational community or a community of teachers instead of developing learning environments (Dillenbourg et al., 2002; Rogers, 2000), others conceptualize VLEs as information and communication technology (ICT) based cyberspaces for dispersed individuals and groups to meet their e-learning goals (Chen & Chen, 2009). The main objective of an educational community or a community of teachers is to provide a discussion forum for the audience. The discussion forum allows audience to discuss their teaching or learning approaches, share experience, answer questions and queries, and to share information about educational tools and resources. However, the promise of VLEs could be much broader and more significant. A VLE should not merely be a space for e-learning; it could very much develop as a cyberinfrastructure for learning and knowledge building activities (Swan & Shea, 2005). A VRE perspective acknowledges the broader impact and significance of virtual learning environments (VLEs). There have been some notably successful VREs, and others that are clearly failures, but most VREs are somewhere in between, with elements that have been quite effective and are extensively utilized and other elements that are not so successful. At this point, it is uncertain what accounts for the functioning and effectiveness of VREs. One approach to understanding VREs is to identify technological, organizational, managerial, and human factors that influence the VRE for good or ill, often in the form of “lessons learned,” but sometimes in more explicit models (Olson, Zimmerman, & Bos, 2008). Another approach, taken by Ahmed and Poole (2011), is to decompose the VRE into specific technologies, identify clusters of technologies, and relate them to VRE processes, products, and effectiveness. Both of these approaches have yielded insights, but they omit an important element – time (Olson et al., 2008; Bos et al., 2007). VREs do not just spring up full grown, they develop over time. A VRE is the product of multiple decisions related to the way in which science will be conducted, technological design and implementation, and who will be involved in the design and governance of the VRE, among other things. These decisions are spaced over time and prior decisions set the context for (but do not fully determine) later ones. The temporal dimension of VREs is critical not only because constructing them takes time, but also because of the absence of a “formula” for an effective VRE (and perhaps there never will be), and therefore all VREs involve experimentation and trial-and-error learning. As with other technologies, much can be learned about VREs by reconstructing the “technological trajectory” by which they develop

40

A Process Method Approach to Study the Development of Virtual. . .

1023

(Clark, 1985; Henderson & Clark, 1990). But it is also important that technology not be studied in isolation from the scientific, social, and organizational processes that form the context of the VRE. This chapter suggests a conceptual framework for understanding the processes by which VREs are developed over time and how these processes contribute to their effectiveness or lack thereof. The chapter first provides a brief description of the process study and its relation to VRE, then introduces the theoretical framework.

Process Research and VRE VREs have been studied extensively and their components are fairly well understood (Olson et al., 2008; Ahmed & Poole, 2011), and so a structured process approach is appropriate to study VREs. Rather than framing explanations in terms of cause–effect relationships among variables (which underlies most factor-based and lessons-learned accounts of VREs), a process theory focuses on a series of events that bring about or lead to some outcome, and attempts to specify the generative mechanism that could produce the event series. Explanations in process theories take the form of theoretical narratives that account for how one event led to another and that one to another, and so on to the final outcome (Mohr, 1982; Langley, 1999; Poole, Van de Ven, Dooley, & Holmes, 2000; Langley & Tsoukas, 2010; www. process-symposium.com). A process theory is tested by identifying or measuring observable events and determining whether the relations among events are what would be expected if the generative mechanism in the process theory was in operation. A process theory yields understanding of how a VRE develops, critical turning points, and how multiple contextual elements interact as the process unfolds. While it is possible to relate the process as a whole to its ultimate effectiveness as judged by various criteria (e.g., Poole & Holmes, 1995), it is often more useful to consider the “ups and downs” of effectiveness of the developing entity at various points in its process. What looks like a roaring success at one point may turn out to be a failure at another or vice versa. Conducting a systematic process study of the development of a set of VREs will yield several advances. First, one key to understanding any phenomenon is to understand how it came to be what it is. A process study will illuminate how VREs attain particular configurations of technological, scientific, and social structures during the “rough and tumble” of development by a community of actors with diverse perspectives and interests over an extended period of time. This will generate insights into the functional and dysfunctional processes that influence VRE development and also into critical points at which action should be taken to avoid problems or capitalize on positive developments. Second, a process study will also identify one or more developmental paths that VREs typically follow. Third, a process study should enable us to sort out the role of several structuring processes (i.e., technological, scientific, social, political) in shaping VREs at various points in their development, thus differentiating VREs on the basis of the critical processes

1024

I. Ahmed and M. S. Poole

that influence their evolution. It is likely that different driving processes lead to differences in VRE effectiveness. Finally, a process study will yield insights into the dynamics of VREs, rather than considering them as relatively stable, set entities. Like all social systems, VREs are continuously changing and a study focused on change processes is uniquely suited to capture this.

Activity Tracks of the Proposed Framework Our framework proposes that VREs are constructed through interactions among critical activity tracks that form a sociotechnical system. Tracks are more or less independently identifiable aspects of the process that exert an influence over it. Tracks are for the most part conceptual devices and empirically they are related to one another. Distinguishing tracks gives structure to the analysis, accounting for key elements in the development process. This framework outlines a process study that distinguishes activity tracks in the developing process (Poole et al., 2000; Poole & Van de Ven, 2010), maps these tracks separately, and then identifies the interrelationships among tracks (see Van de Ven, Polley, Garud, & Venkatraman, 1999; Van de Ven, Angle, & Poole, 2000). Our framework proposes that VREs are constructed through interactions among five critical activity tracks: (a) technological design and implementation, (b) scientific work, (c) the community of VRE users, developers, funders, and other stakeholders, (d) managerial and organizational system, and (e) critical events that impinge on the process. Table 1 provides a brief overview of tracks and their constituents. Activities in each of the tracks proceed according to different developmental processes and at different paces and the interrelationships among them account for the developmental trajectory of the VRE. Effectiveness of the VRE has a complex relationship with the developing tracks. On the one hand, effective coordination among tracks should have a positive impact on effectiveness of the VRE and specific elements within it. On the other hand, periodic evaluations of effectiveness influence how the activities on the tracks are carried out. The VRE also may change various aspects of the tracks (e.g., management structure, technological design) in response to perceived trends in effectiveness. The following sections introduce and define the activity tracks, discuss the developmental processes that are likely to characterize those tracks, and identify various types of possible inter-track relationships and influences.

Technology A VRE is a collection of technical capabilities which is constructed by a complex assembly of hardware and software technology, organized into a particular environment. Some of the technology is general infrastructure, such as the network connectivity, general operating systems, or the database management systems. Other technology may be institutional infrastructure, such as local server farms or middleware shared with other similar projects. Finally, some of the technology will be

40

A Process Method Approach to Study the Development of Virtual. . .

1025

Table 1 VRE tracks and constituents Track Technology

Science

VRE community

Management and organizational structure

Critical events

Constituents Collection of technical capabilities which is constructed by a complex assembly of hardware and software technology, organized into a particular environment. Three types of technical capabilities include: General infrastructure (networks, operating systems) Institutional infrastructure (middleware, local machines) VRE specific technology (tools, workflows, data collections) Specific scientific projects carried out with the VRE. There are two processes within this track: Knowledge generation Social collaboration Various types of stakeholders. Three major groups of stakeholders include: VRE members: scientists, developers, and support personnel Funders VRE users: scientific organizations and individuals The organization of VREs is divided into two different structural sets: Structure for administration and external relations Structure to manage research and scientific activities One-time occurrences that impact developing VRE. Occurrences emerge due to: External environmental influences Reflective evaluation of the VRE

very specific to the VRE itself, including tools, workflows, data collections, and implementations of policies such as access control. It is important to understand how technology specific to the VRE depends on and may be related to other technologies. If the VRE strongly depends on some general purpose technology, it may be necessary for developers to further develop that technology, even if it is not under their control or otherwise relevant to their goals. For example, a VRE that is implemented as portal built with a particular framework may well have to evolve its portal as the framework evolves. Most VREs implicitly depend upon the existence of personal computers connected to the Internet. But, unless the VRE makes unusual demands on these technologies (e.g., they have to work in remote areas “off the grid,” or require very high data rates), these technologies will probably not have impact on the processes of the VRE. On the other hand, a VRE might focus on certain external resources, such as one-of-a-kind instruments or data services not administered by the VRE. In this case, the external systems may need to be considered part of the VRE, along with any processes associated with their use. For example, a VRE which uses a major telescope run by an observatory will depend upon the technical systems driving the instrument, even though they are not implemented or controlled by the VRE. AVRE is not a monolith; it is an integration of hardware, protocols, and software; each with its own life story. Any and all of these components may change during the

1026

I. Ahmed and M. S. Poole

life of the VRE, whether directed by the VRE or not. For this reason, the creation and maintenance of a VRE is not a single process, nor a single schedule. Any given activity in the VRE will explicitly depend upon certain technologies, and also implicitly on a mesh of other technologies. Therefore to understand the VRE, it will be important to discover the most important technological “drivers,” i.e., the technological dependencies that have the most influence on the overall VRE. VREs are usually built primarily through system integration. While the VRE can be conceived as a coherent collection of capabilities, there are usually alternative strategies for realizing each capability, and the development and maintenance of the VRE potentially will have many simultaneous tracks of technological activity. The most basic question in the implementation of a VRE is “buy or build.” It means that a VRE needs to decide whether the technological capability should be met by adopting some existing technology, perhaps with modification, or should it be custom built. The option to “buy” also includes open source and reusable software, as well as commercial software. Answering this question calls into play arguments about the precise definition of the capability, resource constraints, and reference to existing practices. Most capabilities will be built by integrating a number of components from multiple sources, along with a relatively limited amount of “glue” and customization. For this reason, the development process most likely will not resemble software development processes; rather it will be a system integration process, one which may happen gradually over time. So, one would expect to see the system develop in a series of updates to gradually evolve a system as it is used. Creating and maintaining the technology that undergirds the VRE is both a significant process within the VRE and also potentially a driver for other processes. However, it is usually not easy to identify the “edges” of the VRE’s technology, both because it can be difficult to distinguish essential capabilities from accidental features, and, more significantly, because the VRE is necessarily entangled with general infrastructure, institutional infrastructure, and products. For a given VRE, it will be important to analyze the technology to identify the essential capabilities, and also the mesh of technical dependencies and their importance for specific activities within the given VRE. This analysis begins with some fundamental questions, including: (1) what are the key/core technologies in the VRE? (2) What types of technological interdependencies exist? and (3) What processes are used to create, integrate, deploy, and maintain the VRE?

Science Scientific discovery is a sociotechnical process. From a general perspective, activities related to scientific discovery “can be viewed as a communication cycle having three progressive stages: conceptualization, documentation, and popularization” (Lievrouw & Carley, 1990, p. 459) and as a scientific inquiry cycle having four stages: a data representation space, a hypothesis space, an experimental paradigm space, and an experiment space (Schunn & Klahr, 1995; Simon & Lea, 1974).

40

A Process Method Approach to Study the Development of Virtual. . .

1027

Hence, science involves two distinctive collaborative processes: “social collaboration” and “knowledge generation.” Social collaboration depends on peer-to-peer communication in the early stages of scientific inquiry, which implies that interaction occurs in a fairly homogenous group. In later stages, when the findings are disseminated to scientific and other audiences, collaboration expands to heterogeneous groups and moves from small group communication to mass communication. Within initial stages, the communication process depends upon the exchange of a significant amount of social and scientific information. Hence, the effectiveness of group work in the scientific process largely depends upon effective communication as well as effective management of scientific data and information. During the initial stages of scientific collaboration, geographical dispersion changes the nature of communication processes. The particular mode of communication built into the VRE will influence the research process and its progress (Carley & Wendt, 1991). To support the conduct of science, VRE technology should provide adequate access to experimental, modeling, visualization, simulation, and data analysis tools. Scientists should be able to use these tools from their remote locations individually and/or collaboratively. Scientists need to have extensive technologies for documentation. These technologies should enable them to edit reports, research materials, and data. Technology should also provide facilities to disseminate, aid gatekeeping, and safeguarding these documents and materials. VREs incorporate a number of different technologies. Many sites attempt to duplicate traditional information sharing, coauthoring, and e-learning activities. However, a growing number of scientific and educational collaboration sites attempt to go further and offer new capabilities that provide a space for scientists to conduct collaborative research. Studies have identified a comprehensive list of available communication and/or collaboration technologies, including Announcements, Blogs, Calendar, Chat, Discussion boards, Email, Instant messaging, Interactive learning materials, Site to access and manage content, Site to share files, Polling, Shared bookmarks, Video conferencing, Whiteboard, and Wikis (Adelsberger, Collis, & Pawlowski, 2013; Awre & Ingram, 2005). A review of available technologies suggests that the properties of VRE technology will fall under five specific categories: (i) peer-to-peer communication/collaboration technology: technologies that facilitate dyadic and group communication of individuals of the scientific community (i.e., e-mail system, bulletin boards, wikis, ticket systems, content management systems); (ii) scientific instruments: scientific research instruments and tools to generate, collect, manage, and analyze data (i.e., space telescope); (iii) databases and data stores: open, shared, or controlled repository of raw or manipulated data within the VRE system; (iv) internal search systems: systems that facilitate scientific data and document search within the VRE; (v) information sharing systems: information sharing aspects will include the following: (a) document dissemination systems and (b) links to external resources (instruments, databases, websites, blogs, external search engines, and other VREs). The effectiveness of a VRE depends upon technology that supports both aspects of the scientific process, social collaboration, and knowledge generation.

1028

I. Ahmed and M. S. Poole

The VRE Community The VRE community is composed of a set of stakeholders that includes scientists, developers, funders, support personnel, and participating scientific organizations, among others. This community may be structured in different ways, from egalitarian and open involvement to hierarchical organizations. The relative levels of influence of the various sets of stakeholders and of individuals within those sets affect how the VRE evolves. Preece (2000) has discussed usability and sociability as two key dimensions of the design of community spaces for individual users. Usability refers to the requirements for effective human–computer interaction. Sociability refers to the requirements for building and sustaining an effective community. Preece (2000) argues that individual participation in communities is stimulated by assessing community needs with respect to usability and sociability. The key to usability for community design is defining tasks (information exchange, tool access, collaborative writing, etc.), understanding the nature of users (their specific characteristics and needs depending on individual characteristics and also disciplines), and the task-user interactions (e.g., work patterns, access needs). Sociability depends on delimiting the purpose of the community, defining key roles needed to make the community work and populating those roles (as well as designing the technology to accommodate those roles), defining use policies (membership, access, privacy, security, and developing a culture specific to the VRE and its purpose). Putting features, norms, and practices in place that encourage collaboration is a critical aspect of sociability for VREs. Preece (2000) argues that effective design of a community virtual space for usability and sociability requires involvement of users through “community-centered development.” Key activities to be indexed in understanding involving individuals in communities include those that contribute (or detract from) usability and sociability, the design process, and how the community and collaborations typically unfold through the steps of the scientific process. At the organizational level, outline a model that argues that organizations choose to participate in online communities based on connectivity and communality, key dimensions of resources they draw from the community as a public good. Connectivity refers to the ability of the community to network organizations in ways that are rewarding to them, while communality refers to common information stores (and in the larger scheme of things, resources) that the community is able to supply by virtue of the collective contributions of members. Key activities in community building for organizations, then, are those that contribute to (or detract from) connectivity (e.g., engaging in online conferences, facilitating/inhibiting participation of members in the VRE) and communality (e.g., contributing data or tools to the VRE, requiring that certain data or tools have restricted access). Usability and sociability aspects of the VRE are obviously important for effective organizational participation in the VRE, but they are only loosely connected to communality and community drivers of participation. Organizations may require their members to participate in a VRE with poor usability and sociability. However, the VRE is likely to be more active and effective when the two levels – individual and organizational – are

40

A Process Method Approach to Study the Development of Virtual. . .

1029

aligned in such a way that they facilitate one another. Conversely, the VRE is likely to struggle if they are not aligned. Also important to the development of VREs are external organizations and institutions such as government agencies related to education, research, technology, or innovation. The Department of Energy’s Office of Science commenced a major initiative in VRE development in the USA when it developed an environment to share resources among all its laboratories. The National Research Council, NSF, and NASA have also been major forces behind VRE development in the USA. In the UK, a major initiative by the Joint Information Systems Committee (JISC) led to the development of VREs, as have similar programs in Australia (The Middleware Action Plan and Strategy Project (MAPS) and eResearch SA) and in all major countries and regions. These initiatives bring together a community of practice from universities, research institutes, public and private laboratories, and industry. VRE’s often are based in preexisting communities, which have a large imprint on the goals, organization, and practices of the VRE. Though the majority of participants are scientists, technicians, and scholars, a significant number of administrative personnel both from academia/laboratory and funding/agencies are involved. In some cases, these institutions are linked into the VRE directly as part of its community. In other cases, they serve more as external influences and so should be considered part of the fifth track.

Management and Organizational Structure From a management perspective, we can broadly divide the organization of VREs into two different structural sets: (i) organizational structure dedicated to administration and external relations (with funders, public, etc.) and (ii) organizational structure dedicated to managing the research itself. For example, in the Alliance for Cellular Signaling, administrative structure includes an Executive Committee which presides over Finance, Innovation, Policy, and Outreach Groups. Research is structured with a Steering Committee at the top, a Lab/Resources Group that attends to the various tools devoted to bioinformatics, protein analysis, etc., a Technology Development Group, an Editorial Group responsible for publishing results and managing the journals affiliated with the effort, and a large user community (see Gilman et al., 2002; AfCS, 2012, website for detail). There are several project management methods that provides useful model of managing eScience projects. The PRINCE2, developed by UK’s Central Computer and Telecommunications Agency (CCTA), is such an example. This method distinguishes four different levels of management for eScience projects: Corporate/Program Management, Project Board, Project Management, and Product Delivery Management. Not all VREs will have the four levels, and the same manager may engage activities on more than one level, but they represent a good schema for conceptualizing different levels of direction and organizational structure for VREs (Matos & Lopes, 2013; Warr et al., n.d.; Wideman, 2002).

1030

I. Ahmed and M. S. Poole

Achieving VRE goals provide another important aspect of management activities. We can distinguish two levels of goals for the VRE. First, there are goals related to the scientific enterprise that the VRE supports. These might be considered “operational goals” for the VRE. Second set of goals are related to the broader impact of the VRE. These might include education, influencing policy, building communities of scholars and practitioners, or founding a company to commercialize inventions. These two levels interact in that the scientific goals provide the foundation for broader impact goals, while broader impact goals often shape the evolving scientific agenda. Different managers and stakeholder groups are likely to hold different goals, to emphasize the two levels of goals differentially, and to shift in their goals over time. Goals influence the other activities, but activities, in turn, influence and may reshape goals.

Critical Events Critical events refer to one-time occurrences that impact a developing entity such as a VRE. Whereas the other tracks influence the process more incrementally, critical events exert “point influence” on the process and this influence is often significant and wide-ranging. It represents a very different type of influence than the generative mechanisms that motivate the other tracks. The generative mechanisms such as a project management life cycle exert gradual and mostly incremental influence on development and operate throughout lengthy periods or segments of the process. In contrast, a critical event occurs once and results in a major shift in the process. In the case of VREs, there are two types of critical events. One key element influencing any VRE process is the external environment of the VRE. Studies of organizational innovation and change indicate that external “shocks” – events such as budget cuts, changes in external institutions, and even historical events such as the attacks of 911 or a recession – can exert a major influence on the developmental process (Van de Ven et al., 2000). A complete characterization of the developmental process for VREs must include identification of key external events and tracing their impacts across the other tracks. A second important type of critical event is reflective evaluation of the VRE in terms of specific criteria of effectiveness. This may occur on a “schedule,” as when regular evaluation reviews are conducted to satisfy requirements of funders or an external board. Reflective evaluation may also occur more suddenly and irregularly, as when a funder or a major participant in the VRE has a change of management and the new management requires a review of existing commitments and projects.

Developmental Models for the Tracks Since the focus of the framework is on development, we will now discuss some of the basic developmental dynamics that are likely to hold in the five tracks. We will consider these separately for the present and later consider cross-track interactions.

40

A Process Method Approach to Study the Development of Virtual. . .

1031

The general theoretical framework for process research was developed by Van de Ven and Poole (1995; Poole et al., 2000) who posited that there are four basic generative mechanisms for organizational development and change. In the analysis of development of the VREs, this method will test for these generative mechanisms by evaluating evidence for the operation of one or more of the generative mechanisms in each track (see Poole et al., 2000, for specifics on this testing procedure). The four generative mechanisms of change are: 1. A life cycle generates change via a necessary sequence of stages that the developing entity must pass through. The function and content of each stage is determined some natural, logical, or institutional program that predates the cycle and prefigures how it unfolds. Life cycle theories assume that the process always unfolds through the same unitary sequence of stages. 2. A teleological mechanism generates change based on a sequence of goal formulation, implementation, evaluation, and modification of actions or of goals based on deviation of expected outcomes from actual outcomes. There is no necessary sequence of actions, but rather a more or less conscious choice of activities by an individual or group that experiences or manages the change. 3. Dialectical processes are driven by conflicts or tensions within the developing entity. Contradictions, conflicts, and tensions elicit reactions from actors, groups, or organizations, and these reactions shape how the dialectic unfolds. 4. An evolutionary model assumes that a population of entities (e.g., competing technologies) develops through sequences of variation, selection, and retention events. These general generative mechanisms are incorporated in specific developmental process models described below. Table 2 summarizes the developmental process models discussed in this section.

Developmental Model for Technology Two developmental models pertain to technology in VREs. First, the technological trajectory model (Clark, 1985; Henderson & Clark, 1990) posits that technologies like the automobile engine advance via a quasi-evolutionary process through which variants of the technology compete against one another until a dominant design emerges, which is then improved and modified through series of variations within the dominant design. Technology development can thus be modeled as a series of branching trees with some branches being selected out while others survive through an evolutionary process that selects toward fitness in terms of performance, industrial, legal, and economic context, consumer demand and other factors. This model can be readily adapted to the VRE context. Technological performance, developments in related and competing technologies, and the other tracks serve as contexts for the process of variation, selection, and retention.

1032

I. Ahmed and M. S. Poole

Table 2 Developmental processes within the tracks Track Technology

Science

VRE community Management and organizational structure

Critical events

Developmental models Technological trajectory, whereby specific technologies develop in an evolutionary process A stage-wise life cycle model whereby technological trajectories are institutionalized in a three stage process: (1) Knowledge–Awareness; (2) Evaluation–Choice; and (3) Adoption–Implementation A seven stage, cyclic model: (1) conceptualization; (2) data representation; (3) hypothesis space search; (4) hypothesis testing; (5) interpretation; (6) documentation; (7) popularization Multiple replications of this model may run based on the number of separate lines of inquiry that occur in the VRE Network evolution At the project level: A life cycle model of project stages At the organizational level: A dialectic between loose/open structure and tight/closed structure At the goal level: Conflict resolution among competing goals and goal displacement No pattern: Represent external and internal events that affect one or more tracks

The evolutionary technology trajectory operates in concert with a model of technology implementation which governs which branches of the trajectory thrive or are pruned, and the rate that the VRE moves down the branches. As noted in the previous discussion of VRE technology, VREs are built from an assemblage of existing and newly developed technologies (some brand new and others extensions of existing technology frameworks). Hence, a model of the implementation process is required to supplement the technological trajectory model. Meyer and Goes (1988) advance a model of implementation of existing technologies that describes three main stages, each with substages, specifically: (1) Knowledge–Awareness: comprised of apprehension, consideration, discussion substages; (2) Evaluation–Choice: comprised acquisition proposal, technical-fiscal evaluation, political-strategic evaluation substages; and (3) Adoption–Implementation: comprised of trial, acceptance, expansion substages. This is a life cycle model at the macrolevel. However, advancing through their stages and substages requires choice processes. Individual actions based on the freedom of choice constitute microlevel processes. If an organization succeeds in moving through all nine substages of this model successfully, then it has institutionalized the technological innovation. Conversely, how far the organization is able to go in the progression indicates how successful the implementation is: in some cases, the technology gets “stuck” in a given substage and can go no further, thus truncating a given branch completely, while in others it might merely have paused in a stage and pick up again at a later time, perhaps cycling back to redo earlier stage work in order to set the stage for future progression. The stages and substages are listed in logical ordering and a there is a tendency, when things go smoothly, for the organization to move through the stages and substages in order. However, studies of group and organizational decision making

40

A Process Method Approach to Study the Development of Virtual. . .

1033

have shown that they rarely are neat, straightforward processes, and that decisionmakers commonly skip ahead and/or backtrack to earlier stages in response to technical or political problems that arise. So the developmental path of the implementation process is likely to be “messy” and cyclical. Moreover, since multiple technologies are being implemented as the VRE develops, the technological trajectory will have multiple branches, each reflecting a specific technology.

Developmental Model for Science Scientific inquiry can be modeled as a seven-stage process (Simon & Lea, 1974; Dunbar, 1993; Schunn & Klahr, 1995; Okada & Simon, 1997). In the conceptualization stage, scientists explore a problem area, usually by informal interpersonal interaction. In this stage, participants share both scientific and social information. Communication in the conceptualization stage primarily consists of informal interaction focused on the science among people who know and trust each other. The amount of social interaction decreases and scientific information increases as this stage progresses. The nature of scientific communication, therefore, moves from informal to formal communication. This discussion gradually builds into a data representation stage. This stage deals with the representations or abstractions of the data chosen from the set of possible features. In the data representation stage, scientists use existing data to explore the problem area and to generate a hypothesis space. Hypothesis space search builds the structure of a hypothesis and uses prior knowledge or experimental outcomes to develop plausible hypotheses. Once the hypotheses have been developed, the hypothesis-testing stage starts. Existing and new data are used to test the theory or model in question in this stage. In the interpretation stage, the findings are assimilated to existing knowledge base, which is either buttressed or changed by it. A process of documenting initial idea discussions, hypotheses, investigative procedures, results, data analysis, and outcomes start from the initiation of the discovery process and continue throughout the scientific inquiry cycles. The documentation process produces a body of documents that directs the diffusion of new information. Popularization or dissemination of new information to the audience marks the final stage of the scientific discovery process. Therefore, a model of scientific discovery can be seen as stages starting with conceptualization and ending at popularization, and with a documentation phase parallel throughout the stages. These stages are not linear and often repetitive.

Developmental Model for the VRE Community The VRE community can be modeled as a network of stakeholders described in the previous section. Monge et al. (2008) posit that community networks change over time via evolutionary processes. Networks can evolve over time via four different paths: (i) by increasing connectivity among stakeholders, resulting in densification

1034

I. Ahmed and M. S. Poole

of the network over time; (ii) through densification into loosely connected clusters, often of different sets of stakeholders, who interact a good deal with each other and less often with members of other clusters; (iii) toward a core-periphery structure, in which a dense core of key stakeholders who interact often and a periphery or less active members who are connected to one or a few members of the core develop; and (iv) through a schismatic change, in which the network divides into two or more disjoint sets that are more or less opposed to one another. In this case, the particular network that evolves depends on microlevel processes of creation-selection-retention of links among individuals and organizations in the community. These links are created and sustained or broken as a function of usability and sociability characteristics for individual actors who actually like organizations and as a function of communality and connectivity values at the macroorganizational level. There are suggestions that the macrolevel development that results from this evolutionary process conforms to a broader stage-wise process (Bryant & Monge, 2008), but the evidence for this is thin, so this remains a hypothesis at this point.

Developmental Model for Management and Organization Project management often follows a rough life cycle conforming to other processes. It is also possible to observe a dialectical process, because management structures of creative and exploratory endeavors like VREs is fraught with (i) the tension between the loose structures and management practices that tend to support creativity and (ii) the tighter structures and practices necessary to implement and carry out projects envisioned during creative periods. As Poole and Van de Ven (1989) showed, paradoxical demands can be addressed in three ways: 1. Temporally, by alternating between emphasizing one side of the paradox and the other; in this case, one would observe phases in which creativity dominated and phases in which implementation and high structure dominated. 2. Spatially, by placing creative functions in one part of the organization and more structured implementation activities in another part; each part would incorporate different sets of actors with specialized personnel devoted to linking them; developmentally, we would observe stages of the creative process in some parts of the organization and the implementation stages in others. 3. Transcendentally, by developing means to realize both sides of the tension simultaneously; so-called paradoxical organizational forms like the networked organization attempts to transcend these paradoxes (Lewis, 2000; Luscher & Lewis, 2008) describe one method for working through paradoxes while honoring both poles. As the VRE develops, one, two, or all three of these means may be used. One possible pattern is a failed attempt at transcendence followed by a move to either approaches 1 or 2. Another is the temporal alternation of emphases on creativity and structure. A third is a move from approach 1 to approach 2.

40

A Process Method Approach to Study the Development of Virtual. . .

1035

Another developmental process is likely to occur in the goals sub-track. Goals and expected outcomes are likely to change over time at either the operational or the community levels. One important process that drives goal development is the need to reconcile conflicting goals. Goals for the different tracks may vary. For example, a goal in the managerial track may be to maintain data security, while the conduct of science may emphasize open sharing and copying of data. The potential for conflict among these goals poses a challenge for the VRE and how this is handled will affect its development. A second, well-known process is goal displacement, whereby the original goals of the VRE gradually are displaced by other goals without acknowledging such. For example, a VRE originally dedicated primarily to pure science may gradually evolve an educational mission that displaces the scientific emphasis, without ever acknowledging this.

Developmental Model for Critical Events These events will not have a pattern, but would be expected to punctuate the process and influence the other tracks. In some cases, they would represent large “interventions” in the developmental process in a track, introducing a different mechanism that replaced the existing one. For example, a project following a life cycle might suddenly find itself interrupted and a new life cycle started.

Interactions Among the Developmental Models The current framework extends process methods by specifying relationships across tracks using a more extensive and detailed classification scheme than previous research has used. There are three dimensions along which two or more of the tracks may be related (see Poole & Van de Ven, 2004). These are summarized in Table 3 below. First, activities on one track may operate at different levels, with more macrolevel phenomena such as interorganizational fields interacting with lower level processes such as organizational processes. For example, the implementation sequence might form a macrolevel temporal structure within which the evolution of technological trajectories operates (as system developers alter the technology so it will meet users’ needs). Two types of interlevel relationships are relevant in this case, nested and entangled relationships. Tracks are nested when a lower order track is tightly linked with a higher order one, serving functions at the lower level that connect directly with the operation of the higher level track. For example, management of the VRE may be largely driven by the needs of scientific work and monitor it carefully for problems and issues. How management responds to the tensions between creativity and the need for implementation will influence how scientific inquiry is conducted. Tracks are entangled when they influence each other, but are not tightly linked into a single, coherent process. In this case, the tracks operate independently to some degree. In contrast to nested tracks, which have tight coupling, entangled tracks are moderately or loosely coupled. As a result, the tracks run their own courses and

1036

I. Ahmed and M. S. Poole

Table 3 Dimensions of relationships among developmental tracks Type of relationship Level

Valence Temporal

Subtypes Nested: Track A is hierarchically linked to Track B, such that Track B is causally affected by Track A; development on Track A influences or drives development on Track B Entangled: Track A and Track B influence one another, but also have independent development Positive: Two tracks reinforce one another Negative: Two tracks dampen or conflict with one another Velocity: Rate of occurrence of events on a track; Track A may have more events occurring more quickly than Track B Duration: Length of time it takes to complete one cycle on a Track; Track A may take longer to complete than Track B

interact with each other, but are not “in synch” to the extent nested tracks are. If scientific and management tracks are entangled, they operate independently, but a conflict over management might spill over and interfere with scientific inquiry. In either nested or entangled relationships, tracks may have a positive relationship in which they reinforce each other or a negative one in which activity in one track tends to interfere with or dampen activity in another. In a VRE with controlling managers, for example, management may feel a need to monitor scientific activity closely and by so doing interfere with creative advancement, hence dampening the pace of scientific work. Another possible relationship among tracks is entrainment, which occurs when motors at the same or different levels operate independently but come into coordination due to an external pacing factor. For example, regular reviews by outside funders or other authority may result in activities on several tracks being coordinated with one another. Activities along the tracks also exhibit various temporal relationships. One relationship is relative velocity of the track, whether activities on one track tends to move faster than activities on others. For example, there may be a VRE in which scientific work is proceeding very rapidly, but management decisions are slower due to a requirement to consult with an external advisory board. Processes may also vary in terms of their duration. Cycles of scientific work in this same VRE may take only a month, whereas management decision cycles may take a quarter of a year. Depending on which track has a more significant degree of control over the VRE, differences in velocity and duration of processes on two or more tracks can influence how the VRE develops and its effectiveness.

Conclusion Processes are difficult to predict, because of the multiple complex influences upon them. For several tracks, including the technology and management tracks, there are accepted models for development. While some models have been repeated so often

40

A Process Method Approach to Study the Development of Virtual. . .

1037

that they are taken for granted, empirical evidence often shows they are oversimplified as Poole (1981; Poole & Roth, 1989) found in the study of group decision making processes, which proved much more complex than expected based on the existing literature. Discovering tracks that predominate in influencing the development of the VRE can illuminate our understanding of how VREs are built and how to build them so that “science comes first.” VREs have great potential to advance scientific inquiry, knowledge generation, and learning. Understanding how VREs develop and how their effectiveness is related to their development over time would enable designers to improve the development process. Process research involves, by definition, tracing the history of the process. The conceptual framework described here is intended to enable understanding of the generative mechanisms and critical events that drive the process. The framework allows us to conduct in depth longitudinal analyses of the sequences of events involved in the development of VREs along the five tracks and trace inter-track influences. It will allow us to assess the fit of various developmental models to the sequences to determine which generative mechanisms account for the development of the VREs and the coevolution of the tracks. It will relate various features of the developmental process to effectiveness of VREs on several dimensions. Knowledge of the various processes that promote or derail design, implementation, and use would increase the effectiveness of developers and managers of VREs, and will also provide valuable lessons for other cyberinfrastructure developments. Acknowledgments This work was supported by the US National Science Foundation’s (NSF) Virtual Organization as Sociotechnical System (VOSS) Grant Award #1308176.

References Adelsberger, H. H., Collis, B., & Pawlowski, J. M. (Eds.). (2013). Handbook on information technologies for education and training. Berlin, Germany: Springer Science & Business Media. Ahmed, I., & Poole, M. S. (2011). Exploring communication technology configurations in virtual research environments. Urbana, IL: University of Illinois Urbana-Champaign: National Center for Supercomputing Applications. Alliance for Cellular Signaling (AfCS) Web Portal. (2012). Retrieved from http://www.afcs.org/ Awre, C., & Ingram, C. (2005). CREE feasibility study on presenting communication and collaboration tools within different contexts. Retrieved from http://www.hull.ac.uk/cree/downloads/ CREEcommsresults.pdf Bos, N. D., Zimmerman, A., Olson, J., Yew, J., Yerkie, J., Dahl, E., & Olson, G. (2007). From shared databases to communities of practice: A taxonomy of collaboratories. Journal of Computer-Mediated Communication, 12(2), 652–672. Bryant, J. A., & Monge, P. (2008). The evolution of the children’s television community, 1953–2003. International Journal of Communication [Online], 2, 160–192. Retrieved from http://ijoc.org./ojs/index.php/ijoc/article/view/27 Carley, K., & Wendt, K. (1991). Electronic mail and scientific communication: A study of the soar extended research group. Knowledge: Creation, Diffusion, Utilization, 12(4), 406–440.

1038

I. Ahmed and M. S. Poole

Chen, I. Y., & Chen, N. S. (2009). Examining the factors influencing participants’ knowledge sharing behavior in virtual learning communities. Journal of Educational Technology & Society, 12(1), 134. Clark, K. (1985). The interaction of design hierarchies and market concepts in technological evolution. Research Policy, 14, 235–251. Dillenbourg, P., Schneider, D., & Synteta, P. (2002). Virtual learning environments. In 3rd Hellenic conference “information & communication technologies in education” (pp. 3–18). Rhodes, Greece: Kastaniotis Editions. Dunbar, K. (1993). Concept discovery in a scientific domain. Cognitive Science, 17, 397–434. Fifth International Symposium on Process Organization Studies. (2013, June). The emergence of novelty in organizations. Minoa Palace Resort, Chania, Crete, Greece. Retrieved from http:// www.process-symposium.com/ Gilman, A. G., Simon, M. I., Bourne, H. R., Harris, B. A., Long, R., Ross, E. M., . . . Sambrano, G. R. (2002). Participating investigators and scientists of the Alliance for Cellular Signaling. Nature, 420(6916), 703–706. Henderson, R. M., & Clark, K. B. (1990). Architectural innovation: The reconfiguration of existing product technologies and the failure of established firms. Administrative Science Quarterly, 35, 9–30. Langley, A. (1999). Strategies for theorizing from process data. Academy of Management Review, 24, 691–710. Langley, A., & Tsoukas, (2010). Chapter 1: Introducing perspectives on process organization studies. In A. Hernes, T., & H. Maitlis, S. (Eds.), Process, sensemaking and organizing. 1, 1–26. Oxford, UK: Oxford University Press. Lewis, M. W. (2000). Exploring paradox: Toward a more comprehensive guide. Academy of Management Review, 25, 260–275. Lievrouw, L. A., & Carley, K. (1990). Changing patterns of communication among scientists in an era of “Telescience”. Technology in Society, 12, 457–477. Luscher, L. S., & Lewis, M. W. (2008). Organizational change and managerial sensemaking: Working through paradox. Academy of Management Journal, 51, 221–240. Matos, S., & Lopes, E. (2013). Prince2 or PMBOK – A question of choice. Procedia Technology, 9, 787–794. Meyer, A. D., & Goes, J. B. (1988). Organizational assimilation of innovations: A multilevel contextual analysis. Academy of management Journal, 31, 897–923. Mohr, L. B. (1982). Explaining organizational behavior. San Francisco, CA: Jossey-Bass. Monge, P., Heiss, B. M., & Margolin, D. B. (2008). Communication network evolotion in organizational communities. Communication Theory, 18, 449–477. Okada, T., & Simon, H. A. (1997). Collaborative discovery in a scientific domain. Cognitive Science, 21(2), 109–146. Olson, G. M., Zimmerman, A., & Bos, N. (2008). Scientific collaboration on the internet. Cambridge, MA: MIT Press. Olson, J. S., & Olson, G. M. (2013). Working together apart: Collaboration over the internet. Synthesis Lectures on Human-Centered Informatics, 6(5), 1–151. Paré, G., & Dubé, L. (1999, December 13–15). Virtual teams: An exploratory study of key challenges and strategies. Proceedings of the 20th international conference on information systems, ICIS (pp. 479–483). Charlotte, NC: ICIS. Poole, M. S. (1981). Decision development in small groups I; A comparison of two models. Communication Monographs, 48(1), 1–24. Poole, M. S., & Roth, J. (1989). Decision development in small groups V: Test of a contingency model. Human Communication Research, 15(4), 549–589. Poole, M. S., & Holmes, M. E. (1995). Decision development in computer-assisted group decision making. Human Communication Research, 22, 90–127. Poole, M. S., & Van de Ven, A. H. (1989). Using paradox to build management and organization theories. Academy of Management Review, 562–578.

40

A Process Method Approach to Study the Development of Virtual. . .

1039

Poole, M. S., & Van de Ven, A. H. (2004). Theories of organizational change and innovation processes. In M. S. Poole & A. H. Van de Ven (Eds.), Handbook of organizational change and innovation (pp. 374–397). New York, NY: Oxford University Press. Poole, M. S., & Van de Ven, A. H. (2010). Empirical methods for research on organizational decision making processes. In P. C. Nutt & D. Wilson (Eds.), The Blackwell handbook of decision making (pp. 543–580). Oxford, UK: Blackwell. Poole, M. S., Van de Ven, A. H., Dooley, K., & Holmes, M. (2000). Organizational innovation and change processes: Theory and methods for research. New York, NY: Oxford University Press. Poole, M. S., & Zhang, H. (2005). Virtual teams. In S. Wheelan (Ed.), The handbook of group research and practice (pp. 363–384). Thousand Oaks, CA: Sage. Preece, J. (2000). Online communities: Designing, usability, supporting sociability. Chichester, UK: Wiley. Rice, R. E., & Gattiker, U. E. (2001). New media and organizational structuring. In F. M. Jablin & L. L. Putnam (Eds.), The new handbook of organizational communication (pp. 544–581). Thousand Oaks, CA: Sage. Rogers, J. (2000). Communities of practice: A framework for fostering coherence in virtual learning communities. Educational Technology & Society, 3(3), 384–392. Schunn, C. D., & Klahr, D. (1995). A 4-space model of scientific discovery. Proceedings of the 17th annual conference of the cognitive science society. Hillsdale, NJ: Erlbaum. Scott, J. (2000). Emerging patterns from the dynamic capabilities of internet intermediaries. Journal of Computer-Mediated Communication, 5(3). Retrived from http://onlinelibrary.wiley.com/doi/ 10.1111/j.1083-6101.2000.tb00344.x/full. Simon, H. A., & Lea, G. (1974). Problem solving and rule induction: A unified view. In L. W. Gregg (Ed.), Knowledge and cognition. Hillsdale, NJ: Erlbaum. Swan, K., & Shea, P. (2005). The development of virtual learning communities. In S. R. Hiltz & R. Goldman (Eds.), Learning together online: Research on asynchronous learning networks (pp. 239–260). Mahwah, NJ: Lawrence Erlbaum Associates. Teo, H. H., Chan, H. C., Wei, K. K., & Zhang, Z. (2003). Evaluating information accessibility and community adaptivity features for sustaining virtual learning communities. International Journal of Human-Computer Studies, 59(5), 671–697. Van de Ven, A. H., Angle, H., & Poole, M. S. (Eds.). (2000). Research on the management of innovation. New York, NY: Oxford University Press. Van de Ven, A. H., Polley, D., Garud, R., & Venkatraman, S. (1999). The innovation journey. New York, NY: Oxford University Press. Van de Ven, A. H., & Poole, M. S. (1995). Explaining development and change in organizations. Academy of Management Review, 20, 510–540. Warr, A., Lloyd, S., Jirotka, M., de la Flor, G., Schroeder, R., & Rahman, M. (n.d.). Project management in e-science. A report from the “Embedding e-science applications: Designing and managing for usability” project. EPSRC grant no: EP/D049733/1. Wideman, R. M. (2002). Comparing PRINCE2 with PMBoK. Published as part of m4success.com. AEW Services, Vancouver, BC.

Exploring Immersive Language Learning Using Virtual Reality

41

Gary Ka Wai Wong and Michele Notari

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Computer-Mediated Reality in Language Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Development of Computer-Mediated Representations and Visualizations . . . . . . . . . . . . . . . . . . . . Computer-Mediated Tools for Creating CMR Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Affordances and Skills for Concept Representation Using CMR . . . . . . . . . . . . . . . . . . . . . . . . . Language Learning with CMR: Conceptual Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Examples of Technology-Enhanced Language Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Visual Representations of Mental Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developing the eLEARN Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusions and Future Developments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1042 1043 1044 1045 1048 1049 1051 1052 1055 1057 1059

Abstract

Computer-mediated reality (CMR) is a cutting-edge technology that gives numerous opportunities for language teaching and learning. Learners can enhance their language skills through immersion into contextualized environments where they enjoy rich and diverse sensational experiences. Through exposure to specific realistic computer-mediated content, they can learn how to express their authentic experiences. The added value of CMR is its three-dimensional representation, construction, and visualization of the learner’s concept. In the first part of this chapter, common CMR technologies such as virtual reality (VR), augmented reality (AR), and mixed reality are reviewed. The following sections focus on the G. K. W. Wong (*) Faculty of Education, The University of Hong Kong, Pokfulam, Hong Kong e-mail: [email protected] M. Notari PHBern, University of Teacher Education, Bern, Switzerland Institute of Lower Secondary Education, Pädagogische Hochschule Bern, Bern, Switzerland e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_144

1041

1042

G. K. W. Wong and M. Notari

affordances of CMR for learning and the conceptual framework underpinning language learning supported by CMR. Two existing project ideas are presented to illustrate how a CMR environment may be used in different aspects of language learning, i.e., reading, speaking, and writing. These two ongoing projects demonstrate the different affordances of CMR and how to derive meaningful research objectives based on the CMR environment. Toward the end of the chapter, potential research problems are examined to help researchers and practitioners find new research directions for further development. Keywords

Computer-mediated reality · Augmented reality · Virtual reality · Language learning · Concept representation

Introduction In the last few years, computer-mediated reality, such as augmented reality (AR) and virtual reality (VR) interfaces, has shown potential for enhancing teaching and learning. In this process, the physical and virtual worlds are combined to form an artificial reality, leveraging the advantages of both types of CMR (Akçayır & Akçayır, 2017; Azuma et al., 2001; Yannier, Koedinger, & Hudson, 2015). AR and VR have both demonstrated similar but unique affordances for learning. AR technology allows virtual objects (augmented components) to be overlaid onto a real-world scenario so that the user experience is elevated beyond the artificial scene (Chen, Su, Lee, & Wu, 2007). VR technology brings the experience of reality into a virtual environment to simulate our living world (Huang, Liaw, & Lai, 2016; Psotka, 1995). The use of VR technology in education is not a brand-new idea; it originated in the 1980s. Its most frequent use was for flight simulation, as a part of pilot training (Hawkins, 1995; Merchant, Goetz, Cifuentes, Keeney-Kennicutt, & Davis, 2014). However, for many reasons, such as the financial cost of procurement and maintenance, physical and psychological discomfort, and poor instructional design, VR technology has not been completely successful in making a significant impact on education (Merchant et al., 2014). With the latest advancements in and the commercialization of mobile technologies, VR and even AR have become more affordable and accessible to households and schools. Consumer-type head-mounted displays such as Oculus Rift and HTC Vive or VR headsets such as Google Cardboard, working with smartphones as the source of the content projector, have created a new era of AR and VR technologies that can be used for learning (Bickers, 2016; Pang et al., 2007). Educators from different countries have already experimented with AR and VR technology, applying it across different educational sectors (Akçayır & Akçayır, 2017; Hew & Cheung, 2010). In this chapter, we focus on exploring and conceptualizing how CMR can create a new genre of language learning.

41

Exploring Immersive Language Learning Using Virtual Reality

1043

Computer-Mediated Reality in Language Learning A recent systematic review of 68 related studies (Akçayır & Akçayır, 2017) indicated that AR technology, and other computer-mediated reality technology relying on vision glasses, became technologically mature when it began to be used to enhance educational learning performance. Akçayır and Akçayır found that 51% of research projects targeted K-12 learners, and since 2007, 60% of these had developed their learning tools on mobile devices. More importantly, AR has been shown to enhance learning outcomes (e.g., learning achievement, learning motivation, attitude, and confidence), pedagogical design (e.g., collaboration opportunities for students, communication between students and teachers, multisensory learning, self-learning), and different types of interactions (student-student, student-materials, student-teacher) (Akçayır & Akçayır, 2017; Chen, Liu, Cheng, & Huang, 2017). Similarly, in another systematic review, 53 related studies analyzed various educational applications of VR from 1999 to 2009 (Mikropoulos & Natsis, 2011). VR was found to offer unique technological and pedagogical affordances. Despite the improvements in the use of this technology for learning, previous studies have identified several major challenges. These have included the students’ difficulty using it due to poor interface design (Munoz-Cristobal et al., 2015) and cognitive overload from processing an excessive amount of information through AR (Dunleavy, Dede, & Mitchell, 2009). These findings and other recent initiatives have informed us on the future trend of this specific learning environment and the need to find possible solutions to the existing problems with AR technology to it to be used as a new technology-enhanced learning tool in the K-12 sector. Visual cues from computer-mediated reality have already been proven to help students understand abstract concepts in nonlanguage education domains, such as computer programming (Zünd et al., 2015), and visual concepts when studying the anatomy of the human body (Saenz, Strunk, Maset, Seo, & Malone, 2015). Yet this technology has not been fully investigated for its effectiveness and application to language teaching (Chen et al., 2007; Low et al., 2008), and there have only been limited examples. Su (2004), for instance, proposed a new language learning system using the AR environment to promote Chinese phonetic alphabet learning for children aged 6 to 7. By introducing popular cartoon characters into the exercises (avatars and judges in 3D shapes) and providing system-generated feedback, the learners were more motivated and remained focused on the learning topics longer than they did with book reading in a traditional learning setting. Unfortunately, the impediments to this kind of imaginative language learning have often been due to a lack of interdisciplinary collaboration between teachers and researchers studying technology applications in language classrooms at the K-12 level (Zhao, 2003). Even so, Chun, Kern, and Smith (2016) reported on how technology has played an evolutionary role and can benefit language learning and teaching. Indeed, technology offers an innovative way for culture and language to be represented, expressed, and understood. When technology is used in a meaningful way, with clear learning goals and a good understanding of the affordances, its application to language learning encourages critical reflection and engagement. This

1044

G. K. W. Wong and M. Notari

could become the rationale underlying the selection of suitable technological learning. Language teachers have been encouraged to become more competent at teachingaided pedagogy. However, they have been given insufficient room to explore and expand their understanding of how to effectively use instructional technology (Cooke-Plagwitz, 2016). Wu, Lee, Chang, and Liang (2013) found that most AR technology has been designed for science and mathematics in addition to VR (Potkonjak et al., 2016). Nonetheless, only limited effort has been made to integrate AR and VR in language education and specifically to explore how learners can create virtual environments to represent their ideas in the learning process. In addition, regular curricula integration has not been fully explored because most of the existing studies have been short term or outside of the normal school curriculum (Wu et al., 2013). AR/VR technology offers a unique environment within which students can explore the world. Similarly, language education can intuitively take advantage of this technology, allowing students to explore its content from a different perspective. For example, during a lesson on “White Sail and Wood Pulp” in the Primary 4 Chinese language curriculum, students can sail in a boat through virtual space around Hong Kong’s harbor by wearing 360-degree glasses. This would never be possible in actual reality, for safety reasons. With the advanced technology of the accelerometer and gyroscope built into smartphones, students can view VR content in 360 by moving the phone around their head. After exploring the real-life situation, they can be asked to use their language skills to talk about the (virtual) trip. Such experiences can then motivate students to extend their learning experience by reading related materials. Thus, bringing AR and VR into language classrooms, designing such technology-enhanced learning tools, and engaging teachers in the design of new tools fitting their specific pedagogies are worth considering. Although AR and VR display different features, they have commonalities. Accordingly, the following discussion is not limited to using only one of these technologies in CMR. The hypothetical argument in this chapter emphasizes the need for further research into CMR as a medium used to represent and visualize language learning content. In AR viewing, virtual objects are surrounded by physical reality. In VR, learners view objects in a completely artificial environment, but in a similar fashion. Therefore, the following discussion can be generally applied to AR or VR within the CMR learning environment. Before a conceptual argument of the affordances framework is formulated, however, the technology is discussed.

Development of Computer-Mediated Representations and Visualizations CMR is not a new technology. Computer scientists have already invented graphical representations that illustrate reality in a virtual environment (Helsel, 1992; Wickens, 1992). However, the artifacts have only been accessible to skillful users with intensive training in operating high-performance computing facilities and

41

Exploring Immersive Language Learning Using Virtual Reality

1045

digital graphic processing software, such as Virtual Reality Markup Language (VRML) (Moore, 1995). In recent years, mobile technology has become more powerful and user-friendly. Affordable smartphones have become equipped with digital gyroscope modules embedded into their systems. These have enabled the devices to more accurately recognize movement within a three-dimensional (3D) space. The gyroscopes contain accelerometers (acceleration sensors), giving the devices a robust sense of direction and ability to detect motion. With this advanced sensing technology, users can easily rotate the devices and move in different directions while the screen concurrently alters the visual representations to generate different views. This real-time change of visual representations gives users a sense of viewing the real world (Wang, Petrina, & Feng, 2017). These commercialized technologies bring different sensational experiences to users from many accessible applications, available anywhere and at any time, particularly in digital games. Users provided with immersive environments through these technologies can enter into virtual space at different transparency levels, depending on the capacity of the objects being represented in the physical and virtual world.

Computer-Mediated Tools for Creating CMR Environments No doubt, this latest technology brings new learning opportunities, allowing learners to represent their ideas in a virtual environment and then visualize them through a viewer or dedicated head-mounted display such as Google™ Cardboard. The Google™ technology allows a compatible smartphone to be attached, and learners can enter the immersive virtual environment through the lens inside the cardboard (see Fig. 1). Software tools allow learners to create and disseminate the visualization of their ideas with audio, or even video images, in a three-dimensional environment. For example, a free web-based online tool like InstaVR (see http://www.instavr. co/customer-stories/german-university-in-cairo) allows users to create interactive VR apps by using 360-degree images or videos and then publishing them in common VR platforms, including Android, iOS, and web browsers. More experienced learners can use other available open source development tools, such as Open Source Virtual Reality (OSVR) and True Open Virtual Reality (TrueOpenVR). In one case, an InstaVR tool was created by a researcher from a German University in Cairo. An Architecture and Urban Design instructor then introduced it to students so they could create a hyperreality architectural presentation. In the presentation, the historical view of architectural grandeur in the local Cairo area was highlighted using panoramic 3D. In this way, CMR technology allows learners to become engaged in dynamic and interactive learning environments without traveling, which is suited to time-and budget-constrained classroom learning. Taking static images and photos, students can easily produce and upload a 360-degree video or an immersive video using a specially designed camera. With an easy-to-film video production tool, students can readily manage and share their views to others in an app and or a cloud platform and present their ideas by allowing others to become immersed in their 3D virtual world.

1046

G. K. W. Wong and M. Notari

Fig. 1 An affordable Google™ Cardboard viewer for viewing 360-degree videos on smartphones. (The photo is licensed as cc-by-2.0 (see http://flickr.com/photos/12452841@N00/14519574116))

Many online platforms also allow users to create an AR experience and share it with others. For example, the Google Map Street View App (see https://www. google.com/streetview/apps) has recently made it possible for users to create an indoor version of a “street view” with a panorama cam function (see the example of the University of Bern in https://goo.gl/pHLoku), which can also simulate an augmented reality experience by making the smartphone’s screen appear bifocal. By using the Google Cardboard app feature, viewers can have an immersive experience. Google Earth™ has also been enhanced with three different modes of viewing using selected headsets such as HTC™ Vive™ and Oculus Rift™ (see Table 1). Google Blocks (see https://vr.google.com/blocks) is yet another new concept, allowing users to build virtual objects and scenes as 3D models. Using six simple virtual tools, such as Shape, Stroke, Paint, Modify, Grab, and Erase, users can quickly create different models and share them with others. In the future, perhaps users can integrate the built 3D models into a real environment to create an augmented reality experience. This could bring about a paradigm shift for concept representation, opening a new space for learners to represent their concepts and share them with others. As CMR technologies have become more mature, with high computational performance in processing multimedia-rich content on mobile devices, the representation of concepts has become easy for ordinary users to create without problems. As when the Internet was first created and HTML became a popular made-up language used to create webpages, users can easily create webpages with multimedia content to represent their concepts without being skilled in any programming language. It is expected that more learners will become skillful at developing CMR content in the near future (Fig. 2).

41

Exploring Immersive Language Learning Using Virtual Reality

1047

Table 1 Google earth VR viewing mode with selected VR headset Viewing mode Walk around Fly Browse

Descriptions of Google earth VR experience Viewer can stand still while rolling the direction of his/her head to view the virtual landscape; a zoom in/out feature is available when leaning toward or away from a central focal point Viewer can experience flying through the virtual landscape at speed using a handheld controller Viewer can control browsing and switching to different locations while in the virtual environment

Fig. 2 Sample of VR screens created by InstaVR. (The photo is licensed as cc-by-2.0 (see http:// www.instavr.co/solutions/vr-outputs/ios-android-google-cardboard))

CMR content may be the closest representation to what the creator wants to illustrate from his or her own view. Unlike other modes relying on existing CMR research (Potkonjak et al., 2016), it is crucial to emphasize how CMR technologies can help to enhance the representation of concepts that are not only augmentations of existing reality but that also visualize abstract mental models of learning topics. For example, a child learner who has no idea what the dwarves’ house in Snow White looks like may have difficulty imagining it just by reading the book with descriptions and written text or two-dimensional images. Creating an animation could help elevate the child’s understanding. Yet it remains a mono-directional representation of the dwarves’ house based on the fantasy the learner has created from the book. Alternatively, the learner could use a VR tool and represent his or her imaginative concept of what the dwarves’ house looks like from different perspectives. Others could then experience the conceptual representation through a close-toreality environment by putting on the head-mounted VR display, deepening the experience of both the learner and his/her peers who view the representation.

1048

G. K. W. Wong and M. Notari

Creating a VR world enables learners to conceive and display their own representation of what they have read, understood, and learned. This is of course also possible to achieve from a drawing. However, the 3D and multimodal representation might provide a more detailed picture of the learners’ perception of a topic. This hypothetical statement could open new research directions using the emerging CMR technology.

Affordances and Skills for Concept Representation Using CMR In Fig. 3, we present a convergence model based on Robin (2008) to illustrate the use of CMR in concept representation. By using a computer-mediated reality environment for concept representation, we have incorporated a set of affordances, including the skills needed to operate technological tools in the process of CMR creation. Creating the representation through CMR allows learners to interact with peers and teachers and brings them into the designed immersive environment with navigation flexibility. Through the immersion, others can understand the represented concepts visually and audibly. Alternatively, teachers can present content knowledge and represent abstract ideas through 3D visualization, allowing learners to go inside the construction instead of using their imagination. Based on what learners/teachers have read or written, the outcomes can be explicitly presented, compared, and understood by others through the CMR representation, providing a medium through which to disseminate ideas. Multiple literacy, such as organizing, researching, and problem-solving skills, can be illustrated through CMR artifact production in processing. For example, a learner may understand the reading materials first and then organize the ideas into a design and representation of a CMR environment. During the design of the scenes, there

Fig. 3 Affordances and skills framework of computer-mediated reality for concept representation

41

Exploring Immersive Language Learning Using Virtual Reality

1049

Table 2 Descriptions of twenty-first century literacy skills based on Robin (2008) Literacy skills Digital literacy Global literacy Technology literacy Visual literacy Information literacy

Descriptions The ability to communicate with an ever-expanding community to discuss issues, gather information, and seek help through digital and social media The capacity to read, interpret, respond, and contextualize messages from a global perspective The ability to use computers, mobile devices, and other technology to improve learning, productivity, and performance The ability to understand, produce, and communicate through visual images, digital photos, and videos The ability to search, retrieve, filter, evaluate, and synthesize information ethically

could be technical problems and solutions to be searched for online. These processes can certainly develop and encompass multiple literacy skills. By engaging with this technology-enhanced learning experience, the twenty-first century skills Robin (2008) defined in the context of digital storytelling can be promoted with modifications to meet the needs of advanced technology (see Table 2). These affordances substantiate the importance and convergence of the essential skills that help learners directly grapple with the communicated concepts. Moore (1995) argued that VR had critical value for learning and teaching in contrast to traditional education requiring learners to understand complex symbols while concentrating on newly taught concepts. With advancements in VR technology, students can be brought from a traditional brick-and-mortar classroom into a 3D virtual world where they can understand abstract and symbolic concepts in a more visualized and direct way. Comparable to what Moore envisioned in the past, CMR has led to some redefinition of educational theory, methodology, and technology. However, CMR can be used in educational settings in a more affordable, convergent, and impactful way. In this chapter, language learning is brought forth as a potential research direction for CMR in education. In the next few sections, the conceptual framework and two ongoing research projects are presented. These may guide the future research agenda across different educational systems and cultural diversities, setting the research objectives for what may be possible with CMR in future language learning.

Language Learning with CMR: Conceptual Framework In the following section, the design of CMR learning used for language acquisition is formulated and presented. CMR for language learning follows Mayer’s (2005) cognitive theory of multimedia learning (CTML or multimedia learning theory). Mayer explained how multimedia supports human learning and how instructional technology can facilitate the learning process. Multimedia is specifically defined as the combination of words and pictures, where words can be written or spoken and pictures can take on any form of graphical representation, including illustrations,

1050

G. K. W. Wong and M. Notari

videos, photos, and animations (Sorden, 2012). The theory suggests that people learn more deeply with words and pictures than with words or pictures alone. CTML draws from different cognitive learning theories such as Baddeley’s model of working memory, Paivio’s dual coding theory, and Sweller’s theory of cognitive load (Sweller, 1994). It forms the theoretical framework for the relationship between multimedia learning content and learning effectiveness. By using the materials presented in multimedia instruction to explore and perceive, the learner as an active participant is encouraged to build a coherent mental representation and then construct new knowledge. Research has shown that with multimedia presentations, there is an increase in learning transfers to further knowledge or problem-solving skills and learning performance (Mayer, 2009; Sorden, 2012). Sorden (2012) categorized CTML into four key components: (a) a dual-channel structure of visual and auditory channels, (b) limited processing capacity in memory, (c) three memory stores (sensory, working, long-term), and (d) five cognitive processes for selecting, organizing, and integrating. Figure 4 shows the relationship among content representation, memory stores, and the cognitive processes. Suppose we have a multimedia presentation of content, which can be perceived through two channels of sensory memory. Visual sensory memory holds the pictures and printed texts, and auditory sensory memory holds the sound and spoken words. The working memory brings the perceived words or images into the working memory for further processing and integration. In the process of selecting and organizing these representations, verbal models and pictorial models can be integrated with prior knowledge (or schemas) from long-term memory to form new content knowledge. CTML suggests that there is a limitation to our processing capacity in these memory stores. Overloading cognitive capacity and exceeding the limits of memory hinder the learning progress. In the CTML figure, we specifically changed the arrows to be bi-directional between the long-term and working memories. During the integration process, new knowledge must constantly build upon the “prior knowledge” base in the long-term memory. It makes sense that the model should include an updating process and show how new knowledge flows into long-term memory. In this way, the model is more complete. Mayer (2010) states that meaningful learning using multimedia occurs when a learner is engaged with the following five cognitive processes:

Fig. 4 Mayer’s cognitive theory of multimedia learning (Mayer, 2005)

41

1. 2. 3. 4. 5.

Exploring Immersive Language Learning Using Virtual Reality

1051

Selecting relevant words for processing in verbal working memory Selecting relevant images for processing in visual working memory Organizing selected words into a verbal model Organizing selected images into a pictorial model Integrating the verbal and pictorial representations with each other and with prior knowledge

In addition to the science of learning in CTML, Mayer (2009) suggested that research on multimedia instruction should be theory-grounded and evidence-based to become the science of instruction. This means that the development of new instructional principles and pedagogies using multimedia should be derived from theory, and the teaching method and conceptual framework should be supported by replicated empirical findings derived from rigorous research to create a valid and predictable model. CTML explains how learners perceive multimedia presentations in sensory memory, where selected words or images are further processed in the working memory to formulate new schemas stored in long-term memory. Connecting the CTML to the theory of constructivism explains how CMR with AR/VR provides a new space for learners to interpret and construct their realities, which can be represented through virtual environments based on perceptions of experiences (Jonassen, 1994; Moore, 1995). Both cognitive and constructivist varieties of research have considered the value of concepts related to how first-person experiences (through cognitive presence in an immersive virtual environment) can result in the transference of information (Moore, 1995). The CMR environment simply provides a platform to deliver the outcomes of the cognitive processes used to assimilate and encode information. Thus, knowledge can be constructed and represented visually. As far as we know, most of the research around the effectiveness of multimedia for learning has been based on 2D media representations, such as videos, images, texts, and auditory information. Introducing a third dimension could enhance this knowledge in the research field and broaden the understanding of media’s impact on learning. Specifically, our examples draw from Hong Kong language education. This program aims to develop new literacy skills for students to process and create multimodal texts for various modes of communication beyond the ability to effectively read and write (EDB, 2017). Studying how students can facilitate communication using AR or VR aligns with Hong Kong’s educational aims and could form a new learning and teaching paradigm for language acquisition and literature research across different levels and cultures.

Examples of Technology-Enhanced Language Learning In this section, we present two active ongoing projects to illustrate the possibility of learning and implementing design in the primary and secondary school environment using AR or VR.

1052

G. K. W. Wong and M. Notari

Visual Representations of Mental Models The following example not only emphasizes the aforementioned facilitation of learning through 3D representations in a multimedia environment, but it also provides for the creation of such a VR environment. The collaborative creation of a topic-related 3D artifact fosters the conceptual representation of the target’s topic. The interaction among peers to create a virtual representation of a topic to be discussed in a foreign language can provide opportunities for expression and communication with others. The process of building a virtual environment also allows learners to compare concepts related to the topic. When learners compare things and negotiate, it ultimately leads to a deeper understanding of the topic. Like the CMR framework of affordances, the 3D representation helps students immediately visualize and pinpoint divergent understandings. The artifacts created can easily be compared by the entire learning community and assessed by the teachers and leaners. As previously mentioned in this chapter, allowing learners to create a virtual environment gives them an opportunity to represent the concepts they have learned in the classroom. Creating a VR environment may enhance students’ information literacy, digital literacy, and visual literacy as seen in Table 2 and become part of their twenty-first century skills (Robin, 2008). The environment can be portrayed in any language, fostering language learning and conceptual understanding of a specific language topic. For example, Stories360 (http://stories360.org) software was created at the University of Teacher Education in Bern Switzerland for use in Grade 6–12 education, enabling students to build and share their own VR scenes (see Figs. 5 and 6 for the sample interfaces). Stories can be created collaboratively and easily shared among peers.

Fig. 5 The story360.org main page for creating a new VR world

41

Exploring Immersive Language Learning Using Virtual Reality

1053

Fig. 6 Example of a self-created VR scene using story360.org (see http://stories360.org/watch? vr=HkVVgWJpG)

In this example, the aim is to teach secondary school students how to visualize scenes from an English novel as they read it in school by creating a VR environment and allowing them to demonstrate their understanding of the book or scene using the Stories360 tool. The creation of the VR environment can take place in groups of students (i.e., three to four students in a group). The different representations can be immediately shared, compared, and discussed by the entire class, allowing them to sharpen and enhance their understanding of the written text. In this study, we aim to address these potential research questions using a qualitative method. We ask: 1. How does immersive virtual reality environment help learners represent and visualize their ideas? 2. How can the primary language learning process be facilitated by creating a virtual reality environment? The project is currently undergoing design and implementation. One way to design it is to select an experimental group and a control group to gain insight by comparing differences in the flow, as illustrated in Table 3. Because the features of the Stories360 platform are unique to the community, students may not have built a VR representation using a similar type of platform. However, some secondary students may have used a commercial 360-degree camera to capture experiences in their daily encounters, or 360-degree photos in Facebook. These experiences may help them to accomplish the task more easily and can be investigated during data collection. Toward the end of the invention, we can explore the perceptions of students in these two groups through observation logs, VR artifacts, interviews, and selfreflection questionnaires. The skills and literacies can also be addressed and may

1054

G. K. W. Wong and M. Notari

Table 3 Design of research using Stories360 for English language learning Groups Experimental

Control

Intervention of learning activities 1. Read an English novel 2. Instruct students on how to produce VR stories at home based on a selected scene from the book 3. In pairs, allow students to present the story to another student in the form of VR using any head-mounted display (e.g., Google Cardboard) 4. Discuss and negotiate the different representations of Stories360 (e.g., text/ image/video, position, and direction) among groups and change/adapt to the revised design 5. Reconstruct the design of the representation and present it to the class 1. Read an English novel 2. In pairs, instruct students to present a selected scene from the book to another student through plain drawings/photos/videos 3. Compare the stories among the groups and allow the groups to reformulate and adapt the presented story 4. Toward the end, allow students from the control group to explore what the treatment group has done

add value for language learning. Stories360 allows users to insert all sorts of digital artifacts into their VR environments. In addition to text, images, videos, and voice over messages can be specifically positioned in the virtual environment. The conceptual representation of specific content can be visualized and then underpinned by specific audio messages that can be heard once the visitor to the virtual environment watches them in a predefined direction or points toward a specific object within the environment. The audio option within Stories360 allows users to enhance the visual 3D representation through a situated audio comment, explanation, or clarification. It is expected that this learning activity will advance to the theory of CTML and the constructivist learning approach, where students can construct knowledge and visualizations in 3D models while their peers learn the information in a multimedia format. Unlike the common example of immersion within VR environments (Wang et al., 2017), where language learners interact in the 3D virtual world, our exemplary study proposes a paradigm shift. The language learner is no longer a consumer of the virtual environment but the creator of a 3D immersive environment in a way that represents his or her ideas. In language learning, this opens up a new space within which to disseminate conceptual understandings of the written text and a medium through which to project the imagination of learners perceiving ideas from the textual information. Whereas tangible artifacts may be possible to create in some cases (e.g., design a drama around a scene in a book), it may be too costly to demonstrate understanding of written text by constructing stage dramas all the time. Therefore, concept representation through Stories360 or similar VR creation tools can be an affordable alternative for such a purpose. Yet, the impact of this learning design requires further research, which is expected to begin with an intervention in the 2018/2019 academic year after teachers have received training on Stories360.

41

Exploring Immersive Language Learning Using Virtual Reality

1055

Developing the eLEARN Platform In the Enhancing Language Education with Artificial Reality Neo-platform (eLEARN) project funded by the Quality Education Fund (QEF) of the Hong Kong Education Bureau, we propose to create a pilot to develop a set of new mobile learning tools and promote the educational value of AR and VR technology for Hong Kong primary education. Specifically, we are interested in studying how these technologies can facilitate Chinese language education. The project will mainly consist of three parts: 1. Design and develop an AR and VR mobile learning platform for mobile devices (i.e., mobile phones, tablet devices, and 3D glasses) using effective learning designs and pedagogical approaches. 2. Evaluate Chinese language learning effectiveness in primary schools using eLEARN. 3. Provide workshops and training to Hong Kong teachers on how to design learning activities using eLEARN, and produce contextual content for Chinese language education. A questionnaire has already been sent to some selected primary schools in Hong Kong to determine their interest in building an AR/VR learning platform together and to investigate a new pedagogical design with AR/VR content. A total of 18 schools have replied and have shown an interest in actively supporting the project. Five schools were also interested in designing content with our project team. The schools expressed their concern over the limited choices of AR/VR content for local primary language education from free sources and publisher’s content. Further, the school teachers said that a collaborative effort to develop content would support them because they would acquire more technological knowledge during development and implementation. Therefore, instead of building different AR/VR content platforms, they argued it would be more beneficial to build a platform which could be shared by all interested schools. This project was formulated in collaboration with these schools to address the needs of local language education in primary schools. In this project, the development team members work with researchers, teachers, and programmers to develop the eLEARN platform based on the learning content. It is expected that a pilot run of the eLEARN platform will allow learners to become involved in refining their learning experience by improving the learning platform design. The development team should be formed of selected primary school teachers who are experienced in language education. For this eLEARN platform design for language learning, we selected a set of curriculum content for our design and implementation. However, the actual content may be adjusted based on the needs at the time of development. This content is associated with to-be-developed AR and VR content that will enrich the sensations of learners. Traditionally, even with e-learning tools (e.g., 2D games on a screen), learners learn language without connecting to their prior knowledge or bringing their

1056

G. K. W. Wong and M. Notari

experience in physical world into the classroom. Our eLEARN platform emphasizes the use of visual glasses to view AR or VR content by providing an integrated learning environment where students can interact with real nature through their eyes and ears. This immersive participatory experience will enhance the quality of the multimedia presentation and mimic the real world. The computer-mediated reality can also be integrated with CTML and explained by situated learning theory, in which learning is believed to be embedded within and determined by inseparable physical settings (Dunleavy, Dede, & Mitchell, 2009). Thus, the content design and digital environments presented in the eLEARN platform follow situated learning theory, whereas the perception and acquisition of new language skills synthesized with the prior knowledge are explained by CTML. Like the project with Stories360, eLEARN provides a new platform for language teachers to construct and design representations with more sophisticated solutions through the use of development tools. The eLEARN program can be implemented using open sources such as ARToolKit (Kato & Billinghurst, 1999), which is capable of tracking objects and recognizing square printed AR markers as marker-based AR tools. Alternatively, Vuforia by Qualcomm can be used for feature-based AR, where different objects can serve as markers to show AR content. It originally supported a desktop platform, but there are derivative versions that support the smartphone platform. Using smartphones, students can view the AR/VR scene through the screen or VR glasses and interact with the virtual object by tapping or drag-anddrop. For example, students can view a video of Beijing when the eLEARN application recognizes a picture of the Great Wall. Smartphones can provide visual and sound effects in addition to static pictures. With advances in head-mounted devices like HoloLens (Furlan, 2016), wearable AR/VR is feasible in our project. For example, in a writing lesson, a student could look through the VR glasses, and part of the classroom could be turned into the scene of a remote location (e.g., Westlake in Hangzhou). The learner could obtain a first-person feeling for the environment that would stimulate his or her ideas and motivation to write. In the eLEARN environment, the students can still see the real classroom and communicate with the real teacher. This study adopts the design-based implementation research (DBIR) methodology to build artifacts using AR and VR technology (Fishman, Penuel, Allen, & Cheng, 2013). DBIR breaks the barrier between classroom interventions and the implementation of innovative practices, helping researchers and practitioners design effective, sustainable, and scalable solutions for education that improve learning. A classroom intervention that does not consider the granularity of implementation costs to school will not be sustainable. For example, designing an affordable eLEARN platform needs to consider the extent to which school management believes in the CMR-enhanced learning environment and is willing to arrange resources to support the continuation of the pedagogies (i.e., buying and maintaining the devices for AR/VR). As Coburn and Stein (2010) suggested, research and practice should be carried out in a two-way and recursive fashion to recognize the real barriers and challenges and take them into the consideration during the design process. Although researchers work closely with practitioners to design and

41

Exploring Immersive Language Learning Using Virtual Reality

1057

implement interventions, it is suggested that students’ perspectives should also be included in the entire process (Fishman, 2014). Researchers may find additional references and more information on DBIR in (Fishman, Penuel, Allen, & Cheng, 2013), including theories and examples. As such, the project involves teachers with the design and reflection process to improve the system and pedagogical design. It may also involve students in interviews about future improvement. The learning content will be designed based on a selection of curriculum from local teachers and consultations with language experts who have a need to introduce AR and VR content. With learning content on the platform and the mobile devices to deliver it, learning effectiveness can be measured, and challenges can be identified to refine the system and learning design. Ultimately, professional training and workshops will be organized to transfer the technological pedagogical content knowledge (TPCK) to all primary school language teachers. As Mishra and Koehler (2006) suggested, the TPCK is important to the teachers’ integration of pedagogy and technology because it emphasizes the correlation, connection, and interaction among the three domains of knowledge: content (C), pedagogy (P), and technology (T). Technology knowledge, in particular, is often onerous for teachers to acquire, because it shifts from time to time (Mishra & Koehler, 2006). Given the time constraints of teachers, helping them adopt new technology and TPCK can therefore be challenging (Wong, 2016). In this project design, it is crucial to implement teachers’ professional development workshops to help them learn from experienced teachers how to develop TPCK (Koçoğlu, 2009). Through the workshops, the teachers should improve their TPCK and form a strong community of practice in language education with practical e-learning experience. The project may be able to address the following research questions in the future: 1. To what extent can eLEARN tools with computer-mediated reality help children learn the Chinese language in primary education? 2. How should eLEARN tools with computer-mediated reality tools be designed and implemented to align with the language education curriculum and enhance language proficiency? 3. What are the positive or negative effects on children’s learning motivation and achievement in Chinese language skills, including listening, speaking, reading, and writing in primary education? The following tasks in Table 4. should be conducted throughout this study as a part of the evaluation process.

Conclusions and Future Developments CMR as a cutting-edge technology will become part of our realm in the future, because the big tech companies like Apple™, Facebook™, and Google™ are committed to developing and pushing technology and applications onto the masses. Games like “Pokemon GO” help to bring this technology into our daily usage. The

1058

G. K. W. Wong and M. Notari

Table 4 Using the DBIR method to develop an eLEARN platform for primary language education Stage Plan

Development of prototype

Iterative testing and refinement

Reflect

Possible tasks Design the eLEARN platform with researchers and school teachers based on the content and issues that arise when delivering instructions to the students in the key stage 2. Identify the topics and lessons where AR and VR tools can benefit learning effectiveness and motivation. Develop and test the eLEARN platform on an iOS/android platform Deploy and design suitable learning tasks with an eLEARN platform in the classrooms of treatment groups. Adopt a sequential approach for deployment when each topic in the language curriculum is sequentially investigated in each cycle. Train students and teachers to use the tools in the eLEARN platform Perform classroom visits and analyze classroom teaching (both treatment groups and control groups). Analyze and evaluate the performance of both the system and pedagogy. Administer pretests and posttests for each topic sequentially Collect both qualitative and quantitative data through questionnaires, focus group interviews, video recordings of lessons, and pretests/ posttests. Conduct focus group interviews with the treatment group participants (teachers and students). Improve the system and pedagogical performance. Plan for the next development and deployment of topics using the eLEARN platform

common use of CMR, such as VR and AR, has been seen so far in vocational support and in some educational settings (Henderson & Feiner, 2011). Ideas on how to implement and add value to specific learning outcomes and skill acquisition, however, are still uncertain, because CMR technology and its applications are still emerging. In this chapter, we have illustrated two ongoing CMR research projects that support different aspects of language learning. The Stories360 project is about the representation of concepts through the learner’s own creations. The eLEARN platform development project begins with the DBIR approach to study how children can learn the Chinese language through multi-sensational 3D virtual worlds that enrich their experience and enhance their language skills. Both projects may encounter challenges in adoption because school teachers may be affected by different conditions when they try to facilitate technology acceptance and behavioral intentions using technological tools for learning and teaching (Wong, 2016). How to develop TPCK is part of the global educational CMR technology research agenda. Although these technologies are or will become available in the classroom, many teachers may still not have a good sense of how to design with them and use them effectively in their teaching (Robin, 2008). Therefore, future research should investigate how CMR with AR and VR can become more affordable and easily adopted by school teachers and what new challenges CMR technologies may pose in the

41

Exploring Immersive Language Learning Using Virtual Reality

1059

context of language learning as opposed to medical and science education, the mainstream areas of CMR application. Acknowledgments This chapter is supported and funded by the Seed Fund for Basic Research (Ref: 201704159003), of the University of Hong Kong and the Quality Education Fund (QEF) (Project ref: 2016/0318) of the Education Bureau of Hong Kong.

References Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review, 20, 1–11. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21(6), 34–47. Bickers, C. (2016). WA science teacher Richard Johnson, Global Teacher Prize. Retrieved January 7, 2017, from http://www.perthnow.com.au/news/western-australia/wa-science-teacher-richardjohnson-global-teacher-prize/news-story/b33468b153cea0c6812c444e73b11f6e. Chen, P., Liu, X., Cheng, W., & Huang, R. (2017). A review of using augmented reality in education from 2011 to 2016. In E. Popescu et al. (Eds.), Innovations in smart learning (pp. 13–18). Singapore: Springer Singapore. Chen, C. H., Su, C. C., Lee, P. Y., & Wu, F. G. (2007). Augmented interface for children Chinese learning. In Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007) (pp. 268–270). IEEE. Chun, D., Kern, R., & Smith, B. (2016). Technology in language use, language teaching, and language learning. The Modern Language Journal, 100(S1), 64–80. Coburn, C. E., & Stein, M. K. (Eds.). (2010). Research and practice in education: Building alliances, bridging the divide. Lanham, MD: Rowman and Littlefield. Cooke-Plagwitz, J. (2016). Adventures in teaching: Helping language teachers discover the joy of teaching with technology. IALLT Journal of Language Learning Technologies, 37(1), 35–40. Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18(1), 7–22. EDB. (2017). English language education: Key learning area curriculum guide (primary 1 – secondary 6). The curriculum development council. Retrieved May 8, 2018, from http://www. edb.gov.hk/attachment/en/curriculum-development/kla/eng-edu/Curriculum%20Document/ ELE%20KLACG_2017.pdf. Fishman, B. J. (2014). Designing usable interventions: Bringing student perspectives to the table. Instructional Science, 42(1), 115–121. Fishman, B. J., Penuel, W. R., Allen, A. R., & Cheng, B. H. (Eds.). (2013). Design-based implementation research: Theories, methods, and exemplars. New York, NY: Teachers College, Columbia University. Furlan, R. (2016). The future of augmented reality: Hololens-Microsoft’s AR headset shines despite rough edges [Resources_Tools and Toys]. IEEE Spectrum, 53(6), 21–21. Hawkins, D. G. (1995). Virtual reality and passive simulators: The future of fun. In F. Biocca & M. R. Levy (Eds.), Communication in the age of virtual reality L. Erlbaum Associates Inc. Hillsdale, NJ, USA (pp. 159–189). Helsel, S. (1992). Virtual reality and education. Educational Technology, 32(5), 38–42. Henderson, S., & Feiner, S. (2011). Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Transactions on Visualization and Computer Graphics, 17(10), 1355–1368.

1060

G. K. W. Wong and M. Notari

Hew, K. F., & Cheung, W. S. (2010). Use of three-dimensional (3-D) immersive virtual worlds in K-12 and higher education settings: A review of the research. British Journal of Educational Technology, 41(1), 33–55. Huang, H. M., Liaw, S. S., & Lai, C. M. (2016). Exploring learner acceptance of the use of virtual reality in medical education: A case study of desktop and projection-based display systems. Interactive Learning Environments, 24(1), 3–19. Jonassen, D. H. (1994). Thinking technology: Toward a constructivist design model. Educational Technology, 34(4), 34–37. Kato, H., & Billinghurst, M. (1999). Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In Augmented Reality, 1999. (IWAR’99) Proceedings. 2nd IEEE and ACM International Workshop on (pp. 85–94). IEEE. Koçoğlu, Z. (2009). Exploring the technological pedagogical content knowledge of pre-service teachers in language education. Procedia-Social and Behavioral Sciences, 1(1), 2734–2737. Low, J. H., Wong, C. O., Yang, H. K., Jung, K. C., Kim, K. R., & Han, E. J. (2008). Interactive Chinese character learning system through pictograph evolution. In International Journal of Human and Social Sciences 4: 11 2009, SOURCE Proceedings of World Academy of Science: Engineering & Technology; Venice, Italy. http://waset.org/programs/Venice08.pdf. Oct 2008, Vol. 46, pp. 299, 46, 793–799. Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning. New York, NY: Cambridge University Press. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York, NY: Cambridge University Press. Mayer, R. E. (2010). Applying the science of learning to medical education. Medical Education, 44, 543–549. Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70, 29–40. Mikropoulos, T. A., & Natsis, A. (2011). Educational virtual environments: A ten-year review of empirical research (1999–2009). Computers & Education, 56(3), 769–780. Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017. Moore, P. (1995). Learning and teaching in virtual worlds: Implications of virtual reality for education. Australian Journal of Educational Technology, 11(2), 91. Munoz-Cristobal, J. A., Jorrin-Abellan, I. M., Asensio-Pérez, J. I., Martinez-Mones, A., Prieto, L. P., & Dimitriadis, Y. (2015). Supporting teacher orchestration in ubiquitous learning environments: A study in primary education. IEEE Transactions on Learning Technologies, 8(1), 83–97. Pang, A. L. H., Phua, J. Y. C., Wu, W. T., Suriyani, R., Noor, M. Y. B. M., & Pan, A. (2007). Exploratory study on the use of mixed reality for primary science learning. Frontiers in Artificial Intelligence and Applications, 162, 449. Potkonjak, V., Gardner, M., Callaghan, V., Mattila, P., Guetl, C., Petrović, V. M., & Jovanović, K. (2016). Virtual laboratories for education in science, technology, and engineering: A review. Computers & Education, 95, 309–327. Psotka, J. (1995). Immersive training systems: Virtual reality and education and training. Instructional Science, 23(5–6), 405–431. Robin, B. R. (2008). Digital storytelling: A powerful technology tool for the 21st century classroom. Theory Into Practice, 47(3), 220–228. Saenz, M., Strunk, J., Maset, K., Seo, J. H., & Malone, E. (2015, July). FlexAR: Anatomy education through kinetic tangible augmented reality. In ACM SIGGRAPH 2015 Posters (p. 21). ACM. Sorden, S. D. (2012). The cognitive theory of multimedia learning. In B. J. Irby, G. Brown, & R. Lara-Alecio (Eds.), Handbook of educational theories (pp. 1–31). Charlotte, NC: Information Age Publishing. Su, J. Q. (2004). Using AR for children to promote Chinese phonetic alphabet learning. Tainan, Taiwan: National Cheng Kung University.

41

Exploring Immersive Language Learning Using Virtual Reality

1061

Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312. Wang, Y. F., Petrina, S., & Feng, F. (2017). VILLAGE – Virtual immersive language learning and gaming environment: Immersion and presence. British Journal of Educational Technology, 48(2), 431–450. Wickens, C. D. (1992). Virtual reality and education. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics (pp. 842–847). Chicago, IL: IEEE. Wong, G. K. (2016). The behavioral intentions of Hong Kong primary teachers in adopting educational technology. Educational Technology Research and Development, 64(2), 313–338. Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49. Yannier, N., Koedinger, K. R., & Hudson, S. E. (2015). Learning from mixed-reality games: Is shaking a tablet as effective as physical observation? In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 1045–1054). ACM. Zhao, Y. (2003). Recent developments in technology and language learning: A literature review and meta-analysis. CALICO Journal, 21(1), 7–27. Zünd, F., Ryffel, M., Magnenat, S., Marra, A., Nitti, M., Kapadia, M, & Sumner, R. W. (2015, November). Augmented creativity: Bridging the real and virtual worlds to enhance creative play. In Proceeding of ACM SIGGRAPH Asia 2015, Mobile Graphics and Interactive Applications, Article no. 21, Kobe, Japan, 2–6 November 2015.

Dr Gary Wong is an Assistant Professor in the Faculty of Education at HKU. He is currently interested in computational thinking for children, computer-mediated reality for education, and integrated learning in STEM (Science, Technology, Engineering, Mathematics) education. He has published more than 45 book chapters, journal articles, and conference proceedings in these areas. Dr. Wong earned his Bachelor of Science in Computer Science and Mathematics (double major) from Brigham Young University Hawaii, a Master of Philosophy in Electronic and Computer Engineering from the Hong Kong University of Science and Technology, and a Ph.D. in Computer Science from City University of Hong Kong. In addition, he received a Master of Education in Learning Design and Leadership from the University of Illinois at Urbana Champaign and a Master of Law degree in Information Technology and Intellectual Property Law from the University of Hong Kong. He has been the Chair of the IEEE Hong Kong Section (Education Chapter) since 2014. Professor Michele Notari is currently a Professor at the University of Teacher Education in Bern. His research is mainly in computer-supported collaborative learning, collaborative writing, and project-based learning.

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential Learning and the Design of Virtual Patient Simulation Atsusi Hirumi, Benjamin Chak Lum Lok, Teresa R. Johnson, Kyle Johnsen, Diego de Jesus Rivera-Gutierrez, Ramsamooj Javier Reyes, Tom Atkinson, Christopher Stapleton, and Juan C. Cendán

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-Year Project Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary of Year 1–4 Research Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Finding 1: Teams vs. Individuals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Finding 2: Developing History-Taking Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1065 1066 1067 1068 1068

Electronic Supplementary Material: The online version of this chapter (https://doi.org/10.1007/ 978-3-319-17461-7_76) contains supplementary material, which is available to authorized users. A. Hirumi (*) University of Central Florida, Orlando, FL, USA e-mail: [email protected] B. C. L. Lok Computer and Information Sciences and Engineering Department, University of Florida, Gainesville, FL, USA e-mail: [email protected]fl.edu; lok@ufl.edu T. R. Johnson Johns Hopkins University School of Medicine, Baltimore, MD, USA e-mail: [email protected] K. Johnsen College of Engineering, University of Georgia, Athens, GA, USA e-mail: [email protected] D. de J. Rivera-Gutierrez Microsoft Corporation, Redmond, WA, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_76

1063

1064

A. Hirumi et al.

Finding 3: Learning with NERVE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Finding 4: Reflections During Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Finding 5: Placement in Curriculum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Finding 6: Method for Interacting with Virtual Patients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Year 5: Designed-Based Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pedagogical Foundations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conceptual Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . InterPLAY Instructional Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimensions of xLearning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Iterative Cycles of Design and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Integration and Field Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Field Test Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Field Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . R&D Team Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PI and SME (Medical Educator) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PI and Software Engineering Expert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Co-I and Software Engineering Expert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Co-I and Lead Instructional Designer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Statistical Researcher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Graduate Research Assistant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1068 1069 1069 1070 1070 1071 1071 1075 1077 1081 1084 1085 1087 1092 1093 1097 1098 1099 1100 1102 1103 1105

Abstract

Systematic reviews and meta-analyses of randomized controlled studies conclude that virtual patient simulations (VPs) are consistently associated with higher learning outcomes compared to other educational methods, such as lectures, handouts, textbooks, and standardized patients (e.g., Consorti et al., Comput Educ 59(3):1001–1008, 2012; Cook and Triola Med Educ 43(4):303–311, 2009; McGaghie et al., Acad Med J Assoc Am Med Colleges 86(6):706, 2011). However, we cannot assume that students will learn by simply giving them access to the simulations. The instructional features that are integrated before, during, and after the simulations may affect student learning as much as or more so than the simulations. The strategy used to integrate the simulation into the curriculum and evaluate student performance may also have a significant effect on its use and R. J. Reyes Indiana State University, Terre Haute, IN, USA e-mail: [email protected] T. Atkinson Ashford University, San Diego, CA, USA e-mail: [email protected] C. Stapleton Simiosys Real World Laboratory, Orlando, FL, USA e-mail: [email protected] J. C. Cendán College of Medicine, University of Central Florida, Orlando, FL, USA e-mail: [email protected]

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1065

learning. Here, we document the design, development, and testing of NERVE (a VPs created to develop medical students’ ability to examine, interview, and diagnose patients with cranial nerve disorders) in one definitive source and elaborate on what went on in each team members’ mind as the system evolved. Specifically, we examine the skills, knowledge, and dispositions called upon and the key lessons learned by team members during the last year of research and development. Concluding remarks related the individual accounts and discuss common findings to shed further insights on the team’s experience. Keywords

Virtual patient simulations · Medical simulations · Medical education · Simulation-based training · Design-based research · Instructional design · Instructional theory

Introduction Medical students report that limited opportunities to practice and to evaluate their skills are two of the primary reasons they lack knowledge and confidence in their ability to conduct neurological exams (Moore & Chalk, 2009). Learning how to interview, examine, and diagnose patients with cranial nerve (CN) disorders is particularly challenging because CN disorders are rarely seen and difficult to imitate. Supervised encounters with real patients are hard to schedule because CN disorders are uncommon, and trained standardized patients cannot readily reproduce disorders that affect motor nerves. Virtual patient simulations give students nearly unlimited opportunities to practice skills and receive immediate feedback. Virtual patient simulations (VPs) also enable medical educators to give students standardized experiences with rare and difficult to replicate cases as well as present variations and promote mastery within and across institutions in safe environments (Cendan & Lok, 2012; Cook & Triola, 2009). Fueled by its potential and funding from NIH, an interdisciplinary team of faculty, staff, and students in medicine, software engineering, and instructional design from three large universities in the Southeastern United States set out to achieve two goals: (a) develop a virtual environment that enables medical students to rehearse and receive feedback on their patient interviewing, examination, and diagnostic skills and (b) create a tool that enables researchers to study different aspects of VPs design and use. Findings from controlled experiments with the Neurological Exam Rehearsal Virtual Environment (NERVE) completed during the first 4 years of funding are reported by Kleinsmith et al. (2015), Rivera-Gutierrez, Kleinsmith, Johnson, Lyons, Cendan, and Lok (2014), and Johnson, Lyons, Chuah, Kooper, Lok, and Cendan (2013), Johnson, Lyons, Kooper, Johnsen, Lok, and Cendan (2014). In addition, the design and development of NERVE and the testing and integration of NERVE are detailed in Hirumi, Kleinsmith, Johnsen, Kubovec, Eakins, and Bogert (2016a) and Hirumi, Johnson, Reyes, Lok, Johnsen, Rivera-Gutierrez, and Bogert (2016b), and the pedagogical foundations of NERVE are examined in greater depth by Hirumi et al. (in

1066

A. Hirumi et al.

press). This chapter synthesizes findings reported by Hirumi et al. into one definitive source and elaborates on the skills, knowledge, and dispositions called upon by research and development (R&D) team members during the last year of R&D. We begin by characterizing the overall R&D process and summarizing key findings from the first 4 years of the initiative. Then, we describe the adoption and application of the design-based research (DBR) method and use the five features of DBR posited by Cobb, Confrey, diSessa, Lehrer, and Schauble (2003) to synthesize findings and document the design, development, and testing of NERVE. Then, representatives of the R&D team note their role on the project; summarize their professional background; detail the key skills, knowledge, and dispositions they applied during the last year of the project; and reflect on what they learned from the experience. By mining the perspectives of representative team members, we shed light on how different fields see instructional design theory and practice as well as their own areas of expertise before, during, and after a major funded design initiative.

5-Year Project Overview Research and the development of virtual patient technology for communication skills training prior to the 5-year NIH grant contributed to the development of NERVE. Before NERVE, branching multimedia narrative represented the state of the art in the design of virtual patients which had the benefits of scalability, repeatability, and implementation simplicity but lacked the authenticity of real patient communications. Systems that used branching narrative were better-oriented for training clinical decision-making than provider-patient interactions. To address the need for more realistic provider-patient interactions, the first pilot projects to develop and test immersive virtual patient technology dating back to 2005 were initiated, advancing the state of the art along several key dimensions: • Natural language input – Students could formulate their own questions and responses to patients, rather than choosing from a multiple-choice list. By using speech recognition technology, students needed to think of what to say, speak it, and then wait for a response. • Realistic patients – Advancements in computer graphics made it possible to represent patients as interactive 3D models rather than common photo/video patients. The change to 3D models allowed a much wider range of visual interactions with students such as responding nonverbally through posture and eye contact. • Integration with existing educational methods – Natural language input and interactive 3D patients allowed medical educators to employ both the facilities and processes used to examine standardized patients. Hosting virtual patients on computers in examinations rooms simplified the comparison and integration of virtual patient technology with standardized patients. Research and the development of better natural language input and realistic patients, and the integration of these technologies into existing educational methods,

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1067

paved the way for NERVE and the development of VP technology for encounters with patients having abnormal findings.

Summary of Year 1–4 Research Findings Findings from seven studies completed during the first 4 years of the initiative (listed in Table 1) fueled the design of the latest version of NERVE, made accessible to the general public. In short, the seven studies used markedly different versions of NERVE (i.e., OGRE, Panda3D, and Unity) along with alternative measures to examine various aspects of VP design and use. Several studies did not yield practically significant outcomes, and thus the results were neither published nor reported here. For this reference work, we summarize key findings from prior studies and note how they informed the design of NERVE during the last year of R&D. Table 1 Basic attributes of studies completed during the first 4 years of the initiative Study dates 20118–11

Version and CN involved OGRE version CN3, CN5, and CN6

20124–10 to 4–12

OGRE version CN3, CN5, and CN6

2013 1–08 to 1–09

Panda3D version 15 patients CN1–12

20133–12 to 3–13

Unity version 1 CN scenario 2 conditions

2012–07 to 2013–07 20142–03 to 2–05

Unity version 6 CN scenarios 2 factors Unity version VPs: CN0 and CN3 SP: CN6

2013–09 to 2014–08

Unity version CN3 worrisome, CN3 reassuring, CN6 worrisome, CN6 reassuring

Participants 1st-year medical students, n = 60 recruited, 57 attended and consented 1st-year medical students; n = 80, recruited, 78 attended and consented 1st-year medical students; n = 98 recruited, 96 attended, 94 consented 2nd-year medical students, n = 79 recruited, 77 attended, 76 consented 3rd-year medical students during clerkship; n = 130 2nd-year medical students, n = 97

3rd-year medical students during clerkship; n = 82

Treatments 1. Team 2. Individual

1. Team/speech clarifications 2. Individual/speech clarifications 3. Team/chat 4. Individual/chat 1. Instructor + booklet prep/ competition 2. Instructor + booklet prep/no competition 1. Reflection during 2. Reflection after

All students examined 6 virtual patients across 2 sessions (3 during each session) Students accessed the learning resource center and then interviewed 1 VP on day 1 and 1 SP on day 2 or 3 Students interviewed 2 VPs in one session and 2 SPs in another session

1068

A. Hirumi et al.

Finding 1: Teams vs. Individuals Optimal learning and satisfaction outcomes may be achieved by exposing students to NERVE as members of small teams, rather than as individuals. Students working on NERVE in small teams submitted correct diagnoses for certain CN palsy cases to a significantly greater extent than did students working in NERVE as individuals. Moreover, the aptitude-treatment interaction (ATI) observed in the August 2011 study was replicated in the April 2012 study. That is, at certain aptitude levels, students demonstrated higher knowledge scores after they interacted with NERVE as members of three-person teams as compared to students participating as individuals. Based on these findings, we included a statement in the online system overview as well as encouraged students during the initial demonstration of the system to work together in teams of two or more students. Subsequent testing during Year 5 indicated that a significant percentage of students (24%) chose to use the system in pairs as reported later under Field Test Results.

Finding 2: Developing History-Taking Skills Virtual patients may play an important role in developing medical students’ historytaking skills. Students reported increased self-efficacy in history-taking skills specific to patients with cranial nerve palsies following training with VPs. Students also reported reduced anxiety with VPs as compared to standardized patients (SPs) and indicated that VP training reduced their anxiety in subsequent encounters with SPs. However, mean scores in history-taking skills specific to CN palsy cases, as measured through behavioral checklists completed by SPs, were relatively low. Consequently, further investigation into the identification of specific history-taking skills gained from NERVE interactions, and their subsequent transfer to authentic encounters with live patients, including SPs, is warranted. Although we did not seek to identify specific history-taking skills gained from using NERVE during Year 5, a progress report built into the interface informed students if they asked appropriate questions as they interviewed the virtual patients within the system (Hirumi et al. 2016a), and we did find that on average, 93% (14 out of the 15) of the items on the history-taking checklist were observed correctly by the standardized patient during VP/SP hybrid interactions facilitated 1–2 days after using the system (as reported under Field Test Results).

Finding 3: Learning with NERVE Students demonstrated learning of CN palsies through the use of NERVE. Students improved significantly in the frequency of accurate diagnoses in CN palsy cases as compared to a baseline practice case in NERVE. Diagnosis accuracy

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1069

improved significantly following the first VP practice case (pre-VP average of 44%, post-VPs average of about 78%). CN-Specific History-Taking checklist scores were significantly lower (mean = 51%) than General Skills checklist scores (mean = 93%; p < 0.001). In total, students demonstrated learning of CNs with NERVE. Only providing the simulation without additional instruction limited the learning around CN-specific history questions. Training time to mastery, best practices for retention of new content and skills, and transfer of skills to live patient encounters deserve attention in future investigations. Improvements in the accuracy of diagnoses demonstrated the feasibility of the NERVE interface and of the virtual patient interaction which enabled the R&D team to consider alternative methods for enhancing the retention of content and assessing the transfer of skills to live patients. For the last year of R&D, we decided to add content information about CN anatomy, physiology, symptoms, and pathology to the system, along with quizzes that enabled students to self-assess their acquisition of key facts, concepts, and principles. We also decided to formulate a hybrid virtual patient, standardized patient (VP/SP) encounter to assess transfer during the field test.

Finding 4: Reflections During Interactions Students who interact with a VP can reflect after the interaction in a post-interaction debriefing, as well as during the interaction when prompted by the system using dialogue boxes. Students were asked to reflect on their interactions with NERVE either after they used the system (during a guided debriefing) or as they used the system (when dialogue boxes displayed questions to guide reflection). Students engaged in some reflection using both approaches, with students engaging in deeper levels of reflection as they used the system, including an instance of critical reflection. However, the results also showed a statistically significant negative impact in the reported social presence of the VP when learners engaged in reflection during the interaction, highlighting the need to balance the value provided by reflections with the potential of losing realism of the simulated experience. Based on principles of experiential learning and findings from the Year 3 study, we decided not to prompt students to reflect on their experience as they interacted with NERVE; rather, we asked students to reflect on their experience with NERVE during an after action review to facilitate learning (as detailed under the proceeding section on the InterPLAY Conceptual Model).

Finding 5: Placement in Curriculum Students prefer VP simulations of CN palsies before seeing SPs simulating CN palsies. Neurology clerkship students interacted with either a VP first followed by an SP or an SP followed by a VP. The question was, does the order of exposure to VP

1070

A. Hirumi et al.

simulations matter? While performance did not significantly differ, students perceived interactions with VPs before SPs were more effective for interviewing and communications skills training. Moreover, the students who interacted with VPs before SPs reported higher levels of confidence in diagnosing a CN injury. Based on the 2013–2014 findings, we prescribed VP interactions before SP interactions during the last year of R&D and recommend others to first provide VP experiences and frame the experiences as a low-stress environment in which to practice for upcoming SP interactions.

Finding 6: Method for Interacting with Virtual Patients Students who interacted with VPs using a more natural, yet more error-prone approach of typing in questions interacted differently with SPs as opposed to students who interacted with VPs through a pre-populated list of questions. Students interacted with VPs using either a free-response text box or by selecting responses from a pre-populated list. Then students interacted with an SP the following day. The differences in the interactions with the SP were evaluated. A hybrid approach was used in which the students performed the history-taking on the SPs and the physical examination on a VP through a physical examination version of NERVE. The students who interacted with a pre-populated list had a seamless VP interview experience and commented that as they were still learning the right questions to ask. However, approximately half of the students preferred using the free-response text box and did not mind overcoming errors in responses during initial interactions with VPs. Having both the pre-populated list and free-response text box provided additional learning opportunities. The 2014 study listed in Table 1 was designed to help us determine if we should provide students with a closed-ended, menu-driven interface or an open-ended, chatdriven interface to facilitate interactions with the VPs. However, findings pointed to benefits and student preferences for both interfaces. As a result, we decided to give students the option of accessing the VPs in NERVE through either or both interfaces.

Year 5: Designed-Based Research Planning for the last year of R&D began during the early part of Year 4. Our goal was to create a virtual environment that would enable medical students to rehearse and receive feedback on their patient interviewing, examination, and diagnostic skills. Up to this point, the team had focused on creating a tool that would enable researchers to study different aspects of VPs design and use, as described in our earlier summary of Year 1–4 research findings. Until the instructional design (ID) expert joined the initiative at the end of Year 3, the R&D team, comprised of SMEs and software engineers, were not aware of systematic instructional design (ID) and design-based research (DBR) methods that may facilitate both research and the development of the final product. As the team began to plan the last year of R&D,

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1071

the ID expert proposed a series of iterative design studies to create the latest version of NERVE to be released to the public. Subsequently, the team utilized knowledge gained from the first 4 years and adopted DBR as a framework for formulating both theoretical insights and practical solutions with stakeholders in real-world [medical school] context as posited by McKenney and Reeves (2012). Specifically, we worked with medical school students, staff, and instructors to address five features of design experiments posited by Cobb et al. (2003, pp. 9–11), including (a) the development of “a class of theories about both the process of learning and the means that are designed to support that learning,” (b) investigating “the possibilities for educational improvement by bringing about new forms of learning in order to study them,” (c) testing a hypothesized learning process that fosters “the emergence of other potential pathways,” (d) addressing practical problems faced by practitioners, and (e) containing iterative cycles of invention and revision. To summarize the last year of R&D, we begin by describing the pedagogical foundations of NERVE: the class of theories used to guide the design of NERVE that characterized both the process of learning and the means that were designed to support that learning. Then, we describe the iterative cycles of invention and revision that facilitated the design and development of NERVE and discuss how we investigated the possibilities of educational improvement by field testing the hypothesized learning process facilitated by NERVE. Throughout the design, development, and testing of the system, we addressed practical problems faced by practitioners. To experience NERVE prior to reading about its pedagogical foundations, we encourage you to access the system at http://nervesim.com, set up a free account, and take it for a test drive. Below are copies of two short video tutorials embedded in NERVE that give students an overview of the NERVE Learning Center and the NERVE Exam Room (Videos 1 and 2). Additional video tutorials are also embedded in NERVE to help learners use the menu-driven NERVE interface and the open-ended, chat NERVE interface.

Pedagogical Foundations We applied the InterPLAY instructional theory to guide the design of instructional features, and the sequencing of instructional events presented to learners before, during and after interactions with NERVE. We delineate the pedagogical foundations of NERVE by describing the fundamental concepts associated with the theory, key events associated with its corresponding instructional strategy, and its neurobiological underpinnings.

Conceptual Model We believe in experiential learning. We believe in continuity (the idea that students learned from their experiences) and (b) interaction (the notion that students’

1072 Fig. 1 Conceptual elements of the InterPLAY Instructional Theory. Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016b, Copyright 2016 by A. Hirumi. Reprinted with permission)

A. Hirumi et al.

Framing

STORY Characters Events Settings

PLAY Stimulus Response Consequence Experiential Learning Reflecting

Activating

GAME Rules Tools Goals

experiences were derived from their interactions with the environment and other individuals) (Dewey, 1938, p. 25). We also believe that children, adolescents, and adults learn best when presented with relevant, meaningful, and interesting challenges and when skill development and the learning of facts, concepts, procedures, and principles occur in context of how they will be used (Schank et al. 1999). However, experiential approaches to teaching and learning do not explicitly address the role of human emotions and imagination during the learning process (Hirumi et al., in press). They neither explain how human emotions and imagination may affect experiential learning nor posit methods for stimulating human emotions and imagination during the learning process to enhance student engagement and achievement. As a result, the application of experiential learning principles may not realize the potential of emerging technologies to facilitate individual and team performance. Figure 1 depicts a conceptual model of the InterPLAY instructional theory that integrates three universal principles of experiential learning (i.e., framing, activating, and reflecting on the experience) (Lindsey & Berger, 2009) with elements of story (i.e., characters, worlds, and events), play (i.e., stimulus, response, and consequences), and game (i.e., goals, rules, and tools), to evoke emotions, spark imagination, and create engaging and memorable learning experiences. To portray the pedagogical foundations of NERVE, we describe how we applied three principles of experiential learning and the conventions of story, play, and game associated with of the InterPLAY instructional strategy to design the system.

Principle 1: Framing the Experience Principle 1 includes “communicating the instructional objectives, assessment criteria, expected behaviors, and social structure with peers, instructors and the environment beyond the class” (Lindsey & Berger, 2009, p. 124). The objectives varied by study based on unique goals of the researchers for each investigation during the initial 4 years of R&D. To guide the final year of development, the ID

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1073

expert completed a cursory cognitive task analysis and worked with project members to define an explicit set of measurable performance objectives which were made explicit in the introductory screens of NERVE. The application of Principle 1 catalyzed an important discussion between R&D team members. In short, we had to agree on the core outcomes to be presented and assessed and determine how much simulation was enough to achieve the specified objectives. For instance, after early attempts to incorporate haptic (force) feedback, no effort was made to simulate examination tools that required motor perception to limit the scope of the simulation. The performance objectives had to be met with the understanding that there would be reasonable trade-offs for the modeling and simulation of the palsies and of the interviewing and examination tools and techniques used to diagnose the VPs. The recommended social structure, which was also communicated through the initial introductory screens, encouraged students to “explore NERVE individually or learn best by working in teams of two or more students,” as suggested by earlier research findings (Johnson et al., 2014). To further frame the experience, students are shown the two basic components of NERVE – the Learning Center (where they can review CN anatomy, physiology, symptoms, and pathology and practice the use of relevant physical examination tools) and the Exam Room (where they are tasked with diagnosing VPs presenting with symptoms of CN disorders). The initial pages also mentioned the benefits of the experience – learning, practice, and a safe place to explore – to frame what students should gain from the experience. The assessment criteria and expected behaviors are left to the instructor to define and communicate when the system is first demonstrated to students. Performance assessments, in the form of quizzes and students’ diagnoses of virtual patient cases, are included in NERVE, but the system neither specifies which CNs and patients to examine nor prescribes which quizzes to complete. Faculty members are encouraged to define such requirements and communicate expectations for their students when they introduce the system to them. Prior to the last year of R&D, team members thought the planned interactions with NERVE were adequately framed. But, during the initial design studies, we found that students were confused as to what exactly they were to learn from NERVE. Until we conducted multiple design studies focused on improving the system, NERVE was simply an application that was run to interact with VPs with particular CN disorders. The nervesim.com website now frames these experiences and provides a way for individuals and small groups to learn without the direct assistance of an instructor (although instructors are an integral part of the overall process).

Principle 2: Activating Experience Principle 2 recommends (a) providing authentic experiences to facilitate transfer, (b) making decisions that have authentic outcomes, (c) orienting learners so they see the relevance of the specific learning activities in relation to the larger problem, and (d) presenting challenges with optimal levels of difficulty to keep them engaged to activate both prior and newly initiated experiences (Lindsey & Berger, 2009).

1074

A. Hirumi et al.

The propositions associated with Principle 2 were the initial drivers for the project that were most consistently addressed prior to formally adopting the three universal principles of experiential learning. From the beginning, team members sought to facilitate transfer by providing the most authentic experiences possible with VP simulations by asking students to make decisions with realistic outcomes. Throughout the project, team members also worked to present cases with optimal levels of difficulty to keep students engaged and to orient learners so they could see the relevance of using NERVE to refine their clinical skills.

Principle 3: Reflecting on Experience Principle 3 suggests that experience must be analyzed to learn from it. Reflection should involve students answering the questions, “What happened?” “Why did it happen?” “What have I learn?” and “How would I apply this knowledge to future experiences?” (Lindsey & Berger, 2009, p. 129). The R&D team conducted one study, prior to the final year of development that examined the effects of student reflections during the learning process (RiveraGutierrez et al., 2014). Students in the treatment group were prompted with reflective questions during interactions with NERVE. Students in the control group were asked to reflect on their experience after interacting with the system. Results showed that 58% of the learners demonstrated evidence of reflection across groups, but only one learner showed critical reflection in her response. Given that the previous interactions were limited to 20 min, we felt that meaningful reflections would increase if time constraints were removed. Given these results and considering Principle 3, during the last year of R&D, we made reflective learning a formal part of NERVE. To address Principle 3, we planned and implemented an after action review (AAR) for the beta prototype field test, based on tactics. In short, we followed guidelines for planning and conducting AARs recommended by Salem-Schatz, Ordin, and Mittman (2010), adding one additional question about what students learned from their interactions with NERVE to enable the instructor to diagnose and correct misconceptions, fill in gaps, and elaborate on key points. We prescribed an AAR to address one of the universal principles of experiential learning. However, we have yet to fully develop the possibilities of AARs, because students are not asked to complete formal AARs throughout the medical school curriculum. Debriefings between instructors and small groups of students are typical after completing a simulation; introducing an AAR as a specific step in learning after interacting with NERVE led to what was perceived as an additional process for the students. With no prior recognition of its importance, and no equivalent follow-up, the AAR and application of Principle 3 lived in a vacuum. To realize its potential, we need to consider how to better address Principle 3, as well as how to facilitate AARs across the medical school curriculum. We applied the three universal principles to facilitate experiential learning with NERVE. To explain the treatment of the story, play, and game elements of InterPLAY depicted in Fig. 1, we discuss the application of the InterPLAY instructional strategy and the design of its related events.

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1075

InterPLAY Instructional Strategy Figure 2 depicts the latest version of the InterPLAY instructional strategy. Earlier versions posited inquiry and discovery before learners were stimulated to create and experiment with potential solutions to existing challenges (Stapleton & Hirumi, 2011, 2014). Originally, we reasoned that learners had to acquire basic skills and knowledge through play to feed creative problem-solving during subsequent game events. The inquiry and discovery of fundamental skills and concepts were seen as prerequisites as well as drivers for creation and experimentation. However, as we designed NERVE, we found that some learners wanted to interact with the Learning Center as they diagnosed the VPs in the Exam Room (Hirumi et al., in press). We learned that allowing students to access learning resources at any time during the simulation is particularly helpful in sustaining the motivation and positive emotions of learners who would otherwise feel frustrated by having to review information they feel they have mastered (Hirumi et al., 2016a, in press). In other words, like gamers who would rather go straight to playing the game, and read manuals and instructions only when necessary, we found some medical students preferred to go straight to diagnosing VPs in the Exam Room, accessing the resources in the Learning Center only when they felt it was essential. While the efficacy of either approach is an empirical question yet to be answered, the shift from a relatively linear presentation of instructional events to a more “just-in-time” approach marks a significant shift in the basic architecture and design of the system that serves as an alternative approach toward applying the instructional strategy. Fig. 2 Instructional events associated with the InterPLAY instructional strategy

STORY EXPOSE CREATE

INQUIRE

GAME

PLAY

EXPERIMENT

DISCOVER SHARE

STORY

1076

A. Hirumi et al.

Providing access to important facts, concepts, rules, and principles just as learners work through problems, rather than beforehand, is also more consistent with experiential learning principles.

Story Events The InterPLAY story events answer the question, “why should I care?” The first event should expose learners to an objective-driven story experience that motivates them to engage in the subject matter. By using information about students’ demographics (e.g., gender, ethnicity, social economic status) and psychographics (e.g., needs, interests, and habits), the experience should formulate characters, events, and settings that spark the imagination and catalyze an intrinsic desire to know more while framing the learning objectives in authentic context to establish relevance. The second story event (share) provides closure to the learning experience. Reaching new and significant levels of insight often leads to the natural desire to share one’s experience. In addition, many learning experiences are not encoded until students reflect on their experience. Reflecting and sharing one’s experience to enhance learning and transfer are facilitated explicitly during the AAR and are recommended as an integral part of the NERVE integration strategy. During the last year of R&D, we did relatively little to formulate a story that transcended the VPs. The team felt that, with limited time and resources, priority had to be placed on creating the Learning Center. Each VP already had a backstory that is portrayed in real time during the interactions with the VP. Otherwise, the use of the story was limited to (a) text-based case histories, pictures, and simulated interactions of each VP in the Exam Room and (b) the sharing of students’ reflections about NERVE during an AAR. Play Events The InterPLAY play events facilitate learning by answering the questions, “what do I need to know and be able to do?” Play offers learners an open-ended environment with no pressures or expectations of winning or losing or having the right or wrong answer. It presents learners with the stimulus, response, and consequence necessary to “see” cause-and-effect relationships and acquire basic skills and knowledge through inquiry and discovery. The purpose of play is to uncover the dynamic forces and obtain the verbal information, concepts, procedures, and principles necessary to overcome the challenges presented by the game elements of InterPLAY. One of the primary reasons we adopted InterPLAY to guide the design and development of NERVE was the separation of the play and game components of the theory. The distinction made between play and game clearly illustrated the role of the NERVE Learning Center (that learners may want or need to inform the diagnostic process) and VP simulations in the NERVE Exam Room (that enable students to test and refine their diagnostic skills). During the first 4 years of R&D, prerequisite knowledge of basic CN anatomy, physiology, and pathology was covered by medical school faculty in conventional lecture-style format before students were given access to NERVE. InterPLAY illustrated how the addition of content information to the system could enable NERVE to

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1077

become an independent learning platform that medical schools and students could use to cover both the acquisition and application of relevant CN skills and knowledge. With the adoption of InterPLAY, we added the Learning Center to give students the opportunity to: (a) Learn how and when to use the physical examination tools for diagnosing patients with potential CN disorders (b) Review relevant information about CN anatomy, physiology, symptoms, and pathology (c) Explore published case studies about CN disorders (d) Take multiple-choice quizzes about each CN to help them self-assess and monitor their own knowledge A link allowed students to access the Learning Center at any time: before, during, and/or after interacting with the VP cases in the Exam Room. The addition of the Learning Center also made NERVE complete from a student’s perspective. After making a mistake, students can find it very frustrating to hunt for a reliable source of information to correct the misunderstanding. This is especially true in medicine where misinformation is easily available and where reliable texts on specific topics can be too in-depth or difficult to understand. The Learning Center serves as an easily accessible, easy to understand resource that can be used both to teach the information and as a quick reference for students to reinforce knowledge and correct errors in knowledge.

Game Events The InterPLAY game events answer the question, “how do I overcome the given challenge(s)?” The game events provide opportunities to create and experiment with alternative solutions to real-world problems. As we noted earlier while describing the application of experiential learning principles, Principle 2 (Activating the Experience), was the initial, primary driver for the project that leads to the development and experimentation of the VP simulations throughout the 5-year project. During the last year of R&D, the VPs in the NERVE Exam Room represented the game component of InterPLAY, presenting students with authentic, simulated experiences to test, and refined their diagnostic skills with specific goals, rules, and tools. The goal for interacting with the VPs is to properly diagnose specified cases in the NERVE Exam Room. The rules governing interactions with the VPs are to be defined by the instructor and by the system. A variety of physical examination tools and tools for facilitating VP interviews and exams, are integrated into the system. These include a menu of questions and actions provided within the selection NERVE interface, recommendations for posing questions within the chat NERVE interface, and tools for monitoring progress within both interfaces.

Dimensions of xLearning xLearning represents an experiential learning process, based on educational and cognitive neuroscience research, that leads to the expression of an individual’s or

1078

A. Hirumi et al.

Fig. 3 Dimensions of xLearning

team’s skills, knowledge, and dispositions. Five dimensions of xLearning represent fundamental physiological processes that provide a neurobiological foundation for InterPLAY. As depicted in Fig. 3, xLearning begins with perception. When sensory input enters the brain, it is interpreted and filtered by expectations and relevant prior experience. As imagination constructs several possible outcomes, the brain selects the most appropriate response for any given situation. Reflection monitors processes and activates procedures that drive decision-making and behavior. Emotion functions as a catalyst that continuously influences perception, experience, imagination, and reflection.

Perception Perception collects, interprets, and filters sensory input. Perception transforms raw sensory signals into information that guides behavior. As we perceive the world, we use internal mechanisms, such as selective attention and pattern recognition to extract and analyze relevant information. Selective attention focuses on details while filtering out less important information. For example, in a crowded room of individuals, selective attention can focus on a single conversation while ignoring other voices. Pattern recognition allows quick comparison of incoming stimuli to experience. If incoming patterns match our experience, specific actions can be activated based on expectations. If discrepancies exist between what we perceive

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1079

and what we expect, imagination reconciles the differences and predicts alternative outcomes to facilitate decision-making. The result is stored as a new experience that directs future perceptions.

Experience An important goal of xLearning is to filter information in meaningful ways so our experience accurately predicts actions. Despite the common misconception that we learn from failures, the brain focuses on success – experiences that benefit survival. Experience can be categorized into several types of memory: (a) sensory (incoming stimuli), (b) episodic (events), (c) semantic (facts, concepts, and general knowledge), and (d) procedural (action sequences). Procedural memory is similar to episodic and semantic memory. At an unconscious level, skills such as swinging a golf club and singing a song are stored primarily in the cerebellum but also in the motor and premotor cortex as procedures. Procedural memory includes a feedback loop that monitors conditions in the body. When practicing a particular motion, procedures that are processed in premotor and motor cortices activate muscles. As the procedures execute, feedback reports if the muscles are moving appropriately, this contributes to revision for improving the procedure with each repetition. Repetition signals neurons that form long-term potentiation (LTP). In turn, LTP is widely considered one of the principal cellular mechanisms that underlies the development of experience, as memories are encoded through the modulation of synaptic strength (Cooke, 2006). With continued use, relevant information eventually consolidates into more permanent storage by releasing a protein called kinase C (PKC) among specific neurons to form new or strengthen existing synaptic connections. Without rehearsal, experience fades and recall of experience becomes more difficult as imagination tries to resolve missing details. Imagination Although imagination and creativity are often considered synonymous, in xLearning, we distinguish imagination as one of the five fundamental internal physiological processes that lead to the external expression of skills as creativity. Imagination is a continuous process for interpreting perceptions and reconstructing experiences. An extensive network of neural fibers processes relevant sensory input in the associative and motor cortices by resolving differences between our perceptions and experience. Pattern recognition quickly assesses the most appropriate interpretation and expression when given minimal sensory input under urgent conditions, or effectively plans for complex actions, when given time to reflect. During xLearning, imagination reconciles differences between perception and experience and creates probable outcomes to facilitate decision-making and to guide behaviors. For example, when playing golf, effective swing selection requires some ability to predict the effects of future actions (e.g., taking a half swing may hit the ball 60 yards). Before swinging, imagination predicts outcomes given potential actions and selects those that best approximate expectations. As a novice, repeated

1080

A. Hirumi et al.

rehearsal refines and eventually automates procedures to become more expert. So, with the old adage of practice makes perfect, in reality, practice makes actions permanent (Atkinson & Hirumi, 2010). When learning actions incorrectly, replacing them with correct actions can be very challenging.

Emotion Emotion serves as a catalyst for perception, experience, and imagination. In xLearning, emotion invokes automatic processes, primarily at an unconscious level, as well as deliberate conscious processes. In each hemisphere of the brain, sensory input relays through the thalamus and projects to the hippocampus (experience), the prefrontal cortex (PFC), and the amygdala (center for our emotions). When activated, the amygdala signals the cingulate cortex, which quickly projects to the premotor cortex (physical response system) and causes an immediate reaction to the event. This is how incoming stimuli causes us to react in a physical manner (e.g., fight, flight, or freeze) before consciously processing the information to analyze what happened (Rimmele, n.d.). The role of emotions during xLearning lies in its ability to signal the release of neurotransmitters throughout the brain that enhance the strength and number of connections between neurons. Typically, stronger emotions release more neurotransmitters and result in stronger neural connections with long-term potentiation in memory (Rimmele, n.d.). That’s why we are more likely to remember events that elicit strong emotional reactions, regardless if it’s perceived as a negative or positive experience. The valence of our emotions affects our continuing motivation. We tend to shy away from experiences that result in negative emotions and return to goaldirected behaviors that elicit positive emotional responses through reflection (Rimmele, n.d.). Reflection Reflection monitors sensory perception, explores experience, invokes imagination, and assesses emotional state. Following active networks distributed throughout the brain, reflection uses the PFC for cognitive processing with the cerebellum, basal ganglia, hippocampus, associative cortex, thalamus, and other parts of the brain. Colder (2011) suggests all cognitive processing requires the creation and maintenance of accurate emulations (ongoing representations of actions and predictions of a future environment associated with those actions). Emulations offer a mechanism for integrating action selection, perception, attention, learning, and higher cognitive functions into a single theory of cognition. According to Colder (2011), active neurons in the PFC maintain representations of alternate futures that include plans for upcoming actions. When sensations impact cognition, emulations compare the input to the representations. Reflection evaluates the predicted outcomes of the plans to select and express the actions that best realize expectations (Colder, 2011). Guided by emotional state, reflection matches our perceptions of incoming stimuli with experience. When significant differences between perception and experience exist or information is insufficient to make accurate predictions based on

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1081

experience, imagination constructs alternatives with explanations and expectations. Emulations also update associative memory to reduce the likelihood that sensory and expectation errors are repeated. This iterative process continues until our perception is resolved or we abandon the effort (Colder, 2011). For example, when we see a magician perform a trick for the first time, we automatically predict expectations and search our experience to explain how the trick was accomplished. If a solution is not found quickly, the perception is unresolved and eventually abandoned. However, if the magician explains how the trick works, we now have the correct solution and our perception of the trick is forever changed. Whenever we see a similar trick performed, we are more likely to accurately predict or recall the solution from experience – or in other words, from what we learned.

Words of Caution We recognize that others who read the same research may come up with different interpretations of the physiological processes that govern human learning and performance. We also recognize the limitations currently associated with using physiological measures to explain psychological constructs. We believe advances in technology and neuroscience will lead to significant insights into how and why people learn and that xLearning provides a robust theoretical foundation for InterPLAY by accounting for the role of human perception, imagination, experience, emotion, and reflection during the xLearning process. We also believe in the value of grounding the design of simulation-based training in research and theory and conducting iterative cycles of design-based research studies to advance research and theory.

Iterative Cycles of Design and Development During the last year of R&D, we completed two cycles of expert reviews, one-to-one and small-group evaluations to design and develop the system, along with a field test to examine students’ use, reactions, learning, and transfer. The results were used to improve NERVE after each study, demonstrating the iterative cycles of invention and revision characteristic of DRB (Cobb et al., 2003). During the first cycle of design studies, a subject matter expert (SME) and four second-year medical students helped identify the most obvious errors in the initial alpha prototype of NERVE. The SME and students also evaluated the clarity of the instructional objectives, content information, and tutorials and the usability of the user interface and physical exam tools. Comments, observations, and recommendations for improvement were recorded by two team members. The SME’s comments and recommendations for improvement centered on validity of the physical exam tools, the accuracy of the methods used to simulate the CN disorders, and the relative importance of the content (e.g., time spent addressing one CN versus other CNs). The formative evaluation of NERVE by the SME, during both design cycles and the field test, also focused on ensuring that the system

1082

A. Hirumi et al.

addressed practical problems faced by practitioners that are illustrative of DBR. For example, when asked if NERVE addressed challenges in teaching medical students about CN disorders, SME noted that, “CN1 palsies are more common in head trauma, like football players usually have an impaired sense of smell. However, [the system] may be over-emphasizing something that normally isn’t tested—CN1 is usually left out and tested about 5 times for every 1000 physical exams. Should prioritize information somehow.” The R&D team members who recorded comments made by the medical students and compiled the results also formulated and ranked recommended revisions based on perceived impact and cost and presented the recommended revisions to the entire R&D team, who then discussed the findings and recommendations and rated each recommendation as to do, to do if time/resources permit, or take no action. Table 2 presents a sampling of recommendations received from second-year medical students that revolved around the introductory frames (e.g., splash screen, list of objectives, and overview of system), information about the CNs in the Learning Center, and interactions with the virtual patients in the Exam Room. Additional recommendations for improvement were reported and considered by the R&D for each target listed in Column 1 as well as across all CN than depicted in Table 2. A sample of recommendations is presented to illustrate the nature of comments that were reported and considered by R&D team members during the initial cycle of design studies. Several comments lead to significant changes in the design of NERVE. For example, after reviewing the Learning Center, a number of students strongly recommended adding multiple-choice quizzes that would enable students to test and monitor their acquisition of key facts, concepts, and principles that lead to the addition of self-tests. Students also encouraged R&D team members to add tutorials that highlight and illustrate the use of key features of both the menu-driven and openended user interface. One set of comments that resulted in both improvements to the system and significant advances to the InterPLAY instructional strategy focused on providing just-in-time access to CN information contained in the Learning Center. Prior to NERVE, InterPLAY posited our relatively linear sequencing of events, with learners inquiring and discovering basic skills and knowledge prior to playing the game. With the design and development of NERVE, the play elements became instructional events that may be accessed before, during, and/or after game events as discussed in further detail in Hirumi et al. (in press). The advancement in how the play elements are depicted and accessed represented a fundamental change in the InterPLAY instructional strategy and illustrates the development of theory characteristic of DBR which occurred during the first design cycle. During the second cycle of design studies, two SMEs, four first-year medical students, and a small group of ID graduate students evaluated version 2 of the alpha prototype. The experts and students evaluated the revisions made based on cycle 1 findings and again reviewed the objectives, content, and tutorials for clarity and validity and the interface and exam tools for usability. Significant improvements were made to the simulated physical examination tools included in both the NERVE Learning Center and Exam Room based on comments

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1083

Table 2 Sample recommendations for alpha prototype from 2nd-year medical students Target Introduction opening frame

Introduction overview Learning Center Main menu

Learning Center CN1

Learning Center CN2

Exam Room David Jacobs (selection)

Exam Room David Jacobs (chat)

Recommendation Add a frame after log in, before the objectives to better capture learners’ attention at the beginning Consider using a picture of someone with a CN palsy as a background image and asking something like “Can you diagnose this patient?” Rewrite text in 2nd person. Clarify recommended use, and better distinguish purpose of Learning Center and Exam Room Present additional mnemonic for remembering functions (i.e., Some Say Many Other. . .) not just for remembering names Add a sixth column titled, “self-test” with a link in each row to a 10-item quiz that allows students to test their own knowledge of the content information covered on each nerve Add a link to a generic tour of the interact interface (within italicized paragraph) highlighting key features, including a demo on how to get instructions on use of each tool. All four students did not notice soap. Need to provide better indicator of features. Consider how to provide direct access to overview or Exam Room from Learning Center Make sure hand tool is available. Again, need to provide better indicator of what features are available and what to do Text information on use of tool (accessed from “?”) is good but should be made more direct and consistent across tools, stating how to use tool only Check to make sure interview progress works properly. Did not appear to work while students interacted with patient Consider making soap more obvious (label as “soap”) Fix technical problems: No response when smelling soap. Tilt head Provide tutorial/instructions on how to use tool(s) with and without mouse. Include in general tutorial provided on page listing all cases In chat mode, students expressed frustration in asking questions that VP didn’t understand Add searchable question about fatigue. When system did not understand free text, student searched for but could not find any questions about fatigue (relevant for CN0) Add link to Learning Center. Again, students commented that they would like to access information in the Learning Center during and after examining the patient

made by the two SMEs during the second design cycle. The Exam Room features virtual tools (such as ophthalmoscope, tongue depressor, eye chart, and tuning fork) that simulate the use of real tools used during CN exams. The same tools are made accessible in the Learning Center to give students practice at using the tools and insights into when the tools should be used. Usability testing of the tools revealed continued challenges in representing the three-dimensional movement of tools in a two-dimensional computer environment. By concentrating on practical problems faced by practitioners as prescribed by DBR, we realized that NERVE should be designed to teach medical students when to use the various examination tools, rather than how to use them (which is better taught in clinical settings).

1084

A. Hirumi et al.

During the second cycle, the feasibility of implementing the VP simulation was also considered given available time and resources, along with the impact of the VP simulation on students’ attitudes and achievement. Again, the team members who facilitated the studies compiled the results, formulated and ranked recommended revisions, and presented their recommendations to the entire R&D team who decided what revisions to make and when based on perceived impact and cost. Though straightforward in purpose, the underpinnings of NERVE are built on a complex weave of technologies that include natural language recognition, interactive computer graphics, networking, and databases, all developed at multiple physical locations using a myriad of tools that evolved over the course of the project. The complexity, and early mishaps from lack of testing, led our team to implement strategic software engineering policies that included a “code freeze” (no addition of features, only bug fixes) weeks prior to the field test, followed by substantial debugging by all members of the team. We also found it necessary to “stress test” the system by artificially simulating large numbers of users that would be encountered in typical settings. Revisions based on findings from the second cycle of design studies, and efforts to debug the system led to the integration and field testing of the beta version of NERVE.

Integration and Field Testing The beta prototype was integrated into the medical school curriculum and field tested with 120 medical students enrolled in a second-year Neurology course as illustrated in Fig. 4. Patterned initially after Huwendiek, Duncker, Reichert, De Leng, Dolmans, van der Vleuten, and Tönshoff (2013) preferred sequencing of VPs and educational activities, we used existing VPs integration research, input received from the instructor, and the universal principles of experiential learning to hone our strategy for integrating NERVE into the curriculum. The strategy included (a) a lecture on Neurology, (b) a demonstration of NERVE with explicit expectations and

Fig. 4 Key components of the NERVE integration strategy. Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016, Copyright 2016 by A. Hirumi. Reprinted with permission)

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1085

requirements, (c) VPs interactions within NERVE, (d) an instructor-led AAR with the entire class, and (e) a standardized patient/virtual patient (SP/VP) hybrid encounter, as depicted in Fig. 4. The integration and field testing of NERVE illustrates how we tested a hypothesized learning process that fosters “the emergence of other potential pathways” as prescribed by DRB (Cobb et al., 2003, p. 10).

Field Test Method Participants Participants in the field test included 116 of 119 second-year medical students (98.3%) enrolled in the Neurology course at the University of Central Florida, College of Medicine, in Orlando, FL. One student did not consent to the use of data for analysis and reporting purposes, and one student contributed extensively to the development of NERVE and related learning and assessment materials. Field test participants included 58 females (50.4%) and 58 males (49.6%), with a median age of 24 years (IQR = 23–25; range = 22–38), representing one or more diverse ethnicity groups – 34 Asian (29.1%), 5 Black or African-American (4.3%), 7 Hispanic or Latino (6.0%), and 79 White or Caucasian (67.5%). Instruments We designed or adapted tools to assess students’ use of NERVE and students’ reactions, learning, and transfer of knowledge and skills, as described below. Use. Student use of NERVE was tracked directly by the system via unique student login credentials. The system recorded duration of use per page accessed, quiz access and answer selections, interactions with VPs (i.e., questions asked and examinations performed), and final diagnoses. Reactions. We adapted the Instructional Materials Motivation Survey (IMMS) with permission from the author to examine student reaction to NERVE immediately following their 1-week use of the system. The IMMS is based on Keller’s (1987) Attention, Relevance, Confidence, Satisfaction (ARCS) model of motivation. Each of the 4 ARCS areas is represented on the survey with 6 items, for a total of 24 items. Each item is rated on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree), such that subscale scores range from 6 to 30, and total scale scores range from 24 to 120. Cronbach’s coefficient alpha values indicate strong internal consistency for each of the subscales (0.81–0.92) and for the total scale (0.96; Keller 2010). The survey was disseminated to students through Qualtrics. The AAR presented three questions (based on the work of Salem-Schatz et al., 2010) to students during a whole-class discussion 1 week following students’ use of NERVE. Students’ responses to Question 1 (What did you learn from your interactions with NERVE?) was used as a measure of student learning (as discussed in the proceeding section). Questions 2 and 3 (What did you like about NERVE and why? What should be changed to improve NERVE and how?) were used as additional measures of students’ reactions to the system. All three questions were displayed on a screen in the lecture hall and students responded collectively in teams of 10–15

1086

A. Hirumi et al.

students each (predetermined by other curricular activities) via a form in Qualtrics. A faculty member and researcher who facilitated the AAR were able to view team responses in real time to allow for tailored discussion and exploration during the 2-h session. Learning. Student learning was measured through use of quizzes embedded in the NERVE Learning Center, diagnosis of VP cases in the NERVE Exam Room, and student reflections about learning during an after action review (AAR, described above). A quiz containing ten multiple-choice questions (MCQs) was available to students for unlimited attempts for each of the 12 CNs; MCQs were developed, vetted, and revised as needed by members of the R&D team. Following each quiz submission, the system indicated which items were answered correctly and incorrectly and displayed explanatory feedback for why a selected response was incorrect without disclosing the correct response. Diagnosis of each VP and localization of the condition were available in Patient Encounter Notes completed by students after each interview and examination; responses were considered correct when both the diagnosis and localization were identified correctly. Transfer. Transfer of learning was assessed during an SP/VP hybrid encounter 1–2 days following the AAR. Students’ interviewing, interpersonal, and communication skills were assessed by SPs following each encounter through the use of a 15-item checklist developed by medical school faculty members. Checklist items were scored as observed or not observed, and total scores represent percentage of 15 total behaviors observed. Similar to the assessment of learning described above, diagnosis of the patient in the SP/VP hybrid encounter and localization of the condition were available in Patient Encounter Notes completed by students following the encounter; responses were considered correct when both the diagnosis and localization were identified correctly. Students were also asked to identify the underlying pathology. The SP checklist was completed online in the Exam Room through the Simulation Center’s online system, and the Patient Encounter Note was completed by students outside of the Exam Room through Qualtrics.

Procedure Students were exposed to a series of educational, assessment, and feedback events throughout this field study, as follows: (a) presentation of a Neurology lecture, (b) exposure to NERVE demonstration, (c) 1-week use of NERVE with an embedded assessment, (d) completion of the IMMS through Qualtrics, (e) participation in an AAR, and (f) participation in an SP/VP hybrid encounter with related assessment. NERVE was demonstrated to students during the last 20 min of a regularly scheduled Neurology lecture. The Neurology faculty member explained the organization and use of NERVE to students via short videos and communicated expectations for using NERVE over the assigned week. Student consent forms were also completed during this time. Students used NERVE on their own time for 1 week following the demonstration and were permitted to work as individuals, in pairs, or in teams of three to four members each. The IMMS was distributed to students immediately following the 1-week period via an email presenting a link to the survey in Qualtrics. One week after students’ use of NERVE, the Neurology faculty member

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1087

and one member of the R&D team conducted the 2-h AAR in a large lecture hall. Real-time review of student responses in Qualtrics to the first AAR question about learning allowed the faculty member to lead a discussion about CN anatomy, physiology, and pathology. Students were also given instructions related to the upcoming SP/VP hybrid encounter at this time. Due to a limited number of Exam Rooms in the Simulation Center, half of the students participated in the SP/VP hybrid encounter 1 day following the AAR, and the other half participated in the SP/VP hybrid encounter 2 days following the AAR. During the SP/VP hybrid encounter, students interviewed the SP but performed the physical exam on the VP through NERVE on a computer in the Exam Room. Students were permitted to move between interactions with the SP and VP as they deemed necessary throughout the encounter and were given 20 min for completion. Following the encounter, SPs remained in the Exam Room to complete the 15-item checklist on a computer in the room, and students moved to computer stations immediately outside of the Exam Rooms to complete the Patient Encounter Note in Qualtrics.

Data Analysis Categorical variables are presented as frequencies and percentages, ordinal and non-normal continuous variables are reported as medians and interquartile ranges (IQR) and/or minimum-maximum (range), and normal continuous variables are displayed as means  standard deviations (SD) with 95% confidence intervals (CI) of the mean. Cronbach’s coefficient alpha is reported to estimate internal consistency. Bivariate correlations were calculated using Pearson’s correlation coefficient (r). Statistical analyses were conducted using SPSS 22.0 (IBM, Armonk, NY). Qualitative data were examined through thematic content analysis.

Field Test Results Results are organized below to describe students’ use of NERVE and to reflect three of four levels of Kirkpatrick’s model for the evaluation of training effectiveness (1994): Level 1 Reaction, Level 2 Learning, and Level 3 Behavior (Transfer). Correlations between key outcomes measures are also summarized.

Use Most students used NERVE as individuals (53%) or in pairs (24%). Students accessed NERVE an average of six times during the 1-week period and spent 0–8.5 h interacting with NERVE (median = 1.9 h; Table 3); use peaked on the day before the AAR and again on the day before the SP/VP hybrid encounter (Fig. 5). Seventy-nine of 116 students (68%) completed three or more of the VP cases, and 110 of 116 students (87%) completed all five quizzes, as directed by the instructor during communication of expectations for use. Few students (5/116; 4%) elected to complete additional quizzes (Fig. 6).

1088

A. Hirumi et al.

Table 3 Summary of students’ use of NERVE over prescribed 1-week period # of login sessions Expectation: none specified Time Expectation: none specified Primary access mode Expectation: individually or in teams of 2–4

Quizzes completed Expectation: complete minimum of 5 quizzes

VP cases completed Expectation: complete minimum of 3 cases

12 CN content exploration Expectation: whatever necessary to prepare for SP/VP and AAR

Median2 (IQR) range Median2 (IQR) range Individual Pair Team of 3–4 Did not use 0 1–4 5 >5 0 1–2 3 >3 No CN 1–6 CN 7–11 CN All 12 CN

6.0 (3.5–9.0) 0–23 1.9 (1.4–2.9) 0–8.5 h 61 (53%) 28 (24%) 14 (12%) 3 (1%) 6 (5%) 9 (8%) 96 (83%) 5 (4%) 16 (14%) 21 (18%) 71 (61%) 8 (7%) 29 (25%) 54 (47%) 33 (28%) 0 (0%)

Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016b, Copyright 2016 by A. Hirumi. Reprinted with permission)

Time on Task 1:12:00 1:04:48

1:03:13

0:57:36 0:50:24 0:43:12

0:39:20

0:36:00 0:28:48

0:21:36 0:14:24

0:07:12 0:00:00 0:00:22 Day 1

0:03:48 Day 2

0:06:30 0:00:39 Day 3

0 Day 4

Day 5

0:05:55 Day 6

Day 7

Day 8

Fig. 5 Average amount of time students spent interacting with NERVE during the week of prescribe use. Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016b, Copyright 2016 by A. Hirumi. Reprinted with permission)

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1089

Fig. 6 Picture depicting medical students interacting with both SP and VP during hybrid encounter. Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016b, Copyright 2016 by A. Hirumi. Reprinted with permission)

Reactions Instructional Materials Motivation Survey. One hundred ten of 116 students (94%) completed the IMMS. Mean score and SD for the ARCS total scale was 75.6  11.2 (95% CI = 73.5–77.8) and for each of the subscales, as follows: attention, 17.9  3.5 (95% CI = 17.3–18.6); relevance, 20.2  3.1 (95% CI = 19.6–20.8); confidence, 18.9  2.4 (95% CI = 18.5–19.4); and satisfaction, 18.6  4.1 (95% CI = 17.8–19.3). Internal consistency was strong for the total scale (0.89) and moderately strong for each of the four subscales, as follows: attention (0.68), relevance (0.62), confidence (0.61), and satisfaction (0.80). Student responses and summary data by item are presented in Table 4. After Action Review. Students provided reactions to NERVE during the AAR: What did you like and why? (n = 24; see Table 5 for a sample of responses) and What would you improve and how? (n = 24; see Table 6 for a sample of responses). Themes associated with what students liked related to objectives (e.g., NERVE facilitated achievement of the learning objectives), visualization (e.g., labeled diagrams, interactive animations of eye movements), the NERVE Learning Center (e.g., content information, interactive tools, quizzes), and the NERVE Exam Room (e.g., interactions with VPs). Students’ suggestions for improvement primarily related to seven basic issues: integration, technical, content, quiz, diagnostic, interviewing and examining, and interface issues. Learning Quiz Scores. Table 7 presents summary data related to students’ scores on the five required quizzes embedded in the NERVE Learning Center. Some students completed quizzes two or more times, usually achieving higher scores on subsequent attempts.

1090

A. Hirumi et al.

Table 4 Summary of students’ responses to ARCS instructional materials motivation surveys Scale Overall ARCS Attention Relevance Confidence Satisfaction

IC 0.89 0.68 0.62 0.61 0.80

Mean 75.6  11.2 17.9  03.5 20.2  03.1 19.9  02.4 18.6  04.1

Avg. (scale 1–5) 3.6 4.1 3.7 3.1

Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016b, Copyright 2016 by A. Hirumi. Reprinted with permission)

Virtual Patient Diagnosis. Table 8 presents the results of students’ diagnoses, including the name of the VP, the CN that was damaged, the number of students who completed each case, and the number and percentage of correctly diagnosed cases in terms of the CN and localization of the condition. After Action Review. Students also provided information related to their perceived learning during the AAR: What did you learn from your interactions with NERVE? (n = 31; see Table 11 for a sample of responses). Students spent the majority of their time in NERVE examining content related to CNs assessed by the five required quizzes (i.e., CNs 3, 4, 5, 7, 10). Students felt that NERVE helped them understand why a CN injury results in specific clinical signs and symptoms and that interactions with VPs helped them develop recognition skills for CN injuries through observation of abnormal findings (Table 9).

Transfer Standardized Patient Checklist. Percent correct scores on the 15-item SP checklist ranged from 67% (10 items observed correctly) to 100% (median = 93% or 14 items observed correctly; IQR 93.3–100.0). Student performance by individual item is presented in Table 10. Standardized Patient/Virtual Patient Diagnosis. Most students (108/116 93%) correctly identified both the injured CN and affected side as left CN6. Table 11 presents data related to student identification of the underlying pathology. Students correctly identified the underlying pathology in 87% of patients. Relationship Between Outcomes Outcomes were examined for the degree of correlation between students’ use of NERVE, reactions, learning, and transfer. Total number of hours using NERVE over the 1-week period was significantly correlated with the ARCS confidence subscale score (r = 0.20; p = 0.04), ARCS satisfaction subscale score (r = 0.26; p = 0.01), and ARCS total score (r = 0.22; p = 0.02). Students who spent more time interacting with NERVE reported greater confidence, satisfaction, and overall motivation with the use of NERVE than students who spent less time interacting with NERVE. Significant correlations were also found between the total number of hours using

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1091

Table 5 Sample of students’ reactions when asked what they liked about NERVE and why Theme Objectives

Learning Center: Interactive tools

Learning Center

Learning Center: Quizzes

Exam Room: Interviewing patients

Exam Room: Examining patients

Exam Room: User interfaces

Student comments Objective 1 – Recognize the anatomy and physiology of cranial nerves. “The anatomy was very complete during the reading sections for the cranial nerves” Objective 2 – Recognize the pathology and symptoms of cranial nerve dysfunctions. “We think that the simulator did a good job of demonstrating the symptoms of cranial nerve dysfunction” Objective 3 – Distinguish cranial nerve dysfunctions. “The eye ball diagram with the muscles and nerves did a good job of demonstrating the different cranial nerve palsies” “The CN examination tools and maneuvers presented helped us narrow in on what physical exam components were relevant to the presenting complaint. For example, in the patient with blurry vision attributed to CN3 and increased ICP, the expected maneuvers (pupillary reflex, visual acuity, visual fields, sensation in the areas of the facial nerve) helped us understand additional pathology that may be associated with this complaint” “We enjoyed reading through the 12 CN in Learning Center and having key information presented in a high-yield, concise, and consistent manner” “The symptoms and pathology component provided information that allowed us to consider mechanisms of injury that were not directly associated with the head” “We liked how the quiz allowed you to push the button to go back to the reading to explain the answers to the quiz questions” “The quizzes were engaging and challenging. The feedback provided was useful as well” “The CN examination allowed us to learn what the appropriate questions are to ask patients presenting with suspected CN palsies” “In addition, the virtual patient encounter had multiple reminders about empathizing with the patient, which was useful to reiterate its importance in patient communication” “The interactive portion of the examination was very helpful. It was realistic and allowed a unique learning experience” “The convergence tool was also useful and the ability that it had to make this a three-dimensional experience. It was also beneficial to use the ophthalmoscope to see a close up view of the optic nerve” “As a group, we felt that [the menu-driven] VP cases were very helpful especially in solidifying the order in which to ask questions and complete testing procedures” “During the guided physical exam, having all of the questions was helpful in how to word questions when typing questions in the other mode”

Note. Adapted from “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016, Copyright 2016 by A. Hirumi. Reprinted with permission)

NERVE over the 1-week period and students’ quiz scores achieved on the first attempt for CN4 (r = 0.23; p = 0.02) and CN5 (r = 0.32; p = 0.002).

1092

A. Hirumi et al.

Table 6 Sample of students’ reactions when asked what would you change and how Theme Integration

Technical issues

Learning Center

Learning Center: Quizzes

Exam Room: Diagnosing patients

Exam Room: Interviewing and examining patients

Student comments “We haven’t been taught much on diagnoses so it was hard to give a differential diagnosis” “Implement this system during first year when students are learning CN for the first time because it is a really helpful resource and it is nice to have all of the CN information in one place” “Clearer expectations about how much time we should be spending with the program (30 min or 5 h?)” “Our entire team had some technical difficulties, whether during the login process or during the patient interviews themselves and felt that our learning was somewhat compromised by this” “All of our group members encountered some technical difficulties, whether during the login process or in the virtual patient encounter (and the associated software download), and we felt that this could be greatly improved” “When the tools worked, they were very intuitive (but sometimes the tools didn’t work, e.g., tongue depressor)” “Many of us have alternative and varied resources that we are used to using for the past 2 years, so we did not like being forced to use a new resource that we felt we didn’t learn as effectively from. We believe that this should be a supplementary tool, not a mandatory part of the curriculum because we learn different ways” “Grammar and spelling should be reviewed. Quiz questions were sometimes unclear and sometimes had multiple correct answers. The order of true and false in standard examinations is always true first” “Some of the questions did not provide clear feedback, and it was sometimes difficult to discern what the right answer was” “There could be further additions regarding the differential diagnosis for cranial nerve lesions along with the assessment and plan. There should be more information regarding the plan for cranial nerve pathologies” “We learned differential diagnoses for different CN lesions; however, felt that the causes of the lesion could be covered in better detail during the wrap-up following the cases” “We also thought that maybe the history could have been listed out in a summary rather than having us click on every single questions and have a response (a time saver that we believe would not interfere with the learning process)” “Streamline the actions of the physical exam to prevent needing to do certain parts of the physical exam. For example, click “Perform EOM” and watch the motion occur, as opposed to what currently is needed”

Note. Adapted from “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016, Copyright 2016 by A. Hirumi. Reprinted with permission)

R&D Team Perspectives Over the 5-year project period, the composition of the R&D team changed, but the core group of faculty and students from three major public universities in Southeastern United States remained basically intact. The remainder of this chapter

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1093

Table 7 Summary of students’ scores on five required quizzes in NERVE Learning Center Scale CN3 CN4 CN5 CN7 CN10

n 101 102 97 97 99

Mean 82.4 77.6 85.8 76.4 74.9

Range 40–100 30–100 40–100 20–100 10–100

Mode 80 90 90 80 100

Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016b, Copyright 2016 by A. Hirumi. Reprinted with permission)

Table 8 Summary scores for diagnosing virtual patients in NERVE Exam Room VP Bill Jennifer David Molly Monica Cathy

CN 6 4 None 3 3 6

N 57 47 35 30 24 41

Correct 34 (60%) 35 (74%) 19 (55%) 25 (67%) 18 (75%) 36 (88%)

Side Left Left None Left Left Left

Correct 37 (65%) 32 (69%) 17 (49%) 26 (87%) 19 (79%) 34 (83%)

Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016b, Copyright 2016 by A. Hirumi. Reprinted with permission)

focuses on the contributions and evolving understanding of representative R&D team members, including a medical educator, two software engineers, the lead instructional designer, a statistical research specialist, and one graduate research assistant. Specifically, representative members (a) summarize their role on the project; (b) detail the key skills, knowledge, and dispositions they applied during the last year of R&D; and (c) reflect on what they learned from the experience. By mining the perspectives of members on the team, light was shed on how different fields see instructional design practice and theory before, during, and after the overall R&D experience.

PI and SME (Medical Educator) I am the Assistant Dean for Simulation and Chairman of the Department of Medical Education for the College of Medicine at the University of Central Florida. My role in the project was to elucidate the educational goals and objectives that are relevant to first- and second-year medical students with respect to the cranial nerve palsies. In my role as principal investigator, my focus was to act as interlocutor between the programming engineers and the educators. In our group, the programming team included faculty and students, some of whom featured this project as part of their

1094

A. Hirumi et al.

Table 9 Sample of students’ responses when asked what did you learn Theme CN3: Oculomotor

CN4: Trochlear

CN5: Trigeminal

CN6: Abducens

CN7: Facial

CN10: Vagus

Examination and Interviewing Skills

Student comments The difference between myasthenia gravis and CN3 palsy is the symmetric nature of MG vs. the asymmetric nature of CN3 palsy Also, CN3 does not affect the superior tarsal muscle, so the ptosis will be more severe with CN3 palsy CN3 palsy: “down and out” + dilated pupil + ptosis. Loss of direct and consensual reflex in the eye on the affected side Trochlear is the only CN that exits the posterior brainstem, and it is also the longest The most common cause of CN4 palsy in children in congenital. CN4 originates from the posterior midbrain and has the longest intercranial tract. It innervates the contralateral eye, which is unique I learned that otitis media was the most common cause of trochlear nerve palsy and that that Trochlear nerves are associated with ipsilateral pain I learned things such as the trigeminal nerve supplies the anterior 2/3 of GENERAL sensation Different trigeminal branches exit out of different foramen. V1, superior orbital fissure; V2, foramen rotundum; V3, foramen ovale Learned that jaw claudication is associated with trigeminal nerve lesion and with giant cell arteritis We learned that when a patient has a CN6 lesion, binocular, vertical diplopia worsens when the gaze is directed downward I learned the exits of the CNs. I learned how the left vagus and the right vagus travel across the esophagus I learned that myasthenia gravis can mimic any cranial nerve deficiency The chorda tympani is from the facial nerve and does taste for anterior tongue I remembered CN7 had five branches, but I learned a great mnemonic for remembering the names of the branches: To Zanzibar By Motor Car Facial nerve supplies the anterior 2/3 of TASTE sensation Many of us had forgotten that the nodose ganglion was the home for the visceral afferents of CN10 I learned the exits of the CNs. I learned how the left vagus and the right vagus travel across the esophagus Vagus – symptoms would be when saying “Ahh,” the uvula moves toward the intact side I learned how to perform an appropriate exam relevant to the various cranial nerve disorders CN examination tools: most specific ways to assess CN palsies, correlating PE skills (following finger with H pattern) to specific palsies (CN3, CN4, CN6) I learned how to respond to patient’s questions such as “Will I be okay?”

Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016, Copyright 2016 by A. Hirumi. Reprinted with permission)

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1095

Table 10 Students’ performance on standardized patient checklist Item Asked if the patient has any other questions or concerns prior to leaving room Communicated clearly, avoided medical jargon, or explained terms when used Demonstrated genuineness, care, concern, empathy Encouraged patient to develop full and accurate understanding of key messages Explained purpose of encounter within the first 1–2 min Explored how health issues have affected the patient Explored the patient’s worries/fears about cause(s)/ implications Expressed interest in the patient as a person Greeted patient warmly and verified patient’s identity Introduced him-/herself to the patient (first and last name, full title) Listened attentively Provided information related to the working diagnosis and/or next steps Treated the patient with respect Used open-ended techniques that encouraged the patient to tell his/her story Washed hands before patient contact and maintained clean technique

Completed n (%) 108 (93.1)

Not completed n (%) 8 (6.9)

115 (99.1)

1 (0.9)

112 (96.6) 86 (74.1)

4 (3.4) 30 (25.9)

109 (94.0) 100 (86.2) 100 (86.2)

7 (6.0) 16 (13.8) 16 (13.8)

108 (93.1) 115 (99.1) 115 (99.1)

8 (6.9) 1 (0.9) 1 (0.9)

115 (99.1) 109 (94.0)

1 (0.9) 7 (6.0)

116 (100.0) 116 (100.0)

0 (0.0) 0 (0.0)

111 (95.7)

5 (4.3)

Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016, Copyright 2016 by A. Hirumi. Reprinted with permission)

program of study (i.e., doctoral thesis) and their schedule of deliverables required consideration alongside the program’s own work.

Skills, Knowledge, and Dispositions The ability to act as an intermediary between the curricular needs and the available technology turned out to be of critical importance. As we added an instructional designer to the team, the focus of development was pulled in a new and exciting direction. As one of the PIs for the grant, but more importantly as the faculty member responsible for ensuring that the development did not negatively affect the students, I had to familiarize myself with each team member’s expertise. The ability to consider each member’s motivations required open dialogue and frankness with respect to resource utilization and benefits to the program. As the program grew in complexity, the need to maintain our focus on the needs of the particular student group rested largely with me. I was able to reflect on the components of the platform that were relevant to the learner and ensure that the project continued to have these as central targets. With regard to our target student,

1096

A. Hirumi et al.

Table 11 Frequency and percentage of students providing congruent vs. noncongruent diagnoses for the clinical SP case and VP examination hybrid encounter

Not congruent with clinical case and VP examination

Congruent with clinical case and VP examination

Total

Hemorrhagic stroke Transient ischemic attack Optic neuritis Ruptured aneurysm Others (specify in differential diagnosis) Compressive palsy Raised intracranial pressure Ischemic stroke Cranial neuropathy (unspecified)

Frequency 1 1

Percent 0.9 0.9

Cumulative percent 0.9 1.8

3 3 7

2.6 2.6 6.0

4.4 7.0 13.0

13 19

11.1 16.2

24.1 40.3

28 41

23.8 35.9

64.1 100.0

116

100.0

Note. From “Advancing virtual patient simulations through design research and InterPLAY: part II – integration and field test” (By Hirumi et al., 2016, Copyright 2016 by A. Hirumi. Reprinted with permission)

that includes appropriate collection of the patient history, appropriate response to an occasional comment requiring a response, and the physical examination of the patient with a formulation of a clinical diagnosis and plan. My prior experiences as a clinical skills instructor and medical director of a clinical skills laboratory helped me keep these goals in focus.

Personal Lessons Learned I have been involved in medical education and simulation for nearly 20 years. Clinicians learn in a very traditional apprenticeship model and rarely receive any specific training in pedagogy. Through the project, I have acquired new skills with respect to the value of the context in which the simulation was delivered to the students. The project began as a simulation ex vacuo; that is, the students interacted with the virtual patient as if it was a real patient and went on to the next experience. There was no preparatory material, no assessments, and no repository of knowledge for review or inquiry. As the project progressed, the clinicians and engineers recognized that the experience was limited but focused on the virtual reality of the experience and the dialogue exchange. However, it was the addition of an instructional design expert that focused our attention on the entire learning experience, rather than the technical simulation or animation. The initial development team had begun to focus on details that were likely only of interest to those in the specific field, but were no longer helpful as training or assessment tools for learners. The team lacked a theoretical learning framework for improvement. As an educator, this was an important lesson for me, and I am now sensitive to these items as I develop lessons and tools.

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1097

PI and Software Engineering Expert I am a Professor in the Computer and Information Sciences and Engineering Department at the University of Florida and Cofounder of Shadow Health, Inc., an educational software company. My role in the project was to write proposals to fund the research group, advance computer science and virtual reality research through expanding the applicability of virtual reality and virtual human training, and to lead the computer science researchers on the project. I lead the group at a strategic level with respect to the technology and work closely to align technology advancements with PI Cendan’s focus on medical education. I also mentored and directed the work of computer science postdoctoral researchers.

Skills, Knowledge, and Dispositions In the last year, the main skills applied have been software project management and cloud deployment of software. The software project management skills involve identifying the needs of the medical educators and students through processing interviews with the educators, feedback from users of the system, and reviewing the transcripts and logs of user interactions with the virtual patients. From this information, I led discussions about technical improvements, feature development, and user experience design. My previous experience in deploying virtual patients through using cloud deployment systems (specifically Amazon Web Services) was used in the last year to guide the deployment of the NERVE system onto Amazon Web Services as to support a growing number of users of the system. My previous experience with Amazon Web Services helped identify the right services and the right configuration and guided the development team on the architecture of the system. Personal Lessons Learned The primary lesson learned in the last year was a change in disposition on the necessary components of a simulation that are needed for medical educators and students to see value in the system. Through the design-based research method of interviews, development, and iterative testing, it was clear that providing simply the simulation of a virtual patient would result in many people agreeing that the system was interesting; however, there still existed significant barriers to actual adoption and use within a curriculum of the technology. For educators and students to see the value of the system, the system had to incorporate the right content at the right time in the curriculum. Identifying the content and timing was critical and a significant outcome of the DBR approach. This approach identified how to arrange content, that there is a sequence to learn new material, demonstrate competency with the virtual patient, and be evaluated on the content. This sequence allows educators to now see NERVE and quickly identify where they could use this simulation within their curriculum.

1098

A. Hirumi et al.

Co-I and Software Engineering Expert I am an Associate Professor in the College of Engineering at the University of Georgia. My role in the project was centered on advancing the technical state of the virtual patient. In particular, my research group focused on the physical examination of the virtual patient. We also supported the deployment and testing of systems for user studies and were continuously exploring new technologies that could improve NERVE. Many of these technologies were ultimately not adopted (most notably, many of the “immersive technologies” such as head-mounted displays and motion tracking), but some, such as the introduction of Unity 3D as a development tool that enabled widespread deployment through the web, were pivotal to the success of NERVE.

Skills, Knowledge, and Dispositions The final stage of the NERVE project was focused on the delivery of much of what we had learned and built over the course of the project. It represented a merger of two primary lines of research and development, which advanced the feature set of NERVE and studied how to integrate these features into the medical education curriculum. In many ways, this was a humbling experience. We really had to consider what parts of NERVE contributed to student learning and curricular integration, rather than what was “cool” or “innovative” technologically. Moving to an online, desktop-based experience rather than an immersive one was very difficult, but ultimately necessary. During the last year of R&D, we made a concerted effort to make NERVE a comprehensive, coherent, and easy-to-use online experience. During prior years, we focused only on those nerves that had the most visually abnormal findings that could not easily be simulated by standardized patients. However, in most medical schools, students learn about cranial nerves collectively and in the classroom, to a large extent, memorizing the anatomy, physiology, and (ab)normal findings on their own. This required us to develop interactive tools for less visual examinations (e.g., examining the olfactory and auditory nerves). We also worked to reduce the cognitive load of the 3D user interface, seeking to minimize the controls for each physical examination tool while retaining control that allowed students to explore the nuances of each cranial nerve disorder. Personal Lessons Learned My perspective on interface design changed significantly over the course of the last year of the project. Rather than considering the pinnacle of user interface design to be immersive virtual reality with motion controls and lifelike visuals, I have realized that these systems, though impressive, can be far easier to design and implement than “ordinary” user interfaces. For example, the ophthalmoscope was originally movable in 3D space by dragging it with the mouse and moving it in and out with the mouse scroll wheel. In testing, students were confused at how to do this. To improve usability, we reduced the controls to two degrees of movement freedom, such that it can be pointed at any location on a line between the two eyes and can pivot up and

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1099

down. Other tools that were streamlined in a similar manner by automating their use include items such as the hand (to test eye tracking), the eye chart, tongue depressor, tuning fork, bar of soap, and a ballpoint pen. On-screen instructions were added to further facilitate their unaided use. In addition, I have learned there is immense value in adaptable, reliable, and resilient systems. Designing computer systems that work across platforms, that don’t behave unexpectedly and crash, and that are not severely impacted by changes in context is difficult (and rarely a part of the academic software development process), but necessary for dissemination.

Co-I and Lead Instructional Designer I am a Professor of Instructional Design and Technology at the University of Central Florida. Toward the end of the third year of the project, I was invited to join the R&D team to improve the educational components of NERVE. Initially, I provided input on the instructional design of the system as well as the design of a few controlled studies that were completed to examine different aspects of VP simulation design and use. Then, at the end of Year 4, my status was elevated to Coinvestigator and I was asked to lead the last year of R&D.

Skills, Knowledge, and Dispositions The primary skills I used during my time on the NERVE project centered on instructional design (ID) and design-based research (DBR). In terms of ID, I completed goal and subordinate skills analyses to help define learning goals and objectives, used my knowledge of learner assessment to help create conventional multiple-choice quiz questions that were integrated into NERVE, define performance-based assessment criteria that were used to field test NERVE, and ensure all assessments created by the R&D team were aligned to specified objectives. I also used my disposition toward grounded design and my knowledge of the InterPLAY instructional theory to convince R&D team members to apply experiential learning principles and the InterPLAY instructional strategy to guide the design of NERVE. My knowledge of, and disposition toward, DBR also helped guide the last year of R&D. In short, I convinced R&D team members of the value of DBR as a method for: (a) Formulating both practical solutions and theoretical insights with key stakeholders in real-world (medical school) context (b) Conducting a series of iterative design studies to create and improve alpha and beta versions of NERVE (c) Using Kirkpatrick’s four levels of training evaluation to distinguish students’ use, reactions, learning, and transfer as a framework for field testing and improving the latest version of NERVE released to the general public

1100

A. Hirumi et al.

Personal Lessons Learned When I first joined the team, I read everything I could about VP simulations and learned that unlike many educational technologies, there was consistent and considerable research to support the efficacy of VP simulations and that the majority of studies compared the effectiveness VP simulations versus conventional educational methods. I also learned that like other technologies, the design of instructional events before, during, and after interactions with the VPs may affect learning as much as, or more so, than the simulations, and in addition to design, many curricular factors affect the integration and use VP simulations. During the design and development of NERVE, I learned: (a) It is very difficult to create a system in general and VP simulation in particular that recognize and respond to natural language input in a consistent and pervasive manner. (b) High fidelity is not always desirable when simulating the use of diagnostic tools and techniques. (c) Medical school students like to test their own knowledge and utilize “highyield” learning resources that help to prepare them for annual board exams. Probably the most important lessons learned were the other aspects of instructional design, rather than the systematic design process itself. The key principles and practices of planned change and design-based research were absolutely essential in fulfilling my role on the project. Although I did not formally analyze individual concerns, I believe asking team members what their primary concerns were when thinking about NERVE and using knowledge of Hall and Hord’s (2001) concerns-based analysis model and factors that affect the diffusion of innovations (Rogers & Rogers, 2003) to formulate responses facilitated the adoption and use of DBR and InterPLAY. I also often referred to and sought to implement key features of design experiments (Cobb et al., 2003) and fundamental practices of educational design research (McKenney & Reeves, 2012) to guide the last year of R&D.

Statistical Researcher I am currently an Assistant Professor of Health Sciences Informatics at Johns Hopkins University School of Medicine, as Director of the Office of Assessment and Evaluation. My role in this project was to collaborate with members of the research team in the development of educational research questions and studies, assist in the design of assessment and program evaluation tools, create and distribute tools to students through a web-based system (e.g., SurveyMonkey or Qualtrics), assist with the conduct of research studies (e.g., write IRB protocols, review consent documents with students), compile and analyze data, produce data reports for research team review, and assist in the dissemination of research study findings. Additionally, like all team members, I participated in pilot testing components of the NERVE system with each developmental change, as needed.

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1101

Skills, Knowledge, and Dispositions Requisite technical skills and knowledge that I applied across this project related to assessment and program evaluation design, use of web-based systems to create and distribute assessment and program evaluation tools, data collection and management, quantitative and qualitative data analysis, data reporting, and scientific writing. For example, it was helpful to subject matter experts that I was able to vet assessment items to ensure alignment with best practices (e.g., for constructing multiple-choice questions), and in some circumstances, I was provided detailed content and able to construct multiple-choice questions for subject matter experts to vet. Following delivery of multiple-choice questions, I conducted test and item analyses to examine item performance and identify items in need of revision. Additional skills and knowledge I applied that were helpful to the research team related to research ethics and compliance, IRB protocols and processes, and use of virtual patients in medical education. While multiple members of the research team would be effective in writing IRB protocols and managing the submission, approval, and renewal processes, I addressed many of these activities to maintain consistency across protocols and to allow other research team members to devote time to NERVE tasks that required their specific expertise. Optimizing performance on an interdisciplinary research team also required strong verbal and written communication skills (e.g., explaining complex statistical analyses in layman’s terms), an affinity for working as a member of a diverse team, and a spirit of inquiry to formulate important research questions. Personal Lessons Learned I have been involved in the NERVE project for almost 6 years, and, as a result, I believe I am a better collaborator and communicator. For example, I learned to refrain from assembling lengthy and complex statistical reports that tended to result in team members feeling confused and overwhelmed. I also refined my skills in communicating effectively during telephone conferences, as our research team was spread out across several institutions. The need to persist as a lifelong learner was evident along the way. Over the years, I expanded my prior knowledge of the use of SurveyMonkey, and built my knowledge of the use of Qualtrics from the ground up. I repeatedly observed that real data sets don’t occur “perfectly” as they did when presented in textbooks or in graduate courses, such that data often violate assumptions required of fairly common statistical tests. For example, when data sets from two different projects violated the homogeneity of regression slopes assumption of analysis of covariance, I discovered the field of aptitude-treatment interaction effects and learned a less common statistical analytic method known as the Johnson-Neyman procedure for handling these types of data sets. In the last couple of years of this project, our team has realized the distinct advantages of having a medical student as a member of the research and development team. This team member brought an invaluable perspective to the team on the use of virtual patients in a medical education curriculum, perceptions of peers related

1102

A. Hirumi et al.

to use of NERVE, relevant assessment content, and development of appropriate learning materials or other guides.

Graduate Research Assistant I am now a software development engineer working for Microsoft on the Analog team. My role in the project as a graduate student at the University of Florida centered on the development and implementation of the NERVE Learning Center and the development of the conversational models that power the virtual humans that students interact with in the Exam Room. The project also provided funding for my doctoral research, and the students who used to evaluate the Learning Center were also participants in two of the user studies I conducted as a graduate student. As part of the NERVE project, I balanced my time and effort between (a) developing the features that were deemed necessary for the project and (b) pursuing the goals of my doctoral research. In order to find this balance, I participated in the design of the Learning Center to include the features required for my research. I also collaborated with the rest of the team in the design of the evaluation methods for the NERVE Learning Center to support the needs for my research experiments.

Skills, Knowledge, and Dispositions The final stage of the project required a significant portion of my time and dedication at one of the critical moments of my graduate studies. A significant portion of this stage was improving the NERVE platform to be a stable product that could be used by instructors and students to teach and learn about cranial nerve palsies. This process required a lot of polishing of the product and time spent toward fixing bugs in the product which is time consuming. The key skills required for this stage were time management and openness to criticism. Good time management was critical to the success of the final stage of the project and the achievement of my own academic goals. I had to find time to constantly test the system and keep improving the system while also finding time to finalize my doctoral dissertation. At the same time, I needed to be open to criticism of the platform, as multiple team members were constantly performing tests and reporting new problems. It is common for students to see criticism of a project as a commentary on their own effort, especially when a significant effort has been dedicated to bringing the product to its current state. The last stages of fixing problems with the system can be very deflating as other members point out problems that looked small to them but that felt major to me. Personal Lessons Learned The primary lesson learned for me as a graduate assistant was how to adapt my research to the needs of the project and how to incorporate my research into a project that did not prioritize my research agenda. In a project with a clear goal, as was the case with NERVE, the research needs of a graduate student can often be overlooked.

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1103

It was important for me to find ways to collaborate with other team members and find ways in which my research could enrich the project and benefit from the populations that were testing the system. As part of this team, I learned to listen to the multiple perspectives in the team and make research decisions that addressed the concerns coming from the different team members. While addressing those concerns was not always easy and sometimes was not possible, listening to the perspectives of researchers from other different background strengthened my own research and significantly improved the user experience students had while interacting with the system and the features introduced by my research.

Concluding Remarks The last year of R&D demonstrates the value of DBR and the potential of InterPLAY for designing, developing, and testing a virtual patient simulation. Prior to adopting DBR, the team focused on the second of two major project goals: to create a tool that would enable researchers to study different aspects of VPs design and use based on the research interests and expertise of specific team members. Design-based research gave R&D team members a framework for planning and implementing a series of iterative studies for gathering data from key stakeholders in a systematic manner to improve NERVE in a real-world context based on targeted learner outcomes and the primary project goal: to develop a virtual environment that enables medical students to rehearse and receive feedback on their patient interviewing, examination, and diagnostic skills. Design-based research also gave team members a method for advancing both experiential learning theory and the InterPLAY instructional theory – the means used to facilitate experiential learning. InterPLAY provided a foundation for aligning research, theory, and practice, explaining and predicting results, and for making key design decisions based on research and theory. In turn, out of the design and testing of NERVE emerged a new learning pathway through the elements of story, game, and play that led to significant advancements in the InterPLAY strategy and theory. The reflections by R&D team members were examined to affirm the concluding remarks and lend additional insights into the team’s experience. Reflections on crucial skills, knowledge, and dispositions, along with key lessons learned from the project reveal three common findings, including a change in disposition, the value of DBR in facilitating the change, and the significance of adaptability and interpersonal communication skills across team members. Turning attention toward achieving the fundamental project goal of producing a standalone platform that could be deployed in medical schools across the United States, the leadership team noted a significant change in dispositions toward the design of the simulation. Rather than focusing on the technical design of the user interface based on individual research interests, team members gained an appreciation for key contextual factors to promote the adoption of the system and to facilitate curriculum integration and student learning.

1104

A. Hirumi et al.

During the first 4 years of the initiative, the leadership team asked students to interact with the simulation only to optimize user interface design and stimulate a sense of immersion in the virtual environment with motion controls, lifelike visuals, and natural language recognition. The simulation was used primarily as a tool to advance research on specific technical aspects of VPs design. During the last year of R&D, the leadership team found that it had to pay attention to the needs and requirements of both medical educators and students within the constraints of the broader medical school curriculum. The design and development of the VPs to facilitate adoption and wide-scale deployment required a significant change in disposition and a method to facilitate such change. Team members also reflected on how DBR provided a much needed framework for addressing key contextual factors. In particular, three features of DBR (i.e., the development of a class of theories about the learning process and system used to support learning, addressing practical problems faced by practitioners, and iterative cycles of invention and revisions) helped identify key instructional features that had to be incorporated before, during, and after interactions with NERVE to overcome key contextual factors and to facilitate adoption and integration of the system. Specifically, the application of experiential learning theory directed our attention to the importance of both framing and reflecting on the experience. In addition, the application and refinement of the InterPLAY instructional theory provided a foundation for designing the NERVE Learning Center and integrating information on CN anatomy, physiology, symptoms, and pathology (key components necessary to transform the system into a standalone platform). In turn, the iterative design studies involving medical educators and students revealed practical problems that had to be addressed to facilitate adoption and integration. In particular, the positioning of the system in relation to the overall medical school curriculum and in light of other educational resources used by faculty and students to facilitate learning were vital in determining how and when NERVE was to be presented to students. The change in disposition toward design and the subsequent adoption of DBR as a framework spotlight the significance of adaptability and communication emphasized by R&D team members in their reflections. As noted by one of the PIs, the interpersonal skills we often seek to instill in our students became critical in our own efforts. To a large extent, the overall success of the project was determined by each member’s willingness to listen to multiple perspectives and to gain familiarity with each other’s expertise and motivations. Such understanding required an open dialogue and frankness with respect, along with a readiness to adapt individual research needs and interests to those of the project, particularly in large-scale initiatives that required collaboration across multiple sites and disciplines. Team members’ reflections on background, experience, and interests depict the variety of skills and knowledge that were necessary to design, develop, and implement the virtual patient simulation. The team included experienced professionals and graduate students in neurology, medical education, software engineering, modeling and simulation, research and statistics, and instructional design. At work, we often presume and hope that team members have constructive dispositions and effective

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1105

interpersonal communication skills. But as experienced educators and researchers, we are also aware that such skills and dispositions are not necessarily inherent to all professionals. Reflections by R&D team members remind us of the importance of instilling, demonstrating, and refining interpersonal communications skills, and fostering open and adaptive dispositions, particularly for large-scale design projects that involve members with distinct as well as common goals and interests. The development and production process was representative of a system designed and built to evaluate concepts, to be used by a medical school-sized cohort, and to analyze student performance and data. As is, the development has resulted in a system that is capable of providing an important learning environment in an effective manner. However, there are significant gaps between the resulting system which can be used by educators worldwide to teach cranial nerves and a fully commercial system that would have a strong user experience component (e.g., interface design, visual quality, and flow). In hindsight, the design and development decisions appeared to be appropriate for researchers exploring a new simulation approach as to explore the importance of various factors through user studies. An approach was needed that was adaptable to changing requirements with an emphasis of quick deployments at the expense of design and robustness. Moving forward, any efforts to advance the current system would likely be generated in a manner that would provide a polished, but more fully defined, experience. With respect to improving the development process, it appeared that while the overall strategy toward development was sound, the tactical approach to development (the day to day software development) would have benefitted from leveraging better software design paradigms, including the Agile methodology, a project management tracking system such as Trello, and frequent releases to the entire team to both demonstrate continuous improvement while providing an opportunity for rapid feedback from the rest of the team. Acknowledgments Research reported in chapter paper was supported by the National Institutes of Health (NIH) under award number 1R01LM010813–01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

References Adams, E., Rodgers, C., Harrington, R., Young, M., & Sieber, V. (2011). How we created virtual patient cases for primary care-based learning. Medical Teacher, 33(4), 273–278. Agency for Healthcare Research and Quality (AHRQ). (2013). PAR-11-024 advances in patient safety through simulation research. http://grants.nih.gov/grants/guide/pa-files/PAR-11-024. html. Accessed January 20. Atkinson, T., & Hirumi, A. (2010). The game brain. In A. Hirumi (Ed.), Playing games in school: Using simulations and videogames for primary and secondary education (p. 63). Eugene, WA: International Society for Technology in Education. Bateman, J., Allen, M. E., Kidd, J., Parsons, N., & Davies, D. (2012). Virtual patients design and its effect on clinical reasoning and student experience: A protocol for a randomised factorial multicentre study. Medical Education, 12(1), 62.

1106

A. Hirumi et al.

Bateman, J., Allen, M. E., Samani, D., Kidd, J., & Davies, D. (2013). Virtual patient design: Exploring what works and why. A grounded theory study. Medical Education, 47(6), 595–606. Berman, N., Fall, L., Smith, S., Levine, D., Maloney, C., Potts, M., . . . Foster-Johnson, L. (2009). Integration strategies for using virtual patients in clinical clerkships. Academic Medicine, 84(7), 943–949. Botezatu, M., Hult, H., Tessma, M. K., & Fors, U. G. (2010). As time goes by: Stakeholder opinions on the implementation and use of a virtual patient simulation system. Medical Teacher, 32(11), e509–e516. Cendan, J., & Lok, B. (2012). The use of virtual patients in medical school curricula. Advances in Physiology Education, 36(1), 48–53. Churchill, D. (2007). Towards a useful classification of learning objects. Educational Technology Research and Development, 55(5), 479–497. Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. Colder, B. (2011). Emulation as an integrating principle for cognition. Frontiers in Computational Neuroscience, 5(54), 1–12. Consorti, F., Mancuso, R., Nocioni, M., & Piccolo, A. (2012). Efficacy of virtual patients in medical education: A meta-analysis of randomized studies. Computers & Education, 59(3), 1001–1008. Cook, D., & Triola, M. (2009). Virtual patients: A critical literature review and proposed next steps. Medical Education, 43(4), 303–311. Cook, D. A. (2014). How much evidence does it take? A cumulative meta-analysis of outcomes of simulation-based education. Medical Education, 48(8), 750–760. Cook, D. A., Erwin, P. J., & Triola, M. M. (2010). Computerized virtual patients in health professions education: A systematic review and meta-analysis. Academic Medicine, 85, 15890–11602. Cooke, S. F. (2006). Plasticity in the human central nervous system. Brain, 129(Pt 7), 1659–1673. Dewey, J. (1938). Logic: The theory of inquiry. New York: Holt, Rinehart and Winston. de Jong, T., & von Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68(3), 179–201. Edelbring, S. (2010). A three-fold framework for relating to innovations and technology in education: Learning from, with and about technology. In A. Bromage, L. Clouder, J. Thistlethwaite, & F. Gordon (Eds.), Interprofessional E-learning and collaborative work: Practices and technologies (pp. 23–33). Hershey, PA: IGI Global. Edelbring, S., Broström, O., Henriksson, P., Vassiliou, D., Spaak, J., Dahlgren, L. O., & Zary, N. (2012). Integrating virtual patients into courses: Follow-up seminars and perceived benefit. Medical Education, 46(4), 417–425. Edelbring, S., Dastmalchi, M., Hult, H., Lundberg, I. E., & Dahlgren, L. O. (2011). Experiencing virtual patients in clinical learning: A phenomenological study. Advances in Health Sciences Education, 16(3), 331–345. Fischer, M., Hege, I., Hörnlein, A., Puppe, F., Tönshoff, B., & Huwendiek, S. (2007). Virtual patients in medical education: A comparison of various strategies for curricular integration. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen, 102(10), 648–653. Fullan, M. (1993). Change forces: Probing the depth of educational reform. London, UK: Falmer. Georg, C., & Zary, N. (2014). Web-based virtual patients in nursing education: Development and validation of theory-anchored design and activity models. Journal of Medical Internet Research, 16(4), e105. https://doi.org/10.2196/jmir.2556. Gibbons, A. S., McConkie, M., Seo, K. K., & Wiley, D. A. (2009). Simulation approach to instruction. In C. M. Reigeluth & A. A. Carr-Chellman (Eds.), Instructional-design theories and models: building a common knowledge base (Vol. III, pp. 167–193). New York, NY: Routledge. Gormley, G. J., Mcglade, K., Thomson, C., Mcgill, M., & Sun, J. (2011). A virtual surgery in general practice: Evaluation of a novel undergraduate virtual patient learning package. Medical Teacher, 33(10), 522–527.

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1107

Haag, M., Singer, R., Bauch, M., Heid, J., Hess, F., & Leven, F. (2007). Challenges and perspectives of computer-assisted instruction in medical education. Lessons learned from seven years of experience with the CAMPUS system. Methods of Information in Medicine, 46(1), 67. Hall, G. E., & Hord, S. M. (2001). Implementing change: Patterns, principles and potholes. Boston, MA: Allyn and Bacon. Harden, R. M., Grant, J., Buckley, E. G., & Hart, I. R. (1999). BEME guide no. 1: Best evidence medical education. Medical Teacher, 21(6), 553–562. Harrison, M., Short, C., & Roberts, C. (2003). Reflecting on reflective learning: The case of geography, earth and environmental sciences. Journal of Geography in Higher Education, 27 (2), 133–152. Hirumi, A., Johnson, K., Kleinsmith, A., Reyes, R., Rivera-Gutierrez, D., Kubovec, S., . . . Cendan, J. (in press). Advancing virtual patient simulations and experiential learning with InterPLAY: Examining how theory informs design and design informs theory. Journal of Applied Instructional Design. Hirumi, A., Kleinsmith, A., Johnsen, K., Kubovec, S., Eakins, M., Bogert, K., et al. (2016a). Advancing virtual patient simulations through design research and InterPLAY: Part I – Design and development. Educational Technology, Research & Development, 64(4), 763–785. Hirumi, A., Johnson, T., Reyes, R. J., Lok, B., Johnsen, K., Rivera-Gutierrez, D. J., et al. (2016b). Advancing virtual patient simulations through design research and InterPLAY: Part II – testing and integration. Educational Technology, Research & Development. https://doi.org/10.1007/ s11423-016-9461-6 Huwendiek, S., & De Leng, B. A. (2010). Virtual patient design and curricular integration evaluation toolkit. Medical Education, 44(5), 519. Huwendiek, S., De Leng, B. A., Kononowicz, A. A., Kunzmann, R., Muijtjens, A. M., Van Der Vleuten, C. P., . . . Dolmans, D. H. (2015). Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning. Medical Teacher, 37(8), 775–782. Huwendiek, S., Duncker, C., Reichert, F., De Leng, B. A., Dolmans, D., van der Vleuten, C. P., & Tönshoff, B. (2013). Learner preferences regarding integrating, sequencing and aligning virtual patients with advancing virtual patient simulations through design other activities in the undergraduate medical curriculum: A focus group study. Medical Teacher, 35(11), 920–929. Huwendiek, S., Reichert, F., Bosse, H. M., De Leng, B. A., Van Der Vleuten, C. P., Haag, M., & Tönshoff, B. (2009). Design principles for virtual patients: A focus group study among students. Medical Education, 43(6), 580–588. Illeris, K. (2014). Transformative learning and identity. [electronic resource]. Abingdon, Oxon: Routledge. Issenberg, S. B., McGaghie, W. C., Petrusa, E., Gordon, D. L., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27(1), 10–28. Johnson, T. R., Lyons, R., Chuah, J., Kooper, R., Lok, B., & Cendan, J. C. (2013). Optimal learning in a virtual patient simulation of cranial nerve palsies: The interaction between social learning context and student aptitude. Medical Teacher, 35(1), 899–907. Johnson, T. R., Lyons, R., Kooper, R., Johnsen, K. J., Lok, B. C., & Cendan, J. C. (2014). Virtual patient simulations and optimal social learning context: A replication of an aptitude-treatment interaction effect. Medical Teacher, 36(6), 486–494. Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Journal of instructional development, 10(3), 2–10. Keller, J. M. (2010). Motivation design for learning and performance: The ARCS model approach. New York: Springer. Kleinsmith, A., Rivera-Gutierrez, D., Finney, G., Cendan, J. C., & Lok, B. (2015). Understanding empathy training with virtual patients. Computers in Human Behavior, 52, 151–158. https://doi. org/10.1016/j.chb.2015.05.033.

1108

A. Hirumi et al.

Kononowicz, A. A., Narracott, A. J., Manini, S., Bayley, M. J., Lawford, P. V., McCormack, K., & Zary, N. (2014). A framework for different levels of integration of computational models into web-based virtual patients. Journal of Medical Internet Research, 16(1), e23. Lindsey, L., & Berger, N. (2009). Experiential approach to instruction. In C. Reigeluth & A. CarrChellman (Eds.), Instructional-design theories and models: Volume 3. Building a common knowledge based (pp. 117–142). New York, NY: Routledge. Maran, N. J., & Glavin, R. J. (2003). Low-to high-fidelity simulation – A continuum of medical education? Medical Education, 37(1), 22–28. McGaghie, W. C., Issenberg, S. B., Cohen, M. E. R., Barsuk, J. H., & Wayne, D. B. (2011). Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Academic Medicine: Journal of the Association of American Medical Colleges, 86(6), 706. McKenney, S. E., & Reeves, T. (2012). Conducting educational design research: What, why and how. London, UK: Taylor & Francis Ltd.. Mezirow, J. (1990). Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning. San Francisco. CA: Jossey-Bass. Moore, F., & Chalk, C. (2009). The essential neurologic examination: What should medical students be taught? Neurology, 72(23), 2020–2023. Posel, N., Mcgee, J. B., & Fleiszer, D. M. (2014). Twelve tips to support the development of clinical reasoning skills using virtual patient cases. Medical Teacher, 37(9):1–6. Quinn, C. (2009). Computer-based simulations: Principles of engagement. In M. Silberman (Ed.), The handbook of experiential learning (pp. 138–171). San Francisco, CA: Pfeiffer. Reigeluth, C. M., & Schwartz, E. (1989). An instructional theory for the design of computer-based simulations. Journal of Computer-Based Instruction, 16(1), 1–10. Reyes, R. J., & Hirumi, A. (2016). Analyzing the pedagogical foundations of virtual patient simulations: A review of literature. Poster presented at the annual graduate research forum at the University of Central Florida, Orlando, FL, April 5. Rimmele, U. (n.d.). A primer on emotions and learning. Retrieved May 16, 2017, 2017 OECD. from http://www.oecd.org/edu/ceri/aprimeronemotionsandlearning.htm Rivera-Gutierrez, D., Kleinsmith, A., Johnson, T., Lyons, R., Cendan, J., & Lok, B. (2014). Towards a reflective practicum of embodied conversational agent experiences, IEEE international conference on advanced learning technologies (ICALT). Athens, Greece: IEEE. Rogers, E. M., & Rogers, E. M. (2003). Diffusion of innovations. New York, NY: Free Press. Salas, E., & Gregory, M. E. (2011). Simulation-based training: Beyond the bells and whistles! CyberTherapy Magazine, 4, 18–19. Salem-Schatz, S., Ordin, D., & Mittman, B. (2010). Guide to the after action review (version 1.1). Using evaluation to improve our work: A resource guide. Retrieved March 23, 2015. http:// www.queri.research.va.gov/ciprs/after_action_review.pdf. Salminen, H., Zary, N., Björklund, K., Toth-Pal, E., & Leanderson, C. (2014). Virtual patients in primary care: Developing a reusable model that fosters reflective practice and clinical reasoning. Journal of Medical Internet Research, 16(1), e3. https://doi.org/10.2196/jmir.2616. Schank, R. C., Berman, T. R., & Macpherson, K. A. (1999). Learning by doing. In C. M. Reigeluth (Ed.), Instructional-design theories and models: Vol. 2, a new paradigm of instructional theory (pp. 161–181). Mahwah, NJ: Lawrence Erlbaum Associates. Shank, R. C., Berman, T. R., & Macpherson, K. A. (1992). Learning by doing. In C. M. Reigeluth (Ed.), Instructional design theories and models: A new paradigm of instructional theory (pp. 161–179). Hillsdale, NJ: Lawrence Erlbaum Associates. Stapleton, C., & Hirumi, A. (2011). InterPLAY instructional strategy: Learning by engaging interactive entertainment conventions. In M. Shaughnessy & S. Fulgham (Eds.), Pedagogical models: The discipline of online teaching (pp. 183–211). Hauppauge, NY: Nova Science Publishers.

42

NERVE, InterPLAY, and Design-Based Research: Advancing Experiential. . .

1109

Stapleton, C., & Hirumi, A. (2014). Designing InterPLAY learning landscapes to evoke emotions, spark the imagination, and promote creative problem solving. In A. Hirumi (Ed.), Grounded designs for online and hybrid learning (pp. 159–190). Eugene, WA: International Society for Technology in Education.

Dr. Atsusi Hirumi “2c” is a Professor of Instructional Design and Technology in the College of Education and Human Performance at the University of Central Florida. For the past 20 years, Atsusi centered his research, teaching, and service on the design of online and hybrid learning environments and the design and sequencing of e-learning interactions. His research now focuses on advancing experiential learning and the design of simulation-based training with InterPLAY, an instructional theory that integrate elements of story, play, and game with principles of experiential learning to create engaging and memorable learning experiences. Benjamin Chak Lum Lok is a Professor in the Computer and Information Sciences and Engineering Department at the University of Florida and Cofounder of Shadow Health, Inc., an educational software company. He is also an Adjunct Associate Professor in the Department of Psychiatry and Health Behavior at Georgia Health Sciences University. His research focuses on virtual humans and mixed reality in the areas of computer graphics, virtual environments, and human-computer interaction. Professor Lok received a Ph.D. in 2002 and M.S. from the University of North Carolina at Chapel Hill and a B.S. in Computer Science (1997) from the University of Tulsa. He did a postdoc fellowship in 2003 under Dr. Larry F. Hodges at the University of North Carolina at Charlotte. Dr. Teresa R. Johnson is the Director of the Office of Assessment and Evaluation at the Johns Hopkins University School of Medicine (JHUSOM). In this role, she is responsible for designing and launching initiatives related to the assessment of students and the evaluation of programs in undergraduate, graduate, and continuing medical education, graduate biomedical education, and postdoctoral training. She establishes strong partnerships with faculty members and program administrators to ensure that assessment and program evaluation activities align with learner needs, program goals, accreditation standards, and evidence-based best practices. Dr. Johnson is also an Assistant Professor of Health Sciences Informatics at JHUSOM and serves as a Capstone Advisor for the Master of Education in Health Professions program. In both the Director and Assistant Professor roles, she collaborates with other faculty members on educational research projects and provides consultation to faculty members as a Managing Board Member of the Institute for Excellence in Education. Dr. Johnson serves as the JHUSOM representative in the Society of Directors of Research in Medical Education. Dr. Kyle Johnsen received his Ph.D. (2008), M.S. (2007), and B.S. (2003) in Computer Engineering from the Department of Computer and Information Science and Engineering at the University of Florida. He joined the University of Georgia in 2008 and currently holds a primary appointment as an Associate Professor in the College of Engineering. He studies human-computer interfaces, specializing in the design of virtual environments and information systems, and the application of these technologies to education and sustainability. These efforts have been funded by the NSF, NIH, NOAA, and several industrial partners. In 2016, he became the Inaugural Director of the Georgia Informatics Institutes, a major university effort to expand informatics research and educational opportunities in Georgia. Diego de Jesus Rivera-Gutierrez is a software development engineer working for Microsoft on the Analog team. His research interests include the use of in-action reflective learning interventions

1110

A. Hirumi et al.

during virtual human experiences to develop interpersonal skills. His work also includes topics in human-computer interaction and virtual reality. He obtained his Ph.D. in Computer Science in 2016 and his M.S. in Computer Engineering in 2015 from the University of Florida. He obtained his B.S. in Computer Engineering in 2008 from the Costa Rican Institute of Technology. Prior to his graduate studies, Diego was a software engineer working on game development for Fair Play Labs. Ramsamooj Javier Reyes is a Department Chair and Professor of the College of Technology, Indiana State University. As Graduate Research Assistant while pursuing his doctorate at the University of Central Florida, Javier assisted with expert reviews, one-to-one evaluations, smallgroup evaluations, and field testing. Javier is an Active Duty Air Force Officer with interest in neuroscience research, virtual patient design and integration, and foreign language education. Dr. Tom Atkinson has taught instructional systems design at several universities and specializes in interactive environments for experiential learning based on neurological processes. He chairs the Virtual Worlds committee with the International Association for Communications and Technology (AECT) and presented as keynote and invited speaker at national and international events. As a consultant, he managed workforce and economic development projects with the Louisiana Governor’s Office of Nutrition, Community Services, Department of Transportation, Department of Labor, Northrop Grumman, Novartis Pharmaceuticals, and Cisco Systems. Christopher Stapleton is a Creative Venture Catalyst for Simiosys Real World Laboratories, and he is the Cofounder of the Virtual World Society. He serves as a thought leader in the practice of designing, building, and evaluating the next generation of human experience that melt the boundaries between reality, virtuality, and the imagination. He has received critical acclaim for his pioneering work through his creative direction for global entertainment, education, and technology institutions such as Universal Studios, Disney, Nickelodeon, and NASA. His design research work has launched product innovations across industries and has been sponsored by the US Department of Education, National Science Foundation, Canon USA, and Space Florida. As Affiliate Research Faculty with University of Central Florida’s (UCF) Institute for Simulation and Training, he has served as Principal Investigator for research funded by the National Science Foundation, Department of Defense, and Canon Inc. exploring the frontiers of mixed reality. He was the Founding Faculty Member of the UCF Digital Media Program and launched the creation of transdisciplinary research laboratories. Christopher’s professional training for experience design started on Broadway and independent filmmaking in New York City where he received his Master of Fine Arts in Design for Theater and Cinema from New York University’s Tisch School of the Arts. Dr. Juan C. Cendán joined the University of Central Florida, College of Medicine, in July 2010 as a Founding Faculty Member. His initial efforts were aimed at developing the Clinical Skills and Simulation Center, which he led as the Medical Director until January 2015. He remains the Assistant Dean for Simulation, although his leadership role as Chairman of the Department of Medical Education has now become the focal part of his work. He leads a department of 20 faculty members whose primary mission is to support the undergraduate medical education program at UCF-COM. Dr. Cendán is a board-certified surgeon with a clinical focus on minimally invasive surgery and surgical disorders of the breast. His special interests include medical education and simulation.

Instructional Design as a Moral Ecology of Practice: Implications for Competency Standards and Professional Identity

43

Stephen C. Yanchar

Contents Identity as Self-Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Moral Ecology of Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identity as Moral Self-Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications for Instructional Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Recognizing the Moral Ecology and Moral Realism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Professional Identity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Expanding and Refining Competency Standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1113 1116 1119 1120 1120 1122 1124 1127 1127

Abstract

This chapter offers an analysis of the value of competency standards and other positions on the nature of instructional design practice. Drawing on practice theory as a philosophical perspective – including arguments pertaining to human agency, identity, moral realism, and practical involvement – this chapter suggests how competency standards contribute to the discipline’s broader moral ecology, understood as a practical context for work in the field that includes real moral goods and reference points as guiding values. This chapter suggests that these moral goods and reference points grant the profession direction and purpose, and provide a meaningful context for the development of instructional designers’ professional identities. Given the role that competency standards and related proposals can play in the moral ecology of the field, they warrant serious consideration and discussion. This chapter also discusses the limitations of current competency standards and suggests how, from the perspective of practice theory, they could be usefully expanded.

S. C. Yanchar (*) Instructional Psychology and Technology, Brigham Young University, Provo, UT, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_77

1111

1112

S. C. Yanchar

Keywords

Competence · Identity · Instructional design · Moral · Practice theory · Standards

An important conversation in recent years concerns the nature of instructional design and the designer’s role in this process (Clark & Estes, 1998; Gibbons, 2014; Kerr, 1983; Reiser, 2001; Smith & Boling, 2009; Snelbecker, 1974; Wilson, 2013). In perhaps the simplest terms, as a pragmatic definition would suggest, instructional design is what instructional designers do; they engage in activities that lead to finished learning environments. Simplicity turns to complexity, however, as the details of “what designers do” are scrutinized from diverse perspectives and instructional design becomes a contested enterprise (Campbell, Schwier, & Kenny, 2009; Gibbons, 2014; Smith & Boling, 2009). As a growing literature has suggested, there are multiple ways of conceptualizing instructional design as an educational practice, and there are multiple ways of actually engaging in those practices, which has led to concern about standards for practitioners such as the AECT, CIPD, and ASTD competency models (see also Guerra, 2006; Gulbahar & Kalelioglu, 2015; MacLean & Scott, 2011; Richey, Fields, & Foxon, 2001), change agency (Campbell et al., 2009), and civic-minded instructional design (Yusop & Correia, 2012). Although there is no monolithic approach to instructional design work, such standards and models describe what might be considered the major capabilities required of competent instructional designers and offer frames of reference for understanding the scope and function of the field. Standards typically perform these functions by delineating technical skills, design processes, and competent uses of conceptual tools (Gulbahar & Kalelioglu, 2015; MacLean & Scott, 2011; Richey et al., 2001). Among other things, such standards suggest that practitioners in the field should be in possession of special knowledge and skills that allow for capable professional work. In a sense, these are very practical concerns regarding what designers know, or need to know, as they go about their duties. Indeed, educational technology, as any profession, must have standards regarding what its practitioners can be expected to do and, by extension, the kinds of contributions the field can be expected to make. However, the ways in which the field understands and uses these standards can benefit from careful analysis. Such disciplinary standards and competency models, I will suggest, might be enhanced and expanded in ways that include a greater range of professional concerns, especially those pertaining to the fundamentally moral nature of design work in education. Thus, this chapter offers another perspective on the issue of instructional designer competency, but not by presenting an alternative set of standards or guidelines. It is more philosophical than what has already been proposed in the literature, yet is nonetheless practical in nature. In short, this chapter seeks to explore the issue of designer competence from the standpoint of practice theory – or more particularly, one version of practice theory rooted in ontological hermeneutics (Dreyfus, 1991; Heidegger, 1962; Taylor, 1985, 1989; for general discussions of practice theory, see Schatzki, Knorr Cetina, & von Savigny, 2000) –

43

Instructional Design as a Moral Ecology of Practice: Implications for. . .

1113

a philosophical standpoint that emphasizes human agency, identity, moral action, and practical involvement in everyday life. This chapter, then, will suggest that competency standards are important, if for no other reason than they reveal something significant about instructional design as an aspect of educational technology. What they reveal has to do with the very nature of practice, and, as I will suggest, instructional design as a fundamentally moral undertaking. On the other hand, such standards are not sufficient to specify the details of what instructional designers actually do in particular situations, and thus the conversation regarding competent design practice must extend beyond such general standards and guidelines. In articulating this perspective, this chapter includes two major concepts: first, the notion of identity as self-interpretation, and second, the claim that self-interpretation exists amidst a moral ecology of practice. Finally, this chapter will suggest how these concepts pertain to instructional design and what they imply for work in the field.

Identity as Self-Interpretation Identity has become a significant issue in educational scholarship in recent years (Gee, 2001; Sfard & Prusak, 2005). Though the concept has been defined in various ways and associated with a number of topics such as multicultural understanding (Cifuentes & Murphy, 2000), educational reform (Luehmann, 2007), and economic aspects of public schooling (Akerlof & Kranton, 2002), it is commonly associated with situated, participation-oriented theories of learning (Sfard, 1998). These are theories that emphasize the cultural construction of identities as equated with, or a principal mechanism of, the development of knowledge and skill (Hung & Chen, 2007; Lave & Wenger, 1991; Sfard & Prusak, 2005; Wenger, 1998; Wortham, 2006). Learning, in this sense, is about something different than acquiring information; it is about becoming a skilled cultural participant – that is, becoming one who sees the world, uses equipment, and generally participates in particular cultural ways. To develop capability, from this perspective, is to develop an identity of fit and belonging. Much could be said about the connection between identity and learning as conceptualized in situated approaches; for my purposes, it is enough to suggest that they offer an important contrast to acquisition-oriented views which hypothesize a private realm of mind inhabited by memory mechanisms, mental representations, models of reality, and so on – a general epistemological viewpoint that has been the subject of effective critical scrutiny (Dreyfus, 1991; Guignon, 1983; Ingold, 2000; Packer, 2011; Sfard, 1998). As suggested above, identity-oriented views focus on capability and becoming in cultural contexts rather than mentalism and representationalism; practice theory, however, considers identity and participation at a deeper level and offers several significant insights regarding the nature of cultural involvement. Advocates of practice theory would concur with situated approaches that identity formation is best viewed as a phenomenon of becoming, but clarify this concept by

1114

S. C. Yanchar

considering it a kind of continuous position taking. Put concisely, from this perspective, identity is the stand one takes on matters of cultural significance as he or she engages in the world in particular ways, though often not explicitly or as a result of focused deliberation (for more on this point, see Brinkmann, 2008; Sugarman, 2005; Taylor, 1989). According to this version of practice theory, humans as cultural participants experience a kind of “existential concern” about the nature of their lives that shows up in their activities, use of resources, ways of relating to the world, and so on. Understood this way, humans are beings who take a position on their own existence by engaging in some form of participation that matters to them from available cultural possibilities, and in this sense, their concernful involvement is based on the difference that given forms of cultural participation make to themselves and others. This concernful, yet often tacit and inarticulate position taking, is the primary characteristic by which personhood is distinguished from other forms of existence (objects, animals, etc.; for more on this view of cultural participation and agency, see Guignon, 2002; Heidegger, 1962; Sugarman, 2005; Taylor, 1985, 1989; Yanchar, 2011). Notwithstanding the abstract, philosophical nature of this description of personhood, examples of what it means are not difficult to identify. One might consider a college student, for instance, who selects a particular major because of the difference she sees it making in her life personally (perhaps even selfishly) or in the lives of others. She may see the work of that field as interesting and easily relates to it; she may also see it as a way of making a living while contributing something of value to society, and will thus pursue academic success in ways that work best for her. Her motives may be varied, but in any event, she is acting in a way that makes a difference in at least her own life and places her on a trajectory regarding what matters amidst the available cultural options. It might be said, in this regard, that how one views possibilities and engages in certain cultural activities can be metaphorically thought of as a response to, or commentary on, those cultural activities; and such a response or commentary is one’s identity (Guignon, 1983; Heidegger, 1962). Identity, then, primarily has to do with the manner in which one fits into a particular cultural context and is committed to the pursuit of certain ends; it is – through one’s practical involvement – a tacit affirmation of certain ways of engaging in cultural practices and, thus, a stand on how cultural forms of life might be managed. In this sense, it might be said that one is what one does; but it must also be understood that what “one does” is a particular way of expressing how to be a person under certain circumstances. Thus, from this perspective, it would be more accurate to say that one is how one engages in the affairs of life (Dreyfus, 1991; Guignon, 1983). An underlying assumption of practice theory, which gives rise to the claim that identity is a kind of continuous position taking, concerns the basic nature of human existence. From this perspective, each human must take a stand on her or his own identity because there is no prior ontological given regarding the nature of one’s character and action; an individual’s life is, in this sense, existentially ungrounded and in need of direction or content provided by his or her own concernful involvement in the world (Brinkmann, 2008; Guignon, 1983; Heidegger, 1962; Taylor,

43

Instructional Design as a Moral Ecology of Practice: Implications for. . .

1115

1989). One’s identity is what becomes expressed in cultural context and, thus, one constitutes oneself via certain kinds of participation against a background of cultural possibilities. From this perspective, there is no psychological reality or level of explanation running deeper than meaningful, agentic involvement in cultural practices. Clearly, from this perspective, culture can be said to offer a basis for action and, more generally, direction in one’s life; thus, people participate in culturally relevant activities such as going to school, working jobs, engaging in common recreational activities, and so on, in ways that one might generally expect within a particular cultural context. But culture does not specify the details of how one will engage in those pursuits moment by moment and, as a number of philosophers have argued, does not necessarily exert a structural determinism by which human action is reduced to a mere expression of sociocultural forces (e.g., Dreyfus, 1991; Gadamer, 1989; Giddens, 1979; Taylor, 1985). As cultural participants, people clearly act in cultural ways, and those ways offer meaningful patterns for living; but how a cultural participant goes about those activities is to be determined, to some significant degree, by that participant and may differ substantially from how those activities are managed by others. Human identity, in this regard, involves giving one’s own life direction by taking up cultural practices in certain ways and thus taking a stand on the possibilities that a given culture offers. As Guignon suggested (2012, p. 104), this view holds that one’s relation to culture and tradition is ambivalent – a person can “respect and even ‘revere’ the possibilities of being that make it the entity it is, while at the same time challenging and rethinking that tradition in defining its own existence.” In this regard, someone fits into cultural contexts by virtue of how she takes up the possibilities it offers and expresses herself as a particular kind of person; and this existential responsibility to become some kind of person through one’s practical, cultural involvement is necessitated by the lack of a basic human essence that predetermines identity and action. Some have used the term self-interpretation (Brinkmann, 2008; Guignon, 1983; Sugarman, 2005; Taylor, 1985) to refer to this tacit project of pursuing one’s own identity via cultural position taking. The claim that identity is a form of self-interpretation, and that self-interpretation involves action as a kind of position taking, offers a general framework for understanding particular human practices. Considering instructional design in particular, one can see that to be a designer is to take on, or grow into, a certain professional identity. That is, one only is an instructional designer if one engages in the kinds of activities that such professionals tend to engage in, sees the world as they tend to see it, and so on. This is, in the first instance, an issue of position taking, in that one can only take on the identity of an instructional designer if the practice of instructional design is available as a cultural possibility and viewed as a worthwhile endeavor in some sense; thus, one takes a position on the value or good of instructional design by pressing forward into the possibilities it offers as a form of professional practice. At this broad level of analysis, it is fairly clear that competency standards can play a role in revealing the value and possibilities of instructional design, though they do so in fairly abstract ways, offering propositions and generalities. The value and

1116

S. C. Yanchar

possibilities of instructional design are revealed more concretely through practical involvement in the work that these professionals actually do. One becomes aware of the nature of the field and engages in a kind of tacit position taking by performing instructional design tasks in certain ways – for example, seeking to produce quality instruction by conducting learner analysis, formulating learning objectives, using effective theoretical principles to inform design decisions, developing assessments that align with objectives, and so on. This is, generally speaking, a kind of position taking in the form of committing to professional practices in certain ways. As many have contended, however, instructional design practice calls for more than a commitment to general standards or even more particular resources such as ADDIE, ISD, events of instruction, first principles, and so on (Gibbons, 2014; Rowland, 1993; Smith & Boling, 2009; Yanchar & South, 2009). As some have suggested (Wilson, 2013; see also Bichelmeyer, Boling, & Gibbons, 2006; Yanchar, South, Williams, Allen, & Wilson, 2010), theories and related formalisms within the field underspecify practice and cannot offer detailed direction for the numerous design decisions to be made in the unfolding creation of learning environments. Thus, designers working in isolation or on design teams will bring a project to completion by virtue of their own unique ways of being instructional designers. From a practice theory perspective, this personal, practical contribution to the work of design might be thought of as designer identity, understood as a way of participating in the work of the field; it is a designer’s particular way of being involved in the tasks of design, emphasizing what she takes to be the best ways to solve problems and make decisions en route to completing a project. Moreover, from a practice theory perspective, this personal, practical way of being an instructional designer – that is, this identity as a stand on design practice – is a commentary or response to the myriad of possibilities entailed within the work of the profession; it is a designer’s position, among the many that she could take, on what is most important in the work of instructional design and what kind of designer she wishes to be. From this perspective, then, a designer’s strategic decisions suggest something significant about her stance regarding the most effective or helpful ways to go about design work, at least under a given set of circumstances – for example, that certain tasks must be completed while others are optional, that certain theoretical principles are more effective than others in certain cases, that some decisions must be made early in the process while others can wait, and so on. From the perspective of practice theory, this participational notion of identity – that is, taking a stand on matters of import by pressing into possibilities in certain ways – is an inevitable part of being human and engaging meaningfully in practices.

The Moral Ecology of Practices A unique aspect of human existence, according to practice theory, is this concernful involvement – that is, engaging in practices according to what about them is meaningful and what seems worthy of one’s attention and energy; and that concernful involvement is bound up with identity; it is the stand one takes on issues

43

Instructional Design as a Moral Ecology of Practice: Implications for. . .

1117

of practical relevance (Brinkmann, 2008; Taylor, 1989). But identity and concern, from this perspective, are not to be understood as merely subjective sentiments, meaningful only within one’s own private consciousness; rather, they exist as part of one’s fully-embodied, situated action as a part of the lived world. Moreover, the forms of concernful involvement one engages in, and thus the stands one takes in the face of cultural possibilities, are not granted warrant merely on the basis of one’s own idiosyncratic preferences. They are situated amidst a preexisting world in which meanings and value are very real features. What one values and pursues, in this sense, occur as part of a context in which moral oughts and values are real, already exist in the world, and play a crucial role in what people do and the stands they take. They are real in that they exist as a part of in-the-world practices, and thus must be somehow addressed or managed by that practices’ practitioners; that is, they don’t merely vanish based on personal preference; they have to be dealt with in some way (Taylor, 1989). The notion that moral claims are real and actually exist in the world of practical human involvement has been termed moral realism and implies what have been referred to as moral ecologies (Brinkmann, 2004). This notion suggests that one does not construct one’s own moral universe; rather, one comes to participate in-theworld with others in such a moral ecology provided by standards that are also in-the-world, though one does not necessarily act in perfect harmony with those standards all of the time. In an admittedly conceptual sense, then, it is erroneous to assume a fundamental split between an inner subjectivity of private values and an outer reality of morally-neutral events; rather, fully-embodied participation in practices itself exists in the “outside” world, and there is no “inside” strictly speaking. Participants and values exist in the same real-world space. As one might expect, practice theorists postulate a strong connection between the notion of cultural practices and the notion of a moral ecology. From this perspective, one’s participation occurs in light of moral oughts and standards, even if that participation is a reaction against them in some way. More specifically, any practice, as a set of culturally situated activities that have certain purposes, will entail expectations – whether implicit or explicit – regarding patterns of conduct such as how to perform work correctly, or with quality, and so on; and this applies to practices as diverse as parenting, gardening, participating in a sport, work in a profession, citizenship, and so on. Practices exist amidst a cultural matrix and are expected to make some contribution, or more generally, to perform some function; and the value-laden expectations they entail offer a sense of moral directedness that informs how practitioners should go about their activities for that contribution to be realized. A moral ecology, in this sense, offers a vision of the good practitioner. The practice of instructional design, for instance, is implicitly or explicitly guided by values regarding how practitioners ought to engage in their work. These values grant purpose and meaning to the professional skills needed, the support that designers seek to offer to learners, and whatever dispositions must be developed to achieve competence or excellence in the profession. Under this practice theory view, these moral-practical standards contribute significantly to the meaning of instructional design. More specifically, design skills reflect the implicit or explicit values of

1118

S. C. Yanchar

instructional design and are what enable some form of professional participation to be seen as skillful practice in the first place. In this sense, the values of practice entail morally acceptable ways of going about one’s professional duties. But how one goes about them will not necessarily reflect those values in every instance. Individual performance will surely vary in terms of how well it lives up to practice-internal values. An example from the literature concerns informal, workplace learning in instructional design settings. As inquiry in this vein has suggested (Yanchar & Hawkley, 2014, 2015), designers routinely encounter novel situations and adapt to the demands of specific design projects, which involves continuous learning in a variety of senses (e.g., learning new tools and skills, developing new ways of solving problems, or learning about the context surrounding a project to produce relevant and customized instruction). From a moral realist perspective, practitioners must be involved in their work in this way to meet expectations of competence or excellence, which means that they must continually learn in ways that allow them to rise to the challenges of their work and satisfy the moral demands of practice; they must deliver the goods, so to speak, where “goods” are understood as the moral goods of their work, or more specifically, the quality outcomes they are expected to produce, such as engaging learning experiences, and thus what others value and perhaps depend upon. If learning in this sense is crucial to excellent design work, as this inquiry suggests, then it functions as an important aspect of the moral ecology of the field itself; that is, it figures into what designers ought to do to achieve competence or excellence in their craft, aside from personal preferences. In this sense, it might be said that designers are called by the moral demands of their profession to engage in this sort of continuous, in situ professional development. Understood this way, the term moral concerns something larger than formal ethical codes of conduct or kindness toward others. While good practice might entail such codes or kindness, the notion of a moral ecology suggests ways of being oriented to the demands of cultural life in general; it provides “qualitative distinctions of worth” (Taylor, 1985, p. 17) that make standards of excellence possible; it offers value-laden points of reference for participation – for example, how to do one’s work well (Macintyre, 1984; Taylor, 1989). Guiding values or moral oughts inhere in practices, then, and those practices are real aspects of the historical-cultural world that situates one’s participation. Perhaps more importantly, practice theory advocates have argued that one cannot make sense of human forms of life without such moral ecologies (Taylor, 1989). Cultural practices provide points of reference that allow for coordinated social conduct – that is, they provide general patterns, so to speak, for everyday activity – because they offer distinctions regarding quality of conduct, better or worse ways of doing things, and expectations for particular kinds of practical involvement. Moreover, because such moral ecologies are real parts of the lived world, they provide a horizon of significance from which to appraise human impulses and imaginings. What an individual does, in this regard, will be viewed in light of a moral ecology that offers such distinctions of worth. Moral ecologies, then, provide a framework for understanding one’s participation; or, said differently, the nature of

43

Instructional Design as a Moral Ecology of Practice: Implications for. . .

1119

one’s participation will only make sense against a cultural background in which there are better and worse ways of going about particular practices. By offering such value-laden points of orientation, moral ecologies function as a basis for judgments – what Taylor referred to as “strong evaluation” (1985, p. 16) – regarding the quality of one’s motivations, forms of participation, and contributions. Without moral ecologies and the possibility of strong evaluations, one is left without a frame of reference to understand the value of engaging in practices in particular ways. If one takes the notion of moral ecologies seriously, as suggested by practice theory, then an emphasis within educational technology on competency standards is revealing. That various sets of standards and related proposals have been formulated suggests that many within the field are, in some way, sensitive to the same issues as practice theory advocates, though perhaps in a less philosophical way. While such standards surely are a disciplinary expediency, in that they provide a general framework regarding practitioner expectations and thus communicate something important about instructional design professionalism, they are also signs of the need for moral ecologies to meaningfully situate and give direction to work within the field. Competency standards may be, in some sense, a formalized reconstruction of what instructional design practitioners actually do, but they nonetheless provide a thematized way of demarcating the field’s professional boundaries and offer a vision of what it means to be a good instructional designer within the larger configuration of the professions; they offer a means of making sense of the discipline and what its practitioners do. Moreover, as expectations regarding the nature and quality of instructional design practice, competency standards provide points of reference that exist apart from any individual’s putative construction of reality; they provide a real basis for recognizing instructional design as a profession, but also a real standpoint for making judgments about the nature of design work and thus constitute, at least in part, a moral ecology of practice for work in the field.

Identity as Moral Self-Interpretation Given that cultural practices function as moral ecologies, it follows that by engaging in those practices in certain ways – and thus taking a stand on the possibilities that such practices offer – one is taking a stand on aspects of that moral ecology, or put more straightforwardly, one is taking a moral stand. Identity, then, is not just a position taking or a self-interpretation; as suggested above, it is a moral position taking or a moral self-interpretation that will have implications for matters such as how work should be conducted, how one’s life should be lived, and so on. It is in this sense that identity as self-interpretation is a moral phenomenon; it is an expression of oneself in this moral sense; and such a moral expression is human identity from this practice theory perspective (Brinkmann, 2008; Taylor, 1989; Yanchar, 2016). On this account, to be human at all means to be some kind of cultural participant, which in turn means that a person will be fundamentally engaged in some form of practice that functions as a moral undertaking. Indeed, being human, in this sense, is an inescapably moral project.

1120

S. C. Yanchar

With regard to instructional design, this means that one’s identity as a designer involves taking a stand on aspects of cultural-professional practices that constitute a moral ecology, and that designing instruction is, in the sense expressed by practice theory advocates, a moral endeavor. Designer identity as a moral stand will be expressed in terms of how an instructional designer goes about her activities, the values and priorities she will be committed to as she makes design decisions, the standards of quality she aspires to amidst contextual constraints, and the contributions that she can make as she manages the daily challenges of her profession. In this regard, there is a clear sense in which formal competency standards, as moral points of reference, provide at least part of a real moral ecology by which designers may orient themselves to the kind of contribution they should make to the learning of others. Designers will thus do their work in light of these distinctions of worth; they will take positions, or act as commentaries, on those visions of the good designer by virtue of how they perform their work in real situations; they can, then, evaluate their own performance, or be evaluated by others, in light of those distinctions. But since competency standards underspecify practice, designers will do more than follow design process formalisms that roughly cohere which such standards. As I have noted, designers adapt to the unique challenges of particular design situations (Bichelmeyer et al., 2006; Rowland, 1993; Wilson, 2013; Yanchar & Hawkley, 2014); and through experience, designers will arrive at progressively more developed designer judgment, practical wisdom, and sets of preferred techniques (Yanchar & Hawkley, 2014). As they do so, they are constituting their identity; they are taking a stand on the nature of design work in the field and on how to manage particular situations, which in turn suggests a significant connection between designer identity and personal agency, and more specifically, that one’s identity is a stance to be worked out in the details of one’s work in the field. One may be an instructional designer in general, but each practitioner in the field is also a particular kind of designer, with a design sense and perspective unique to them, expressed in the work they produce. From this perspective, a designer is what he or she does; but it must be recognized that what he or she does is a form of moral position taking as a commentary on the field’s practices and possibilities.

Implications for Instructional Design Recognizing the Moral Ecology and Moral Realism The claims of practice theory regarding moral ecologies and moral self-interpretation are far reaching. With respect to competency standards and related issues within educational technology, practice theory makes clear why scholars and practitioners should be concerned with expectations for practice at a level deeper than platitudes. There is a felt sense of importance associated with standards for quality work and a desire to make significant contributions, even if such standards are not explicitly viewed as parts of a moral ecology per se; and there is a felt sense that something is at stake in what the field can produce within various social contexts. It seems clear that

43

Instructional Design as a Moral Ecology of Practice: Implications for. . .

1121

learning is essential to flourishing human life and thus the facilitation of learning in others counts as a moral good of the highest order. If moral ecologies constitute a basis for practice – that is, if moral ecologies allow people to make sense of their involvement in the world and have a sense of direction in their work – then competency standards that function as part of a moral ecology help give shape and meaning to the field of educational technology; they help clarify what the field should be able to contribute. Although moral ecologies are not typically expressed in formal statements, and such statements do not fully capture what moral ecologies entail, the task of considering and producing competency standards helps create needed guidelines, which, as suggested above, function as moral reference points. And without such reference points, one cannot make sense of cultural forms of practice, including design practice. Moreover, as noted above, this variant of practice theory is based on a form of moral realism in which such valueladen standards are inherent in practices and thus actually exist in the world itself (i.e., they constitute a moral ecology). The standards that figure into a moral ecology are thus not mere subjective impressions projected onto a neutral, external world. They are actual parts of the lived world of human engagement and must be dealt with in this sense. Given the moral ecology argument of practice theory, and the role that competency standards play in the field’s moral horizon, it seems clear that the conversation regarding competency standards and related issues is important. In fact, in light of the significance of moral ecologies, the topic of competency standards should be granted a high priority, examined from many perspectives, and treated as a continuous topic of conversation. Importantly, however, this is not a call for unanimity in the field regarding what counts as a quality performance or a doctrine that all designers would be expected to follow. Some in the conversation may emphasize civic-minded design practice (Yusop & Correia, 2012), while others may advocate technical rationality (Clark & Estes, 1998), change agency (Campbell, Schwier, & Kenny, 2009), or other standards (MacLean & Scott, 2011; Richey et al., 2001), but in any case, the root issue is the inevitability of a moral horizon that will give design work meaning and provide a general sense of what best practices and quality contributions entail. In this sense, the practical is moral and the moral is practical (Bernstein, 1983; Taylor, 1989). Ultimately, this argument is an acknowledgement that moral horizons are inescapable and that it is the field’s responsibility to navigate them by earnestly pursuing relevant moral reference points in the midst of practice. Additional insight into the moral ecology of instructional design may be gained by inquiry into the actual practice of designers, with a focus on how they manage their duties in light of the explicit or implicit moral demands that seem operative in everyday work settings. This would be inquiry focused on the challenges faced by instructional designers as they seek to balance the many demands of their work (seen as moral demands), including an examination of what appear to be inescapable yet challenging moral reference points, contextual factors that hinder or facilitate designers’ pursuit of excellence, and motivations and values of designers (or other stake holders) that may not mesh with broader disciplinary notions of good practice. Given the cultural and practical intricacies of instructional design work across

1122

S. C. Yanchar

professional niches (K-12, corporate, adult education, military, e-learning, etc.), this kind of inquiry might be best conducted via ethnographic-oriented approaches formulated to explore everyday, cultural contexts of practice and meaning, though other approaches may also yield fruitful insight into the dynamics of moral ecologies (Yanchar & Slife, 2017). Ultimately, inquiry such as this can provide rigorous, data-based explications of the moral ecologies of instructional design, which can provide insight into related issues such as designer training, continuing professional development, and everyday informal learning on the job. That is, with a better understanding of what participation in the field really involves – from a moral ecologies standpoint – training of various kinds could better adapt to the lived realities of designers seeking to maximize the good they can do within institutional constraints. Moreover, such inquiry could provide insight into the nature and direction of the field itself. For example, inquiry of this sort could be help reveal conflicts among implicit moral demands that create challenges for designers and impede good practice. Knowledge produced by this form of inquiry could then provide a basis for refinements or adjustments in expectations for design teams and possibly a basis for revised views of what good practice in the field generally ought to entail.

Professional Identity From a practice theory perspective, instructional design as a practice only makes sense when viewed in the context of a moral ecology; it is the moral ecology of instructional design that provides reference points for the activities of designers, how those activities should be conducted, standards of excellence, and so on. As noted, not all aspects of a moral ecology can be easily translated into propositions, but what can be translated offers direction regarding how designers may view their work in light of a broader professional horizon. Moreover, general principles such as competency standards do not dictate the details of practice in context and leave designers on their own, so to speak, regarding problem solving and decision making in particular design situations. Theories and process models may facilitate these design tasks in context, but, as I noted earlier, they also underspecify what occurs in the actual step-by-step production of learning environments (Wilson 2013) and thus leave designers in need of their own ways of designing, amidst the moral ecology and equipped with conceptual resources that may provide insight. It is here that designer identity as moral self-interpretation becomes most salient. As a particular way of being involved in the tasks of design, designer identity emphasizes what one takes to be the best ways to solve problems and make decisions in a given design project, particularly with respect to details that cannot be managed purely on the basis of general principles and models. That is, a designer’s decisions at strategic junctures are what make the design finally come to fruition, and those decisions – which may be informed by theories and models in some fashion – are a reflection of the designer’s identity, that is, a reflection of the designer’s stance regarding best practices under particular circumstances. In this sense, from a practice

43

Instructional Design as a Moral Ecology of Practice: Implications for. . .

1123

theory perspective, a designer’s efforts to produce a learning environment will be a commentary on the practical work of instructional design; but it is a moral commentary, in that a designer’s identity will be expressed in terms of certain values and priorities as she makes design decisions intended to facilitate the learning of others. From this perspective, something (or someone) specifies what competency standards, process models, and theories cannot, namely, the practicing instructional designer who embodies certain practical-moral stands in his design work. A focus on designer identity tells only part of the story, however, because identities, from this perspective, are dynamic and open-ended, more akin to an unfolding life narrative (Guignon, 2002) than a thing with static properties. As suggested above, theorists concerned with identity as a dynamic phenomenon have typically connected identity formation with learning in some way, suggesting that learning is in large measure a process of identity formation (Lave & Wenger, 1991; Wortham, 2006; Yanchar, 2016). Likewise, from a practice theory perspective, one’s identity can be said to unfold through formal and informal learning experiences. Practitioners’ ways of seeing situations and taking action, and thus the stands they take amidst a moral ecology of design, develop as they continually solve problems and manage tasks in the midst of everyday activities, enabling them to engage in increasingly capable practice over time (Yanchar & Hawkley, 2014; see also Dreyfus & Dreyfus, 2004; Yanchar, Spackman, & Faulconer, 2013). Formal professional development experiences may also contribute to a designer’s sense of their craft and ability to best perform their labors. In either case, however, professional development can be seen as a kind of moral becoming or identity formation – that is, becoming the kind of designer who takes a particular stand on various aspects of instructional design work within a moral ecology. Professional development of whatever sort, in this sense, is the development of self-interpretation (or identity) and entails the ways designers take up practices informed by values and standards of conduct, either implicitly or explicitly. Furthermore, how (or whether) a designer seeks to learn and thus improve her capability in certain ways depends on the extent to which she implicitly or explicitly cares about becoming a certain kind of professional, involved in certain kinds of practices, and making certain kinds of contributions. In this practice theory sense, how one goes about learning constitutes a stand on this moral becoming. With regard to professional development, then, designers’ efforts to refine or augment their skills would optimally be informed by their commitments in this moral respect. A practicing designer, for instance, might reflect on her work and identify important areas for professional improvement as a response to the moral ecology of the field and her own positioning with it. For example, she might come to the conclusion that with greater development skills she would be in a position to contribute more to the design team, have a clearer understanding of what is (and is not) possible in the creation of a learning environment, and become aware of technologies that help produce more engaging learning experiences. She would thus view her own professional development as a striving toward practical-moral excellence in these respects; and that without the acquisition of greater technological skill, she would be limited, perhaps significantly, in her ability to achieve this excellence.

1124

S. C. Yanchar

Expanding and Refining Competency Standards As I noted earlier, competency standards in the field tend to focus on technical skills and processes, which are surely important and figure into designers’ work identities as they participate within a moral ecology of practice. But such standards offer little guidance for complex decision making in the midst of everyday challenges. Indeed, authors of the 2000 IBSTPI standards straightforwardly encourage practitioners to tailor standards to individual circumstances. In this sense, factors such as assumptions, values, practical wisdom, and judgment, in addition to contextual constraints such as deadlines, budgets, resource limitations, and client expectations, provide the primary basis for problem solving and decision making in context (Boling & Gray, 2014; Nelson & Stolterman, 2012; Rowland, 1992,1993; Yanchar et al., 2010). But which standards apply to the navigation and management of such contextual factors? Unfortunately, as a review of standards such as those produced by AECT or ISPI suggests, there are none. That is, as important as published standards are, for reasons I have already suggested, additional standards that offer expectations of some sort at this crucial level of practice are nonexistent. Moreover, the issue of competency standards per se has not been addressed from the perspective of moral ecologies and moral self-interpretations. While organizations such as AECT and ISPI have published codes of professional ethics that address issues related to professional integrity, use of validated principles, contributions to society, and so on, they do not address practice per se as a fundamentally moral enterprise. Codes of ethics may be helpful and, again, point to the need for moral points of orientation; but such codes offer only brief statements of proper conduct and treat ethics as merely one aspect of a broader field, perhaps even as an add-on to the more fundamental issue of design competencies per se. In this respect, such codes fail to acknowledge that instructional design itself constitutes a moral ecology of practice and that designers will take particular moral stances as part of their everyday work in the field, even when their challenges seem largely unrelated to codified ethical concerns. For example, one’s position on the nature of learner agency (i.e., one’s ability to freely or meaningfully choose and act) constitutes a moral stance from this perspective – given that what they assume about this issue will have implications for the quality of instruction and for the lives of learners – but codes of ethics are mostly silent on this and numerous other concerns pertaining to the basic nature of learners per se. One might wonder, from a practice theory perspective, about the implications of this practical-moral view of human life for instruction; that is, should learners be treated as if their learning and activity in general are mechanistically determined by natural laws; and if so, what are the consequences of this conceptual stance? These kinds of questions become salient when practice is viewed in this moral realist sense. Given what I have suggested here, such practical-moral considerations should be a primary issue in discussions of how practitioners might be oriented toward their work. How, it might be asked, does the good designer deal with such assumptions, manage the exercise of design judgment, or grapple with the uncertainty of new design projects that call for creative solutions? Granted that assumptions and values

43

Instructional Design as a Moral Ecology of Practice: Implications for. . .

1125

are inescapable, some have argued that an important critical thinking task is to become aware of these often implicit influences, to the extent possible, and make them the subject of critical scrutiny (e.g., Brookfield, 2012; Slife & Williams, 1995; Yanchar & Gabbitas, 2011). These are practical-moral tasks that require practitioners to take a moral stand, in the sense described here, and thus should be part of the conversation regarding competency standards; that is, such standards would call for practitioners to be aware of, and to be able to reflect critically on, these important aspects of their work. Critical attention paid to underlying assumptions regarding learner agency and responsibility, for instance, could help designers gain a deeper sense of what this concept might entail and what its implications might be for solving instructional problems. If learner agency is as important as some suggest (e.g., Ferguson, Phillips, Rowley, & Friedlander, 2015), then the stand one takes on it as an educational professional, implicitly or explicitly (though usually implicitly), will make a difference in what is produced. An historical-critical analysis of programmed instruction, for instance, suggested that its deterministic (i.e., non-agentic) assumptions were an important contributing factor in the decline of this movement (McDonald, Yanchar, & Osguthorpe, 2005). If this is the case, and if designer’s assumptions regarding agency do make a practical difference in their work, then it behooves the field to take this issue seriously and emphasize it appropriately in its competency standards and training. In this sense, the good designer would be aware of his or her own assumptions regarding the nature of learning and learners that lead to quality work. And, of course, the same would be true of other key assumptions related to learning, instruction, and design, such as those pertaining to instructor or designer responsibility, the influence of technology, the role of design formalisms such as process models, and related issues. While it is difficult or impossible to prove theoretical assumptions true (Slife & Williams, 1995; Yanchar, Slife, & Warne, 2008), professionals in the field can at least be aware of them, be able to defend them, and revise them as needed based on unfolding experience and relevant scholarship. The broader point, however, is that competency standards might be fruitfully expanded by placing emphasis on critical thinking about these kinds of assumptions, values, and judgment, so that design team members are more aware of the stands they take within the moral ecology of the field and how they might continually pursue good practice in this respect. (Critical thinking resources that facilitate this reflective work are available in various social science and education literatures; see, for example, Brookfield, 2012; Gabbitas, 2009; Slife & Williams, 1995; Yanchar & Slife, 2004). Moreover, any proposed competency standards will themselves be historicallyculturally situated and thus need to be examined with regard to the practical-moral vision they provide. From what standpoint are any standards offered, what do those standards presuppose, and what are their implications for design work as a moral enterprise? Recent discussions about the nature of instructional design work and the role of the designer are a case in point. As I have suggested, design itself has become a contested issue in the field. While emphasis was traditionally placed on ID process models, more recent analyses have emphasized designer assumptions (Gray et al.,

1126

S. C. Yanchar

2015; Yanchar & Gabbitas, 2011), judgment (Gray et al., 2015; Nelson & Stolterman, 2012; Rowland, 1992, 1993), and innovation (Clinton & Hokanson, 2012; Yanchar & Hawkley, 2014). An important question then concerns how insights that result from this ongoing discussion can be incorporated into standards and reflect emerging views of the nature of the practice of instructional design. It seems that issues such as assumptions, values, judgment and innovation would have relevance for a number of competencies, but perhaps none more than #13 of the 2000 IBSTPI standards, which reads as follows: “Select, modify, or create a design and development model appropriate for a given project” (Richey et al., 2001, p. 51). Here it might wondered if this standard appears somewhat narrow in its focus, at least by contemporary lights. As this ongoing discussion has suggested, formalisms such as models may play an important role in design work, but not to the extent, or in the way, that is implied by this standard, at least as it is articulated in the 2000 version. How might this competency be revised to better reflect the lived realities of design practice in which assumptions, values, judgment, and innovation play key roles? There is no clear and uncontested answer to this question; but there is, from a practice theory (and moral realist) perspective, good reason to take it seriously and engage in the kinds of scholarship that can move the field closer to a satisfactory response, namely, one that clarifies the fit of assumptions, values, and so on into the moral ecology of the field, or in specific work contexts. Finally, the practical-moral nature of professional identities is an issue in designer training and continuing education as practitioners – that is, in the continuous moral becoming of designers. What is a designer seeking to become, even if only tacitly? What moral vision provides a backdrop for her development as a practitioner? How does she tacitly come to fit within a moral ecology? Again, extant competency standards have not addressed these issues straightforwardly and offer little, if any, guidance regarding the nature of professional development as an ongoing process. Of course, no mandated set of rules can control how these issues play out in the field; but a conversation at this level can generate awareness of professional learning as a moral phenomenon and offer a more robust vision of what such learning might entail. One aspect of that vision, as I have suggested, concerns an emphasis on critical self-reflection in designers’ formal training and in their professional growth as a form of continuous identity development. If designers’ professional practices, as moral stances, are as important as practice theory would suggest, then this topic should receive at least as much attention as other worthy topics such as developing skill in the use of process models, design theories, development tools, and project management principles. This suggests an important frontier for the field: developing a vigorous conversation regarding professional identities and practices as moral stances, and moreover, allowing the relevance of these moral stances to be seen by designers on a wide scale through professional training and more concentrated scholarly attention. Research on the actual practices of designers and the contexts in which they work have been produced (e.g., Sugar, 2014), but a stronger emphasis on this topic and deeper analyses of the assumptions, values, and judgment of designers as moral phenomena is warranted from this perspective.

43

Instructional Design as a Moral Ecology of Practice: Implications for. . .

1127

Conclusion General discussion about competency standards performs an important function in the field. From a practice theory perspective, they reveal the fundamentally moral nature of instructional design as a practice; that is, they reveal how moral ecologies provide a meaningful backdrop for such practice and clarify what the field aspires to; or stated differently, design practices only make sense when viewed as activity amidst a moral ecology that entails expectations of competency or excellence. This chapter has suggested that the contribution made by competency standards can be strengthened by addressing considerations that go beyond general statements about skills and process models, such as critical self-reflection and continuous professional learning. The inescapable question facing instructional designers, from a practice theory perspective, is: “Where do I stand, in light of the historical and possibly novel cultural-professional-moral possibilities of the field, and why?” These questions require consideration of deeper issues than those currently addressed in competency standards. But standards that acknowledge these issues can allow for a greater awareness of instructional design as a moral practice and a clearer vision for how designers might specifically go about their work.

References Akerlof, G. A., & Kranton, R. E. (2002). Identity and schooling: Some lessons for the economics of education. Journal of Economic Literature, 40, 1167–1201. Bernstein, R. J. (1983). Beyond objectivism and relativism: Science, hermeneutics, and praxis. Philadelphia: University of Pennsylvania Press. Bichelmeyer, B., Boling, E., & Gibbons, A. S. (2006). Instructional design and technology models: Their impact on research and teaching in instructional design and technology. In M. Orey, V. J. McClendon, & R. M. Branch (Eds.), Educational media and technology yearbook (Vol. 31, pp. 33–73). Littleton, CO: Libraries Unlimited. Boling, E., & Gray, C. M. (2014). Design: The topic that should not be closed. TechTrends, 58(6), 17–19. Brinkmann, S. (2004). The topography of moral ecology. Theory & Psychology, 14(1), 57–80. Brinkmann, S. (2008). Identity as self-interpretation. Theory & Psychology, 18(3), 404–422. Brookfield, S. D. (2012). Teaching for critical thinking: Tools and techniques to help students question their assumptions. San Francisco, CA: Jossey-Bass. Campbell, K., Schwier, R. A., & Kenny, R. F. (2009). The critical, relational practice of instructional design in higher education: An emerging model of change agency. Educational Technology Research and Development, 57, 645–663. Cifuentes, L., & Murphy, K. L. (2000). Promoting multicultural understanding and positive selfconcept through a distance learning community: Cultural connections. Educational Technology Research and Development, 48(1), 69–83. Clark, R. E., & Estes, F. (1998). Technology or craft? What are we doing? Educational Technology, 38(5), 5–11. Clinton, G., & Hokanson, B. (2012). Creativity in the training and practice of instructional designers: The design/creativity loops model. Educational Technology Research and Development, 60(1), 111–130. Dreyfus, H. L. (1991). Being-in-the-world: A commentary on Heidegger’s being and time, Division 1. Cambridge, MA: MIT Press.

1128

S. C. Yanchar

Dreyfus, H. L., & Dreyfus, S. (2004). The ethical implications of the five-stage skill-acquisition model. Bulletin of Science, Technology, & Society, 24(3), 251–264. Ferguson, R. F., Phillips, S. F., Rowley, J. F. S., & Friedlander, J. W. (2015, October). The influence of teaching: Beyond standardized test scores: Engagement, mindsets, and agency. Retrieved from http://www.agi.harvard.edu/projects/TeachingandAgency.pdf. Gabbitas, B. W. (2009). Critical thinking and analyzing assumptions in instructional technology (Unpublished master’s thesis). Provo, UT: Brigham Young University. Gadamer, H. G. (1989). Truth and method (2nd revised ed.). New York: Continuum. Gee, J. P. (2001). Identity as an analytic lens for research in education. Review of Research in Education, 25, 99–125. Gibbons, A. S. (2014). An architectural approach to instructional design. New York: Routledge. Giddens, A. (1979). Central problems in social theory. London: Macmillan. Gray, C. M., Dagli, C., Demiral-Uzan, M., Ergulec, F., Tan, V., Altuwaijri, A. A., . . . Boling, E. (2015). Judgment and instructional design: How ID practitioners work in practice. Performance Improvement Quarterly, 28, 25–49. Guerra, J. A. (2006). Standards and ethics in human performance technology. In J. A. Pershing (Ed.), Handbook of human performance technology (3rd ed., pp. 1024–1046). San Francisco: Pfeiffer. Guignon, C. (2002). Ontological presuppositions of the determinism-free will debate. In H. Atmanspacher & R. Bishop (Eds.), Between chance and choice: Interdisciplinary perspectives on determinism (pp. 321–337). Charlottesville, VA: Imprint Academic. Guignon, C. B. (1983). Heidegger and the problem of knowledge. Indianapolis, IN: Hackett Publishing. Guignon, C. B. (2012). Becoming a person: Hermeneutic phenomenology’s contribution. New Ideas in Psychology, 30, 97–106. Gulbahar, Y., & Kalelioglu, F. (2015). Competencies for e-instructors: How to qualify and guarantee sustainability. Contemporary Educational Technology, 6(2), 14–154. Heidegger, M. (1962). Being and time. New York: Harper Collins Publishers. Hung, D., & Chen, V. (2007). Context-process authenticity in learning: Implications for identity, enculturation and boundary crossing. Educational Technology Research and Development, 55, 147–167. Ingold, T. (2000). The perception of the environment: Essays in livelihood, dwelling, and skill. New York: Routledge. Kerr, S. T. (1983). Inside the black box: Making design decisions for instruction. British Journal of Educational Technology, 14, 45–58. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York: Cambridge University Press. Luehmann, A. L. (2007). Identity development as a lens to science teacher preparation. Science Education, 91(5), 822–839. MacIntyre, A. (1984). After virtue (2nd ed.). Notre Dame, IN: University of Notre Dame Press. MacLean, P., & Scott, B. (2011). Competencies for learning design: A review of the literature and proposed framework. British Journal of Educational Technology, 42(4), 557–572. McDonald, J. K., Yanchar, S. C., & Osguthorpe, R. T. (2005). Learning from programmed instruction: Examining implications for modern instructional technology. Educational Technology Research and Development, 53(2), 84–98. Nelson, H. G., & Stolterman, E. (2012). The design way: Intentional change in an unpredictable world (2nd ed.). Cambridge, MA: MIT Press. Packer, M. (2011). The science of qualitative research. New York: Cambridge University Press. Reiser, R. A. (2001). A history of instructional design and technology: Part II: A history of instructional design. Educational Technology Research and Development, 49(2), 57–67. Richey, R., Fields, D., & Foxon, M. (2001). Instructional design competencies: The standards (3rd ed.). Iowa City, IA: International Board of Standards for Training, Performance, and Instruction.

43

Instructional Design as a Moral Ecology of Practice: Implications for. . .

1129

Rowland, G. (1992). What do instructional designers actually do? An initial investigation of expert practice. Performance Improvement Quarterly, 5(2), 65–86. Rowland, G. (1993). Designing and instructional design. Educational Technology Research and Development, 41(1), 79–91. Schatzki, T. R., Knorr Cetina, K., & von Savigny, E. (Eds.). (2000). The practice turn in contemporary theory. Florence, KY: Routledge. Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. Educational Researcher, 27(2), 4–13. Sfard, A., & Prusak, A. (2005). Telling identities: In search of an analytic tool for investigating learning as a culturally shaped activity. Educational Researcher, 34(4), 14–22. Slife, B. D., & Williams, R. N. (1995). What’s behind the research? Discovering hidden assumptions in the behavioral sciences. Thousand Oaks, CA: Sage. Smith, K. M., & Boling, E. (2009). What do we make of design? Design as a concept in educational technology. Educational Technology, 49(4), 3–17. Snelbecker, G. E. (1974). Learning theory, instructional theory, and psychoeducational design. New York: McGraw-Hill. Sugar, W. (2014). Studies of ID practices: A review and synthesis of research on ID current practices. New York: Springer. Sugarman, J. (2005). Persons and moral agency. Theory & Psychology, 15(6), 793–611. Taylor, C. (1985). Human agency and language: Philosophical papers (Vol. 1). New York: Cambridge University Press. Taylor, C. (1989). Sources of the self: The making of the modern identity. Cambridge, MA: Harvard University Press. Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New York: Cambridge University Press. Wilson, B. G. (2013). A practice-centered approach to instructional design. In M. M. Spector, B. B. Lockee, S. E. Smaldino, & M. Herring (Eds.), Learning, problem solving, and mind tools: Essays in honor of David H. Jonassen (pp. 35–54). New York: Routledge. Wortham, S. (2006). Learning identity: The joint emergence of social identification and academic learning. New York: Cambridge University Press. Yanchar, S. C. (2011). Participational agency. Review of General Psychology, 15(3), 277–287. Yanchar, S. C. (2016). Identity, interpretation, and the moral ecology of learning. Theory & Psychology, 26(4), 496–515. Yanchar, S. C., & Gabbitas, B. W. (2011). Between eclecticism and orthodoxy in instructional design. Educational Technology Research and Development, 59, 383–398. Yanchar, S. C., & Hawkley, M. (2014). “There’s got to be a better way to do this:” A qualitative investigation of informal learning among instructional designers. Educational Technology Research and Development, 62, 271–291. Yanchar, S. C., & Hawkley, M. N. (2015). Instructional design and professional informal learning: Practices, tensions, and ironies. Educational Technology & Society, 18(4), 424–434. Yanchar, S. C., & Slife, B. D. (2004). Teaching critical thinking by examining assumptions. Teaching of Psychology, 31, 85–90. Yanchar, S. C., & Slife, B. D. (2017). Theorizing inquiry in the moral space of practice. Qualitative Research in Psychology, 14(2), 146–170. Yanchar, S. C., Slife, B. D., & Warne, R. T. (2008). Critical thinking as discipinary practice. Review of General Psychology, 12, 265–281. Yanchar, S. C., & South, J. B. (2009). Beyond the theory-practice split in instructional design: The current situation and future directions. In M. Orey, V. J. McClendon, & R. Branch (Eds.), Educational media and technology yearbook 2009 (pp. 81–100). New York: Springer. Yanchar, S. C., South, J. B., Williams, D. D., Allen, S., & Wilson, B. G. (2010). Struggling with theory? A qualitative investigation of conceptual tool use in instructional design. Educational Technology Research and Development, 58, 39–60.

1130

S. C. Yanchar

Yanchar, S. C., Spackman, J. S., & Faulconer, J. E. (2013). Learning as embodied familiarization. Journal of Theoretical and Philosophical Psychology, 33(4), 216–232. Yusop, F. D., & Correia, A. (2012). The civic-minded instructional designers framework: An alternative approach to contemporary instructional designers’ education in higher education. British Journal of Educational Technology, 43(2), 180–190.

Stephen C. Yanchar, Ph.D., is a Professor in the Department of Instructional Psychology and Technology in the David O. McKay School of Education at Brigham Young University. He is primarily interested in theoretical and philosophical issues in education and psychology, especially those pertaining to instructional design practices, human agency and learning, and qualitative inquiry. He is currently an Associate Editor of the Journal of Theoretical and Philosophical Psychology and a Consulting Editor of New Ideas in Psychology. He is a past-president of Division24 of the American Psychological Association (the Society for Theoretical and Philosophical Psychology) and a recipient of the Benjamin Cluff Jr. Award for Excellence in Educational Research in the McKay School of Education. He has published in journals such as Educational Technology Research and Development, Educational Researcher, Journal of Computing in Higher Education, Qualitative Research in Education, Journal of Theoretical and Philosophical Psychology, Theory & Psychology, Journal for the Theory of Social Behaviour, Review of General Psychology, Journal of Humanistic Psychology, New Ideas in Psychology, and Qualitative Research in Psychology. He is co-editor of the Routledge International Handbook of Theoretical and Philosophical Psychology.

Narrative or Expository Video Cases: Exploring the Influence of Video Cases on Junior Staff’s Attitude and Reflection

44

Jingbo Huang

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Review of the Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Narrative and Expository Videos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reflection: Connecting Experiences to Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Video Cases and Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Career Self-Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Research Questions and Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Statistical Analysis of Attitude Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overall Effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparison of Effectiveness between Expository and Narrative Video Types . . . . . . . . . . . Regression Analysis for the Effect of Video Type on Attitude Changes toward Future Careers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reflection Levels and Video Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Innovative Design and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Knowledge Transfer across Generations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Innovative Applications of Research Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Limitations and Future Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1132 1133 1133 1134 1134 1136 1136 1137 1137 1137 1139 1140 1140 1140 1144 1144 1146 1146 1147 1149 1149 1150

J. Huang (*) United Nations University Research Institute in Macau, Macau, China e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_148

1131

1132

J. Huang

Abstract

Video cases are powerful tools. When used properly, they can help shape attitudes and reflexivity development among adults. This study examines the effects of utilizing video stories of senior staff to help shape the career attitudes and reflexivity development of entry-level staff in the same working environment. This study compares the effects of narrative and expository video approaches on entry-level participants’ career attitudes and reflections. To investigate the use of differing types of video stories, 121 staff at the career entry phase in an international organization participated in the study. Participants represent 60 different nationalities and are located in more than 10 different work locations around the world. They were requested to participate in an online survey with embedded video stories, and were randomly assigned to watch either narrative or expository videos regardless of gender. Participants’ responses to the open-ended pre-test and posttest were the major data source. The results indicate that the narrative video type produced greater improvement in participants’ attitude toward their current careers and in their reflection levels than did the expository video type. The findings offer practical and innovative guidance on the pedagogical use of video in the context of career development training. Keywords

Video cases · Narrative · Corporate training · Career self-management

Introduction Contemporary careers are viewed as boundaryless, as individuals are more likely to move geographically and functionally within or across organizations (Arthur, Khapova, & Wilderom, 2005). Junior staff in an organization may not be clear about career prospects, and they gain such insights mainly through observation of senior staff’s career paths. Senior staff, especially those close to retirement age, is willing to and able to offer their rich career experiences to the younger generation to help the latter reflect their own careers. This study collects career stories of the senior staff, captures them in videos, and transfers the institutional memory to junior staff to help them make informed career decisions. Specifically, this study aims at using videos containing the career stories of senior staff in an international organization to help shape positive career attitudes and foster the development of reflexivity in staff in the career entry phase. Furthermore, the effects of narrative and expository videos will be examined in terms of the learning outcome of career self-management. Video has considerably influenced the field of education; when used properly, video can be a powerful learning technology. Research has shown that different types of videos can lead to different learning outcomes, and learners can see, say, do or engage in different ways with different types of videos (Schwartz & Hartman, 2007). Video can present a list of facts or advice that explicitly instruct learners what to do or it can tell stories, leaving the viewers to decide what to do. The expository and narrative video

44

Narrative or Expository Video Cases: Exploring the Influence of. . .

1133

approaches hold different potentials in leading different learning outcomes. Narrative videos have been proven to be more effective than expository videos in influencing people’s attitude in the fields of health communication, advertisement and gender studies (see Boeijinga, Hoeken, & Sanders, 2017; de Graaf et al., 2016; Hinyard & Kreuter, 2007; Kim et al., 2012; McQueen & Kreuter, 2010; Pietri et al., 2017; Shen, Sheer, & Li, 2015; Stitt & Nabi, 2011; Winkler et al., 2017). The studies all show that narrative video content influences people’s attitude. However, scant research has reported on the effects of video approaches on career self-management, a process where individual employees are aware of their personal goals, and make informed career decisions to drive their own careers within or outside of their current organization.

Review of the Literature The theoretical framework of this study consists of four pillars: narrative videos, reflection, video cases and reflection, and career self-management.

Narrative and Expository Videos Narrative approach involves story telling. Kreuter Holmes, Alcaraz, Kalesan, Rath, Richert, McQueen, Caito, Robinson, and Clark (2010) define narratives in health communication as “a representation of connected events and characters that has an identifiable structure, is bound in space and time, and contains implicit or explicit messages about the topic being addressed”(p. S7). Researchers in health communication and advertisement found through experiments the persuasive effects of narratives (Kim et al., 2012; Phillips & Mcquarrie, 2010). Green and Brock (2000) claimed that narratives transported audience into an “integrative melding of attention, imagery and feelings” (p. 701), so audience would focus on the events or life experiences in the narratives rather than making counterarguments. A meta-analysis study examined the persuasive effects of narratives in health communication (Shen et al., 2015). The results showed that narratives can influence people’s attitudes, intentions and behaviors, particularly when the narrative messages were conveyed in audio or video. As a contrast, the expository or rhetoric approach often implies explicitly presenting facts or information, and leaving the audience to agree or disagree. The effects of narrative and expository approaches were compared. Moss-Racusin, Pietri, Hennes, Dovidio, Brescoll, Roussos, and Handelsman (2018) examined the effects of three types of videos on gender bias literacy toward women in science. The first set of videos employed narratives, which were entertaining, emotionally evocative stories to illustrate gender bias in sciences. The second set of videos used expository approach, which featured experts in interviews describing the same empirical findings shown in narratives. The third group used a hybrid approach with both narrative and experts’ interviews. Compared to the control group, all three experimental groups increased awareness of bias, and influenced knowledge of gender inequality, and the everyday recognition of bias in everyday situations. However, the narrative videos increased participants’ immersion in the videos and identifications with the

1134

J. Huang

video characters, whereas expert interview improved logical thinking and perception of gaining knowledge on gender bias. The hybrid condition reflected the strengths of both of the narrative and experts’ interviews, but sometimes weaker.

Reflection: Connecting Experiences to Learning The Analects of Confucius states: to learn without reflecting is fruitless; to reflect without learning is dangerous (Confucius, 2007, p. 8). For learning to take place, reflection is indispensable. Woerkom (2003) posited that reflection was concerned with people’s action in specific situations. By reviewing such experiences, people could analyze causes and effects, and draw conclusions for future action (Woerkom, 2003, p. 40). The nature of reflection is not only rational, but emotional, as Boud, Keough, and Walker (1985) pointed out: Reflection is an important human activity, in which people recapture their experience, think about it, mull it over and evaluate it. Reflection in the context of learning is a generic term for those intellectual and affective activities in which individuals engage to explore their experiences in order to lead to new understanding and appreciations (p. 19).

Mezirow (1990) interpreted reflection as an evaluation of how or why we have perceived thought, felt or acted. York-Barr, Sommers, Ghere, and Montie (2001) defined reflection as: . . .a deliberate pause to assume an open perspective, to allow for higher-level thinking processes. Practitioners use these processes for examining beliefs, goals, and practices, and to gain new or deeper understandings that lead to actions that improve learning for students. Actions may involve changes in behavior, skills, attitudes, or perspectives within an individual, partner, small group, or school (p. 6).

This interpretation makes three points: first, it focuses on the outcome of reflection – change. Second, it implies that reflective practice is the catalyst to transform experiences into learning. Third, reflective practice can take place not only within an individual, but also at the organizational level.

Video Cases and Reflection Video plays an essential role in multimedia case instruction, as video not only constitutes an integral part of multimedia case systems and video-based curricula, but also encourages learning outcomes directly or indirectly. Schwartz and Hartman (2007) connects the types of designed video to categories of learning outcomes to indicate the potential uses of video in learning (Schwartz & Hartman, 2007). The type of learning outcomes are saying, seeing, doing, and engaging, as illustrated in the first ring of the wheel of learning for the use of designed video (Schwartz & Hartman, 2007, p. 338) (see Fig. 1). The wheel’s second ring refines the learning

44

Narrative or Expository Video Cases: Exploring the Influence of. . .

1135

Fig. 1 Extended space of learning for the use of designed video. The “seeing,” “doing,” “engaging,” and “saying” portions of the figure are from the original model (Schwartz & Hartman, 2007, p. 338). The “reflection” portion is a suggested addition to the model by this dissertation. The circular design of the figure pays homage to the instructional technologist, Romiszowski (1981)

goals based on the general learning outcomes. The third ring lists behavioral indicators and the fourth presents the types of video that can encourage such behaviors. Given the attention that has been granted to reflection in the video studies, a reflection portion should be added to Schwartz and Hartman’s space of learning for the use of designed video to represent the increasingly important role of reflection in video studies in education. Reflecting should appear as the fifth learning outcome (see Fig. 1) in addition to saying, engaging, doing and seeing (Schwartz & Hartman, 2007, p. 338). The specific learning targets associated to reflection would be twofold: full comprehension and application. Full comprehension means that learners not only understand what the video presents, but also comprehend the logic, reasons and outcomes behind the surface events. Application is a higher

1136

J. Huang

level of reflection outcome, which indicates that learners are able to make personal connections to the video cases, and apply the learning into their own practice. Subsequently, requesting learners to make decisions for video scenarios, justify the reasons, and interpret the outcomes and implications of their decisions could assess full comprehension. In terms of application, learners can be asked to produce a personalized event associated with video learning. For example, after watching career stories of senior employees of the same organization, new employees can be asked to make their own career self-management plans in light of the video stories. Some video genres that could be used for learners to develop deep understanding of a particular video message and/or to produce their own instances based on the videos are autobiography, interview, narrative, event and simulation. Traditionally, cases have been used in medical, legal and business education to train reflective practitioners. Later, learning with video cases also proved relevant in teacher education (see Baker, 2009; Boling, 2007; Bransford et al., 1990; Kinzer et al., 2006) and health communication (see de Graaf et al., 2016; Hinyard & Kreuter, 2007; Kim et al., 2012; McQueen & Kreuter, 2010; Pietri et al., 2017; Shen et al., 2015; Stitt & Nabi, 2011). Video cases supported teachers’ development from novices to experts, and proved effective in persuasive effects in health communication. However, few empirical studies report the use of video cases in the field of career self-management, or evaluate the effectiveness of video interventions. This study aimed to fill these gaps.

Career Self-Management Career self-management is a core concept in vocational psychology. Career selfmanagement refers to the behaviors that individuals pursue while attempting to achieve their desired career outcomes. Individuals engage in these behaviors throughout the course of their careers, not only at the initial career establishment stage (King, 2004, pp. 112–133). The motivation for career self-management is that people want to believe that they are the owners of their own careers, and that their efforts to direct their careers ensure their agency throughout their professional lives (King, 2004, p. 113). Particularly nowadays, as the nature of organizations is changing, and the character of work becomes unpredictable, career self-management seems to be a way that people can take control of their own careers and navigate through professional uncertainty. Career self-management can provide insights into all kinds of career patterns and trajectories (King, 2004, p. 114).

Research Questions and Hypothesis Based on the above literature review in the field of learning with cases, reflection, and career self-management, this study addresses the following research questions.

44

Narrative or Expository Video Cases: Exploring the Influence of. . .

1137

Research Questions Do career story videos of senior staff affect the career attitude of staff at the career entry stage? Does an expository or narrative approach in the video stories affect the career attitude of staff at the career entry stage? Does an expository or narrative approach in the video stories affect the reflection of staff at the career entry stage?

Hypothesis The hypotheses are: Career story videos can help staff at the career entry stage develop concrete and specific career action plans. A narrative structure will be more effective in the changing career attitudes of staff at the career entry stage. An expository structure will be more effective in fostering the reflection of staff at the career entry stage.

Methodology A total of 121 staff members at the career entry phase in the United Nations (UN) Secretariat and peacekeeping missions participated in this study over a period of 2.5 months in 2011. Participants represent 60 different nationalities and are located in more than 10 different global duty stations. They were requested to participate in an online survey with embedded videos, and were randomly assigned to watch either narrative or expository videos regardless of gender. Utilizing an online survey enabled staff members from different geographical UN office locations to participate in the study, including staff members located in New York, Geneva, Vienna, Bangkok, Santiago, Beirut, Nairobi, Addis Ababa, Brindisi, Incheon City, and Mexico City. There were also participants from UN peacekeeping operations in the Sudan, Uganda, Ivory Coast, and East Timor. The participants represented 60 different nationalities. Table 1 below shows the baseline characteristics of participants within the expository and narrative groups. As p > 0.05, the two groups are not significantly different. This implies that if changes take place between the two groups after the video intervention, they should not be due to the group differences. Users had control over who and what specific questions to watch. The selection of video interviewees and questions might be because participants felt close to someone who was in the same occupational group, or shared the same cultural background. Research has found that user-control improved users’ ability to glean insights and

1138

J. Huang

Table 1 Baseline characteristics of participants by study group Characteristic Number Demographic Gender Female Male Age 20–25 26–30 31–35 36–40 41–45 46–50 >51 Education High school Bachelor Master Doctor Years of working experience Job categories Professional General service and others Pretest current attitude Pretest future attitude

Percent or mean (SD) by group Expository Narrative 62 59

p

67.74 32.26 3.21 (1.22) 3.23 22.58 45.16 16.13 8.06 1.61 3.23

59.32 40.68 3.19 (1.37) 5.08 27.12 38.98 13.56 6.78 5.08 3.39

0.93a

4.84 17.74 70.97 6.45 2.48 (1.98)

1.69 25.42 66.10 6.78 2.15 (1.86)

0.61a

58.06 41.94 0.26 (0.72) 0.82 (0.61)

59.32 40.68 0.12 (0.81) 0.78 (0.83)

0.89a

0.91a

0.35b

0.32b 0.75b

Note. a Pearson’s chi-square test; b Student t-test

enhanced the effectiveness of learning with multimedia cases (Baker, 2009, pp. 259, 262). It took approximately 1 h to complete the online survey. The survey consisted of five sections: study description and informed consent, demographic information, pre-test, video watching (either N or E), and posttest (Fig. 2). Participants’ responses to the open-ended pre-test and posttest were the major data source. The qualitative responses to the survey questions were coded, quantized and analyzed. As the questions of pre-test and posttest served more as a means of soliciting thoughts than as categorization of themes, the responses to the pre-test were coded as an entity, so were the posttest responses. Coding was carried out in two phases. For the first phase, the qualitative data was analyzed inductively using grounded theory via the utilization of an open coding process (Glaser & Strauss, 1967). Some common themes emerged from the data, such as attitude changes and the formation of action plans. The coding process was recursive until saturation, when all the qualitative data fit into major themes. Then, the themes were re-examined, merged, hierarchized or eliminated. At this stage, NVIVO 9, qualitative analysis software, was used for the coding

44

Narrative or Expository Video Cases: Exploring the Influence of. . .

1139

Fig. 2 Online survey flow

of qualitative data. Second, themes about reflection and attitude were quantized. The qualitative data about reflection was coded based on a four-category scheme for coding the level of reflection in written work (Kember et al., 2008). The data on attitude were assigned scores based on a 5-score Likert scale developed for this study. A coding book was developed to guide qualitative data coding and quantization. Another coder independently used the coding book to code 20% of the data, and a high reliability rate (81%) was established, which ensured the reliability of the study. The non-parametric Wilcoxon signed rank test (Kember et al., 2008) was used for comparison of attitude mean scores between the narrative and expository groups, as well as for the comparison of reflection scores between these two groups. This statistical test procedure is supposed to apply to discrete random variables, and a paired t-test is applied to continuous random variables. As both the reflection scores and the attitude scores are discrete variables, the non-parametric Wilcoxon signed rank test was applied to analysis of both. Additionally, to test whether the video type was associated with the degree of attitude changes toward current and future careers following the intervention, a multinomial logistic regression model was established. Some demographic factors (e.g., gender, age, and years of experience) were examined first to see whether they were correlated to an improvement in attitude, then the model was re-fitted.

Results The data analysis focused on whether or not the use of the video cases, particularly the expository and the narrative approaches, had any differential effects on participants’ attitudes, their action plans, and their reflection levels. The results suggested that narrative videos were found to be four times more effective in improving participants’ attitudes toward their current careers than expository videos, and that narrative videos were more likely to stimulate reflection.

1140

J. Huang

Statistical Analysis of Attitude Change Individual textual responses were coded on a 5-score scale, from
2 to 2,
2 being “very negative,”
1 “negative,” 0 “neutral,” 1 “positive,” and 2 “very positive” as illustrated below.
2 very negative


1 negative

0 neutral

1 positive

2 very positive

Individual pretest and posttest scores were compared to generate a tendency score (tendency ¼ post score – pre score) to measure individual attitude change.

Overall Effectiveness The non-parametric method, the Wilcoxon signed rank test, proved that the posttest score is significantly higher than the pretest score (p < 0.005) for attitude changes for both current and future careers. This analysis indicated that watching the video clips improved staff members’ attitude toward their current and future careers.

Comparison of Effectiveness between Expository and Narrative Video Types The following statistical models were built to assess the difference in the effectiveness between expository and narrative types of video clips on participants’ current and future career attitudes. For the sake of model simplicity and interpretation, the original data was transformed in the following ways: For the variable “tendency” (tendency ¼ post attitude score – pre attitude score), the original samples were re-stratified into three groups to balance the sample size. Specifically, tendencies of 2 and 3 were merged into a new group, which was called Tendency2 (strong attitude improvement). Similarly, tendencies of
1 and 0 were merged, and this group was denoted as Tendency0 (no improvement in attitude). Tendency1 included only the people whose attitude change equaled to 1 (little attitude improvement). For variable “age,” the original sample was divided into two groups to balance the sample size: people who were 35 and younger (35 years). The variable labeled “treat” (treatment) was used to define the video type, where “treat ¼ 0” stood for the group that watched the expository clips and “treat ¼ 1” for the group that watched the narrative clips. To visually represent the effect of the narrative approach on strong attitude improvement toward current careers, mosaic plots (Fig. 3) were utilized to compare the proportions of participants from the expository and narrative groups in the no-

44

Narrative or Expository Video Cases: Exploring the Influence of. . .

1141

attitude-change group (Tendency0) and the strong-improvement group (Tendency2). Figure 4 illustrates that the proportion of those who watched narrative clips was much larger in Tendency2 group than that in Tendency0 group. This suggested that video type was associated with the degree of attitude improvement, and that, particularly, narrative videos tended to precipitate more strong improvements in attitude than expository videos. Regression analysis for the effect of video type on attitude changes toward current careers.

Fig. 3 Mosaic plots to compare the proportion of participants from the expository and narrative groups in the no-attitude-change group (Tendency0) and strong-attitude-improvement group (Tendency2). The areas represent the proportions of people in each video type group and in the Tendency0 and Tendency2 groups (Numbers from Table 8)

1142

J. Huang

Fig. 4 Proportion of participants in the expository and narrative groups who receive different levels of reflection scores (1–3). E ¼ Expository; N¼Narrative

The following multinomial logistic regression model was established for the whole dataset to test the abovementioned assumption of the effect of video type (narrative or expository) on attitude change toward participants’ current careers. Prð  Tendency 1Þ ¼ exp ðα1 þ β11  treat þ β12  gender þ β13  age þ β14  yearÞ, Prð  Tendency 0Þ

ð1Þ Prð  Tendency 2Þ ¼ exp ðα2 þ β21  treat þ β22  gender þ β23  age þ β24  yearÞ, Prð  Tendency 0Þ

ð2Þ

44

Narrative or Expository Video Cases: Exploring the Influence of. . .

1143

In this model, Pr(  Tendency2) represented the probability of having strong attitude improvement, Pr(  Tendency1) represented the probability of having slight attitude improvement, and Pr(  Tendency0) represented the probability of no attitude change. α and β stand for coefficients. The fitted coefficients for Eqs. 1 and 2 were summarized in Tables 2 and 3. It was observed from Eq. 1 that the fitted coefficients were not statistically significant ( p > 0.05). This suggests that after watching the video clips, the number of people who experienced a slight degree of improvement in their attitudes following the intervention was not statistically different from the number of people who maintained their attitudes. In addition, neither the video approach variable nor the demographic factors reported by participants (gender, age, and years of experience) were correlated with the number of people who indicated a slight improvement in attitude (Tendency1). As such, the slight attitude improvement in participants’ career attitudes could be viewed as an increase that resulted from watching video clips, regardless of the type of video viewed. However, the fitted coefficients for Eq. 2 indicated that, in addition to the video type variable, gender was also a factor that was correlated with a strong improvement in attitude. Women were more likely to experience a strong improvement in attitude than men after watching the video clips. Other demographic factors such as age and years of working experience were not reliable indicators of strong improvement in attitude. Consequently, the data has been refitted to a new multinomial logistic model, which was based upon the previous model, by eliminating the covariates of age and years of working experience. The new fitted model was the following:

Table 2 Fitted coefficients for Eq. 1

Equation 1 Intercept Treat (narrative) Gender (female) Age (>35) Year (>4)

Mean (SD) 0.95 (0.83)
0.30 (0.41) 0.41 (0.42)
0.35 (0.47)
0.86 (0.54)

95% CI
0.68, 2.58
1.10, 0.50
0.41, 1.23
1.27, 0.57
1.92, 0.20

p values 0.26 0.46 0.32 0.46 0.11

Table 3 Fitted coefficients for Eq. 2

Equation 2 Intercept Treat (narrative) Gender (female) Age (>35) Year (> 4)

Mean (SD)
4.52 (1.65) 1.75 (0.73) 1.52 (0.73)
1.48 (0.88)
0.37 (0.95)

95% CI
7.75,
1.29 0.32, 3.18 0.09, 2.95
3.20, 0.24
2.23, 1.49

p values 0.006 0.016 0.037 0.094 0.700

1144

J. Huang

Table 4 Summary statistics results for Eq. 3

Equation 3 Intercept Treat (narrative) Gender (female)

Mean (SD) 0.22 (0.68)
0.37 (0.40) 0.49 (0.41)

95% CI
1.11, 1.55
1.15, 0.41
0.31, 1.29

p values 0.75 0.35 0.23

Table 5 Summary statistics results for Eq. 4

Equation 3 Intercept Treat (narrative) Gender (female)

Mean (SD)
4.94 (1.45) 1.65 (0.71) 1.45 (0.71)

95% CI
7.78,
2.10 0.26, 3.04 0.06, 2.84

p values 0.006 0.020 0.042

Prð  Tendency 1Þ ¼ exp ð0:22
0:37  treat þ 0:49  genderÞ, Prð  Tendency 0Þ

ð3Þ

Prð  Tendency 2Þ ¼ exp ð
4:95 þ 1:65  treat þ 1:45  genderÞ, Prð  Tendency 0Þ

ð4Þ

The 95% confidence intervals (CI) for the coefficients in Eq. 3 were(
1.11, 1.55), (
1.15, 0.41), and(
0.31, 1.29). All the coefficients contained 0; the 95% confidence intervals (CI) for the coefficients in Eq. 4 were(
7.78,
2.10), (0.26, 3.04), and (0.06, 2.84). These intervals and corresponding p values are shown in Tables 4 and 5. In the fitted coefficient and Eq. 4, the probability that participants from the same gender group tended to strongly improve their attitudes were approximately [exp (1.65)-1] ¼ 4 times greater after watching narrative videos than watching expository ones. These results were statistically significant (see p values in Table 5).

Regression Analysis for the Effect of Video Type on Attitude Changes toward Future Careers Similarly, the same models (Eqs. 1 and 2) and multinomial logistic regression were applied to analyze the changes in attitude toward future careers. The fitted results indicated that none of the factors, including age, gender, years of working experience, and type of video, were statistically significant (p > 0.05), as shown in Tables 6 and 7. It was therefore concluded that watching neither expository nor narrative videos affected participants’ attitudes toward their future careers.

Reflection Levels and Video Approaches Reflection Coding Since the participants were randomly assigned to either the narrative or expository group, the reflection levels of the two groups should be equalized. Consequently, the posttest reflection scores of the two groups could be compared to determine whether

44

Narrative or Expository Video Cases: Exploring the Influence of. . .

1145

Table 6 Fitted coefficients for Eq. 1 toward future attitude

Equation 1 Intercept Treat (narrative) Gender (female) Age (>35) Year (> 4)

Mean (SD)
0.07 (0.61)
0.21 (0.39) 0.27 (0.41)
0.67 (0.48)
0.39 (0.52)

95% CI
1.28, 1.13
0.99, 0.56
0.54, 1.08
1.60, 0.27
1.92, 0.20

p values 0.90 0.59 0.51 0.16 0.46

Table 7 Fitted coefficients for Eq. 2 toward future attitude

Equation 2 Intercept Treat (narrative) Gender (female) Age (>35) Year (> 4)

Mean (SD)
2.94 (1.61) 1.45 (1.16) 0.05 (0.96)
0.89 (1.25)
0.61 (1.28)

95% CI
6.10, 0.22
0.82, 3.72
1.83, 1.93
3.33, 1.55
3.12, 1.90

p values 0.069 0.210 0.959 0.476 0.636

or not there was difference in reflection level between the two groups following the intervention. Participants’ posttests were coded to assess their reflection levels, and Kember et al.’s coding scheme was used in this analysis (Kember et al., 2008, pp. 372–375). A coding unit could be a sentence or a paragraph, and each unit received a reflection score of 0 to 3: non-reflection (0), understanding (1), reflection (2), and critical reflection (3). As all the coding units may not fit exactly into one of the four categories, intermediate scores are inevitable (Kember et al., 2008, p. 376). Therefore, in the posttest coding, 1.5 and 2.5 were introduced to represent intermediate reflection levels. The posttest writing was treated as a whole after the coding of each unit, and the highest reflection score was used as the final judgment of this written work (Kember et al., 2008, p. 376).

Effects of Video Approach on Reflection After the coding process, a participant received a reflection score, which represented the highest score of all coding units in the posttest. Table 8 shows the reflection score distribution within the two video approach groups. The last two rows demonstrate the percentages of the people in the narrative and expository groups receiving a particular reflection score out of the total number of people in both groups who received such a score. For example, among those who received the reflection score of 1.5, there are 16 from the narrative group and 28 from the expository group. Therefore, the percentage of the narrative group is 36.36% (16/44), and the percentage of the expository group is 63.64% (28/44). It is also noted that with the increasing of the reflection level, the percentages of these scores found in the narrative group also increases. Figure 4 below visually represents this pattern. In Fig. 4, the dark color blocks illustrate the expository group’s proportions within the group that received the same reflection scores, and the light color blocks demonstrate the proportions of the narrative group. The top horizontal axis denotes the reflection scores recorded. The areas of the blocks embody proportionally the

1146

J. Huang

Table 8 Posttest reflection scores distribution by video approach

E (counts) N (counts) Total counts % of N % of E

0 0 0 0 0.00 0.00

Reflection Scores 1 1.5 9 28 0 16 9 44 0.00 36.36 100.00 63.64

2 22 28 50 56.00 44.00

2.5 2 9 11 81.82 18.18

3 1 6 7 85.71 14.29

Note. E ¼ Expository; N¼Narrative. 0: non-reflection, 1: understanding, 1.5: understandingreflection, 2: reflection, 2.5: reflection-critical reflection, 3: critical reflection

numbers of people in the respective groups. Figure 4 illustrates that with the increase of the reflection scores, the proportions of the light color blocks (narrative) keep increasing. In other words, the higher the reflection score received, the more people proportionally come from the narrative group. To test whether this finding was statistically reliable, a non-parametric Wilcoxon rank sum test was conducted. The results confirm that participants who watched narrative videos have an increasingly larger proportions, when the reflection level rises (one-sided test, p < 0.05). This implies that overall, narrative videos tend to precipitate higher levels of reflection.

Innovative Design and Applications The innovative features of this study are twofold: on the one hand, video captures the institutional memory of the older generation in an organization, and transfers the tacit knowledge to the younger generation for them to reflect on their careers. On the other hand, the findings of the research, especially those related to gender, can inform the design of corporate staff training.

Knowledge Transfer across Generations On the first point of innovation, many organizations do not preserve institutional memory from their staff close to retirement. Those who do tend to preserve the historical milestones or technical knowledge. In this study, video cases of career stories were collected from long-term staff, some of which were around retirement age. These career stories captured the institutional memory of staff, which witnessed the history and the development of an organization to the present, and held hope for the organization’s future. The organizational legacy from the older generation is of special significance to the younger generation, who have just started their careers in the organization and who need support to effectively navigate the organization and plan their careers. In this study, older generation’s career path stories were captured as video cases, which the younger generation watched reflect upon. By transferring

44

Narrative or Expository Video Cases: Exploring the Influence of. . .

1147

the institutional memory of the older generation to the younger, the tacit form of knowledge from the older generation was not only preserved but also being utilized by the younger to plan their careers. Ultimately, the tacit knowledge would help chart a better future for the organization. One participant from the pilot study appreciated the knowledge sharing from the older generation, “. . .as someone from a cultural upbringing that values the word of the elders, I am grateful for this opportunity to have viewed a video word from these [organizational] ‘elders’.”

Innovative Applications of Research Findings The second innovative feature refers to the practical implications of the findings, particularly those related to gender. The results showed that, overall, after watching the career story videos of senior staff, junior staff changed their attitudes toward their current and future careers significantly ( p < 0.005). This suggests that watching the videos improves junior staff’s attitudes toward both their current and future careers. Furthermore, within the same gender group, narrative videos are statistically more effective than expository videos in precipitating strong improvement in attitude toward participants’ current careers ( p < 0.005). Another finding is that narrative videos are more likely to precipitate higher level of reflection than expository videos. One interpretation is that narrative messages may have several advantages over more informational and expository forms of communication in changing people’s attitudes. While expository messages list facts, reasons and arguments to support a decision or a course of action, narrative style uses storytelling to depict events, experiences, and their consequences (McQueen & Kreuter, 2010, p. S15). Narrative messages facilitate identification with the characters in a video (Cin, Zanna, & Fong, 2004, p. 188), help information processing by capturing attention, and enhance understanding (Kreuter et al., 2010, S7), thus increasing engagement and positive thought about the narrative presented in a video story (Kreuter et al., 2008, p. 40). Consequently, narrative is more likely to produce more attitude change than expository messages (Stitt & Nabi, 2011, p. 20). Since the narrative videos appear less “aggressive’ than the expository videos, narrative videos can be effective in reducing counter-arguing, which is referred to as the rejection process (Green & Brock, 2000, p. 702), and increasing persuasion. The narrative videos are less overtly persuasive, and therefore are more likely to generate acceptance and overcome resistance by minimizing viewers’ perceptions that they are being persuaded to behave in certain ways, as seen in expository videos (MoyerGusé, 2008, p. 415). The narrative video’s motivating effect can be applied in the new staff orientation programs in corporate training to prepare positive attitudes for the new jobs and organizations. Most of the corporate orientation programs are based on practical information, where guest speakers from relevant departments would present rules, regulations, and practices to the newcomers. Participants may often be overloaded with the large amount of information intake and do not have a dedicated time to reflect and react within the orientation programs. Information-based orientation may

1148

J. Huang

also lose an important opportunity to foster positive attitudes for its new employees at their career entry stage, which is one of the best timings to shape such an attitude. One recommendation to address this issue is to separate the informational orientation from the inspirational one. The corporate information can be made available in the format of an e-kit, a website or a wiki page on the intranet, so that new staff can keep referring back to the information when needed. To shape the positive attitudes, organizations can conduct video interviews of the senior staff to capture their career stories. Then these videos can be incorporated into the orientation programs, containing a video learning portion both individually and collectively. Individually, new staff can watch video career narratives of the senior staff in the organizations, and reflect on their own. Collectively, new staff can discuss the learning with their peers as well as the current employees to validate their career plans. Or the entire orientation program can take narrative approach. Participants listen to stories from long-term staff, and reflect within the group. Many findings of this study are related to gender. These results are worth a closer look and warrant further discussion. Gender is correlated with a strong improvement in attitude. Females are more likely to experience a strong improvement in their attitudes toward their current careers than males. This tendency coincides with the self-report results, where a higher proportion of females reported that their attitudes were affected by the videos than did males. Additionally, females are more likely than males to report being affected by the video. They are more likely than males to adjust their career goals, to see career possibilities, to admit that their actions and ideas have been reinforced by the video, to consider learning as a career selfmanagement strategy, and to use a positive attitude to self-motivate their career development. It can be assumed from the above findings that females may be more engaged and more absorbed in the videos than males, and therefore may be more susceptible to changing their ideas with interventions. In addition, the finding about the narrative approach confirms Slater and Rouner’s (2002) theoretical model for processing narrative persuasive content embedded in narrative and corresponds to Green and Brock’s finding (Green & Brock, 2000) indicating that greater absorption in narrative increases persuasion and affects beliefs. The above findings on gender have the following implications in development of career development programs in organizations. First, more career development programs should be developed to help females take control of their careers and progress professionally. For example, organizations can interview corporate female leaders in a narrative way. The content that carries positive attitudes should be highlighted, with the support of a narrative about their experiences. Consequently, female viewers may relate to the attitudes presented in the video, respond emotionally and cognitively to the video narrative, and be persuaded to take career self-management actions. Second, learning videos can focus more on the future and potential benefits of a career strategy for the female audience, such as the positive consequences of career self-management behavior or belief. If females believe in the video messages, they are more likely to change their own beliefs and behaviors thereafter.

44

Narrative or Expository Video Cases: Exploring the Influence of. . .

1149

To summarize, knowledge transfer across generations and the applications of the findings in corporate training design are two major points of innovation. Other innovative applications of the research findings can be experimented in the future to test their effects.

Limitations and Future Studies The findings of this study are consistent with the literature of health communication, advertisement and gender studies on the effects of informational and narrative videos. A future study could be conducted among a sample of the same participants to examine whether and how video approaches have a long-term effect on attitude, cognition, and career self-development behaviors. In addition, this study consisted of participants from 60 nationalities with different cultural backgrounds. Participants with different cultural perspectives may interpret the video learning experiences differently. Future studies could add culture as an analytical dimension, and look at how participants from different cultures learn from expository and narrative videos. This would be particularly valuable in international organizations, because the findings could suggest effective video type(s) for different cultural groups to help achieve different learning goals.

Conclusion The objectives of this study were twofold: from the practical perspective, this study aimed to provide career self-management support to staff at their career entry stage in organizations. This was done by providing them with videos of the career stories of their predecessors within the same working environment. From the academic perspective, this study looked at whether narrative and expository video cases of career stories had different impacts on the attitudes and reflective practices of the participants with regard to career self-management. This study identified important insights into the use of video cases in career selfmanagement. The findings suggest that narrative videos are significantly more effective in strongly improving participants’ attitudes toward their current jobs, and are more effective in fostering reflection on career planning. The innovative design of the study allows career stories of the older generation to be preserved, and the younger generation can reflect on the videos and plan their own careers. Additionally, the findings related to narrative videos and gender can lead to innovative design of corporate training programs. Learning with video cases holds great potential, not only for staff at the career entry stage but also for adult learners in general, to help them with their career self-management. Future studies may wish to evaluate to what extent narrative and expository videos affect staff’s attitudes, action plan development, and reflective practices over a longer period.

1150

J. Huang

References Arthur, M. B., Khapova, S. N., & Wilderom, C. P. M. (2005). Career success in a boundaryless career world. Journal of Organizational Behavior, 26(2), 177–202. https://doi.org/10.1002/ job.290 Baker, E. A. (2009). Multimedia case-based instruction in literacy: Pedagogy, effectiveness, and perceptions. Journal of Educational Multimedia and Hypermedia, 18(3), 249–266. Boeijinga, A., Hoeken, H., & Sanders, J. (2017). Risk versus planning health narratives targeting Dutch truck drivers: Obtaining impact via different routes? International Journal of Communication, 11, 5007–5026. Boling, E. C. (2007). Linking technology, learning, and stories: Implications from research on hypermedia video-cases. Teaching and Teacher Education, 23(2), 189–200. https://doi.org/10. 1016/j.tate.2006.04.015 Boud, D., Keough, R., & Walker, D. (1985). Reflection: Turning experience into learning. In D. Boud, R. Keough, & D. Walker (Eds.), Kogan Page. London, England: Routledge. Bransford, J., Sherwood, R., Hasselbring, T., Kinzer, C. K., & Williams, S. M. (1990). Anchored instruction: Why we need it and how technology can help. In R. Spiro (Ed.), Cognition, education and multimedia: Exploring ideas in high technology (pp. 114–141). Hillsdale, NJ: Lawrence Erlbaum. Cin, S. D., Zanna, M. P., & Fong, G. T. (2004). Narrative persuasion and overcoming resistance. In E. S. Knowles & J. A. Linn (Eds.), Resistance and persuasion. Mahwah, NJ: Lawrence Erlbaum Associates. Confucius. (2007). The analects of Confucius: Parallel English and Chinese. Charleston, SC: Forgotten Books. Retrieved from http://www.forgottenbooks.org/info/9781605064000 de Graaf, A., Sanders, J., Hoeken, H., & De Marchis, G. P. (2016). Characteristics of narrative interventions and health effects: A review of the content, form, and context of narratives in health-related narrative persuasion research. Review of Communication Research, 4(4), 88–131. https://doi.org/10.12840/issn.2255-4165.2016.04.01.011 Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine Publishing Company. Green, M. C., & Brock, T. C. (2000). The role of transportation in the persuasiveness of public narratives. Journal of Personality and Social Psychology, 79(5), 701–721. Retrieved from http://psycnet.apa.org/journals/psp/79/5/701 Hinyard, L. J., & Kreuter, M. W. (2007). Using narrative communication as a tool for health behavior change: A conceptual, theoretical, and empirical overview. Health Education & Behavior, 34(5), 777–792. https://doi.org/10.1177/1090198106291963 Kember, D., McKay, J., Sinclair, K., & Wong, F. K. Y. (2008). A four category scheme for coding and assessing the level of reflection in written work. Assessment & Evaluation in Higher Education, 33(4), 369–379. https://doi.org/10.1080/02602930701293355 Kim, H. S., Bigman, C. A., Leader, A. E., Lerman, C., & Cappella, J. N. (2012). Narrative health communication and behavior change: The influence of exemplars in the news on intention to quit smoking. The Journal of Communication, 62(3), 473–492. https://doi.org/10.1111/j.14602466.2012.01644.x King, Z. (2004). Career self-management: Its nature, causes and consequences. Journal of Vocational Behavior, 65(1), 112–133. https://doi.org/10.1016/S0001-8791(03)00052-6 Kinzer, C. K., Cammack, D. W., Labbo, L. D., Teale, W. H., & Sanny, R. (2006). Using technology to (re) conceptualize pre-service literacy teacher education: Considerations of design, pedagogy and research. In M. C. Mckenna, L. D. Labbo, R. D. Kieffer, & D. Reinking (Eds.), International handbook of literacy and technology (Vol. II, pp. 211–231). Mahwah, NJ: Lawrence Erlbaum Associates. Kreuter, M. W., Buskirk, T. D., Holmes, K., Clark, E. M., Robinson, L., Si, X., Rath, S., . . . & Mathews, K. (2008). What makes cancer survivor stories work? An empirical study among African American women. Journal of Cancer Survivorship: Research and Practice, 2(1), 33–44. https://doi.org/10.1007/s11764-007-0041-y

44

Narrative or Expository Video Cases: Exploring the Influence of. . .

1151

Kreuter, M. W., Holmes, K., Alcaraz, K., Kalesan, B., Rath, S., Richert, M., McQueen, A., . . . & Clark, E. M. (2010). Comparing narrative and informational videos to increase mammography in low-income African American women. Patient Education and Counseling, 81 Suppl, S6–S14. https://doi.org/10.1016/j.pec.2010.09.008 McQueen, A., & Kreuter, M. W. (2010). Women’s cognitive and affective reactions to breast cancer survivor stories: A structural equation analysis. Patient Education and Counseling, 81(Suppl), S15–S21. https://doi.org/10.1016/j.pec.2010.08.015 Mezirow, J. (1990). How critical reflection triggers transformative learning. In Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning (pp. 1–20). San Francisco, CA: Jossey-Bass. Moss-Racusin, C. A., Pietri, E. S., Hennes, E. P., Dovidio, J. F., Brescoll, V. L., Roussos, G., & Handelsman, J. (2018). Reducing STEM gender bias with VIDS (video interventions for diversity in STEM). Journal of Experimental Psychology: Applied, 24(2), 236–260. https:// doi.org/10.1037/xap0000144 Phillips, B. J., & Mcquarrie, E. F. (2010). Narrative and persuasion in fashion advertising. Journal of Consumer Research, 37(3), 368–392. https://doi.org/10.1086/653087 Pietri, E. S., Moss-Racusin, C. A., Dovidio, J. F., Guha, D., Roussos, G., Brescoll, V. L., & Handelsman, J. (2017). Using video to increase gender bias literacy toward women in science. Psychology of Women Quarterly, 41(2), 175–196. https://doi.org/10.1177/0361684316674721 Romiszowski, A. J. (1981). Designing instructional systems: Decision making in course planning and curriculum design. New York, NY: Nichols Publishing. Schwartz, D. L. L., & Hartman, K. (2007). It is not television anymore: Designing digital video for learning and assessment. In Video research in the learning sciences (pp. 349–366). Hillsdale, NJ: Erlbaum. Retrieved from http://aaalab.stanford.edu/papers/Designed_Video_for_Learning.pdf Shen, F., Sheer, V. C., & Li, R. (2015). Impact of narratives on persuasion in health communication: A meta-analysis. Journal of Advertising, 44(2), 105–113. https://doi.org/10.1080/00913367. 2015.1018467 Slater, M. D., & Rouner, D. (2002). Entertainment-education and elaboration likelihood: Understanding the processing of narrative persuasion. Communication Theory, 12(2), 173–191. Stitt, C. R., & Nabi, R. L. (2011). The persuasive impact of narratives: A comparison across message types and modalities. In Paper presented at the annual meeting of the International Communication Association. New York, NY. Retrieved from http://www.allacademic.com/ meta/p13181_index.html Winkler, P., Janoušková, M., Kožený, J., Pasz, J., Mladá, K., Weissová, A., . . ., & Evans-Lacko, S. (2017). Short video interventions to reduce mental health stigma: A multi-Centre randomised controlled trial in nursing high schools. Social Psychiatry and Psychiatric Epidemiology, 52(12), 1549–1557. https://doi.org/10.1007/s00127-017-1449-y Woerkom, M. (2003). Critical reflection at work: Bridging individual and organisational learning. PhD thesis, Twente University. York-Barr, J., Sommers, W. A., Ghere, G. S., & Montie, J. J. (2001). In J. York-Barr, W. A. Sommers, G. S. Ghere, & J. Montie (Eds.), Reflective practice to improve schools: An action guide for educators. Thousand Oaks, CA: Corwin Press.

Jingbo Huang is the Director of the United Nations University Research Institute in Macau, which focuses its research at the intersection of digital technologies and sustainable development. In the past 20 years, Dr. Huang has been dedicating her professional career on learning, development, and research, and playing managerial roles in five United Nations organizations (United Nations Secretariat, United Nations Development Programme, United Nations Educational, Scientific and Cultural Organization, United Nations System Staff College, and United Nations University). Jingbo received her Doctor of Education degree in Communication, Computing and Technology in Education from Teachers College, Columbia University.

Learning Environments for Academics: Reintroducing Scientists to the Power of Creative Environment

45

Julia Figliotti, Maggie Dugan, and Donnalyn Roxey

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Creative Person . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Organizers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mentors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Facilitators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Creative Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . “Best” Practices: Or Are They? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Strategies to Overcome “Best” Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Creative Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Creative Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1154 1155 1156 1157 1157 1158 1160 1161 1161 1163 1165 1166 1167 1167

Abstract

Learning environments are relevant to more than just student content and education. In theory, any environment in which learning takes place, from a classroom to a kitchen, could be considered a learning environment. Likewise, any person who opens his or her mind to new knowledge, perspectives, and theories can be

J. Figliotti (*) Knowinnovation, Portland, OR, USA e-mail: julia.fi[email protected] M. Dugan Inclusive Innovation, Barcelona, Spain e-mail: [email protected] D. Roxey Knowinnovation, Columbus, OH, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_87

1153

1154

J. Figliotti et al.

considered a learner. In this chapter, the authors discuss the type of learning environment that is inherent in innovation workshops: the creative learning environment. Innovation workshops are commonly attended by scientists and educators, many of whom have been teaching and researching for so long that they may not normally consider themselves learners. This chapter features Mel Rhodes’ (1961) Four Ps of Creativity model, which breaks creativity down into the creative person, creative process, creative press (environment), and creative product. Using these four facets of creativity, the authors outline techniques and best practices to encourage creativity and creative learning environments for scientists and educators who attend innovation workshops. Keywords

Creative learning environment · Facilitation techniques · Group dynamics · Problem solving

Introduction When most people hear the phrase “learning environment,” they think of classrooms, laboratories, or lecture halls. Much of the research on the topic focuses on learning as a student-centered activity, usually within an educational establishment (see Lage, Platt, & Treglia, 2010; Lizzio, Wilson, & Simons, 2010; Mäkelä & Helfenstein, 2016). However, the authors believe that there is more to a learning environment than scholarly settings and student status alone. In fact, an often-overlooked contributor to a learning environment is the level of creativity present. A creative environment “provides a cognitive basis for idea generation and encourages the actions required for implementing these ideas. . . [and] demonstrates acceptance and recognition for the individual’s creative efforts” (Mumford & Gustafson, 1988, p. 37) and “supports the development, assimilation and utilization of new and different approaches and concepts” (Isaksen, Lauer, Ekvall, & Britz, 2001, p. 172). It is therefore less than surprising that creative environments can have tremendously powerful impacts on learning. Indeed, there is an entire field of literature surrounding creative learning environments, here defined as environments that encourage the education and practice of creative thinking skills, as discussed in the systematic literature review on the topic by Davies et al. (2013). At Knowinnovation, a company focused on accelerating scientific innovation, facilitators have been engineering creative learning environments around scientific and academic challenges for over a decade. The company first entered the world of scientific innovation workshops in 2003, when the UK’s Engineering and Physical Sciences Research Council (EPSRC) developed the concept of the Sandpit. The goal of a Sandpit is to inspire more innovative, interdisciplinary research proposals, to be considered for government funding. In the early 2000s, Knowinnovation began

45

Learning Environments for Academics: Reintroducing Scientists to the. . .

1155

running these workshops as a startup company. When the process was adapted by the National Science Foundation (NSF) in the United States, an alternate title was established: the Ideas Lab. Sandpits and Ideas Labs are designed to be 5-day events with 20–30 participants from different disciplines and institutions. These participants are brought together to generate potential research proposals around a designated, complex challenge. The ultimate goal is the creation of novel, interdisciplinary, and potentially high-risk research projects. The purpose of these workshops is ultimately to encourage and infuse creativity into a novel output that accelerates scientific innovation. In order to do this, the facilitating team must bring each participant from a “knower” mindset, where everything must be backed up with facts and data, to a “learner” mindset, where new ideas are explored with curiosity and judgment is deferred. It is only with the latter mindset – an openness to new thinking and a suspension of assumptions (Bohm, 2004; Senge, 2006) – that participants can fully experience the benefits of a creative learning environment. To encourage the learner mindset, it is integral to provide an open, trusting, and creative learning environment to organically draw out novel ideas and partnerships between introverts and extroverts, participants from different institutions, and individuals with diverse areas of expertise. Over the years, the organization has put together a compilation of internal documents, known as The Ideas Lab Organizer’s Secret Handbook (Dugan, 2016), which provides facilitators access to different tools, techniques, and tips for running a successful workshop. The process is grounded in Mel Rhodes’ (1961) germinal work, which is still a foundational work for practitioners in the field of creativity. Rhodes introduced the Four Ps Model of Creativity as a guide for how to cultivate creativity and a creative learning environment. This model is made up of the Creative Person, the Creative Process, the Creative Press (Environment), and the Creative Product (Fig. 1). While his work continues to be researched and new thoughts continue to emerge in this area (see Amabile, 2012; Puccio, Mance, & Murdock, 2011; and West & Richter, 2008), it is in part the conscious focus of the organization on building workshops around the Creative Person, Process, Environment, and Product that has allowed it to accelerate interdisciplinary scientific innovation beyond traditional classroom environments.

The Creative Person It takes more than an experienced company to run a workshop around innovative science and interdisciplinary collaborations. There is a balance of diverse scientific expertise which helps to build the vibrant community needed for workshop success. Our typical workshop includes organizers, a director, mentors, participants, and facilitators, all of whom should be willing to collaborate, and all of whom are vital to the establishment of the creative learning environment.

1156

J. Figliotti et al.

Person

Product

Process

Press (Environment): Physical Psychological (Rhodes, 1961)

Fig. 1 Mel Rhodes’ (1961) Four Ps of Creativity

In order for an innovation workshop to be successful, it is helpful if every person in attendance – from the organizers to the participants – has some key characteristics that make the creative learning environment that much stronger. The attributes include: • • • • •

Intellectual credibility and expertise in his or her field Impartiality and objectivity Strong interpersonal skills Low ego or the ability to self-manage one’s ego Capacity to tolerate ambiguity and to let the workshop process unfold

Organizers The primary role of the organizing committee is to substantiate the proposed grand challenge for the workshop. They have the financial ownership of any proposed solutions from the workshop through their funding of the participants’ proposed output. The organizers fund the workshop around this specific topic and recruit the facilitation team and other roles (see below). They then work with the facilitation team many months before the workshop even occurs to produce a call to action; decide on an ideal venue; choose the director, mentors, and participants; and make sure the planning process stays on track. The fact that funding may be awarded at the end of the event is part of what creates the energy and momentum of the innovation workshop experience. It is important – and useful – if one or two staff members from the funding organization are present. Having people to help with the administration and logistics, especially during the first day when there is much to do to kick off the event, is a delightful

45

Learning Environments for Academics: Reintroducing Scientists to the. . .

1157

bonus. During the week, it is good to have the funding organization represented to track the development of the projects and to answer any questions about criteria for funding or how funding will be distributed.

Director The director acts as the leader of the event. S/he is appointed by the organizers and serves as the scientific guide for the course of the workshop, managing scientific output expectations. The director sets the stage, clarifies the vision, and steers the participants to stay aligned with that vision over the course of the workshop. In addition to the characteristics that are beneficial to all workshop attendees, the specific attributes of a good workshop director include the following: • • • •

Expertise related to the workshop topic Credibility and contacts to attract good mentors and stakeholders Ability to be a peer reviewer, mentor/subject guide, and leader, all at once Enthusiasm about the workshop process and new science, and a willingness and ability to communicate that enthusiasm • Preparation to promote the event externally, to academia and industry, the press and media, and the public The director needs to instill a sense of ambition in the participants and be an enthusiastic supporter of the process. There can be difficult times during the workshop and the director needs to be willing to challenge the group to continue pushing for novel and exciting science. As well, s/he needs to be able to lead a disparate group of mentors, setting the tone as they interact with participants while always modeling the type of behavior that supports creative thinking. After the event the director remains engaged, reviewing final proposals and monitoring the development of the projects.

Mentors The mentors (also referred to as subject guides) are sometimes analogous to peer reviewers but with a much more creative role: to connect and catalyze. They should be a diverse group, providing expertise on various dimensions of the problem and sparking new thinking in the participants. At the start of the event, their job is to encourage new ideas by asking questions, highlighting ideas that seem exciting, and making connections between participants and to the wider body of knowledge. By the end of the event, the mentors might be asked to adopt a more critical perspective, in order for them to be able to assist in the funding recommendations. Mentors work with the director to provide objective advice and input to the participants, but they have a potential role selecting the participants and commenting

1158

J. Figliotti et al.

on the research projects that emerge from the workshop, with the overall aim to ensure that the event leads to high-quality innovative research. A mentor has many of the same characteristics as the director. Several of their unique qualities include: • • • • • •

Knowledge of some aspect(s) of the subject area Capacity to instruct and guide by asking provoking questions Ability to act as a peer reviewer and a subject guide Enthusiasm for the workshop topic and the event Willingness to collaborate with a facilitation team General desire to see the participants succeed in coming up with novel proposals

Mentors, like the director, need to have credibility with the participants. They should represent a range of different academic perspectives, be curious and ambitious about the science, and be very good at connecting and catalyzing. This is also critical: they have to stay neutral and avoid getting too involved in any of the ideas. During the workshop, their role is vital to the growth of many potential ideas.

Participants The deliberate thought behind the building of a workshop with a creative learning environment can only be fully realized through a group of participants. The participants are of such significance that much of the preworkshop effort is put into the selection of who will be in the room. Ideally, a participant at an innovation workshop should only know a handful of the other participants. This isn’t a weird wish to maintain secrecy, but rather is the key driver of the novel ideas, interactions, and conversations that make up the core of innovation workshops. For this to happen, recruiting with diversity in mind is essential. At the same time, in order for people to have a conversation around, for example, photosynthesis, it is necessary for everyone to understand enough about the subject to engage in the discussions. Because of the delicate balance required between interdisciplinarity and a shared understanding of content, each workshop provides its own unique learning experience. Usually, in the midst of the different perspectives colliding around a complex question, it becomes apparent which additional perspectives and disciplines might be useful in augmenting the challenge and generating ideas. The interdisciplinarity inherent in these workshops makes room for that insight to take place (Cooke & Hilton, 2015; Pellmar & Eisenberg, 2000; Stokols, Misra, Moser, Hall, & Taylor, 2008): nothing encourages experts to suspend judgment, abandon assumptions, and open their minds to new thinking quite like being surrounded by individuals with separate yet relevant areas of expertise. The introduction of deeper, critical knowledge from an outside field to the problem at hand serves as a reminder that even experts do not always have all of the answers – sometimes it takes a question or suggestion from a separate domain to add the final, necessary piece to the puzzle.

45

Learning Environments for Academics: Reintroducing Scientists to the. . .

1159

Once the workshop is complete, groups will have a better understanding of who they wish to work with, and what additional areas of expertise are needed for their project proposals to become reality. Given the knowledge that certain domains of expertise will be inadvertently omitted, and keeping in mind the delicate balance of nonoverlapping deep content expertise, how might an innovation workshop boast an appropriate range of disciplines in recruitment? A brief literature review can identify all of the disciplines that have explored the topic. By simply discussing the challenge with program officers from different divisions, agencies, and institutions and attempting to frame the solicitation in a way which attracts participants from these disciplines, a sufficient and necessary diversity (Ashby, 1956; Bassett-Jones, 2005; Wang, Fussell, & Cosley, 2011) can be reached to achieve innovative products and research. The tricky part is to recruit from disciplines apart from the “usual suspects,” and a large part of that challenge is simply making potential attendees aware of the solicitation. The people who attend an innovation workshop go through an application process that assesses their skills, expertise, and capacity for collaboration. Depending on the subject, a diverse range of academics might be invited to apply: physical scientists, engineers, biologists and chemists, mathematicians, statisticians, physicists, computer scientists, social scientists, architects and artists, musicians, and medical specialists. The more diversity, the more novel the ideas are likely to be (Bassett-Jones, 2005; Hoever, van Knippenberg, van Ginkel, & Barkema, 2012; Milliken, Bartel, & Kurtzberg, 2003; Wang et al., 2011). When participants have to translate the terminology (and jargon) of their own fields to others, new connections occur and this is often a major driver toward novelty and innovation. An innovation workshop demands a lot from the participants. They are asked to keep open minds, meet, and connect in a short period of time with new people from different fields, and quickly form teams and solve problems together. The participants will ultimately be competing with these same newfound peers for funding. It can be a stressful experience, so collecting the right kinds of participants is key. Participants should be: • • • • • • •

Generally curious, and willing to explore new or different aspects of a question Collaborative and ready to work in teams Open to new ideas and new ways of thinking and working Intrigued by the innovation workshop process Willing to take responsibility for their own level of participation Capable of contributing constructively to their own teams and to others’ Good sports: in the end, happy to have participated, whether funded or not

Because the stakes of these workshops are so high, it can be invaluable to bring in an outside organizational psychologist – Bharat Maldé – to review the applications. The innovation workshop is not for everyone, and Bharat has a unique talent of being able to read the applications and consistently assess each participant’s likelihood to contribute productively to the event. It’s not that everyone has to get along; it

1160

J. Figliotti et al.

can be useful to have a few participants who create some friction, who enjoy challenging assumptions – if they’re capable of doing so in a positive way. Managing difficult people steals time and distracts the participants. The truth is that without the presence of grand-standers (Senge, 2006) and people unwilling to work in teams, the innovation workshop can yield much stronger results with a lot less stress. Even when it is not possible to involve an organizational psychologist, it is incredibly helpful for organizers to consider this perspective during the selection process.

Facilitators While the director and mentors are responsible for the content of the workshop, and the participants are charged with creating output, the facilitators are responsible for the process and work hard at establishing and maintaining the creative learning environment. Specifically, our facilitators: • • • • • • • • •

Develop the agenda and overall process design Schedule activities and exercises Create a climate that is conducive to creative thinking and teamwork Hold true to the context of the meeting (Bohm, 2004; Senge, 2006) Model the expected behavior at all times to encourage the flow of thinking and novel ideas (Bohm, 2004; Senge, 2006) Provoke people to stretch their thinking and learning Guide the mentors and the director in their role overseeing the content Attend equally to participant needs, whether they be resource-, emotional-, or process-based Frame the entire workshop so participants understand the flow of the event

The facilitators work closely with the director, the mentors, and the funders to design the schedule. At the workshop itself, they take the lead, explaining the process and directing the activities. In addition, they will facilitate smaller groups as needed and in general keep an eye on how the workshop is advancing in terms of the process and how the participants are feeling and reacting. The facilitation team has to be extremely flexible, often making decisions very quickly to alter the agenda in response to the progress of the group and the observations of the mentors and the director. This ensures the creative learning environment is maintained. The participants are the main focus of every one of our workshops. The director, mentors, and facilitators all share one common role: to help support the participants throughout the week, and to sustain the creative environment that has been established. The goal is to take a room full of knowers – participants with knowledge and experience in different areas – and transform them into learners – open-minded individuals who actively seek out partnerships and connections with individuals from different disciplines, fields, or perspectives.

45

Learning Environments for Academics: Reintroducing Scientists to the. . .

1161

Fig. 2 General structure of the authors’ workshops

The Creative Process As previously outlined, our facilitators are process experts who work together with content experts in order to provide the best possible experience for each and every attendee. The workshop itself is never directed, but instead is guided by the facilitators, mentors, director, and funders. Though no two workshops are identical, most follow a similar structure (Fig. 2). One of the most notable aspects of the process that the facilitators employ is that, although it draws parallels between certain aspects of a variety of creative problemsolving processes such as Creative Problem Solving (CPS, Isaksen & Treffinger, 1985; Osborn, 1953; Parnes, 1981), design thinking (Simon, 1969), lateral thinking (de Bono, 1970), and brainstorming (Osborn, 1953), it is ultimately unique to our organization. As practitioners, we have developed a methodology framework that we have found to be successful based on concrete experience and qualitative observation from over 50 workshops. In the 12 years since the organization has been running workshops, facilitators have interacted with participants of all sorts of behaviors. Though the facilitators encounter their fair share of personality traits and quirks, it is important to recognize that many of these observed behaviors are simply human nature – and not at all conducive to innovative thinking. While the facilitation team tries to encourage an environment of openness, learning, and somewhat engineered serendipity, human nature can sometimes get in the way. However, in response to these human behaviors that have been known to impede creative thinking, several process-oriented strategies were developed to overcome them.

“Best” Practices: Or Are They? Establishing a Routine Human beings are creatures of habit. Doctors stress the importance of healthy routines, and businessmen can predict the stock market based on past activity. So it comes as no surprise that participants adhere to the routines they had prior to the workshop and whatever routines they can create while at the event. Judging by the

1162

J. Figliotti et al.

workshops we have run over the past decade, when presented with open seating at a multiday workshop, most people choose to sit in the same chair, at the same table, every day, and speak primarily with colleagues of shared interests. It is simply habit and human nature to seek out one’s niche.

Making Sure It’s Perfect It’s against everything people have been taught to present an idea that hasn’t been worked all the way through. Everyone has dealt with their fair share of rejection for ideas they thought were golden. Because of that, many people are tempted to work in seclusion until their proposal, prototype, or product is simply perfect. Only then will they show their finished work to anyone else. Looking for the Right Answer Similarly, it can be confusing when participants are told that they shouldn’t be looking for the “right” answer to a problem. By default, humans will strive for a solution that will solve every aspect of a challenge. Typically, when trying to solve a problem, people will try to come up with the one best idea and act on it. Or even if they do come up with a few suggestions, there’s often an urge to select solution quickly. “Quality over quantity” is a phrase that everyone has heard, and that many people take to heart. However, the high-quality (yet low-risk) answer is not always the best answer when innovation is the end goal. Bringing Back the “Best” Idea When a challenge is posed, many people automatically search their memories for a similar problem they have faced in the past. From there, they begin to apply prevailing ideas, technologies, or practices to solve a new challenge. While it may seem logical, and in fact often is, this manner of thinking – applying a preformulated, tested-and-true idea to a brand new challenge – will not typically lead to the types of innovative concepts and projects sought through one of our workshops. Linear Thinking When facing a new challenge, it is common practice to cling to the first great solution that appears. After all, why would one continue looking for answers when the present solution will clearly work? However, thinking in depth about one solution will rarely produce the level of novelty and innovation that can lead to fundamental paradigm shifts, transformational learning, or new applications. Instead, our team encourages participants to think laterally, which encourages problem-solving and motivation by using nonobvious reasoning to reach conclusions that would be difficult to obtain by following traditional logic (de Bono, 1970). Capturing Only the Final Thought It is not uncommon for people – in meetings, in classrooms, in laboratories – to discuss a project in depth and write down the final, end result. Workshop participants are rarely different. However, capturing only the final thought, product, or outcome

45

Learning Environments for Academics: Reintroducing Scientists to the. . .

1163

denies any potential offshoots or new thinking that might be found in the planning stages.

Strategies to Overcome “Best” Practices Human nature – curiosity, ambition, passion – is a beautiful thing. Curiosity drives us to discover and invent, ambition opens us up to worlds of possibilities, and passion keeps us going even when things seem impossible. But our human behaviors can also get in the way of our true potentials. To know innovation, we must first know human nature. By meeting potentially inhibiting behaviors head-on, we are opening ourselves up to something even greater: the novelty that comes with letting go of subconscious, self-imposed restrictions. To overcome the aforementioned “best” practices requires a conscious effort, and it is integral in order to achieve a creative learning environment.

Breaking Routines We believe that in order to achieve novel output in such a short timeframe, participants must be forced out of their routines. Seating arrangements at these events are scrambled as often as possible, usually following participant breaks or excursions. Keeping people moving, changing their environments, and constantly introducing them to new collaborators with different disciplines and levels of professional experience can lead to participant tolerance for new or even strange ideas. It encourages a level of open-mindedness that many are not used to experiencing, and one that many are surprised to enjoy. Draft Sharing Encouraging participants to present their ideas early and throughout the development process in order to receive continuous feedback is one of the greatest gifts that a facilitating team can give at a workshop. This process forces groups to share their progress often and allows other participants to give feedback that may not have been considered using an evaluation tool called the PPCo: Pluses, Potentials, Concerns, and overcomes (PPCo is an affirmative judgment tool developed in the 1980s by Dr. Roger L. Firestien, Dr. Diane Foucar-Szocki and Bill Shepard). By creating a safe environment where ideas are accepted, and using a feedback tool that encourages growth and development, projects are more likely to flourish. Looking for Many Possible Answers The purpose of innovation workshops is to find new and useful solutions to difficult, risky, systemic, or present-day challenges, or those still on the horizon. It is important to remind participants that they are looking for all the possible interesting and unique options that surround a challenge – not just the “right” ones. Once a good number of potential ideas have been thought up, participants can choose a handful of intriguing options to develop. In theory, the more ideas, questions, and opportunities

1164

J. Figliotti et al.

that participants produce, the more likely it is that some of them will be novel and innovative. This theory is known as the third third (Hurson, 2008). The first ideas that people generate tend to be the habitual ones, ideas that have come to mind before. It’s almost as if one must evacuate these initial ideas in order to make room for new ideas to emerge. The next ideas usually have a quality of novelty that the initial ones won’t have. It’s easy to stop there, but our facilitators like to create opportunities to take the group to the third third of ideas. Sometimes this requires using idea generation techniques and detours to help people access this last category, but it is from this pool of ideas that the most novelty often appears. If the first third of the ideas are known to us and the second third are new to us, the third third of ideas are likely to be new to the world and have the potential to be unique and demonstrably innovative (Hurson, 2008; Parnes, 1987; Parnes & Noller, 1972; Puccio et al., 2011; Wang & Horng, 2002).

Developing Brand New Solutions Instead of looking for existing work that might perfectly apply to the problem, participants are encouraged to develop new solutions. Being novel, the ideas may require additional development, but they’re more likely to be breakthrough solutions that showcase real innovation. This is reflected in the aforementioned theory of the third thirds. Lateral Thinking In a creative environment, it is integral to encourage participants to think laterally. Keeping open to new options can be incredibly difficult, especially when one option seems to be the perfect solution to the challenge at hand. Facilitators should frequently remind participants to avoid premature closure and continue producing and considering alternatives, even if a probable solution has been found. One way to encourage lateral thinking is to challenge participants to go work on different projects over the course of a day, lend their expertise to other ideas, and visit different thinking. By the end of the day, participants can gravitate to whichever projects most intrigue them, bringing new thinking that they may not have considered had they not worked on different ideas. Capturing the Entire Process There is a great importance in capturing thoughts along the way. Throughout each of the workshops, participants are encouraged to write down ideas, questions, and challenges as they come up in their many group work and feedback sessions. These notes can be used to supplement the final product and even encourage new paths to take once the end result is reached. An important part of the process – and of capturing output along the way – is the mobility of ideas. For that reason, participants are encouraged to capture their thoughts on Post-it® notes so that as ideas, projects, and solutions emerge, they can be adjusted and regrouped efficiently as needed (Fig. 3). In this way, the Post-it ® notes become Portable Recording Devices, helping to encourage participants to

45

Learning Environments for Academics: Reintroducing Scientists to the. . .

1165

Fig. 3 Portable recording devices – integral to the authors’ creative process

capture thoughts, questions, and ideas at any point (and at any place) during the workshop.

The Creative Environment Creativity has a key role in science, which is something that is often forgotten in the midst of experiments and deadlines. Our team has been able to engineer the workshop environment in a way that supports creativity and still works for the highly intelligent and somewhat skeptical individuals in attendance. As in Rhodes’ (1961) model, it is the creative person, process, and product that are at the core of the creative environment. Previously outlined are the organization’s approaches to the creative person and process, but what is it that sets the creative learning environment apart from other learning environments? This often begins before the workshop even takes place. Many organizers ask the facilitation team to set up an online Hub group for their event (see HUBzero, http:// hubzero.org/). Participants are invited to join the group, which has been populated with resources, information, and stimulating questions to get them into a creative learner mindset. This primes the participants in two ways: by introducing and engaging them prior to the workshop, and by provoking potentially new ways of thinking that encourage open-mindedness. From there, the creative learning environment only grows. For example, on the first day of the workshop, during registration, participants are asked to prepare name tags for themselves. Instead of premade name badges that feature full names and institutions, participants are provided with blank tags and colorful markers. They are given the opportunity immediately to break the mold of a “normal” conference by being asked to write out what they wish to be called throughout the event, and design it in a way that they like. Though it may not seem like a great leap, this colorful introduction frames the rest of the week as one where the possibilities are limitless.

1166

J. Figliotti et al.

Fig. 4 Small toys to help keep participants focused

When they enter the plenary room, participants are greeted with tables that seat up to six. It is important to note that the tables are round (never in rows), which compels participants to look at and get to know one another. This breaks the routine of a regular classroom environment and encourages discussion throughout the event. The tables are stocked with modeling clay, 3-D puzzles, and small toys, all of which help participants deal with cognitive overload by giving them a way to expend some mental and physical energy. These props also give tactile learners a means of movement, therefore further engaging them in the workshop (Fig. 4). The facilitation team strives to build a creative environment within each workshop by bringing a level of comfort, humor, and openness to the participants. In order to do so, the facilitators must share roles relatively interchangeably: making suggestions for activities and prompts, keeping an eye on participants’ reactions and emotions, and event documentation, to name a few. These roles aren’t necessarily assigned, but are instead fluid. This facilitation style models the type of collaboration that participants must exhibit in order to effectively function within their groups throughout the workshop. By modeling and expecting collaboration and openmindedness to this degree, the facilitation team is creating and supporting a creative learning environment.

The Creative Product Why do the facilitators do what they do? After all, people solve problems and invent solutions without the assistance of facilitators every single day. The fact of the matter is that some challenges require more creativity than others. When problems are objectively difficult and require risky, unique, interdisciplinary, and innovative solutions, we get involved. Often times these are situations where the current field is stuck or making very slow progress, or where new opportunities have come to light (either through technological advancements or changes in policy) which offer a range of potential futures with no clear guidelines as to the best approach. In these

45

Learning Environments for Academics: Reintroducing Scientists to the. . .

1167

situations, the goal of innovation workshops is to produce novel and high-risk ideas and projects for potential solutions. The company’s focus is on the people in the room, the process of developing innovative solutions, and the environments that allow for creative thinking and outputs to thrive – and together, that can all lead to a viable, robust, and creative product.

Conclusion The authors of this chapter are not experts in the content of the workshops that they facilitate. In fact, it is the very removal from the content that makes this process so fluid. By focusing on the process of eliciting and encouraging innovative and novel ideas, each workshop run by the facilitators is able to host an environment that encourages a learner mindset in the true content experts: the participants. The facilitators employ a tried and true combination of diverse creative problemsolving practices and experiential instinct to support the creative learning environment best suited to each innovation workshop. As most events are unique from one another, so too is each creative learning environment unique to its respective workshop. For this reason, facilitators must be constantly open to adapting the methodology to best suit the people in the room and, of course, the clients’ desired outcomes. While the creation of the creative learning environment may be lauded as the reason for success in many innovation workshops, it is simply a key. Facilitators at such an event are only as successful as their counterparts allow. While an open environment and a learner mindset are encouraged, it is the talent, knowledge, and innovation already present – in the director, the organizers, the mentors, and the participants – that makes a true learning environment. In short, and to once again reference Rhodes (1961), it is the creative process, people, and environment working synchronously toward a novel, creative product that is the hallmark of the successful innovation workshop.

References Amabile, T. M. (2012). Componential theory of creativity (No. 12–096). Boston, MA: Harvard Business School. Ashby, W. R. (1956). An introduction to cybernetics. London, UK: Chapman & Hall. Bassett-Jones, N. (2005). The paradox of diversity management, creativity and innovation. Creativity and Innovation Management, 14(2), 169–175. Bohm, D. (2004). On dialogue (2nd ed.). New York, NY: Routledge. Cooke, N. J., & Hilton, M. L. (Eds.). (2015). Enhancing the effectiveness of team science. Washington, DC: The National Academies Press. Davies, D., Jindal-Snape, D., Collier, C., Digby, R., Hay, P., & Howe, A. (2013). Creative learning environments in education: A systematic literature review. Thinking Skills and Creativity, 8, 80–91.

1168

J. Figliotti et al.

de Bono, E. (1970). Lateral thinking: Creativity step by step. New York, NY: Harper Perennial. Dugan, M. (2016). The ideas lab organizer’s secret handbook (2nd.). (Unpublished internal document, Knowinnovation). Hoever, I. J., van Knippenberg, D., van Ginkel, W. P., & Barkema, H. G. (2012). Fostering team creativity: Perspective taking as key to unlocking diversity’s potential. Journal of Applied Psychology, 97(5), 982–996. Hurson, T. (2008). Think better: An innovator’s guide to productive thinking. New York, NY: McGraw-Hill. Isaksen, S. G., & Treffinger, D. J. (1985). Creative problem solving: The basic course. Buffalo, NY: Bearly Limited. Isaksen, S. G., Lauer, K. J., Ekvall, G., & Britz, A. (2001). Perceptions of the best and worst climates for creativity: Preliminary validation evidence for the situational outlook questionnaire. Creativity Research Journal, 13(2), 171–184. Lage, M. J., Platt, G. J., & Treglia, M. (2010). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education, 31(1), 30–43. Lizzio, A., Wilson, K., & Simons, R. (2010). University students’ perceptions of the learning environment and academic outcomes: Implications for theory and practice. Studies in Higher Education, 27(1), 27–52. Mäkelä, T., & Helfenstein, S. (2016). Developing a conceptual framework for participatory design of psychosocial and physical learning environments. Learning Environments Research, 19(3), 411–440. Milliken, F. J., Bartel, C. A., & Kurtzberg, T. R. (2003). Diversity and creativity in work groups: A dynamic perspective on the affective and cognitive processes that link diversity and performance. In P. B. Paulus, B. A. Nijstad, & B. A. Nijstad (Eds.), Group creativity (pp. 32–62). Bethesda, MD: Oxford University Press. Mumford, M. D., & Gustafson, S. B. (1988). Creativity syndrome: Integration, application, and innovation. Psychological Bulletin, 103, 27–43. Osborn, A. F. (1953). Applied imagination: Principles and procedures of creative problem-solving. New York, NY: Scribners. Parnes, S. J. (1981). The magic of your mind. Buffalo, NY: Bearly Limited. Parnes, S. J. (1987). The creative studies project. In S. G. Isaksen (Ed.), Frontiers of creativity research: Beyond the basics (pp. 156–188). Buffalo, NY: Bearly Limited. Parnes, S. J., & Noller, R. B. (1972). Applied creativity: The creative studies project: Part II – Results of the two-year program. Journal of Creative Behavior, 6, 164–186. Pellmar, T. C., & Eisenberg, L. (Eds.). (2000). Bridging disciplines in the brain, behavioral, and clinical sciences. Washington, DC: National Academies Press. Puccio, G. J., Mance, M., & Murdock, M. C. (2011). Creative leadership: Skills that drive change (2nd ed.). Thousand Oaks, CA: SAGE. Rhodes, M. (1961). An analysis of creativity. Phi Delta Kappan, 42(7), 305–310. Senge, P. (2006). The fifth discipline: The art & practice of the learning organization (2nd ed.). New York, NY: Doubleday. Simon, H. A. (1969). The sciences of the artificial. Cambridge, MA: MIT Press. Stokols, D., Misra, S., Moser, R. P., Hall, K. L., & Taylor, B. K. (2008). The ecology of team science: Understanding contextual influences on transdisciplinary collaboration. American Journal of Preventive Medicine, 35(2), S96–S115. Wang, C., & Horng, Y. (2002). The effects of creative problem solving training on creativity, cognitive type and R&D performance. R&D Management, 32(1), 35–45. Wang, H. C., Fussell, S. R., & Cosley, D. (2011, March). From diversity to creativity: Stimulating group brainstorming with cultural differences and conversationally-retrieved pictures. In Proceedings of the ACM 2011 conference on computer supported cooperative work (pp. 265–274). ACM.

45

Learning Environments for Academics: Reintroducing Scientists to the. . .

1169

West, M. A., & Richter, A. (2008). Climates and cultures for innovation and creativity at work. In J. Zhou & C. E. Shalley (Eds.), Handbook of organizational creativity (pp. 211–236). New York, NY: Lawrence Erbaum Associates.

Julia Figliotti is a Creative Specialist at Knowinnovation with a Bachelor of Arts in Writing and a Master of Science in Creativity. She has facilitated and provided technical assistance for numerous Ideas Labs, Innovation Labs, and other innovation workshops. Julia codesigned and currently manages a Massive Open Online Course (MOOC) titled Ignite Your Everyday Creativity. She has worked directly with organizations such as the National Science Foundation (NSF), NASA, the United Nations Population Fund (UNFPA), and the National Institutes of Health (NIH), and is a certified FourSight facilitator. Julia’s academic work has been published with The Partnership for twenty-first Century Skills (P21), and the Big Questions in Creativity book series. She is coauthor of the educational books Weaving Creativity into Every Strand of Your Curriculum and What Works: Weaving Mentoring into Teaching, Research, and Service. Maggie Dugan is a facilitator and a trainer; her specialty is designing and facilitating events that provoke innovation: creativity workshops, leadership development programs, and helping scientists develop more innovative research proposals. She’s part of the facilitation team at Knowinnovation (KI), and head of its newest division, Inclusive Innovation, which uses the same methodology to activate innovation within the field of economic development. Maggie is a member of the community of practitioners that grew out of the Creative Education Foundation (CEF) in Buffalo, NY, USA. She received the CEF’s Distinguished Leader Award and has been a leader/presenter at creativity conferences worldwide, including the Creative Problem Solving Institute (CPSI) in the United States, the European Creativity Association conference (CREA) in Italy, European Association for Creativity and Innovation (EACI), the South African Creativity Conference, Creativity Istanbul, and Mindcamp in Toronto, Canada. Donnalyn Roxey received her Bachelor of Science in Biological Sciences from the University of Maryland, College Park. After graduation, she began a career in research administration at The Ohio State University. She has served as a sponsored program officer, clinical trial contracts, and budget analyst, as well as a senior grants manager within Ohio State’s College of Medicine. Most recently at Ohio State, she led the administrative side of two Ph.D. graduate programs and a multicollege, interdisciplinary center, The Center for Applied Plant Sciences (CAPS). Donnalyn’s passion for creativity, coupled with her long-standing involvement in biological research, led to her focusing her energy on the science of team science. She spent 5 years in CAPS working closely with teams of scientists and was fascinated by how research teams are formed, operated, and supported and recognized, and how they can work even more effectively. After 10 years in academic administration, Donnalyn left the university to join Knowinnovation and pursue her passion in facilitating scientific innovation. She is currently finishing her Master of Science in Creative Studies from SUNY Buffalo State.

Section V Transformative Learning

Transformative Learning: A Section Introduction

46

Kaushal Kumar Bhagat, C. Halupa, Demetria L. Ennis-Cole, Princess M. Cullum, and Konrad Morgan

Abstract

At a time when digital technologies and techniques are subjecting every aspect of our lives to increasing levels of uncertainty, the relevance of transformational learning to adult education has never been greater. The pioneering work in the field by Mezirow (Transformative dimensions of adult learning. Jossey-Bass, San Francisco, 1991), Boyd (Personal Transformation in small groups. Routledge, London, 1991), and Cranton (Understanding and promoting transformative learning: a guide for educators of adults. Jossey-Bass, San Francisco, 1994, Transformative learning in action: insights from practice. Jossey-Bass, San Francisco, 1997) has provided educators with a framework that can enable learners to experience genuine transformations of their understanding toward many of the core assumptions that had underpinned their lives. Therefore, this section overviews transformative learning from theory to practice. K. K. Bhagat (*) Advanced Technology Development Centre, Indian Institute of Technology Kharagpur, Kharagpur, India e-mail: [email protected] C. Halupa A.T. Still University, Kirksville, MO, USA e-mail: [email protected] D. L. Ennis-Cole Department of Learning Technologies, College of Information, University of North Texas, Denton, TX, USA e-mail: [email protected] P. M. Cullum Cancer Treatment Center of America, Newnan, GA, USA e-mail: [email protected] K. Morgan Eduvate, Havant, UK © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_132

1173

1174

K. K. Bhagat et al.

Keywords

Transformative Learning · Adult Education · Transformative learning theory · Instructional practices · Digital technology

This section provides examples of how transformative Learning can be applied to different adult learning contexts. Each chapter provides a different perspective, showing how meaning structures can be understood and developed through some directed reflective practices, such as reflection on the content of a problem, the process of problem-solving, or the premise of the problem itself. All can lead to improved domain learning and a transformation in a learner’s understanding of self, which can dramatically change lives for the better. The section commences with a chapter from one of the pioneers of transformative Learning, Patricia Cranton, who described her experience in helping develop the field and seeing it evolve over the past decades. It was an enormous shock to everyone preparing this section that Dr. Cranton tragically passed away shortly after completing her submission. Her contribution to the field of education was significant, and she will be greatly missed by those who had the privilege of working with her. We are therefore honored to have one of Patricia’s final contributions included with this collection. ▶ Chap. 48, “Preparing for High-Tech Jobs: Instructional Practices, Adults with Autism Spectrum Disorders (ASD), and Video Game Design,” by Demetria Ennis-Cole and Princess Cullum. It provides a highly relevant example of how Transformative Learning can be applied to the specialized curricula for adult learners with Autism Spectrum Disorders (ASD) in training for the challenging domain of video game design. ▶ Chap. 49, “Are Students and Faculty Ready for Transformative Learning?,” by Colleen Halupa describes the background to applying transformative learning within the context of faculty development and provides valuable insights into how such innovative curriculum change can transform both educators and students. ▶ Chap. 50, “Clicker Interventions in Large Lectures in Higher Education,” gives an overview of research on clicker interventions in lectures in higher education and discusses how such interventions can provide students and lecturers with formative feedback. In ▶ Chap. 51, “Transformative Learning in an Online Doctoral Programme: Autoethnography as a Pedagogical Method,” Kyungmee Lee discusses “autoethnography” as a pedagogical approach to transformative learning experiences of online doctoral students. ▶ Chap. 52, “Operationalizing Transformative Learning: A Case Study Demonstrating Replicability and Scaling,” by Jeff King and Brenton Wimmer shared a case study about the impact of an initiative, Student Transformative Learning Record (STLR), on students’ academic performance at The University of Central Oklahoma. ▶ Chap. 53, “Transformative Learning and the Affordance of Flexible Habits of Mind,” by Michelle L. Maiese describes that transformative learning depends on educational environments and pedagogies that leads to the development of flexible habits such as openness, empathy, curiosity, and imagination.

46

Transformative Learning: A Section Introduction

1175

In ▶ Chap. 54, “Achieving Education for Sustainable Development (ESD) in Early Childhood Education Through Critical Reflection in Transformative Learning,” the authors explore critical reflection’s role in examining the potential of transformative learning in a capacity-building project. ▶ Chap. 55, “Transformative Experience: A Critical Review and Investigation of Individual Factors,” presents a review that focuses on how individual factors such as emotions, task values, dispositions, and personality traits affect the transformative learning experience. ▶ Chap. 56, “Transformative Learning for Sustainability in a Business School Through the Analysis of Students’ Critical Reflection,” presented an example of transformative learning in a business school. This study analyzed the students’ critical reflection in transformative learning for sustainability. Finally, the section closes with an overview of transformative learning theory within the context of educational technology.

Transformative Learning: A Narrative

47

Patricia A. Cranton

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . In the Beginning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Next Developments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transformative Learning: A Comprehensive Theory of Adult Education . . . . . . . . . . . . . . . . . . . . Expansion of Transformative Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fragmentation of Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Influence of the Journal of Transformative Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Influence of the International Conference on Transformative Learning . . . . . . . . . . . . . . . . . How Will This Narrative Continue? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1178 1178 1180 1180 1182 1183 1185 1187 1187 1188

Abstract

In this chapter, I present the story of the development of transformative learning theory, a story that unfolds over 40 years. Stimulated by his wife Edee’s experience, the story begins with Jack Mezirow’s early research on women’s reentry into college. This led to his conceptualization of perspective transformation. By 1991, he had developed a comprehensive theory of adult learning based on transformation. Scholars critiqued Mezirow’ work on a variety of bases: it was too cognitive, it neglected social change, and it was based on a misinterpretation of Habermas’s theory. Over time, these critiques led to alternative perspectives, and in turn, these alternatives created a problematic fragmentation of the theory of transformative

Patricia A. Cranton died in 2002. P. A. Cranton (*) Adult Education, University of New Brunswick, Fredericton, PA, Canada e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_37

1177

1178

P. A. Cranton

learning. I trace the influences of the International Transformative Learning Conference and the Journal of Transformative Education in this narrative. Keywords

Narrative, perspective transformation · Transformative learning theory · Critiques of transformative learning theory · International Transformative Learning Conference · Journal of Transformative Education

Introduction In this chapter, I tell the story of transformative learning: how it began and how it developed into a comprehensive theory of adult learning. I explore the troubled and sometimes problematic growing pains of the theory and how scholars responded. I am especially interested in how the fragmentation of the theory occurred as a result of the development of alternative perspectives. In some cases, this seems to have led to a loss of meaning, with scholars even calling for an abandonment of the theory. The Journal of Transformative Education had a strong influence on the field, but it also exacerbated the fragmentation by creating a less-than-clear distinction between transformative learning and transformative education. From here, I turn to thoughts on transformative learning and transformative education, and then to the future of transformative learning theory, including research goals for theory development.

In the Beginning In the early 1970s, Edee Mezirow returned to college. This was the time of what we now call the “second wave” of feminism. The first women’s study program was established in 1970. Betty Friedan had published The Feminine Mystique in the 1960s; Kate Millet published Sexual Politics in 1970. The “women’s liberation movement” was changing women’s lives. Edee was part of that movement. Unfortunately we do not know a great deal about her life, especially through the literature. I feel very fortunate to have known her and talked to her about her experiences. She was intrigued by the events her classmates discussed in relation to their roles as wives and mothers. And, of course, she discussed her insights with her husband, Jack Mezirow. Jack Mezirow (1991) writes: About the same time [the early 1970s], Edee, my wife, decided to return to college to complete her education after several years away from formal schooling. Interested as I was in attempting to understand both her and adult learning, I found her dramatically transformative experience, which led to a new career and lifestyle, both fascinating and enlightening. Her

47

Transformative Learning: A Narrative

1179

experience influenced my decision to undertake an ambitious national study of women returning to college and the workforce (Mezirow, 1978) . . . out of which evolved my earliest concept of perspective transformation. (p. xvii)

In his exploration of the experiences of women returning to college, Mezirow encountered the writings of Paulo Freire and Ivan Illich, and he had the opportunity to work with the psychiatrist, Roger Gould (Mezirow, 1991, pp. xvi-xvii). These experiences shaped the development of his theory of transformative learning. Mezirow’s research plan was based on a grounded theory methodology (with a goal of creating a description of transformative learning inductively) (Mezirow, 1978, p. 56). There were 12 programs from across the United States involved in his research. The sample was diverse in terms of class, urban/rural locations, and ethnicity (p. 56). From these data, Mezirow developed his now well-known patterns (although they have changed somewhat over the years) of reentry for women going back to college: “(1) disorienting dilemma; (2) self-examination, (3) a critical assessment of sex-role assumptions and a sense of alienation from taken-for-granted social roles and expectations; (4) relating one’s discontent to a current public issue; (5) exploring options for new ways of living; (6) building competence and selfconfidence in new roles; (7) planning a course of action and acquiring knowledge and skills for implementing one’s plans; (9) [sic] provisional efforts to try new roles; and (10) a reintegration into society on the basis of conditions dictated by the new perspective” (Mezirow, p. 12). In the 1978 report, the patterns are illustrated with helpful quotations, most of which refer to the women’s movement in some way. Here are some of the voices of the women in Mezirow’s original study: It is true that the very ways we find to conceptualize experience are in large measure given us by the culture in which we learn “how to think and feel” or even learn what thinking of feeling are. But people are also continually straining against the boundaries given by that culture—and seeking the means to understand and express the many experiences for which it does not suffice. This is true of all people. For women today it is a pre-eminent factor. (p. 12) I’m seeing a lot of uptight women going to school who are pushing themselves to the umpteenth degree. I hear them say, “I’m going to shove hot dogs into the kids tonight, I’m going to take tranquilizers and lock the door and get this paper done.” I see lots of pressure. (p. 13) I had all the old-fashioned ideas, and I found some of the new idea difficult to follow through on. I took the program to pick a career and go to work. But I didn’t. Instead, I found that I didn’t know myself, didn’t know the times—women’s lib—and I just couldn’t adjust to it. You can’t just turn off a lifetime of doing things one way. The most important thing I got out of the course was to look at myself. (p. 16)

Based on the data collected in this original study, Mezirow (1978) defines “perspective transformation as the process by which adults come to recognize culturally induced dependency roles and relationships and take action to overcome them” (p. 17). Everyone loves to criticize a new theory, and the critiques came in, especially after Mezirow (1991) published his comprehensive theory of transformative learning, Transformative Dimensions of Adult Learning. I’ll return to this later, but it was interesting that the critics ignored Mezirow’s (1978) publication and the explicit way in which it was embedded in the women’s movement of the time.

1180

P. A. Cranton

The Next Developments Mezirow began to explore transformative learning theory from the perspective of J€ urgen Habermas, a German social philosopher. Habermas (1971) describes three kinds of knowledge: technical, practical, and emancipatory. Technical knowledge is the kind of knowledge that we use to control and manipulate our environment (building highways and buildings, creating new technology); practical knowledge is about how we understand ourselves, others, and the social context within which we live; emancipatory knowledge is related to how we work to become free from constraints. Mezirow renamed technical knowledge as instrumental knowledge and practical knowledge as communicative knowledge. It was in 1981 that Mezirow used Habermas’s work to create what he called a critical theory of adult education. He equated “perspective transformation” (1978) with emancipatory learning. He was interested in the “psycho-cultural assumptions” (Mezirow, 1981, p. 6), the way in which a person’s past constrains his or her perceptions of the self and relationships with others. This was the beginning of his theory of transformative learning theory, published in 1991. Next, Mezirow (1985) went on to connect perspective transformation to self-directed learning. I was curious about the relationship between Mezirow’s work and Knowles’ work. They were both writing about similar topics from the mid-1970s to the mid-1980s. As far as I know, Knowles never cited Mezirow’s writing. Mezirow (1991) only cited Knowles once. He described Knowles as writing about “logical reasoning” (p. 103). Knowles (1975, 1980) wrote about self-directed learning from an instructional design perspective, that is, how learners could engage in the process of instructional design by making the decisions about objectives, strategies, and evaluation themselves. It seems unfortunate that these two theorists did not find a way to relate to each other’s work. In his article on self-directed learning, Mezirow related perspective transformation to self-directed learning. Perspective transformation involved making assumptions explicit, contextualizing them, validating them, and acting on them. Self-direction came into that picture since it involved the ability to understand experiences; in other words, a self-directed learner was an individual who engaged in transformative learning. This connection seems to have been lost in the years that followed the publication of this chapter. I raised this issue much later, when I proposed that learner empowerment (which includes self-directed learning) was both a prerequisite and an outcome of transformative learning (Cranton, 2006). The next part of the narrative is the publication of Mezirow’s (1991) book, Transformative Dimensions of Adult Education.

Transformative Learning: A Comprehensive Theory of Adult Education Mezirow (1991) recognized the “fault line that separates theories of adult learning from the practice of those who try to help adults learn” (p. xi). His goal was to create a theory that synthesized the different theories that adult educators use and, in turn, to

47

Transformative Learning: A Narrative

1181

use that comprehensive and integrated theory to help educators guide their practice. He summarized his goal in this way: A learning theory centered on meaning, addressed to educators of adults, could provide a firm foundation for a philosophy of adult education from which appropriate practices of goal setting, needs assessment, program development, instruction and research could be derived. (p. xii)

Mezirow described his theory as a “constructivist theory of adult learning addressed to those involved in helping adults learn” (p. xx). He brought together a variety of theoretical perspectives in order to achieve this goal. This moved his work from a model (perspective transformation) to a theory. It is interesting that the titles of Mezirow’s books and articles most often did not refer to “transformative learning theory.” Instead, he referred to “transformative dimensions of adult learning” (1991), “learning as transformation” (2000), and “learning to think like an adult” (2000). He saw his theoretical work as a theory of adult learning, and he was careful to describe it within that context in everything he wrote. In the development of his comprehensive theory of adult learning, Mezirow (1991) integrated two ideas: the cultural context of learning, including socialization as the foundation for early learning, and the central role of making meaning from experiences in learning. Socialization involves internalizing and personalizing the assumptions, beliefs, and values that are communicated by parents, teachers, the community, and the culture. When a person encounters perspectives that contradict that early socialization, he or she may question the currently held perspectives and revise them. Mezirow (1991) explains that the “formative learning of childhood becomes the transformative learning in adulthood” (p. 3). This distinction comes up often in Mezirow’s writing and addresses the questions people often raise about whether children can engage in transformative learning. The role of making meaning of experiences plays an important role in Mezirow’s (1991) theory. He describes meaning as an interpretation. We experience something, and we interpret or understand that experience. Mezirow says, “Meaning is constructed both prelinguistically, through cues and symbols, and through language” (p. 4). Perspectives that are uncritically assimilated (without thought) form habits of expectation (which Mezirow (2000) later calls “habits of mind”). Transformative learning involves revising limited and distorted meaning perspectives through reflection on assumptions that have been uncritically assimilated. The concept of meaning perspectives was central to Mezirow’s writing from 1991 to 2000. He originally defined three types of meaning perspectives: epistemic (those related to knowledge and how we acquire knowledge), sociolinguistic (based on social norms, cultural expectations, and how we use language), and psychological (derived from how people see themselves – self-concept, needs, inhibitions, anxieties, and fears – particularly those perceptions that come from childhood experiences). In 2000, Mezirow added three more kinds of meaning perspectives, and, by this time, he was calling them “habits of mind:” moralethical (incorporating conscience and morality, how people define good and evil), philosophical (based on a worldview, philosophy, or religious doctrine), and aesthetic (including values, attitudes, tastes, judgments, and standards about beauty).

1182

P. A. Cranton

Mezirow (1991) places his comprehensive theory of adult learning within the context of a variety of other theories of adult learning, including associative bond theory (based on the behaviorist notion of stimulus–response bonds), information processing theory (which emphasizes how information is stored and remembered), and contextual theories (where learning and memory are related to the psychological, social, cultural, and physical contexts within which they occur). He emphasizes the role of language in transformative learning. In his 1981 article, as has already been mentioned, Mezirow called on Habermas’s (1971) work on kinds of knowledge as a framework for his comprehensive theory. He expanded this framework in his 1991 book. Unfortunately, this analysis is not especially clear, even for a reader who is familiar with Habermas’s book. Mezirow’s early work was largely ignored in the literature prior to his 1991 book. But when that book was published, the critics all came out from behind the bushes. He was criticized for not including issues to do with social action, power, and cultural context. He was criticized for misinterpreting Habermas. And the most long-lasting critique was that he was “too rational,” ignoring role of intuition and emotion in the process of transformative learning. He had presented his work as a “theory in progress” (Mezirow, 2000), and he invited critiques. However, he was not always accepting the critiques. He maintained his stance that transformative learning is a cognitive, rational process. He wrote: “Transformative learning is understood as a uniquely adult form of metacognitive reasoning. Reasoning is the process of advancing and assessing reasons, especially those that provide arguments supporting decisions to act. Beliefs are justified when they are based on good reasons” (Mezirow, 2003, p. 58). I return to this later, but the initiation of the transformative learning conference needs to be mentioned here, as it is a key part of the story. This conference, then called “The First National Conference on Transformative Learning,” was organized and held in 1998. Jack Mezirow, Victoria Marsick, and Colleen Wiessner planned to invite about 25 people who were working with transformative learning for exchanging informal ideas and sharing suggestions for future work. Word spread quickly among the relatively small community of scholars interested in the topic, and more than 100 people arrived at Teachers College for the conference. The contributors to this conference were invited to write chapters for Learning as transformation (Mezirow & Associates, 2000). The resulting chapters included topics related to the cognitive rational perspective of transformative learning, the developmental perspective, the connected knowing (or relational) approach, ideology critique, individual differences, small group learning, and organizational learning. This conference also led to the development of the Journal of Transformative Education. This journal came to have an important role to play in the shaping of transformative learning theory.

Expansion of Transformative Learning Theory In light of the critics of transformative learning theory as proposed in response to Mezirow’s (1991) and Mezirow’s (2000) description of transformative learning as a “theory in progress,” alternative interpretations of the theory emerged. Various

47

Transformative Learning: A Narrative

1183

authors have classified the alternatives in different ways. Dirkx (1998) describes four lenses through which we can view transformative learning: Freire’s (1970) perspective of having liberation from oppression as a goal; rational thought and reflection, as suggested by Mezirow; a developmental approach in which the process is based on individual change and growth over time; and a spiritual journey or soul work. In 2006, I presented five alternative perspectives on transformative learning: connected knowing, based on women’s ways of knowing and especially Belenky and Stanton’s work (2000); social change, including Freire’s work, social movements such as the Antigonish Movement and the Highland Folk School, and Brookfield’s ideology critique; groups and organizations, where organizations are seen to engage in transformative learning; the ecological view, proposed by O’Sullivan (2003) – a broad vision of transformation that spans the individual, relational, group, institutional, societal, and global perspectives – and the extrarational approach in which imagination, emotions, and the Jungian concepts of individuation create a process of discernment rather than cognitive reflection (Dirkx, 2012). Ed Taylor (2008) took this further. He sees the alternative conceptions of transformative learning theory including spirituality, positionality, emancipatory learning, and neurobiology. He then went on to add three new perspectives. He calls Dirkx’s’ extrarational point of view the psychoanalytic point of view – a lifelong journey of coming to understand oneself through reflecting on (Dirkx would not use the word “reflecting”) the psychic structures. He added a psychodevelopmental perspective – a view across the life span, reflecting gradual growth and change over time. And the third perspective he added was a social emancipatory one. This is similar to the social change described in the previous paragraph. Next, Taylor (2008) added four more views of transformative learning: neurobiological, cultural-spiritual, race-centric, and planetary. The neurobiological perspective is interesting and is just beginning to develop. We will need to watch this development closely. The planetary perspective fits in with O’Sullivan’s work, as described here. The spiritual perspective has been in the literature for several years, but the link between “cultural” and “spiritual” is not clear. The race-centric view puts people of African descent at the center of transformative process; does transformative learning vary dependent on race?

Fragmentation of Theory Here I rely on a chapter Ed Taylor and I wrote for the Handbook of transformative learning (2012). As mentioned in the opening of this chapter, scholars and theorists tried to make meaning of the development of transformative learning theory by distinguishing one approach from another and categorizing accordingly. Early on, in response to Mezirow’s (1991) work, he was criticized for ignoring the implications of transformative learning for social change (even though his research was conducted within the context of the women’s movement of the 1970s). Within the focus on individual transformation, further splinters are immediately visible. Set up in contrast to Mezirow’s cognitive approach is the extrarational approach or, as labeled by others, the depth psychology approach. In this approach, the Jungian concept of

1184

P. A. Cranton

individuation, in which individuals bring the unconscious to consciousness as they differentiate self from other and simultaneously integrate self with the collective (Boyd & Myers, 1988; Dirkx, 2001), is central to transformative learning. In the developmental perspective, shifts are described in the way we make meaning – moving from a simplistic reliance on authority to more complex ways of knowing or higher orders of consciousness (e.g., Kegan, 2000). Within the focus on social change, some theorists see race and power structures as pivotal to ideology critique (Johnson-Bailey & Alfred, 2006). Tisdell and Tolliver (2003) add spirituality, symbolism, and narrative to what has been called the social-emancipatory approach. And there are those theorists who are interested in how groups and organizations transform (Yorks & Marsick, 2000). So, what does transformative learning mean? When we use the phrase, what are we talking about? Some recent work is focusing on integration and holistic understandings in order to overcome a problematic plunge into a fragmented theory. At the 2005 International Conference on Transformative Learning, Dirkx and Mezirow engaged in a debate (Dirkx, Mezirow, & Cranton, 2006) that modeled an integrative process. They each presented their point of view, then looked for commonalities, overlap, and ways in which the two quite different perspectives could coexist without contradiction. Dirkx indicated that he was not denying the rational process of transformative learning; rather, he was simply more interested in the subjective world and the shadowy inner world. Mezirow acknowledged the significance of this dimension and added that there must also be a critical assessment of assumptions to ensure that they are not based on faith, prejudice, vision, or desire. There was a meeting of minds in this discussion. Gunnlaugson (2008) advocates working with a meta-analysis of what he calls the first-wave and second-wave contributions to the field of transformative learning in order to integrate perspectives. He sees the first wave of contributions as those that build on Mezirow’s original theory. And then, second-wave contributions are those that yield integrative, holistic, and integral theoretical perspectives. Gunnlaugson suggests that Taylor’s (2007) integrative overview of the field is one example of how this new picture of transformative learning theory can emerge. One of the dangers of the fragmentation of transformative learning theory is that the theory becomes meaningless. It means so many different things that it seems not to mean anything. Newman (2012), for example, suggests that transformative learning is nothing more than good learning. He writes: “perhaps there is no such thing as transformative learning; perhaps there is just good learning” (p. 37). Newman goes through several aspects of good learning to make his point: • Based on papers from the Sixth International Conference on Transformative Learning, he concludes that learning was not transformative (it represented significant change, but all learning represents change). • Transformative learning can only be verified by the learners themselves, which has no guaranteed validity. • That transformative learning is a different kind of learning is based on a false premise.

47

Transformative Learning: A Narrative

1185

• No distinction is made between identity and consciousness in the literature; a lot of what is described as transformative learning is related to identity and it “tinkers with our being” (p. 42) rather than the continuing creation of our consciousness. • That transformative learning is described as finite experience is based on a false assumption; this is untenable if the learning is engagement with consciousness. • The centrality of discourse in the process of transformative learning is problematic especially when it comes to having empathy with how others think and feel and using consensus as a test of validity. • Mobilization (or taking action) is often misunderstood as transformative learning. • Spirituality is associated with transformative learning, but the generalizations made in the literature do not stand up to scrutiny. • The indiscriminate use of the term “transformative learning” and its presentation as a universal theory of adult education leads to the term applying to everything and thereby losing its meaning. Stuckey, Taylor, and Cranton (2014) attempted to address some of these issues in their development of their Transformative Learning Survey. They defined four outcomes of transformative learning that followed from the various perspectives: (a) acting differently, (b) having a deeper self-awareness, (c) having more open perspectives, and (d) experiencing a deep shift in worldview. The survey then included several transformative learning processes which all led to the four outcomes: dialogue, emotions, imaginal, spiritual, support, soul work, action, critical reflection, disorienting dilemma, discourse, experience, empowerment, social action, unveiling oppression, and ideology critique. These processes were not viewed as different kinds of transformative learning, but rather different ways of getting to the same place. Statistical analyses supported the structure of the survey. However, this solution does not respond to the problematic aspects of the fragmentation of transformative learning theory; it could even be seen as supporting fragmentation. We need to move toward an integrated theory, one that is inclusive of the different points of view present in the current literature. And this integration needs to be reflected in our approach to research as well as practice. I return to this discussion, but first, I discuss the influence of the Journal of Transformative Learning and the International Conference on Transformative Education.

The Influence of the Journal of Transformative Education The Journal of Transformative Education was initiated in January of 2003, sponsored by the Fielding Graduate Institute and the Link Foundation. Laura Markos and Will McWhinney were the first editors of the journal. The idea came out of the first Transformative Learning Conference in 1998. Invitations were sent out to scholars in 20 educational organizations in the United States and Europe and (although they did not acknowledge this in their editors’ notes) in Canada.

1186

P. A. Cranton

The choice of the title reflected a shift in thinking. The editors described transformative education as follows: Transformative education (TE) is practiced in a number of contexts: as transformative learning, new career training, programs for humanitarian service, rehabilitation, and spiritual renewal. It is supported in local reading groups, community colleges, universities, training centers, experiential and travel groups, correctional and rehabilitation facilities, and spiritual organizations. (Markos & McWhinney, 2003, p. 4)

This set the stage for fragmentation of transformative learning theory. In the first issue, under the heading “Submission Guidelines,” the journal was said to cover adult development; adult education; change, transition, and transformation; corporate education; educational psychology, experiential education; holistic education; humanistic psychology; lifelong learning; management education and development; organizational development; organizational learning; organizational psychology; rehabilitation; social change; and transformative learning. Transformative learning was last in a long list of general topics, not all of which were even related to education. By 2015, this list of topics was reduced to adult development; adult education; change and transformation in individuals, communities, and organizations; experiential education; holistic education; lifelong learning; rehabilitation; social change; and transformative learning. The list is shorter, but the problem remains. This leads me to contemplate the difference between transformative learning and transformative education and to wonder where the Journal of Transformative Education fits. In general, education is associated with formal learning taking place within institutions or systems. Education is usually planned and prepared by educators. Education is “external, handed to or down to the learner, and is time limited, that is, educating about something that has a beginning and an end.” Learning is internal, or initiated from within, and is lifelong. So, transformative education would be education that has the goal of fostering transformative leaning. However, we know that we cannot ensure that transformative learning takes place in any one setting. We can set up an environment where there is the potential of transformative learning taking place. Where does this leave us? Educators cannot transform learners. Transformative learning theory is a learning theory. It describes the process of transformation in any context, including but not restricted to an educational context. The majority of transformative learning occurs outside of formal educational contexts. The learning is informal and nonformal (Mejiuni, Cranton, & Taiwo, 2015; Taylor, 2012). If we follow this thinking, transformative learning theory does not apply to organizations, groups, or schools. For example, what is called “organizational transformation” usually refers to a process by which staff buy into the vision of the organization (usually related to increased productivity) and not to the critical questioning of assumptions within the organization. Tara Fenwick (1998) pointed this out, but she was mostly ignored. I wonder if we have been misled by the Journal of Transformative Education. Perhaps, at least, the journal has confused our thinking about transformative learning

47

Transformative Learning: A Narrative

1187

and transformative education. In spite of the title of the journal, the majority of the articles published are related to transformative learning theory, research, and practice.

The Influence of the International Conference on Transformative Learning The International Conference on Transformative Learning generally has been focused on learning and has required proposals to be grounded in transformative learning theory. In some years, having an emphasis on which perspective on transformative learning the proposal takes has been considered to be important. Usually, the conference asks for two kinds of submissions: paper presentations (based on research or theory) and experiential sessions (based on putting ideas into practice in the conference session). There are also keynote addresses, panel presentations, and the usual trappings of academic conferences. The conference has contributed to a more inclusive understanding of transformative learning. Paper presentations are grouped into a time slot where participants share some common interests, and participants are asked to contact each other in advance of the contrast to discuss how they will share the time they have. Debate and discussion have been encouraged at the conference in constructive and helpful ways. Most years, time is made available for reflection and dialogue every day. There are many sessions where participants talk to each other, and there are sessions in which presenters debate their ideas (e.g., Mezirow, Dirkx, & Cranton, 2006). Some of the conferences have had reflection groups who met at the end of the day to talk about their experiences during the day; others have held small group sessions led by a facilitator to help participants to debrief their experiences at the conference. How can we take this collaborative and inclusive approach into the future developments of transformative learning theory?

How Will This Narrative Continue? I am hopeful that the next part of the transformative learning theory narrative will be to recognize that the different perspectives currently in the literature can easily coexist and lead to an integrated theory. Surely, we do not need to view “individual and social” and “cognitive and intuitive” and “autonomous and relational” as irreconcilable differences. I can see these facets of transformative learning as coexisting, and I can see the possibility of an emerging definition that takes into account the various fragments that now exist in the narrative. We need to work together to create an integrated theory. Originally, I had planned to end this chapter with an integrated definition of transformative learning, but when I thought further about this, I realized it would be more appropriate to call on my colleagues to respond to a call to create an integrated perspective. We need to consider what is “not included” (everything is not transformative, as Michael Newman points out so well). And, we need to seriously consider what is transformative and why it is transformative.

1188

P. A. Cranton

From there, we can look to the future. What are the research goals for theory development? How can we develop practice that reflects an integrated approach to transformative formative learning? And how can we understand transformative learning that takes place outside of formal contexts?

References Belenky, M. & Stanton, A. (2000). Inequality, development, and connected knowing. In J. Mezirow & Associates (Eds.), Learning as transformation: Critical perspectives on a theory in progress (pp. 71–102). San Francisco: Jossey-Bass. Boyd, R. D., & Myers, J. B. (1988). Transformative education. International Journal of Lifelong Education, 7, 261–284. Cranton, P. (2006). Understanding and promoting transformative learning: A guide for educators of adults (2nd ed.). San Francisco: Jossey-Bass. Dirkx, J. (1998). Transformative learning theory in the practice of adult education: An overview. PAACE Journal of Lifelong Learning, 7, 1–14. Dirkx, J. (2001). Images, transformative learning, and the work of soul. Adult Learning, 12(3), 15–16. Dirkx, J. (2012). Nurturing soul work: A Jungian approach to transformative learning. In E. W. Taylor & P. Cranton (Eds.), The handbook of transformative learning: Theory, research, and practice (pp. 116–130). San Francisco: Jossey-Bass. Dirkx, J. M., Mezirow, J., & Cranton, P. (2006). Musings and reflections on the meaning, context, and process of transformative learning: A dialogue between John M. Dirkx and Jack Mezirow. Journal of Transformative Education, 4(2), 123–139. Fenwick, T. (1998). Questioning the concept of the learning organization. In S. M. Scott, B. Spencer, & A. M. Thomas (Eds.), Learning for life: Canadian readings in adult education. Toronto: Thompson Educational Publishing. Freire, P. (1970). Pedagogy of the oppressed. New York: Herder and Herder. Gunnlaugson, O. (2008). Metatheoretical prospects for the field of transformative learning. Journal of Transformative Education, 6(2), 124–135. Habermas, J. (1971). Knowledge and human interests. Boston: Beacon. Johnson-Bailey, J., & Alfred, M. (2006). Transformative teaching and the practices of Black Women adult educators. In E. W. Taylor (Ed.), Fostering transformative learning in the classroom: Challenges and innovation (New Directions in Adult and Continuing Education, no. 109). San Francisco: Jossey-Bass. Kegan, R. (2000). What “form” transforms? A constructive-developmental approach to transformative learning. In J. Mezirow & Associates (Eds.), Learning as transformation: Critical perspectives on a theory in progress. San Francisco: Jossey-Bass. Knowles, M. (1975). Self-directed learning: A guide for learners and teachers. Chicago: Follet. Knowles, M. (1980). The modern practice of adult education: From pedagogy to andragogy. Cambridge: New York. Markos, L., & McWhinney, W. (2003). Editors’ perspectives. Journal of Transformative Education, 1(1), 3–15. Mejiuni, O., Cranton, P., & Taiwo, O. (Eds.). (2015). Measuring and analyzing informal learning in the digital age. Hershey, PA: IGI Global. Mezirow, J. (1978). Education for perspective transformation: Women’s re-entry programs in community colleges. Teachers College, Columbia University: Center for Adult Education. Mezirow, J. (1981). A critical theory of adult learning and education. Adult Education, 32, 3–24. Mezirow, J. (1985). A critical theory of self-directed learning. In S. D. Brookfield, (Ed.), Selfdirected learning: From theory to practice (New Directions for Continuing Education, no. 25). San Francisco: Jossey-Bass.

47

Transformative Learning: A Narrative

1189

Mezirow, J. (1991). Transformative dimensions of adult learning. San Francisco: Jossey-Bass. Mezirow, J. (2000). Learning to think like an adult. In J. Mezirow & Associates (Eds.), Learning as transformation: Critical perspectives on a theory in progress (pp. 3–34). San Francisco: JosseyBass. Mezirow, J. (2003). Transformative learning as discourse. Journal of Transformative Education, 1 (1), 58–63. Mezirow, J, & Associates (2000) (Eds.). Learning as transformation: Critical perspectives on a theory in progress. San Francisco: Jossey-Bass. Newman, M. (2012). Calling transformative learning into question: Some mutinous thoughts. Adult Education Quarterly, 62(1), 36–55. O’Sullivan, E. (2003). The ecological terrain of transformative learning: A vision statement. In C. A. Wiessner, S. R. Meyer, N. Pfhal, & P. Neuman (Eds.), Transformative learning in action: Building bridges across contexts and disciplines. Proceeding of the Fifth International Conference on Transformative Learning, New York: Teachers College, Columbia University. Stuckey, H., Taylor, E. W., & Cranton, P. (2014). Assessing transformative learning processes and outcomes. Journal of Transformative Education, 11(4), 211–228. Taylor, E. W. (2008), Transformative learning theory. In S. B. Merriam (Ed.), Third update on adult learning theory (New Directions on Adult and Continuing Education, no. 119) (pp. 5–16). San Francisco: Jossey-Bass. Taylor, E., & Cranton, P. (2012). Looking back and looking forward. In E. Taylor & P. Cranton (Eds.), The handbook of transformative learning, (pp. 555–574). San Francisco: Jossey-Bass. Taylor, E. (2007). An update of transformative learning theory: A critical review of the empirical research (1999–2005). International Journal of Lifelong Education, 26(2), 173–191. Tisdell, E., & Tolliver, E. (2003). Claiming a sacred face: The role of spirituality and cultural identity in transformative adult higher education. Journal of Transformative Education, 1(4), 368–392. Yorks, L., & Marsick, V. (2000). Organizational learning and transformation. In J. Mezirow & Associates (eds.), Learning as transformation (pp. 253–281), San Francisco: Jossey-Bass.

Patricia A. Cranton’s interests are in transformative learning and authentic teaching. She is a retired professor of adult education affiliated with the University of New Brunswick. Her most recent books include The Handbook of Transformative Learning (2012), coedited with Ed Taylor; Stories of Transformative Learning, coauthored with Michael Kroth (2015); A Novel Idea: Researching Transformative Learning in Fiction, coauthored with Randee Lawrence (2015); Measuring and Analyzing Informal Learning in the Digital Age (2015), coedited with Olutoyin Mejiuni and Olufemi Taiwo; and A Guide to Research for Educators and Trainers of Adults (2015), coauthored with Sharan Merriam. In 2014, Patricia A. Cranton was inducted into the International Adult and Continuing Education Hall of Fame. In 2016, she was a recipient of the Order of Canada.

Preparing for High-Tech Jobs: Instructional Practices, Adults with Autism Spectrum Disorders (ASD), and Video Game Design

48

Demetria L. Ennis-Cole and Princess M. Cullum

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Goals of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Institute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conceptual Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Literature on Employment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Thematic Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Individualized Instruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Teachers Who Understand ASD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Shared Interest in Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Curriculum Support and Accessibility of Computers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interview Questions for Students at the Technology Institute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interview Questions for Founders of the Technology Institute . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1192 1193 1194 1194 1195 1199 1199 1200 1201 1202 1202 1203 1205 1205 1206 1206 1207 1207 1208 1208 1209

D. L. Ennis-Cole (*) Department of Learning Technologies, College of Information, University of North Texas, Denton, TX, USA e-mail: [email protected] P. M. Cullum Cancer Treatment Center of America, Newnan, GA, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_40

1191

1192

D. L. Ennis-Cole and P. M. Cullum

Abstract

Technology is an amazing tool that can be used for electronic communication (email, text, blogs, IM, etc.) and social engagement (Facebook, Twitter, chats, discussion forums, etc.). The forms of information and communications technology (ICT) listed above require little to no verbal expression or physical interaction between users; this makes them a good fit for adults with autism spectrum disorders (ASD) who may lack prowess in social skills and verbal expression. Employment with fewer social and verbal demands may be beneficial for adults with ASD. Occupations designing and testing software and hardware can be solitary in nature, require less social contact than traditional jobs, and produce rewards based on the products created. As a result, high-tech jobs may be ideal for adults with ASD, but the literature indicates that this group tends to be underemployed or unemployed even though they are able to use technology to improve their abilities (Hetzroni O, Thannous J, J Autism Dev Disord 34(2):95–113, 2004; Moore D, McGrath P, Thorpe J, Innov Educ Train Int 37(3):218–228, 2000; Oberleitner R, Ball J, Gillette D, Naseef R, Stamm B, J Aggress Maltreat Trauma 12(1–2):221–242, 2006; Panyan M, J Autism Dev Disor 14(4):375–382, 1984; Pennington, 2010; Schall CM, McDonough JT, J Vocat Rehabil 32(2):79–80, 2010). Uneven job histories, the inability to secure positions, and problems interacting socially are usually cited as reasons for unemployment. This study examines the instructional practices of an institute designed to teach adults with ASD video game and app design. Preferred instructional practices, curriculum support, and professional development are revealed through semi-structured interviews with participants. Findings indicate that individualized instruction with technology, a structured learning environment, a feedback loop, and the elimination of stressors were preferred by students and conducive to learning video game and app design. Keywords

Video game design · Autism spectrum disorders · Instructional practices

Introduction Most adults in the autism spectrum function well with support and guidance from caring sources, and most are very capable of succeeding in academic and employment settings performing a variety of administrative, technical, or computing tasks (Howlin, Alcock, & Burkin, 2005). Adult-directed teaching, positive reinforcement, visual support, family involvement in interventions, structured learning environments, individual support, and specialized curriculum content are effective practices for working with individuals with autism spectrum disorders (Iovannone, Dunlap, Huber, & Kincaid, 2003; Myers, Mackintosh, & Goin-Kochel, 2009; National Autism Center,

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with. . .

1193

2009; Odom et al., 2003; Rivers & Stoneman, 2003; Russa, Matthews, & OwenDeSchryver, 2015; Simpson, 2005). Without consistent support throughout their life span, individuals with ASD face a very uncertain future. Many adults in the spectrum, even those who are high functioning, are often unemployed; many live socially isolated lives with few opportunities for advancement (Baldwin, Costley, & Warren, 2014; Billstedt, Gillberg, & Gillberg, 2005; Gerhardt, 2007; Holwerda, van der Klink, Groothoff, & Brouwer, 2012). Myles described outcomes for 114 adults with Asperger syndrome. A good outcome meant the individual was employed or engaged in educational or vocational training, lived independently, and had two or more friends or steady relationships. A fair outcome indicated that the person was employed or engaged in educational or vocational training and lived independently, and a poor or very poor outcome meant the person was not employed, was not engaged in educational or vocational training, and did not live independently (Myles, 2008). Most of the adults, 92%, had a poor or very poor outcome; only 8% of the 114 participants had a fair outcome, and 0% had a good outcome. Similar results were reported by Howlin, Goode, Hutton, and Rutter (2004). These results are dismal, and they must change to allow the gifts and talents of individuals with autism spectrum disorders (ASD) to be realized, cultivated, and used in their communities. Many persons with ASD are bright and capable, but they struggle with behavior, social skills, and language/communication – all areas that must be addressed at all stages of their development. Individuals in the autism spectrum can learn and have productive employment. Some individuals with ASD possess skills that make them a genius in a narrowly defined area that relies on memorization, pattern recognition, computation, musical, or artistic talent (savants). Others are borderline in their intellectual capability, and some are intellectually disabled. Although this variation mirrors society, it also makes autism an extremely complex spectrum of disorders with few generally applicable solutions. Many individuals with ASD have skills and talents in a variety of areas, and many are visual learners who can understand and use technology tools (Hetzroni & Thannous, 2004; Moore, McGrath, & Thorpe, 2000; Oberleitner, Ball, Gillette, Naseef, & Stamm, 2006; Panyan, 1984). Why not prepare as many individuals with ASD as possible for meaningful employment? Employment is a challenge, but it is possible with personcentered planning, differentiated instruction, job training, and support.

Goals of the Study The purpose of this study was to gain insight into the educational experiences of adults with ASD and document the practices they find helpful for learning to design video games and apps. Additionally, strategies for teaching technical skills to adults with ASD were determined from both student interviews and interviews with founders of a special institute designed to teach adults with ASD high-tech skills. The research questions for this inquiry are as follows:

1194

D. L. Ennis-Cole and P. M. Cullum

1. What instructional practices are preferred by adults with ASD in a technical training environment? 2. What curriculum support is needed to help adults with ASD learn technical skills? The sections that follow describe the institute where this research occurred, the conceptual foundation for this study, the literature on employment and ASD, the methodology for the study, the participants, the procedures followed, and the findings.

Institute The technical institute that hosted this study was founded in 2008 by two parents. Each parent has a son with ASD who is interested in computers and game consoles; both parents sought to use their son’s interest to teach them technical skills that could be used for employment and personal growth. In keeping with the theme of personal development and contributions to the workplace, the parents outlined core values to guide programming and outreach. The core values include belonging, demonstrating mutual respect, discovering and growing, engaging in the community, promoting mental and physical health, and achieving personal and professional goals. The institute began with core values and one student – very quickly more students were added and one became eight. Space on a college campus was secured, and within 8 months, enrollment grew to 52 students. A year later, the institute secured additional space to support 80 students. Currently, 180 crew members work together to create products and develop video games and apps. Additional programs and a residential facility are future plans of the institute as it continues its mission to teach adults with ASD industry standard video game and app design programs and techniques.

Conceptual Framework The preparation of adults with ASD for careers in gaming is timely, and it can also be a lucrative, self-supporting endeavor. Games are a pastime for many adults who enjoy escaping reality, entertaining themselves, competing with others, solving problems, and testing their skills. As the population of adults with ASD increases, efforts must be made to integrate this population into the workforce in order for society to benefit from the products, services, talents, and skills of these individuals. Through this qualitative inquiry, insight was gained on the preferred instructional practices and curriculum support needed by adults with ASD who were learning to create high-tech games. Game building and design for adult learners with ASD are situated in cognitive constructivism; this theory allows the learner to build individual knowledge at his or her own pace through interactions that take place in social, physical, and technical environments. In cognitive constructivism, learning is a dynamic and ongoing process

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with. . .

1195

where learners build knowledge from concepts and ideas with which they are familiar; they continue to acquire and refine a better understanding of the external world through their individual thought processes and social interaction (Powell & Kalina, 2009). Content in the learning environment is individualized, and it involves hands-on activities, projects, discovery, and self-paced learning. The traditional roles of teacher and student are reversed, and assessment methods are based on authentic practice rather than rote memorization and lecture. Creativity, discussion, collaboration, and cooperative learning activities help students build their knowledge and competence. Social interaction with others helps the learner develop skills needed for success. Vygotsky believed that education was impacted by culture and the individual needs of learners. As a result, he indicated that instruction should begin with the learner’s zone of proximal development (ZPD) and advance through social interaction with more capable peers or adults that move the novice to a level of potential development determined by independent problem-solving (Roblyer & Doering, 2010). Learning takes place through interpersonal communication and shared activities between a more experienced peer or adult and a novice learner. Through interaction and shared activities, the knowledge and behaviors of the more learned guide are conveyed to the novice. As a result, the ZPD is the distance between a learner’s actual development and his or her potential level of development. Scaffolding was a strategy that was observed in this study. The adults who participated in this study performed at their developmental level, and a more experienced peer or adult provided assistance or guidance as the learner mastered tools, features of software, and design techniques. This assistance faded as the learner’s competence and skill increased. More advanced peer instructors demonstrated the capabilities of design tools and helped novice users learn features of the tools and use those to create apps and design video games.

Literature on Employment Employment designing high-tech games commands high wages and requires specialized skills. In order to secure these types of positions, individuals with ASD need supportive environments that are sensitive to their challenges. Employment fulfills a number of personal, professional, and financial needs. It allows an individual to feel a sense of purpose, make a contribution, and satisfy the need to be productive and participatory. Supportive employment improves cognitive performance, increases participation in community work, improves the quality of life, adds personal dignity and worth to the individual, and supplies assets to an employer (Garcia-Villamisar, Wehman, & Navarro, 2002; Hendricks & Wehman, 2009). Many individuals with ASD have the ability to work in a variety of capacities in business and industry (Hendricks, 2010; Hendricks & Wehman). These competitive environments reduce the cost of healthcare through the provision of insurance and allow an employee to earn benefits (retirement, medical and dental care, and prescriptions) at a reduced cost. In addition to financial compensation and benefits, the employee contributes

1196

D. L. Ennis-Cole and P. M. Cullum

taxes to state and federal governments, and in so doing, he or she contributes to society and shares the responsibility of citizenship. Employment provides a sense of purpose. Days can be long and boring without a positive undertaking. Going to work each day can eliminate boredom, create opportunities for personal and professional growth, and allow an employee to feel a sense of accomplishment. If there are good working relationships between co-workers, the workplace can be enjoyable and valuable. Participation in employment contributes to better health and well-being; the opportunity to collaborate with others on projects can benefit both the organization and the individual employee (Holwerda et al., 2012). Employees benefit financially and socially, and their need for acceptance, esteem, and self-worth can be satisfied. Work utilizes human capital and decreases the need for funding from federal or state programs that provide cost of living support for individuals with disabilities. Despite all these benefits, many individuals with ASD are unemployed or underemployed (Chappel & Somers, 2010; Hendricks & Wehman, 2009; Nesbitt, 2000). Intellectual, social, communicative, and behavioral challenges are factors which make employment difficult for many persons with ASD. Taylor and Seltzer (2011) reported on 66 young adults with ASD who had recently exited the secondary school system and found low rates of employment. Most of the young adults (56%) were spending their time in day activity centers or sheltered workshops. The young adults with ASD who had an intellectual disability were spending their time in sheltered workshops or activity centers, while those without an intellectual disability were more likely to spend their time living at home with their parents. The authors’ findings suggest that the current system of services is inadequate to support the needs of young adults with ASD who do not have an intellectual disability. In an examination of a subset of the 66 young adults who had exited high school between 2004 and 2008, only 12 individuals were employed. They were between 19 and 25 years of age. Four of the young adults were competitively employed, and eight were involved in community employment with support. The adults who were competitively employed lived at home with their parents or caregiver and worked 10–30 h each week as either a bus boy at a restaurant, a dishwasher who replaced dirty glasses with clean glasses, a worker at The Salvation Army, or an entrepreneur in a self-owned business. The eight individuals in the subset with supportive employment held jobs in restaurants rolling silverware into napkins, shredding information, washing dishes, working in a grocery store, and folding towels in a hotel. In both cases, the majority of employees were underemployed in unskilled positions. The bulk of adults in the group of 66 had an intellectual disability (73.5%) and attended adult day services. There was a significant relationship between employment and day activity. The presence of an intellectual disability was measured by the Wide Range Intelligence Test and the Vineland Screener. A score of 70 or below on both instruments meant the adult had an intellectual disability. Taylor and Seltzer confirmed earlier studies which indicated that adults with ASD have a low rate of employment, and those who are competitively employed experience underemployment (Burke, Andersen, Bowen, Howard, & Allen, 2010; Howlin et al., 2004; Hurlbutt & Chalmers, 2004).

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with. . .

1197

Some individuals with ASD may have an intellectual disability (ID) which will require additional support: more time to learn the job tasks, different methods for learning content, cueing, additional prompting, visual and auditory support, repetition, a Picture Exchange Communication System, or procedural information (Chappel & Somers, 2010; Schall, 2010). This type of support helps the individual compensate for deficits in processing speed, memory, recall, spatial orientation, sequencing, and executive functions. Individuals with ASD who also have an intellectual disability (ID = IQ  70) have less optimal social functioning as compared with those with higher IQ scores (Taylor & Seltzer, 2011). An ID affects a person’s ability to learn from experience, think abstractly, reason through problems, and plan; it may be accompanied by adaptive behavior problems which impact social ability and the application and conceptualization of information. As a result, the person’s ability to function well in daily life and his or her ability to respond to his or her environment may be moderately or severely impaired. Chappel and Somers (2010) reported that 6% of persons with ASD are employed in full-time positions. The authors quote Lawer, Brusilovskly, Salzer, and Mandell (2009) who indicate that the uneven cognitive and social skills of individuals with ASD require more challenging and more expensive vocational rehabilitation services. These services may be denied to individuals with ASD because of the belief that their disability is more severe than other types of disabilities. Without targeted services for adults with ASD, social difficulties, a lack of communication and understanding, a disconnect between the individual’s skills and the job requirements, a lack of support, stereotypical behaviors, and inflexibility hinder job success, cause confusion and misunderstandings, and lead to termination (Chappel & Somers; Hurlbutt & Chalmers, 2004). Individuals with ASD may misunderstand social norms, misinterpret communication, fail to read nonverbal communication signals and body language, and use a tone of voice that is inappropriate. Many times, they are unaware that others have different goals, ideas, thoughts, and beliefs from themselves; they lack theory of mind (Baron-Cohen, 2008). Reciprocating appropriate social exchanges, sharing interests with others, and being attentive to the needs of others foster positive working relationships with co-workers and supervisors. Many individuals with ASD have challenges relating to others throughout adolescence and adulthood (Bregman, 2005; Schall & McDonough, 2010). Most individuals with ASD have some degree of language impairment – receptive, expressive, or pragmatic. Schall and McDonough (2010) present a profile of a young adult with ASD (“Mary Ann”) who has communication issues. They describe her as follows: “sounds different when she speaks,” “vocal prosody is at times pedantic and monotone,” and “her laugh is very giddy and louder than expected.” The young lady is further described as having “fleeting and uncoordinated eye contact with conversation.” Instead of looking at a person, “Mary Ann” seems to look through the person. Her conversation is not spontaneous or self-directed; she relies on her conversation partner to maintain dialog. She does not ask questions to learn more, express genuine interest, or extend the conversation. This description reflects some of the communication challenges of some individuals with ASD. Other examples of communication obstacles reported by Hendricks (2010) include asking

1198

D. L. Ennis-Cole and P. M. Cullum

too many questions, difficulty understanding verbal directions, an inability to “read between the lines” to understand hidden meanings and personal agendas, and difficulty interpreting meaning from situations. There are individuals with ASD who are nonverbal, and they communicate through sign language or Picture Exchange Communication Systems. These individuals are employable, but they, like their verbal counterparts with ASD, require support and understanding from their co-workers and supervisors. Employment is critical for improving the quality of life for persons with ASD. The ability to secure and retain a well-paying job can transform the frame of reference or worldview of both employers and their employees with ASD by opening both up to new possibilities and removing stereotypical assumptions and judgments about capabilities and talents (King, 2000; Mezirow, 1997). Our frames of reference can be limited, because they are based on cultural assimilations and influences of primary caregivers; the reevaluation of these through reflection and discourse can transform our thinking and learning and shift our conscious thoughts about our world and those with whom we interact (King, 2000; Mezirow, 1997). Several practices for preparing individuals with ASD for employment include person-centered planning (PCP), community-based instruction, paid work experiences, career planning, and the supported employment model (Hurlbutt & Chalmers, 2004). The model that best describes individuals with ASD and video game and app design is person-centered planning. In PCP, the emphasis is placed on self-advocacy, self-determination, and active involvement in the process. With person-centered planning, the individual’s goals and capacities are explored from a variety of perspectives, and the support needed for success is also considered. This approach is natural and holistic, and it allows the individual with ASD and his or her significant others to express a shared vision of the future. Person-centered planning for employment has been linked with increased community participation and enhanced social support (Hagner, May, Kurtz, & Cloutier, 2014). Video game and app design require individual and collective activities where coders work with layers of software to learn specific features that allow them to create the environment, the audio, the functions of the game or app, the test cases for functions, and the modules that integrate their code within a larger system. Game designers create their own procedures and use their ideas to create the code that will be the foundation for the game. This type of individualization is well suited for persons with ASD. Designing video games and apps can involve a variety of activities from creating a story, using a storyboard, designing characters, building the “world” for the story, action, and characters to mapping the game environment and creating, coding, and building 2D and 3D interactive experiences. This breadth of activity requires a vast skill set, and individuals with ASD may excel at one or more of these tasks. They may be creative writers who can create storyboards. They may be able to sketch the details of the “world” needed for a game environment or create the intricate detail for 3D characters. They may have the ability to design the digital exoskeleton of a character, apply colors and layers, or map texture within the 3D game environment. Additionally, they may excel at building and using code to make game elements

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with. . .

1199

work together. A code engine generates complex code for geometric shapes and uses artificial intelligence components to create the logic and physics of the game world and the interaction and movement of objects. All of these tasks are important, and an individual with ASD may find his or her niche performing these design tasks. Many persons with ASD have a detailed focus and the ability to recognize patterns easily. They tend to be visual thinkers and learners who do not tire of repetitive tasks, so testing and the iterative nature of design and development can be ideal situations for them. Additionally, many persons with ASD excel at musical, computational, artistic, and spatial tasks. Couple this with their ability to learn and use technology and video and app design makes sense. Their abilities are assets that can speed the development process of a production team.

Methodology The purpose of this phenomenological inquiry was to discover the instructional preferences and support needed by adults with ASD in a technology institute. Instructional preferences are generally defined as strategies for content delivery, procedures, and lesson formats that promote student learning. Support is generally defined as a helpful technique or auxiliary aid that facilitates the completion of tasks and promotes student success. Phenomenology was selected as the research method because it allowed the researcher to gather data on the essence of the lived experiences of several individuals with ASD who were learning video game and app design (Creswell, 1998, 2014). Qualitative data collection and analysis techniques were used to understand the teaching and learning of adults enrolled in the technology institute. Contact was made via email with the president of a technical institute which provides training in video game design, design engine features and tools, map creation, and 3D modeling to adults with ASD. A request was made to conduct face-to-face interviews with students and founders of the institute in an attempt to learn about teaching strategies, environmental considerations, and employment outcomes for students. The request was received favorably, and a subsequent request was made to the University IRB for approval to conduct this study. Once approval was granted, semi-structured interviews were scheduled with seven adult students and two founders.

Participants The institute permitted the primary researcher to interview seven diverse participants: one Asian American male, one Caucasian female, two African American males, and three Caucasian males. The institute verified a diagnosis of ASD for each of the participants; all individuals at the institute are required to have valid testing indicating a diagnosis of ASD from a licensed professional before admittance. The founders of the institute selected participants in different stages of their course work. The interviewees were chosen based on their articulation ability and their

1200

D. L. Ennis-Cole and P. M. Cullum

willingness to talk with the researcher. Six of the students were between the ages of 18 and 26, and one participant was 30+. The participants were asked a series of questions. All interview questions were reviewed by the institute before contact with participants was granted. In addition, the founders of the institute shared their insight on the creation of the educational environment, curriculum, and necessary support systems. Three of the participants were attending local community colleges prior to their enrollment in the institute. With the right skill set, participation in and completion of post-secondary programs can be a reality which opens doors to self-fulfillment and employment. Vanbergeijk, Klin, and Volkmar (2008) indicate that many individuals with ASD are intellectually capable of obtaining a university education and that can be realized if academic and supportive accommodations are provided as they are needed. Several examples include institutional fit (size, comfort, and safety), course selection, number of courses taken, peer training and support, and accommodations that support learning and development (the inclusion of technology tools for lecture notes, organizational skills, chunking large assignments, and visual support). Adjustments in the physical environment may be needed to reduce overstimulation and sensory issues; the institution should be responsive and have mechanisms to assess and address student needs. More study is needed on the educational needs of postsecondary adults with ASD (Vanbergeijk et al. 2008; Taylor & Seltzer, 2011).

Procedures In order to determine the preferred instructional practices and support at the postsecondary institute participating in this study, interviewing and creating memos, clusters, themes, and summaries occurred. Two semi-structured interviews were conducted to gather information about the lived experiences of adults with ASD in several areas: instructional practices, teaching, and support. The interview protocol contained questions about the technologies participants were currently learning, their previous educational experience and how it compared with their experience at the institute, the things they enjoyed about their classes, future goals, the number of classes they had taken, how the classes were helping them learn to design video games, what helped them learn new content, and what they were doing before entering the institute. The last question was open ended and asked if the interviewee had any additional information to share. See Appendices A and B for the interview protocols. The following practices were undertaken to ensure ethically conducted interviews: informed consent, assurances to participants, and addressing concerns of founders of the institute. At the initial face-to-face interview with each of the seven adult students, informed consent was obtained, and each person was informed that they could cease participating at any time. No coercion of any kind took place. Participants were informed of the purpose of the study, and they were assured that

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with. . .

1201

any information they provided would be held in the strictest confidence. They were further notified that any data reported on the outcomes of the study would be aggregated, and their identity would remain confidential. No physical, psychological, or mental stress was imposed. Many individuals with ASD suffer from anxiety disorders and other comorbid conditions, so the researcher was patient and allowed participants to answer questions as fast or slow as they needed. To ensure successful interviews, the founders of the institute selected participants that could best articulate their experiences, verified a diagnosis of ASD for each interviewee, selected, and scheduled the location for the interviews. Each interview was transcribed and returned to the interviewee for verification. A face-to-face follow-up interview was scheduled to gain clarification and additional information. All interviews took place in a quiet environment; interviewees were allowed to answer questions in a relaxed atmosphere and use as much time as they needed. Follow-up interviews were conducted and verified by interviewees as a member check to ensure validity. The primary researcher was careful to use the same procedure during each interview and follow-up, ask interviewees the targeted questions, engage in memoing (reflective note-taking) of additional comments the participants shared, and review the recorded interviews and follow-up as soon as possible after the interviews.

Thematic Analysis Themes emerged from the data, and the analysis began with the transcription of audio recordings and memos (field notes) from the interviews. Each transcript was prepared from the audio recording as soon after the interview as possible, and both the transcript and the field notes were organized and reviewed several times. The procedure for analyzing the data was based on Hycner’s (1985) guidelines for phenomenological analysis: bracketing and phenomenological reduction (becoming open to the phenomenon and suspending researcher assumptions and interpretations about the instructional support and preferences of adult learners with ASD), delineating units of meaning (extracting statements from the transcripts that illuminated the phenomena), having another researcher independently verify relevant meaning (eliciting another individual to follow the same procedures with the transcripts to improve reliability), eliminating redundancies (examining the units of meaning to determine redundancies and noting the number of times items appear), clustering units of relevant meaning into themes (determining whether or not the units of relevant meaning are a natural fit and express the essence of the clusters), and summarizing each interview and conducting follow-up with the participant. The member check or follow-up interview was performed to determine whether or not the transcription and themes accurately reflected the initial interview (Creswell, 1998, 2014). The essence of the lived experiences of adults at the technical institute is shared in the next section.

1202

D. L. Ennis-Cole and P. M. Cullum

Findings Statements from the adult students and founders interviewed allowed the researchers to gain insight on students’ educational needs and curriculum support. The researchers determined that the following themes were most frequently mentioned by adult students as important factors that contribute to their success: individualized instruction, teachers who understand ASD, and being around others with ASD who share an interest in technology. Data collected from the founders’ interviews revealed individualized instruction and attention to student comfort and safety as important features that need to be embedded within the curriculum for student success.

Individualized Instruction Several comments helped the researchers identity individualized instruction as a primary theme. Adult learners (57%) indicated that they preferred instruction that was individualized, based on their needs and their aptitude. They indicated that they did their best when there was individual follow-up and encouragement. They preferred structure, flexible lesson formats, and the absence of homework. Centering instruction on student needs, interests, and ability and self-paced learning experiences were preferred by students: Self-paced learning, getting support or help, reducing the stress, because it increases the anxiety – My experience at [the institute] has been Good!

Self-paced learning helped adults with ASD feel comfortable and discover features of software. Students spoke about learning the technology on their own and enjoying learning at their own pace. One indicated that “self-paced really lightens up the load.” One student said, “I learn technology on my own, and these classes teach new features in the layers of Photoshop, and I did learn several things about Hammer.” Self-paced learning aids discovery and a personal understanding of content. Founders of the institute indicated that individualized instruction and student comfort were important considerations in the institute’s curriculum. The categories that emerged from the themes found in the founder’s interviews included one-on-one instruction, considering student needs and sensory issues, and increasing student engagement. Other comments included “using students’ strengths, making sure students experience success, providing down-time, focusing on practical results, not a grade, reducing intimidation, and personal and physical separation.” Many students with ASD found the lecture format uncomfortable. Elements in the environment (lighting, AC, other noises), constant verbal information, and the need to make eye-to-eye contact can be problematic for learners with ASD. As a result, lecture formats are not used at the institute; instead of watching an instructor,

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with. . .

1203

students watch their monitor to see what the instructor is doing with the program. This direct teaching with technology takes place one on one with the instructor and student sitting opposite each other observing separate monitors connected to the same system unit. One founder describes it best by indicating that “We have total visual delivery with a split screen configuration that has 2 monitors attached to 1 computer.” The instructor demonstrates features of the software, explains procedures, and responds to a student’s questions while both are watching the instructor create maps, scenes, stories, and music, and explore menus prompts, and other features of the software. This is very different from traditional instruction in design and coding courses where students attend lectures and engage in trial and error with the software to learn its features. At the institute, students observe cause and effect, ask questions, have the answers demonstrated, and learn by doing. Their learning takes place in the moment as they watch demonstrations, ask questions, and follow their instructor’s actions within the software. One-on-one instruction was mentioned as a differentiating factor between this technological institute and other programs of study. One student with a negative experience in another program contrasted that experience with instruction at this training institute: Other experiences used a lecture format. [This institute] uses one-on-one and delivers instruction as fast or as slow as needed by the student. At [this institute], you don’t have to look at the instructor, and that’s good, it’s more comfortable. Some people with Autism are nervous and ask that you don’t look at them. Some don’t care. The focus on technology takes that away.

Being able to learn new technology without a ceiling and for some students not being told to learn this, because this is my job indicated that students with ASD are able to expand and learn more in a self-directed manner. As one student with ASD noted, “I can work at my own pace and get more done.” Another student highlighted that “It’s one-on-one training.” More than one student agreed that “[This institute] is nothing like another school I’ve attended.” “Flexible, structured, and based on students’ needs and aptitudes” were stated as reasons why students were comfortable with their current training environment.

Teachers Who Understand ASD During the interviews, most students mentioned the frustration of not being understood in the educational settings they attended before coming to the institute. Their negative feelings were replaced by relief and gratitude, because they are in a program where the instructors are encouraging and offer positive praise. As a result, a student noted, “I like to make him [my instructor] happy.” Most interviewees felt the instructors wanted to help them succeed. In part, the positive perception of instructors at the technological institute was due to instructors who teach each student differently and measure success by the projects my students create. The majority of students expressed negative experiences with instructional programs they attended

1204

D. L. Ennis-Cole and P. M. Cullum

prior to coming to the institute. Their attitude was much improved, as indicated by the comment below: My experience is better. People understand Autism and most have Autism. I can gain skills and bring those into the work world. I used to forget to turn my homework in, and online classes were not so helpful to me. I need face-to-face instruction to remember to get the work done. My professors didn’t care if students passed or failed. At [the institute] students are encouraged to succeed and do their best. No follow-up, and having a project due every week was a problem in college, which was my previous environment.

Another interesting facet of instruction at the institute is its hiring practice. Three (42.8%) of the interviewees were advanced students who have been trained at the institute and hired there as either full- or part-time instructors. They are uniquely qualified to deliver instruction, because they understand the software and use it proficiently, and they have ASD. Their perspectives are identified below: When I teach, it is easier when I know the person and the material. If [a learner] is not getting it, switch what you’re doing and find a different avenue to get there. They [learners with ASD] aren’t being obstinate; their creative ability may be lacking and they can’t do it. They may be stuck and can’t move around the problem. Aspies will stay stuck longer; they don’t know how to look at things differently. They have one mode of thinking. Adults with ASD need patience. They need detailed explanations. Immediately after you present something, explain it in detail. Adults with ASD need visual references, and they love to talk. I am teaching at [the institute]. The hardest part of teaching is delivering instruction, because I have to adapt my teaching style to the needs of a particular student and that student can be obsessed or uninterested. Teaching depends on students’ needs, and those change.

Founders indicated that they both have sons with ASD, and they uniquely understand both the issues involved in training adults with ASD and computer programming. Each founder has 18–20 years of professional experience in software development/systems architecture or programming/technical training. Their understanding of the social and communication deficits in ASD allowed them to include a “Social Room” into the experience of adult students with ASD. The Social Room is where specific games are played. It is set up for interaction, discussions of gaming, and informal instruction. The room was designed to help students learn collaborative skills and practice social skills. Both founders also indicated that adults with ASD need “patient and compassionate instructors” in order to achieve their instructional goals. One founder indicated that “Visual Tools are good tools for adults with ASD, and adults with ASD are very rule-based.” This comment further describes his understanding of the characteristics of adults with ASD. Additionally, one founder explained instructional delivery for students with ASD: Training is presented at the student’s pace. This process increases the student’s engagement as the student engages in professional development activities designed to help him or her master and functionally use the design tools in our curriculum.

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with. . .

1205

A Shared Interest in Technology Without exception, each student identified being around others with ASD and a shared interest in technology as a positive experience. One student offered this statement: The fact that I’m surrounded by gamers like me – The whole organization revolves around Autism and video gaming. This microworld is easier than things in the real world.

Another student shared “here at [the institute] everybody is on the same page.” “That makes it easier to learn when everybody’s on the same page.” Another stated: “We work together to get this program a float and help it improve on itself.” One student indicated that common interests created an environment where collaborative programming and problem-solving occurred. The majority of the interviewees indicated that they like the socializing part and creat[ing] stuff that makes a selfsustaining program. One founder indicated that students have expectations in their peer group beyond showing up and leaving. They are required to work through problems, share ideas, and create in their production teams. The activities involved in creating a product with others mirrors the workplace environment where students will be expected to interact in a reciprocal way to develop products for a prospective employer. Employment in video game design is a strong possibility for these adults as one founder stated: We’ve been approached by several large companies like Google, Valve Corporation, and Microsoft about hiring graduates from out program, but our students have not completed the entire program.

Curriculum Support and Accessibility of Computers Computers were always accessible through fixed training sessions each week with unlimited lab access for students. In addition, students have optional courses in writing, social skills training, advocacy training, and other courses geared toward building workplace and social skills. One founder commented that there was consistent exposure to technology, but it was not overwhelming, since the curriculum content was presented throughout the week. The courses in the curriculum were designed by the institute based on tools the professional technology industry uses to create media. Technologies for courses were selected based on popularity, accessibility, and cost-effectiveness. Lab time was flexible and available on demand. As a part of content delivery and management, an administrative system was created for students using cloud computing to assist them by providing a scheduler and feedback loop. In addition, reinforcement is supplied to increase

1206

D. L. Ennis-Cole and P. M. Cullum

and maintain student motivation. One founder describes the curriculum as follows: I designed a self-supporting administration and Connect System for student use available through Cloud Computing. The Connect System has a scheduler and feedback loop which tracks a student’s progress. Students are reminded of the things they need to complete, and they are reminded to turn work into their instructor. As a form of reinforcement for completing curriculum tasks, students gain points for completed work (rather than receive grades). The points they earn can be used in [the institute’s] store to purchase items with the school’s logo or purchase time on video games.

Limitations Two limitations are noted in this phenomenological inquiry. The first is the selection of interviewees by staff members of the participating institute. The staff randomly selected participants based on their availability, their willingness to talk with the researcher, and their ability to articulate their concerns and ideas. A second limitation is the lack of employment data. A perspective from former employers might have added additional insight and provided an opportunity to triangulate the data based on the founders’ interviews, students’ perspectives, and employers’ insight. The combined perspectives may be helpful in further studies.

Conclusion This study examined practices implemented at a technological institute for adults with ASD to determine the attendees’ preferred instructional practices and the curriculum support they needed. Learner perspectives were documented to determine how video game and app design should be presented to adults with ASD. Clustered units of meaning revealed the following key themes: individualized instruction with technology, teachers who understand ASD, and being around others with ASD who share an interest in technology. Careers in video game and app design are desirable for and attainable by adults with ASD. These learners excel with technology because it is visual, motivating, and reinforcing and provides few barriers in the way of communication or social skills. Employers can benefit from the technical ability of adults with ASD and use their talents to fill vacancies in information technology. Teaching adults with ASD video game design and other technology skills can be accomplished by patient and compassionate instructors who individualize learning, support students, and create an environment that reduces stress and anxiety. Technology instructors, employers, and others working with adults in the Autism Spectrum may benefit from the utilization of the themes identified in this chapter. These findings should be considered in both educational and employment settings to support adults with ASD so that they may achieve success through enhanced performance, production, and inclusion.

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with. . .

1207

This work can be expanded to examine the perspectives of both employers and colleagues of individuals with ASD. The findings can also be applied to curriculum design for both children and teens with ASD who may be learning video game design and other content areas. In addition, future research is needed on reducing the barriers to employment for individuals with ASD and creating comfortable work environments that include individuals with ASD, facilitate their success, and support training for both employers and colleagues of adults in the autism spectrum. Finally, additional research is needed on the educational needs of post-secondary adults with ASD.

Appendix A Interview Questions for Students at the Technology Institute

1208

D. L. Ennis-Cole and P. M. Cullum

Appendix B Interview Questions for Founders of the Technology Institute

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with. . .

1209

References Baldwin, S., Costley, D., & Warren, A. (2014). Employment activities and experiences of adults with high-functioning autism and Asperger’s disorder. Journal of Autism and Developmental Disorders, 44(10), 2440–2449. Baron-Cohen, S. (2008). Theories of the autistic mind. The Psychologist, 21(2), 112–116. Billstedt, E., Gillberg, I., & Gillberg, C. (2005). Autism after adolescence: Population-based 13-22year follow-up study of 120 individuals with autism diagnosed in childhood. Journal of Autism and Developmental Disorders, 35(3), 351–360. Bregman, J. (2005). Definitions and characteristics of the spectrum. In D. Zager (Ed.), Autism spectrum disorders: Identification, education, and treatment (pp. 3–46). Mahwah, NJ: Lawrence Erlbaum Associates. Burke, R., Andersen, M., Bowen, S., Howard, M., & Allen, K. (2010). Evaluation of two methods to increase employment options for young adults with autism spectrum disorders. Research in Developmental Disabilities, 31(6), 1223–1233. Chappel, S., & Somers, B. (2010). Employing persons with autism spectrum disorders: A collaborative effort. Journal of Vocational Rehabilitation, 32, 117–124. Creswell, J. (1998). Qualitative inquiry ad research design: Choosing among five traditions. Thousand Oaks, CA: Sage. Creswell, J. (2014). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage. Garcia-Villamisar, D., Wehman, P., & Navarro, M. (2002). Changes in the quality of autistic people’s life that work in supported and sheltered employment. A 5-year follow-up study. Journal for Vocational Rehabilitation, 17(4), 309–312. Gerhardt, P. (2007). Notes from the field: Effective transition planning for learners with autism spectrum disorders approaching adulthood. Journal for Vocational Special Needs Education, 27(2), 35–37. Hagner, D., May, J., Kurtz, A., & Cloutier, H. (2014). Person-centered planning for transition-aged youth with autism spectrum disorders. Journal of Rehabilitation, 80(1), 4–10. Hendricks, D. (2010). Employment and adults with autism spectrum disorders: Challenges and strategies for success. Journal of Vocational Rehabilitation, 32(2), 125–134. Hendricks, D., & Wehman, P. (2009). Transition from school to adulthood for youth with autism spectrum disorders: Review and recommendations. Focus on Autism and Other Developmental Disabilities, 24(1), 78–88. Hetzroni, O., & Thannous, J. (2004). Effects of a computer-based intervention program on the communicative functions of children with autism. Journal of Autism and Developmental Disorders, 34(2), 95–113. Holwerda, A., van der Klink, J., Groothoff, J., & Brouwer, S. (2012). Predictors for work participation in individuals with an autism spectrum disorder: A systematic review. Journal of Occupational Rehabilitation, 22(3), 333–352. https://doi.org/10.1007/s10926-011-9347-8. Howlin, P., Alcock, J., & Burkin, C. (2005). An 8 year follow-up of a specialist supported employment service for high-ability adults with autism or Asperger syndrome. Autism, 9(5), 533–549. Howlin, P., Goode, S., Hutton, J., & Rutter, M. (2004). Adult outcome for children with autism. Journal of Child Psychology and Psychiatry, 45(2), 212–229. Hurlbutt, K., & Chalmers, L. (2004). Employment and adults with Asperger syndrome. Focus on Autism and Other Developmental Disabilities, 19(4), 215–222. Hycner, R. (1985). Some guidelines for the phenomenological analysis of interview data. Human Studies, 8(3), 279–303. Iovannone, R., Dunlap, G., Huber, H., & Kincaid, D. (2003). Effective practices for students with autism spectrum disorders. Focus on Autism and Other Developmental Disabilities, 18(3), 150–165.

1210

D. L. Ennis-Cole and P. M. Cullum

King, K. (2000). The adult ESL experience: Facilitating perspective transformation in the classroom. Adult Basic Education, 10(2), 69–89. Lawer, L., Brusilovskiv, E., Salzer, M., & Mandell, D. (2009). Use of vocational rehabilitation services among adults with Autism. Journal of Autism and Developmental Disabilities, 39(3), 487–494. Mezirow, J. (1997). Transformative learning: Theory to practice. New Directions for Adult & Continuing Education, 1997(74), 5–12. Moore, D., McGrath, P., & Thorpe, J. (2000). Computer-aided learning for people with autism – A framework for research and development. Innovations in Education and Training International, 37(3), 218–228. Myers, B., Mackintosh, V., & Goin-Kochel, R. (2009). Brief report: My greatest joy and my greatest heartache: Parents’ own words on how having a child in the autism spectrum has affected their lives and their families’ lives. Research in Autism Spectrum Disorders, 3(3), 670–684. Myles, B. (2008). Autism spectrum disorders: Understanding the cycle of tantrums, rage, and meltdowns. 17th Annual Texas Conference on Autism. Arlington. 5–6 December 2008. National Autism Center. (2009). National standards project-addressing the need for evidencebased practice guidelines for autism spectrum disorders. Randolph, MA: National Autism Center. Retrieved from http://bestpracticeautism.blogspot.com/2010/02/national-autism-cen ters-national.html Nesbitt, S. (2000). Why and why not? Factors influencing employment for individuals with Asperger syndrome. Autism, 4(4), 357–369. Oberleitner, R., Ball, J., Gillette, D., Naseef, R., & Stamm, B. (2006). Technologies to lessen the distress of autism. Journal of Aggression, Maltreatment & Trauma, 12(1–2), 221–242. Odom, S., Brown, W., Frey, T., Karasu, N., Smith-Canter, L., & Strain, P. (2003). Evidence-based practices for young children with Autism: Contributions for single-subject design research. Focus on Autism and Other Developmental Disabilities, 18(3), 166–175. Panyan, M. (1984). Computer technology for Autistic students. Journal of Autism and Developmental Disorders, 14(4), 375–382. http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_ nfpb=true&_&ERICExtSearch_SearchValue_0=EJ798602&ERICExtSearch_SearchType_0= no&accno=EJ798602 Pennington, R. C. (2010). Computer-assisted instruction for teaching academic skills to students with Autism Spectrum Disorders: A review of literature. Focus on Autism & Other Developmental Disabilities, 25(4), 239–248. Powell, K., & Kalina, C. (2009). Cognitive and social constructivism: Developing tools for an effective classroom. Education, 130(2), 241–250. Rivers, J., & Stoneman, Z. (2003). Sibling relationships when a child has autism: Marital stress and support coping. Journal of Autism and Developmental Disorders, 33(4), 383–394. Roblyer, M., & Doering, A. (2010). Integrating educational technology into teaching (5th ed.). Boston, MA: Allyn & Bacon. Russa, M., Matthews, A., & Owen-DeSchryver, J. (2015). Expanding supports to improve the lives of families of children with autism spectrum disorder. Journal of Positive Behavior Interventions, 17(2), 95–104. Schall, C. (2010). Positive behavior support: Supporting adults with autism spectrum disorders in the workplace. Journal of Vocational Rehabilitation, 32(2), 109–115. Schall, C. M., & McDonough, J. T. (2010). Introduction to special issue autism spectrum disorders: Transition and employment. Journal of Vocational Rehabilitation, 32(2), 79–80. Simpson, R. (2005). Evidence-based practices and students with autism spectrum disorders. Focus on Autism and Other Developmental Disabilities, 20(3), 140–149.

48

Preparing for High-Tech Jobs: Instructional Practices, Adults with. . .

1211

Taylor, J., & Seltzer, M. (2011). Employment and post-secondary educational activities for young adults with autism spectrum disorders during the transition to adulthood. Journal of Autism and Developmental Disabilities, 41(5), 566–574. Vanbergeijk, E., Klin, A., & Volkmar, F. (2008). Supporting more able students on the autism spectrum: College and beyond. Journal of Autism and Developmental Disabilities, 38(7), 1359–1370.

Demetria Ennis-Cole is a Professor of Learning Technologies at the University of North Texas. She worked in industry as a Programmer for International Business Machines, and she worked as a Computer Analyst at Louisiana State University before accepting a faculty position with the University of North Texas. Ennis-Cole is included in Outstanding Young Women of America, and she is a Patricia Roberts Harris Fellow, an Image Award Recipient, a recipient of the TCEA Area 10 Excellence with Twenty-First Century Tools Award, and a recipient of ISTE’s Inspire by Example Award. Her research interests include technology utilization by special populations (mature adults, primary and secondary students, and students with autism spectrum disorders), preservice teachers and technology training, software evaluation, and artificial intelligence in education. She is the author of Technology for Learners with Autism Spectrum Disorders. Princess M. Cullum is a Senior Manager in Leadership and Culture at Cancer Treatment Centers of America. She is a doctoral candidate in Applied Technology Performance Improvement in the Department of Learning Technologies at the University of North Texas. Her research focuses on workplace development, leadership development, and instructional design. She travels throughout the United States and the Caribbean teaching seminars on leadership development, diversity and inclusion, and team building.

Are Students and Faculty Ready for Transformative Learning?

49

C. Halupa

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transformative Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Faculty Resistance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Student Resistance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Faculty Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Student Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix A: Transformative Learning Readiness Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Instructional Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix B: Transformative Learning Readiness Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Faculty Personal Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix C: Student Transformative Learning Readiness Assessment . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1214 1215 1217 1221 1222 1225 1227 1229 1229 1229 1231 1231 1232 1233

Abstract

In order for transformative learning in higher education to occur, both students and faculty must be ready to transform. However, students may not be ready to engage in self-directed, reflective learning, and faculty may not be ready to change their pedagogical practices to facilitate this transformation. This chapter will include information on the challenges faculty face in an attempt to use transformative learning theory in their classrooms, as well as the challenges students face in trying to attain the level of learning desired in transformative

C. Halupa (*) A.T. Still University, Kirksville, MO, USA Dean Online Learning, East Texas Baptist University, Marshall, TX, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_70

1213

1214

C. Halupa

learning. Best practices in transformative learning implementation theory and assessment will be discussed. In addition, this chapter will include a selfassessment for both students and faculty to test their readiness to engage in transformative learning practice. Keywords

Transformative learning · Instructional strategies · Student-centered pedagogies · Faculty and student resistance to change

Introduction Transformative learning is “the process of effecting change in a frame of reference” (p. 5) according to Jack Mezirow, the father of transformative learning. A frame of reference includes a student’s habit of the mind, as well as a personal point of view. The habits of mind are affected by previous learning experiences and cultural norms, while the points of view are the student’s personal beliefs and attitudes (Mezirow, 1997). Mezirow identified four processes of learning: • • • •

Elaborate an existing point of view Establish new points of view Transform previous point of view Transform habits of the mind

When a learner first engages with learning content or begins an assessment, he or she tends to look for evidence that supports his/her own beliefs and preconceived notions. However, the educational experience begins to transform the student, and he starts to examine alternate points of view. These alternate points of view may then replace or be added to the existing point of view to create a new point of view. This transforms into a habit of the mind when the learner can learn to look at things differently. This includes acknowledging potential biases of previous, as well as new points of view (Mezirow, 1997). In higher education this process is continual. But in order for the process to be effective, it is crucial both the faculty member and the student are willing to transform and evaluate their personal points of view. This can result in the transformation of a habit of the mind for both the student and faculty member. Ultimately, this leads to much higher levels of learning and knowledge that is retained both short and long term. However, for these higher levels of learning to occur, both students and faculty must be willing to do what it takes to facilitate transformative learning. This includes changes in practice and expectations for both parties. Faculty may find transformative assessment much more time-consuming to grade, while students may find it takes much more time and effort to complete. But if the end product is true knowledge and learning, transformative education is an excellent method for use in higher education.

49

Are Students and Faculty Ready for Transformative Learning?

1215

Transformative Learning Transformative learning essentially means to effect a change. In higher education, students primarily learn from faculty; however, faculty also learn from their students. Education is not based on commensalism – students can in fact effect a transformative change on a faculty member through events that take place in the classroom or throughout the student/faculty relationship during a program of study. Paolo Freire (1970) developed some initial theories on which Mezirow built upon to formulate the theory of transformative learning. Freire called the practice of faculty primarily lecturing to students in higher education the “banking method” where professors deposit information and students accept it. This view identifies students as empty vessels which need to be filled without any regard to the student’s beliefs or experiences. This also makes the faculty member the exclusive “keeper” of knowledge. In today’s society where the answer to most any question can be “Googled” in just a few seconds, this is absolutely not true, if it even ever was. Mezirow (1990) hypothesized when learning occurs, the student interprets the new information based on previous experience. This best happens as a product of reflection on the learning itself. According to Mezirow, reflection on learning includes making inferences, discriminating how the information meets or challenges preconceived notions, evaluating the information itself, and, last, solving a problem or dilemma. This last stage can include deciding if the information meets the students’ morals and ethical beliefs or if they challenge these beliefs. This process is continual, and throughout their lives, humans continue to evaluate both knowledge as actions. Mezirow specifically discriminates between active interpretation of knowledge and reflective interpretation. Active interpretation happens very quickly before all of the facts and nuances are evaluated. Reflective interpretation of learning takes longer and is usually less biased because most or all aspects have been evaluated and filtered through the learner’s experiences and beliefs. In essence, learning which happens too quickly may not be as easily processed or maintained. In 1978, Mezirow identified ten steps that are required in order for transformative learning to occur. These ten steps are: (a) a disorienting dilemma; (b) selfexamination; (c) discontentment, realizing others are also discontent and have changed; (d) evaluation of potential options; (e) critical assessment of personal assumptions; (f) experimenting with new roles; (g) planning a course of action; (h) attaining knowledge and skills to realize action plan; (i) attainment of competence in new role; and (j) reintegration of new perspective. Not all of these steps are required for a learner to learn transformatively; some steps may be omitted (Mezirow, 2000). However, when faculty are designing curriculum and wish to use transformative learning principles, these ten steps should be considered. Sterling (2011) reported that not everything a student learns spurs them to action, no matter how the faculty member has tried to include transformative education principles. Certain concepts will speak to certain students, while some students will be able to relate to others. Ison and Russell (2000) identified two levels of change that are driven by learning: first and second order change. First order change is the type of change

1216

C. Halupa

that occurs with traditional pedagogies where lecture and testing are the primary modalities used. In first order learning, students may learn the content for a test, but it never really makes any long-term impact on their lives. What they have learned is quickly forgotten. Second order change impacts both the way a student thinks and believes and the way he acts. This may be a service learning experience that assists a student in realizing what he wants for his career or something at a much less significant level such as a student who practices and reflects on how to do algebra problems and, after much practice and struggle, finally understands the concepts. This understanding is likely not to be easily forgotten and will be retained for much longer. Sterling had different terms for these two levels of learning. He called first order change cognition and second order change metacognition. He noted multiple levels exist between cognition and true metacognition. But essentially these two theories from Ison and Russell, as well as Sterling, are very similar. However, Sterling did add a third level, a type of third order change which he called epistemic learning. This type of learning changes a student’s worldview. Mezirow (1990) wrote, “Reflection is generally used as a synonym for higher order mental processes” (p. 8). It allows students to gain new understandings and appreciation. He noted reflection is different from thinking because it requires additional analysis. This integration of reflection and relevant, problem-based assessment may not seem difficult at first glance; however, it can be very challenging for faculty to do because this type of assessment takes a great deal of time to create. In addition, it may be very different from the type of assessment students are used to. Reflective exercises are used as a key educational strategy in transformative learning to assist students in reaching second or even third order change. Reflective exercises can be done in most subject areas, although it is much easier to develop them in some disciplines than in others. However, outside of education and psychology, reflective exercises may not be viewed as “real assessment” because these types of activities rarely require APA style and references. However, even without these two requirements, reflective exercises are high-level evaluation assessments if designed properly. It has already been noted transformative learning is difficult to incorporate into teaching strategies for some faculty. However, integrating transformative practices is really no different than integrating state education requirements or integrating programmatic accreditation requirements which is already being done by most faculty. How and when it is integrated is dependent on the discipline and the university where the faculty is employed. Kang (2013) noted transformative learning in faculty is even more critical in Christian higher education where professors not only have to teach the subject matter but also have to attempt to seamlessly integrate Christian principles, ethics, and morals in the curriculum at the same time. This skill in integration is a key characteristic of transformative educators. If transformational learning results in a more robust, meaningful, long-term learning experience for the students in higher education, then why does resistance to it occur? This is a multifaceted answer. Resistance occurs in faculty for a multitude

49

Are Students and Faculty Ready for Transformative Learning?

1217

of reasons. In addition, resistance also occurs in students. Since it occurs more frequently, faculty resistance will be discussed first.

Faculty Resistance Armstrong (2014) noted pedagogical change is a disruptive change which naturally leads to resistance. Fink (2003) noted although faculty want students to reach higher levels of learning, they continue to use primarily lecture-based teaching which does not assist students in developing critical thinking and problem-solving skills. In addition, students retain less information at the end of a lecture-based course. India Lane (2007, p. 87) listed several factors which contribute to faculty resistance to changes in academia: (a) strong existing traditions or paradigms, (b) lack of perceived need for change, (c) autonomy and independence of individuals involved, (d) strong professional or discipline identification, (e) department or disciplinary protection of curricular time, (f) conservative education practices, (g) skepticism of educational theory or alternate pedagogy views, (h) perceived attack on training or current teaching practice, (i) lack of experience or hard data to support change, (j) lack of rewards for teaching innovation or change, (k) lack of time to study or implement changes, (l) ineffective curriculum committee structure, (m) fear of loss of resources, (n) fear of loss of accreditation, and (o) fear of impact on students’ exam performance (i.e., certification and licensure). Tahiri (2010) noted several related reasons why professors do not engage in pedagogical change: (a) fear of losing their jobs, (b) fear of endorsing and making genuine change, (c) disinterest, (d) low self-efficacy, (e) resistance to change their attitudes, (f) rely on outdated teaching styles, (g) prefer authoritarian teaching environment, and (h) resistance to attending in-services or professional courses on pedagogy (p. 151). Tagg (2012) suggests faculty resist change because they are humans. Innately to ensure survival of the human race, humans were risk adverse. Some of these has carried over even today. Some early innovator faculty may be risk takers, but only a small number will step out to try something new. The remainder will adopt an attitude of wait and see; in other words, to see what happens to the faculty who stepped up to initiate changes first. This is part of the survival instinct. If the results for the early innovators are negative, then the others never have to change. Tagg also notes the pedagogies in traditional higher education are in conflict with human cognition and the way humans learn. In addition, faculty have not been routinely interested in course quality. He posits, “Loss aversion and the endowment effect add up to the status quo bias, a pervasive preference for leaving things as they are” (p. 5). He goes on to surmise most faculty link tenure to research rather than quality teaching. Kitchenham (2015) supports Tagg’s view. He says: In my experience, there is very little variety in assessment practices in the Academy as professors are stuck on what they have used in the past (and in many cases, for the last 20 years) rather than examining what content has changed and how students have changed.

1218

C. Halupa

I still see courses that use three “midterms” (an oxymoron) worth 30 percent each with a token 10 percent for “participation). In other words, the professors are not even considering that this form of assessment relies on the false assumption that learning can be demonstrated through a 100-item multiple-choice examination offered three times in a term rather than realizing that their choice is much more about the ease of marking (p. 15).

Keeling and Hersh (2012) noted students would experience higher education differently if a culture of learning was truly established and learning always came first. They note this would include “elevated expectations and support for students,” as well as “rigorous and comprehensive assessment of student learning (p. 2). They noted this should include formative and summative assessment, as well as learning experiences inside and outside the classroom. They also note student learning should be an integral part of every faculty and staff member’s annual evaluation and purposeful closing of the assessment loop must be done to ensure students can do what the university said they should be able to upon graduation. To sustain these robust changes, continuing faculty development is needed. Oleson and Hora (2013) noted a mantra in higher education is “faculty teach the way they were taught” (p. 2). This was also noted by Baran, Correia and Thompson (2011), as well as Kreber and Kanuka (2006). In Oleson and Hora’s study of 53 STEM faculty, they identified four themes that influenced faculty teaching: experiences as a student, experiences as a teacher, experiences as a researcher, and personal experiences. They note teaching practice is not a linear process. They recommend exposing faculty to bodies of literature on innovative teaching practices so the one which most approximates the faculty member’s desired teaching style can be adopted. Brownell and Tanner (2012) note faculty are resistant to pedagogical changes due to a lack of time, training, resources, and incentives. They note institutional change and pedagogical change are difficult essentially because this change is viewed by faculty as an indication that they have been doing something wrong the last several decades they have been teaching. However, although higher education remained essentially the same for centuries, it has undergone more changes in the last two decades than it did in the two centuries before. Brownell and Tanner point out these changes can negatively impact faculty members’ professional identities as well. This includes how they view themselves, their discipline, their work, and their students. They note many faculty do not feel well equipped to change the way they teach and may revert back to what is comfortable rather than what may be most effective. In addition, faculty may truly feel that emerging pedagogies are not proven and may not truly be effective (Brownell & Tanner, 2012; Miller, Martineau & Clark, 2000; Winter, Lemons, Bookman & Hoese, 2001; Yarnall, Toyama, Gong, Ayers & Ostrander, 2007). When faculty go to professional meetings and continuing education, they may become excited about new teaching methods, yet when they return to the university they may face resistance from colleagues and administration. Brownell and Tanner (2012) note pedagogical change is an “iterative and ongoing process” (p. 340). They also note the difficulty of carving out time to experiment with new types of assessments while trying to juggle teaching loads, research, and

49

Are Students and Faculty Ready for Transformative Learning?

1219

university and community service. Miller et al. (2000) noted creating curriculum that challenges and engages students requires much more preparation time than traditional lecturing. This is because lecture is more spontaneous because the faculty is an expert in the field, while assessments and activities that build critical thinking often need to be scaffolded. This requires faculty to plan out activities well in advance which many faculty do not like to do. Brownell and Tanner (2012) note even if incentives are given to faculty to change the pedagogical methods and premises they use in the classroom, faculty have to have a predisposing reason to want to do it. The types of incentives that can be used include overload pay (if the curricular changes are done while teaching a full load), course releases, faculty recognition, and additional monies for scholarship. Researchers such as Wilson (2010) and Anderson et al. (2011) have noted these incentives are not widespread in US higher education. Overall, particularly in large universities, research is valued more than teaching. This is likely because it brings funds into the university and yields personal recognition for faculty members in a “publish or perish” environment. In fact, Anderson (2007) noted using new pedagogical methods may easily result in poor student evaluations which can impact tenure. This is likely because students are not used to these new methods and in many cases may find them much more difficult and challenging. Transformative activities are designed to increase knowledge and critical thinking rather than just regurgitate information to get a good grade. This regurgitation pattern is what many students are used to because of the standardized test-focused K-12 environment in the United States. Some students may welcome the challenge, while others just do not want to have to work that hard. Brownell and Tanner (2012) note even in a perfect situation where faculty had time, incentives, and support, this does not necessarily mean pedagogical change will happen. This is because university faculty tend to be resistant to change. Other factors such as peer pressure and even faculty motivation and desire for the status quo can prevent pedagogical change. Ultimately faculty who teach in the university setting are often just not taught to teach; it is a tertiary consideration after expertise in the discipline and the desire for promotion and tenure overall. They noted faculty who identify as teachers, rather than experts in a discipline, can undermine their professional status. Faculty often do not perceive being a teacher and a practitioner equally in a discipline; however, both roles can successfully coexist. Teaching may be what faculty do, but their personal identity is tied to their discipline as scientists, engineers, writers, historians, etc. Brownell and Tanner note in the field of science teaching ranks low among the tasks and personal identities of faculty. Beath, Poyago-Theotoky, and Ulph (2012) noted in scientific disciplines teaching ranks well below research and interaction in the scientific community. Lloyd Armstrong, Provost, and Professor Emeritus at the University of Southern California, in a publication of the TIAA-CREF Institute (2015), reported higher education institutions, and faculty are historically slow to change. He notes, “In higher education, success in the realm of research has a strong influence on overall institutional reputation” (p. 6). Teaching and research are intertwined and in a constant battle

1220

C. Halupa

with one another in regard to a faculty member’s time and attention. Institutional reputation unfortunately is not built on an institution’s teaching reputation although perhaps it should be. Brownell and Tanner (2012) suggest pedagogy education should be integrated into discipline-specific programs at the doctoral level to increase teaching skill and decrease resistance to pedagogical change. This way in addition to being experts in their discipline, future faculty will at least have a rudimentary knowledge of how to teach. Sabagh and Saroya (2014) report teaching strategies should be innovative and engage students. However, faculty may fear failure with new techniques because it may have a permanent impact on how students can view learning. Armstrong (2014) noted many faculty can perceive pedagogical change as decreasing quality of the educational experience. Professors have to clearly understand new pedagogies, and one of the most effective ways for education is provided for faculty is through professional development activities. However, they note the teaching of improvement in educational practices and activities has overall made little impact in the way faculty teach overall. But small pockets of success can develop, and this success can become pervasive. Sabagh and Saroyan found in their survey of over 1,600 university professors in Canada almost 50% of professors perceived high workload as a barrier to implementing pedagogical changes. They called for additional teaching incentives to encourage and assist faculty in making these changes. A second problem with enacting pedagogical change is the extensive use of adjunct faculty in colleges and universities. The American Association of University Professors (AAUP) reports more than 50% of faculty are part time or adjunct. Kezar and Maxey (2014) also reported faculty who were not on a tenure track tended to use less student-centered and active teaching approaches. Part-time and adjunct professors often do not receive continuing education funds from the universities where they teach; many have full-time jobs and cannot attend university professional development sessions and are not as invested in the university as full-time faculty. Although there are excellent adjunct faculty who are interested in teaching methods that are optimal for their students, many adjuncts also teach for multiple universities. This may result in a high teaching load in order to meet their personal financial needs. This may leave them little time to create assessments and activities that are student centered, problem based, and transformative. Particularly in the online environment, adjunct faculty may teach standardized courses that have been written by someone else, and they cannot change the curriculum. Universities often pay adjunct and part-time faculty to write such courses since their full-time faculty already have full teaching loads. Because it is cost-prohibitive, this may lead to less frequent updating to standardized online curriculum. Adjunct faculty also often do not feel truly a part of the university, and this may result in these faculty not recommending changes in the curriculum for fear of losing the teaching position. However, resistance does not only occur in faculty.

49

Are Students and Faculty Ready for Transformative Learning?

1221

Student Resistance Faculty are not the only ones that can be resistant to pedagogical change. Students can be resistant to pedagogical changes as well. A course that is lecture based requires much less from a student than one that contains authentic assessments. In transformative learning, these assessments will include problem-based learning and reflective assignments that may take a great deal of student time and planning. Peerto-peer learning is also an important part of authentic assessment since students will be required to collaborate in the real world on the job. Yet many students do not like to engage in group work for a multitude of reasons with the main one being that not all group members pull their weight in the group assessment process. Blaise and Eisden-Clifton (2007) noted when changes were made that transformed a course into one that enhanced critical thinking and was more student centered, students can rebel, particularly with group work. When students were supposed to be working in groups, some left campus and did not interact. Students were also resistant to meeting outside of class time. Blaise and Eisden-Clifton noted although they (the researchers) thought the revised curriculum was exciting, some of the students did not. Instead, some students questioned the relevancy of the material and the assessments they had to do. This was a form of student resistance to a different pedagogical approach. The authors noted critical pedagogy is often met with resistance in the classroom. In their study students also complained the assessment methods would prevent them from doing well in the course. This reflects the premise that many students in higher education today are much more concerned with grades than learning. Although students may feel better grades will “get” them a job, the fact is true learning of what is required in their field will allow them to keep it, progress, and succeed. Baumgartner (2001) noted students who consider a faculty member as a significant authority figure may be unwilling to engage in the type of discourse that is present in transformative learning. Faculty who are viewed as too much of an authoritarian figure may be perceived by students as unwilling to accept and respect beliefs other than his/her own. This can occur whether this is true or not. Blin and Munro (2008) reported students have a predisposed expectation of how they should learn. They note faculty have specific expectations as well, and they may not always be the same. Implementing emerging pedagogies such as transformative education is a disruptive innovation (Christensen, Horn & Johnson, 2008). In addition to being disruptive for faculty, they are also disruptive for students. Because a great deal of the assessment in secondary education in the United States is primarily standard objective-type testing such as multiple choice and true false questions, students may have difficulty adapting to completing assignments which require them to apply and evaluate, rather than just recall and comprehend. This can also cause fear in students related to both their performance in the class and their overall grade. Duarte (2010) noted student cynicism can be addressed through transformative education. Cynicism is also student resistance in many cases; however, cynicism can also drive students to go on a quest for the truth about a particular issue. This journey

1222

C. Halupa

is transformative education as the student explores, evaluates, and reflects on his/her beliefs about an issue. This journey may result in either a change in beliefs or an even a stronger conviction of previous beliefs. Duarte notes this process can be emancipatory. Even though resistance may exist in both faculty and students, the benefits of transformative learning well outweigh the risks. With transformative learning, the benefit is long term as well as short term.

Faculty Transformation Teaching was the hardest work I had ever done, and it remains the hardest work I have done to date. –Ann Richards

Patricia Cranton, who has published extensively about transformative learning, noted in 1994 that faculty themselves are adult learners who have little training in instructional strategies and often do not view themselves primarily as teachers. She recommends professional development for faculty to learn to use transformative education practices to model the creation of meaning perspectives and reflection. Why does a faculty member teach the way he/she does? What is the true reason, and is that reason a good one or does the faculty member just emulate a teacher he/she had, or decide to be different from a teacher he/she struggled with as a student? She notes it is crucial that faculty truly see themselves as teachers and not just practitioners. Cranton noted when faculty are called upon to truly reflect on their teaching behaviors and their consequences, they tend to externalize the cause blaming students and administration rather than internalizing it and looking to themselves and what they could do better. Kucukaydin and Cranton (2013) note transformative learning is a theory in progress. They proposed transformative learning is an extra-rational, postmodern epistemology where learners can critique each other’s perspectives and knowledge with an open mind and effectively communicate differences. They note knowledge is subjective, and through reflection of one’s own knowledge, truth is sought. However, one’s truth should be open to questioning. This can include the creation of assignments that use deductive logic; however, one of the challenges for faculty is these types of assessments are time-consuming to create, as well as grade. In addition, students coming out of a secondary school environment where they are drilled and treated as passive vessels waiting to be filled may not be ready for these types of assessments. Using Vygotsky’s theory of proximal development, one solution may be to create basic assessments that include nominal increments of deductive knowledge to teach students to evaluate knowledge more critically. These assessments can be scaffolded until students adapt to the new learning style. Heddy and Pugh (2015) suggested since transformative learning can be difficult for faculty to implement, transformative experiences can be implemented on a micro level.

49

Are Students and Faculty Ready for Transformative Learning?

1223

These transformative experiences can cross over course boundaries and eventually lead to transformative learning. According to Mezirow (1997), faculty must change roles from a lecturer who delivers knowledge to a facilitator who teaches students to learn, explore, and evaluate on their own with faculty assistance. However, before faculty members can do this, they must first critically reflect on their current teaching practices. This includes content reflection which is also called instructional knowledge. This is a reflection of why the content is selected for a course and also includes reflection on the appropriateness of delivery methods. The following is evidence of faculty content reflection as identified by Kreber and Canton (2000): (a) discussing materials and methods with students and peers, (b) reading professional journals, (c) keeping a journal of methods that worked and did not work, (d) administering learning style inventories to students, (e) keeping up with educational theory, (f) keeping current with trends in higher education, and (g) adding a rationale to course syllabus. The second type of reflection recommended by Kreber and Cranton (2000) is process reflection. This is knowledge on how to design a course structurally to enhance student learning, as well as meet accreditation standards. As part of the reflective process, the faculty member must ask herself some questions. What is it that I do well? What have I done that works well with some classes and not for others? Why? What is my personal philosophy of education? If a faculty member has not created a written teaching philosophy in the past, the creation or revision of one can be an effective way to guide this reflection. Evidence of process reflection can include: (a) collecting data on students’ perception of course materials and teaching methods, (b) asking peers to review course material, (c) comparing the findings in the classroom to research results, and (d) gathering feedback from students on the learning process. The last type of reflection is premise reflection which is transformative in nature. This includes reflecting on the quality of the course itself and how it fits in the university goals and/or program of study. Premise reflection is a key component of formal program evaluation. According to Kreber and Cranton (2000), this can include the following: (a) utilizing alternate methods and assignments to obtain course goals, (b) critiquing methods of teaching, (c) challenging institutional norms in regard to teaching methods, (d) participating in philosophical discussions about teaching and learning, and (e) asking for feedback from employers to determine if graduates are meeting their needs. Faculty can also use Mertonian criteria to their teaching performance. These criteria complement transformative education. These criteria are as follows: (a) expert content knowledge, (b) innovative approach to material delivery, (c) elaboration of applicability of knowledge, and (d) highlighting the relevancy of the work (Kreber & Cranton, 2000). In essence, students need to be taught to be in control of their own learning. A faculty member who meets these criteria can better help facilitate this process. This can improve the educational process for everyone. According to Cranton (2002) there “is no particular teaching method that will guarantee transformative learning” (p. 66). Instead, a variety of methods may have to

1224

C. Halupa

be tested until the faculty member determines how best the majority of students respond to different types of authentic assessment. However, transformative faculty do not give student pointless work. Students need to develop skills they will need to succeed on the job and in life, and the best way to do this is for them to practice these things in a safe environment while they are completing their education. Sterling (2011) recommends in order for transformative learning to occur, large classes of students must be broken down into small groups that can effectively interact. In addition, he notes the faculty member must create an environment conducive to online learning which may result in the faculty member being viewed less as an authority figure and more as a coach. Sterling also listed an intensive residential experience as an environmental factor that can enhance transformative learning. However, transformative learning can and does work in the asynchronous online environment as well. Effective discourse can be done with synchronous meeting technology, as well as robust discussion forums that include Socratic questioning. Based on Mezirow’s multiple publications on transformative learning, the following assessments can be used in transformative learning: (a) assessments that challenge student assumptions; (b) engaging in effective discourse (oral and written); (c) completing reflective assessments (such as journaling or a reflective paper); (d) assessments where all of the answers are not in the text or provided course material so students can learn to find answers for themselves; (e) evaluative assessments where students have to filter the question posed through the lens of their own personal perceptions, beliefs, and values; (f) assessments which address scenarios or situations a student will face in their personal lives or on the job to teach critical thinking; (g) critical incident training; and (h) a plethora of different types of problem-based and authentic assessments (Mezirow, 1978, 1985, 1997, 1998, 2000, 2003, 2006). Edward Taylor (2007) wrote transformative teaching is a process of constant change and adjustment. Institutional needs, faculty needs, and student needs may change over time. In addition, instructional methods must be carefully selected to match the content being assessed. He notes faculty must learn to trust transformative learning and realize it can work, but it is not an absolute science. Rather, it is the art of finding what enhances learning best in each group of students. Because transformative learning is a qualitative measure of beliefs and perceptions, it is difficult to measure in a quantitative manner. However, the faculty selfassessments can assist faculty in determining their readiness to transform their teaching practices from traditional lecture-based learning to transformative learning which is more student centered and authentic. These assessments were created based on the work of Mezirow and other authors and the citations accompany each of the questions, as does the scoring mechanism. The first assessment was designed to evaluate faculty instructional factors (see Appendix A), and the second was designed to evaluate faculty personal factors (see Appendix B) in regard to transformative learning. Transformative learning requires the synergy of both the faculty member and the student. How can faculty assess if their students are ready for transformative learning

49

Are Students and Faculty Ready for Transformative Learning?

1225

and instructional strategies? Why are some accepting and others resistant? This will be discussed in the next section. Pedagogical change is often voluntary, but there may be instances where it is not voluntary and administration has decided to adapt emerging pedagogies to either improve teaching or to provide a market edge. Sinclair and Faltin Osborn (2014) reported there are four key themes to faculty resistance to pedagogical change that is imposed by administration: (a) fear and anxiety, (b) encouragement without support, (c) insufficient training, and (d) student resistance to new pedagogies. One of the major concerns about student resistance to new pedagogies in the classroom is the faculty evaluation. Faculty members fear bad evaluations because it can impact their careers. In order to decrease this problem, faculty can assist students to transform their beliefs about learning and education. One last thought on faculty transformation. . . I have learned that, although I am a good teacher, I am a much better student, and I was blessed to learn valuable lessons from my students on a daily basis. They taught me the importance of teaching to a student – and not to a test. –Erin Gruwell

Student Transformation Much education today is monumentally ineffective. All too often we are giving young people cut flowers when we should be teaching them to grow their own plants. –John W. Gardner

Most students who come to the classroom, particularly in undergraduate education at the freshman and sophomore level, are not transformative learners. They have extensively been exposed to traditional methods of teaching and assessment with the current concentration on performance on standardized tests in secondary education. Initially, a faculty member’s goal is to teach these students metaliteracy skills and how to become independent learners since so many students come to higher education as totally dependent learners. The Alpha Omega Academy’s blog listed the following characteristics of independent learners: (a) curiosity, (b) motivation, (c) ability to self-examine, (d) accountability for their actions, (e) ability to think critically, (f) can comprehend material with little or no instruction, and (g) persistence (n.d.). Tahiri (2010) notes in order for transformative education to occur students must: (a) acknowledge they are equal partners with the professor in the learning experience, (b) be open for change, (c) be willing to determine their own reality, (d) be willing to share their life event, (e) be willing to engage in critical reflection, and (f) show maturity on dealing with change (pp. 152–153). Tahiri notes this process is risky for students because they may not get the external reward (grades, etc.) they do in the passive learning process. Courses that

1226

C. Halupa

are transformative are more challenging, and it may be more difficult to perform at a high level on all assessments. In addition to teaching students to become independent learners, another purpose of higher education is to prepare students for a job and for the challenges of life. Essentially, education changes a student’s worldview at least in some ways although it may not impact personal ethical and moral beliefs. Brock (2009) noted transformative learning can be gradual or “cataclysmic” (p. 124). As a student matures, learning becomes more reflective and critical. This requires maturity in mental processes, and some students mature more quickly than others. Wald, Borkan, Taylor, Anthony, and Reis (2012) noted reflection is not intuitive in students even at the graduate level. The earlier students learn to reflect, the more effective the educational process can be. Schon (1983) noted reflective capacity progresses through the following stages: (a) knowing in action, (b) surprise, (c) reflection in action, (d) experimentation, and (e) reflection on action. This final stage is when students become engaged in transformative learning. Transformative learning is not passive; it is active. The journey is different and personal for each and every student and faculty member. Some students still do not reach the final step of the transformative education process noted by Mezirow until after the completion of an associates or bachelors’ degree. In some, it does not occur until the former student has been employed for a while. In a few this final step will never be realized (Kegan & Miller, 2003). Moore (2005) postulated that students who find transformative education content delivery and assessment more challenging than the lower level recall and comprehension educational practices they are used to can experience frustration and anxiety. They are not used to the level of education transformative education requires. This changing of a student’s worldview can be a difficult process for both the student and the faculty member. However, this transformation does create a better global citizen who can carefully evaluate problems and issues to make more effective decisions. It also strengthens and helps embed moral character as well. Transformative learning is specifically designed to present students with disorienting dilemmas that challenge their knowledge and beliefs. This is not an easy transition for students who have never been exposed to these strategies before. However, it is what is needed in today’s complex world with complex problems. Kezar and Maxey (2014) reported interactions between faculty and students “improve the quality of student learning and their educational experiences” (p. 30). In addition, they noted collaborative learning can further increase academic success. Faculty can assist in instilling a sense of passion and motivation for learning. Faculty can do this by valuing student contributions and their quest for knowledge. Working together with faculty on something such as a research project can increase the attainment of higher level cognitive skills in students. White and Nitkin (2014) and Bamber (2016) suggest student service projects within the community or throughout the world can be transformative and cause students to view the world differently because they are often presented with a disorienting dilemma. As Tahiri (2010) postulated, students must gain personal self-confidence and realize they have something to contribute to the learning process. Students must be willing

49

Are Students and Faculty Ready for Transformative Learning?

1227

to take risks and accept change hoping that on the other side of the experience they will be a better potential employee and perhaps a better person. Transformative education requires motivation and the maturity to really reflect on the education experience. Most of all, it requires students to place learning above all other competing priorities in their college life; this is perhaps the most difficult step of all. Students have to be willing to evaluate both sides of an issue and create their own learning and meaning in the educational process. Although transformative learning is relatively new when compared to centuries of traditional learning, there have been some success stories. It can be theorized that transformative education ability increases with the level of education, as well as exposure to transformative practices. King (2011) found there were no significant differences in student empowerment in transformative education based on gender and age. Brock (2009) looked at a sample of undergraduate students over a 2-year period and found at least 66% of them experienced some form of transformative learning during this period. Stevens-Lang, Shapiro, and McClintock (2012) studied transformative factors in doctoral students up to 5 years post-graduation. These 393 graduates identified the transformative practices which took place in their doctoral program that most impacted their ability to be successful on the job. Three factors were noted with the first being curriculum that required them to look at all aspects and perspectives of an issue. The second was practice in developing interpersonal relationships where effective discourse can take place even if parties do not agree. The last was providing learning that was experiential which assisted them in becoming reflective, self-directed learners. Faculty members who are interested in trying transformative approaches may want to assess where their students are in the journey of transformative learning. This can assist the faculty member in tailoring instructional strategies for groups of students who are ready for transformative learning and prevent a faculty member from introducing too much transformative learning content and assessment when a group of students is not ready for it. A companion instrument to the faculty assessment instrument to assess student readiness and place in the transformational learning journey can be found in Appendix C. This assessment is just a guide to assist faculty in using transformative strategies in the classroom. As these strategies are used, the hope is students will continue to advance in their educational journey.

Recommendations The one exclusive sign of thorough knowledge is the power of teaching. –Aristotle

First, as Brownell and Tanner (2012) suggested, in order for new emerging pedagogies to be more globally adopted, teaching practice should be incorporated into doctoral programs to provide additional electives for those who plan to become faculty rather than advanced practitioners in the field. This would provide future

1228

C. Halupa

faculty a level of baseline knowledge that could be built upon at the university level through professional development and continuing education offerings. At the university level, Sabagh and Saroyan (2014) recommend if administration values pedagogical changes, then changes must be initiated university wide. This includes encouraging the building of pedagogical skills through educational opportunities, incentives for teaching, and a concentration on teaching even more than on research and service. These formalized efforts may reduce faculty-perceived barriers to these changes. This is best done one department at a time by building a community of learning and practice in each department. When departmental “wins” are recognized and even incentivized, the practices may disseminate. This dissemination may occur due to faculty competition and peer pressure, as well as students’ call for a more meaningful educational experiences. As with any initiative in higher education, change must be encouraged, supported, and perhaps even incentivized, but not dictated to increase the chances for success. This can include such things as weighing teaching significantly more in faculty evaluations than research and service. In addition, administration should also inform students of this movement to advanced pedagogical practices to enhance the quality of their educational experience. This can increase student buy-in, as well as decrease student resistance and cynicism. At the department level, deans can incentivize faculty to utilize transformative techniques by offering course releases or monetary incentives (if possible) to provide additional time to create robust, transformative assessments. Although it would be wonderful to believe all faculty would do these things because enhancing student learning is the right thing to do, pragmatically faculty time is limited and incentives may be necessary. If administration is highly supportive of pedagogical changes, then additional funds for these incentives may be provided. Definitively the most significant changes overall happen at the personal level for faculty and students. However, these may not occur if changes are not made at the systemic, university, and department level. Illerris (2015) points out transformative learning cannot be taught; he reflects it is an “internal process” and the educator can only try to facilitate the “environment, situations, procedures, content, and teaching in ways which optimize or promote the probability of transformative learning” (p. 46). One key thing to remember is as faculty members, it is not possible to help every student transform. Most of the time faculty never find out the true impact they may have made in just one student’s life. Faculty can spur students on to success or embed unrealistic expectations about life that can eventually lead to defeat at some level. In a perfect world to educate transformatively, faculty would educate themselves about learning on a frequent and ongoing basis and would always put learning before student complaints, annual evaluations, personal ease, personal gain, and monetary gain. But we do not live in a perfect world in higher education, and faculty must often make do with what they have. Faculty often feel they have so little time and they have to cram so much content into a short period. This feeling is exacerbated by traditional teaching methods where faculty are the deliverers of material and also the authoritarian who has decided if the students have “cut it” or

49

Are Students and Faculty Ready for Transformative Learning?

1229

not. But no matter what the constraints, faculty can still begin to implement what Heddy and Pugh (2015) called micro-level changes. Transforming teaching is literally done one class at a time or one assessment at a time. It can be as simple as slowing down and encouraging students in one class to really talk and reflect on what they have learned so far and what it means. It can be substituting a group project for a standardized test or changing one assignment from an APA style paper with ten references to a paper where students are required to reflect on what they have learned so far and what they can do with that knowledge now and in the future. Then the next time the course is offered, a few more changes can be initiated. This eases the transition for faculty and for the students. Last, students must be willing to mature and realize higher education is not high school. In life, they will not be given easy assignments to get a good grade. They need to be willing to accept more challenging assessments that require them to apply, analyze, evaluate, and create rather than just recall basic facts. Overall, they need to know how to truly learn for life.

Conclusion Transformative learning is a journey, not a process, for faculty and students. It is constantly being improved upon. The most difficult part of this journey, as with any journey, is the first step. In addition, a key realization for both parties is the long-term benefits are much more important than short-term benefits such as grades. By overcoming fear and preconceived notions, together students and faculty can work together to assist students to be better learners, better future employees, and better citizens.

Appendix A: Transformative Learning Readiness Scale Instructional Factors This assessment is answered and scored using the following Likert-type scale. Please answer the questions using the number that best matches your instructional practices. 1. Never 2. Rarely 3. Sometimes 4. Often 5. Always 1. I create student assignments that cause them to reflect on the topic they are leaning and create new meaning (Mezirow, 1998). 2. I require my students to explore the ethical values of the field they are studying (Mezirow, 1998). 3. I require my students to evaluate their own ethical values (Mezirow, 1998). 4. I incorporate intuitive assignments in my courses that allow students to explore problems in new and different ways (Mezirow, 1998). 5. I spend a great deal of time in the classroom lecturing (Freire, 1973). 6. The assignments I give allow my students to grow on an intellectual level (Mezirow, 2006).

1230

C. Halupa

7. I encourage my students to evaluate what they know in order to learn something new (Mezirow, 2006). 8. I include assignments that make students look at things in new and different ways (Mezirow, 2006). 9. Most of my assignments are problem based (Mezirow, 2006). 10. I frequently use objective tests such as multiple choice and true/false questions (Kitchenham, 2015). 11. When I design my instruction, I consider students’ different learning styles (visual, auditory kinesthetic) (Fleming, 2001). 12. When I design my instruction, I consider student interpersonal and intrapersonal learning intelligences. (Gardner, 1985) 13. I utilize publisher test bank questions frequently to assess my students (Buckes & Siegfried, 2006; Roofe-Steffen, Shmaefsky & Griffin, 2014; Rutgers University, 2016). 14. When I design my instruction, I consider student linguistic and mathematical intelligences (Gardner, 1985). 15. I encourage my students to discuss and dialogue to solve complex issues (Mezirow, 1990). 16. I pose dilemmas and have my students find a variety of solutions to evaluate them (Mezirow, 1998; Miller, 2012). 17. I teach my students how to make technology work for them in the learning process (Miller, 2012). 18. I believe students must provide the predetermined answer on assessments in order to succeed in my class (Friere, 1970). 19. I primarily use textbooks to teach without additional supplementary resources. 20. I use multiple types of methods to deliver instruction to my students (Miller, 2012). _____ TOTAL Part I Scoring Reverse code #5, 10, 13, and 19 Interpretation Score 0–35 35–54 55–71 72+

Interpretation of instructional strategies Traditional Slightly transformative Utilizes some transformative instructional techniques regularly Very transformative in assessment and presentation of content

49

Are Students and Faculty Ready for Transformative Learning?

1231

Appendix B: Transformative Learning Readiness Scale Faculty Personal Factors Please answer the questions using the following Likert-type scale (which corresponds to the points allotted to the answer to each question). 1. Strongly Agree 2. Agree 3. Neutral 4. Disagree 5. Strongly Disagree 1. I reflect on how I impart knowledge to others as an educator (Freire, 1973; Mezirow, 1990). 2. I reflect on what I know (Freire, 1973; Mezirow, 1990). 3. I reflect on what I do not know (Freire, 1973; Mezirow, 1990). 4. I find I often do not know what I thought I knew (my beliefs have been challenged) (Freire, 1973; Mezirow, 1990). 5. It is my job as an educator to deliver the information (Kitchenham, 2008). 6. I know each of my student’s strengths (both academic and personal) (Kitchenham, 2008). 7. I know each of my student’s weaknesses (both academic and personal) Kitchenham, 2008). 8. After I teach a course, I alter my curriculum based on what worked and did not work with that section of the class (Kitchenham, 2008). 9. Student learning is of great concern to me (Kitchenham, 2008). 10. I teach the way I do primarily for (Kitchenham, 2008): a. Myself; I am the subject matter expert (5 points) b. The most intelligent students in the room (4 points) c. The struggling students (3 points) d. The students in the middle who are not excelling nor struggling (2 points) e. All students (1 point) Scoring Reverse code #5 Interpretation Score 0–25 26–40 41+

Interpretation Transformative Somewhat transformative Traditional

1232

C. Halupa

Appendix C: Student Transformative Learning Readiness Assessment Please answer the questions below using the following Likert-type scale. The number before the answer that best approximates your beliefs of actions is the score to each question. 1. Strongly Disagree 2. Disagree 3. Neutral, 4. Agree 5. Strongly Agree __________________________________________________________________ 1. When I am learning about a topic in a classroom, I research information above and beyond what is required regarding the topic for class (Kitchenham, 2008). 2. I am dependent on the teacher to give me the information I need to pass a class (Kitchenham, 2008). 3. When I am learning something new, I reflect on the topic and how it relates to things I already know (Mezirow, 1990, 1991, 1997, 2000, 2003). 4. I like to ask why something is or works the way it does when I learn something new (Kitchenham, 2008). 5. If I struggle with a topic, I work harder until I understand it (Mezirow, 1998). 6. I am responsible for my own learning (Mezirow, 1990, 1991, 1997, 2000, 2003; Kitchenham, 2008). 7. I seek to learn “beyond the syllabus” (Kitchenham, 2008). 8. I am willing to consider ideas and points of view that are different than my own (Mezirow, 1990, 1991, 1997, 2000, 2003; Kitchenham, 2008). 9. I am concerned more with knowing the facts than the purpose or reason behind the facts (Kitchenham, 2008). 10. I like to find rather than memorize or know information (Kitchenham, 2008). 11. I enjoy discussion where interpretations of concepts can be discussed (Mezirow, 1990, 1991, 1997, 2000, 2003). 12. In my learning I make and interpret my own meaning (Mezirow, 1990, 1991, 1997, 2000, 2003). 13. I am an independent learner (Mezirow, 1990, 1991, 1997, 2000, 2003). 14. I want my professor to tell me what is expected (Kitchenham, 2008). 15. I tend not to focus on the big picture (Mezirow, 1990, 1991, 1997, 2000, 2003). Scoring Questions #2, 9, 14, and 15 must be reverse coded before calculating the final score. This means a score of 5 will become 1 and a score of 4 will become 2 and vice versa.

49

Are Students and Faculty Ready for Transformative Learning?

1233

Interpretation Score Less than 30 30–43 44+

Interpretation Student has likely not been exposed much to transformative instructional strategies. Small numbers of transformative strategies should be introduced Student has had some exposure to transformative instructional strategies. Additional transformative strategies can be incorporated into the curriculum Students are transformative and are reflective and self-directed. They are ready for the use of additional transformative instructional strategies

References Alpha Omega Academy Blog. (n.d.). 7 characteristics of independent learners. Retrieved from http://aoaacademy.com/blog/trends-and-tips/7-characteristics-of-independent-learners/ American Association of University Professors. (2016). Background facts on contingent faculty. Retrieved from https://www.aaup.org/issues/ contingency/background-facts Anderson, W. A., Banerjee, U., Drennan, C. L., Elgin, S. C. R., Epstein, I. R., Handelsman, J.,. . . & Strobel, S. A. (2011). Changing the culture of science education at research universities. Science, 331(6014), 152–153. https://doi.org/10.1126/science.1198280. Armstrong, L. (2014, November). Barriers to innovation and change in higher education. Teachers Insurance and Annuity Association of America: College Retirement Equities Fund Institute. Retrieved from https://www.tiaainstitute.org/public/pdf/barriers-to-innovation-and-change-inhigher-education.pdf Bamber, P. M. (2016). Transformative education through international service-learning: Realising an ethical ecology of learning. London, England: Routledge. Baran, E., Correia, A., & Thompson, A. (2011). Transforming online teaching practice: Critical analysis of the literature on the roles and competencies of online teachers. Distance Education, 32(3), 421–439. https://doi.org/10.1080/01587919.2011.610293. Baumgartner, L. M. (2001). An update on transformational learning. New Directions for Adult and Continuing Education, 89, 15–24. https://doi.org/10.1002/ace.4. Beath, J., Poyago-Theotoky, J., & Ulph, D. (2012). University funding systems: Impact on research and teaching. Economics, 6, 2012-2. https://doi.org/10.5018/economics-ejournal.ja.2012-2. Blaise, M., & Elden-Clifton, J. (2007). Intervening or ignoring: Learning about teaching in new times. The Asia-Pacific Journal of Teacher Education, 35(4), 387–407. https://doi.org/10.1080/ 13598660701611404. Blin, F., & Munro, M. (2008). Why hasn’t technology disrupted academics’ teaching practices? Understanding resistance to change through the lens of activity theory. Computers & Education, 50(2), 475–490. https://doi.org/10.1016/j.compedu.2007.09.017. Brock, S. E. (2009). Measuring the importance of precursor steps to transformative learning. Adult Education Quarterly, 60(2), 122–142. https://doi.org/10.1177/0741713609333084. Brownell, S. E., & Tanner, K. D. (2012). Barriers to faculty pedagogical change: Lack of training, time, incentives, and tensions with professional identity? Cell Biology Education-Life Sciences Education, 11(4), 339–346. https://doi.org/10.1187/cbe.12-09-0163. Buckles, S., & Siegfried, J. J. (2006). Using multiple-choice questions to evaluate in-depth learning of economics. The Journal of Economic Education, 37(1), 48–57. https://doi.org/10.3200/ jece.37.1.48-57.

1234

C. Halupa

Christensen, C. M., Horn, M. B., & Johnson, C. W. (2008). Disrupting class: How disruptive innovation will change the way the world learns. New York, NY: McGraw-Hill. Cranton, P. (1994). Self-directed and transformative instructional development. The Journal of Higher Education, 726–744. https://doi.org/10.2307/2943826. Cranton, P. (2002). Teaching for transformation. New Directions for Adult and Continuing Education, 61–71. https://doi.org/10.1002/ace.50. Duarte, F. (2010). Addressing student cynicism through transformative learning. Journal of University Teaching & Learning Practice, 7(1), 4. Fink, L. D. (2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco, CA: Jossey-Bass. Fleming. (2001). The VARK Questionnaire. VARK learning styles Website. Retrieved at http:// www.vark-learn.com/english/page.asp?p= questionnaire Freire, P. (1970). Pedagogy of the oppressed. New York, NY: Herter and Herter. Freire, P. (1973). Education for critical consciousness. New York, NY: Continuum. Gardner, H. (1985). Frames of mind: The theory of multiple intelligences. New York, NY: Basic Books. Heddy, B. C., & Pugh, K. J. (2015). Bigger is not always better: Should educators aim for big transformative learning events or small transformative experiences? Journal of Transformative Learning, 3(1), 52–58. Illeris, K. (2015). Transformative learning in higher education. Journal of Transformative Learning, 3(1), 46–51. https://doi.org/10.1177/1541344614548423. Ison, R., & Russell, D. (2000). Agricultural extension and rural development: Breaking out of traditions, a second-order systems perspective. Cambridge, MA: Cambridge University Press. Keeling, R. P., & Hersch, R. H. (2012, May 15). Culture change for learning. HigherEdJobs Authors in Residence. Retrieved from https://www.higheredjobs.com/blog/post Display.cfm? post=344 Kegan, R., & Miller, M. (2003). The value proposition of development. In Proceedings from the 4th international conference on transformative learning, New York, NY. Kezar, A., & Maxey, D. (2014). Faculty matter: So why doesn’t everyone think so? Thought & Action, 30, 29. King, K. (2011). Teaching in the age of transformation: Understanding unique technology choices which transformative learning affords. Educational Technology, 51(2), 4. Kitchenham, A. (2008). The evolution of John Mezirow’s transformative learning theory. Journal of Transformative Education, 6, 104–123. https://doi.org/10.1177/1541344608322678. Kitchenham, A. D. (2015). Transformative learning in the academy: Good aspects and missing elements. Journal of Transformative Learning, 3(1), 13–17. Kreber, C., & Cranton, P. A. (2000). Exploring the scholarship of teaching. The Journal of Higher Education, 71(4), 476–495. https://doi.org/10.2307/2649149. Kreber, C., & Kanuka, H. (2006). The scholarship of teaching and learning and the online classroom. Canadian Journal of Continuing Education, 32(2), 109–131. https://doi.org/ 10.21225/d5p30b. Kucukaydin, I., & Cranton, P. (2013). Critically questioning the discourse of transformative learning theory. Adult Education Quarterly, 63(1), 43–56. https://doi.org/10.1177/ 0741713612439090. Lane, I. F. (2007). Change in higher education: Understanding and responding to individual and organizational resistance. Journal of Veterinary Medical Education, 34(2), 85–92. https://doi. org/10.3138/jvme.34.2.85. Mezirow, J. (1978). Perspective transformation. Adult Education, 28(2), 100–110. https://doi.org/ 10.1177/074171367802800202. Mezirow, J. (1985). A critical theory of self-directed learning. In S. Brookfield (Ed.), Self-directed learning: From theory to practice (pp. 7–30). San Francisco, CA: Jossey-Bass.

49

Are Students and Faculty Ready for Transformative Learning?

1235

Mezirow, J. (1990). How critical reflection triggers transformative learning. In J. Mezirow and Associates (Eds.), Fostering critical reflection in adulthood (pp. 1–20). San Francisco, CA: Jossey-Bass Publishers. Mezirow, J. (1991). Transformative dimensions of adult learning. San Francisco, CA: Jossey Bass Publishers. Mezirow, J. (1997). Transformative learning: Theory to practice. New Directions for Adult and Continuing Education, 74, 5–12. Mezirow, J. (1998). On critical reflection. Adult Education Quarterly, 48(3), 185–198. doi:10.1177/ 074171369804800305. Mezirow, J. (2000). Learning to think like an adult. In J. Mezirow and Associates (Eds.), Learning as transformation (pp. 3–33). San Francisco, CA: Jossey-Bass. Mezirow, J. (2003). Transformative learning as discourse. Journal of Transformative Education, 1(1), 58–63. https://doi.org/10.1177/1541344603252172. Mezirow, J. (2006). An overview on transformative learning. Lifelong learning: Concepts and contexts, 24–38. Miller, J. (2012). Learning styles: Are you a talker or a thinker? The people equation. Retrieved at http://people-equation.com/learning-styles-are-you-a-talker-or-a-thinker/ Miller, J. W., Martineau, L. P., & Clark, R. C. (2000). Technology infusion and higher education: Changing teaching and learning. Innovations in Higher Education, 24, 227–241. https://doi.org/ 10.1023/b:ihie.0000047412.64840.1c. Moore, J. (2005). Is higher education ready for transformative learning? A question explored in the study of sustainability. Journal of Transformative Education, 3(1), 76–91. https://doi.org/ 10.1177/1541344604270862. Oleson, A., & Hora, M. T. (2013). Teaching the way they were taught. Revisiting the sources of teacher knowledge and the role of experience in shaping faculty teaching practice. Higher Education, 68(1), 29–45. https://doi.org/10.1007/s10734-013-9678-9. Roofe-Steffen, K. Shmaefshy, B. R., & Griffin, M. (2014). How to test and evaluate learning. Teaching for Success National Faculty Success Center. Retrieved from http://teaching forsuccess.com/QC4Mrk14/TFS_Testing Eval_QC-Mrkt.pdf Rutgers University. (2016). Academic integrity for faculty. Retrieved from http://www.business. rutgers.edu/ai/faculty Sabagh, Z., & Saroyan, A. (2014). Professors’ perceived barriers and incentives for teaching improvement. International Education Research, 2(3), 18–40. https://doi.org/10.12735/ier.v2i3p18. Schon, D. A. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books. Sinclair, M. L. (2014). Faculty perceptions to imposed pedagogical change: A case study. The Nebraska Educator: A Student-Led Journal, Paper 20. Sterling, S. (2011). Transformative learning and sustainability: Sketching the conceptual ground. Learning and Teaching in Higher Education, 5, 17–33. https://doi.org/10.1177/0741713611402046. Stevens-Long, J., Schapiro, S. A., & McClintock, C. (2012). Passionate scholars: Transformative learning in doctoral education. Adult Education Quarterly, 62(2), 180–198. Tahiri, A. (2010). Fostering transformative learning: The role of professors and students at the university of Prishtina. Interchange, 41(2), 149–159. https://doi.org/10.1007/s10780-010-9121-4. Taylor, E. W. (2007). An update of transformative learning theory: A critical review of the empirical research (1999–2005). International Journal of Lifelong Education, 26(2), 173–191. Wald, H. S., Borkan, J. M., Taylor, J. S., Anthony, D., & Reis, S. P. (2012). Fostering and evaluating reflective capacity in medical education: Developing the REFLECT rubric for assessing reflective writing. Academic Medicine, 87(1), 41–50. https://doi.org/10.1097/acm.0b013e31823b55fa. White, S. K., & Nitkin, M. R. (2014). Creating a transformational learning experience: Immersing students in an intensive interdisciplinary environment. International Journal for the Scholarship of Teaching and Learning, 8(2), Article 3.

1236

C. Halupa

Wilson, R. (2010, September 5). Why teaching is not priority no. 1. Chronicle of Higher Education. Retrieved from https://www.mica.edu/ Documents/10-0905-FACULTY-Chronicle-Brottman.pdf Winter, D., Lemons, P., Bookman, J., & Hoese, W. (2001). Novice instructors and student-centered instruction: Identifying and addressing obstacles to learning in the college science laboratory. Journal of Scholarship in Teaching and Learning, 2, 15–42. Yarnall, L., Toyama, Y., Gong, B., Ayers, C., & Ostrander, J. (2007). Adapting scenario-based curriculum materials to community college technical courses. Community College Journal of Residential Practice, 31, 583–601. https://doi.org/10.1080/10668920701428881.

Colleen Halupa, Ed.D., is the Dean of Online Learning at East Texas Baptist University and an Associate Professor in the Doctor of Health Professions Education Program at the College of Graduate Health Studies at A.T. Still University. Her doctorate is in curriculum and instruction and educational leadership and management. She has presented and published in the field of online education, curriculum, health, and academic honesty both nationally and internationally. Her interests include the concept of student self-plagiarism, best practices in online learning curriculum, and emerging pedagogies.

Clicker Interventions in Large Lectures in Higher Education

50

Kjetil Egelandsdal, Kristine Ludvigsen, and Ingunn Johanne Ness

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Clicker Interventions: The Classic and the Peer Instruction Approach . . . . . . . . . . . . . . . . . . . . Lectures and Clicker Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Formative Feedback and Clicker Interventions: A Model for Discussion . . . . . . . . . . . . . . . . . . . . Feedback Supporting Students’ Self-Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Students’ Use of Feedback Supporting Their Self-Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . Feedback Enhancing Students’ Understanding of the Subject Matter . . . . . . . . . . . . . . . . . . . . . Feedback to the Lecturer from Clicker Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Clicker Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Clicker Question Enhancing Student Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Clicker Questions Supporting Student Self-Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peer Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peer Discussions Enhancing Student Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peer Discussions Supporting Student Self-Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lecturer Follow-Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lecturer Follow-Up Enhancing Student Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lecturer Follow-Up Supporting Student Self-Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1238 1239 1239 1241 1242 1243 1244 1244 1246 1248 1248 1248 1250 1250 1251 1251 1252 1252 1254

K. Egelandsdal (*) · I. J. Ness Faculty of Psychology, Centre for the Sciences of Learning and Technology (SLATE), University of Bergen, Bergen, Norway e-mail: [email protected]; [email protected] K. Ludvigsen Faculty of Psychology, Department of Education, University of Bergen, Bergen, Norway e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_147

1237

1238

K. Egelandsdal et al.

Abstract

Clicker interventions can be used to transform the learning environment in large plenary lectures from being lecturer-centered to student-active. Such interventions are conducted with the use of a student response system. In each intervention, the lecturer poses a multiple-choice question to the student group; the students discuss the question with their peers and answer individually using a wireless handheld remote control, called a “clicker.” The student answers are then displayed on a big screen for the students and the lecturer to see. Studies have found that clicker interventions can be used to promote student attention, motivation, retention, and performance. Clicker interventions can also support a formative feedback practice aimed at creating activities that make students’ understanding visible, so that students, together with their peers, can adjust their studying and the lecturer adjust teaching. This chapter gives an overview of research on clicker interventions in lectures in higher education and discusses how such interventions can provide students and lecturers with formative feedback. Keywords

Clickers · Student response systems · Dialogue · Sociocultural · Lectures · Higher education

Introduction “Nothing changes unless there’s friction—unless there’s another option out there forcing you to look at it and actually consider it.” (Sanderson, 2017)

The rise of the knowledge economy and the increasing number of students enrolling in higher education has placed more emphasis on effective teaching, “understood as teaching that is oriented to and focused on students and their learning” (Devlin & Samarawickrema, 2010, p. 112). This, in turn, has led to a greater demand for teaching aimed at promoting student activity and social interaction in higher education, which is reflected in criticism against lectures (Mazur, 2009; Wieman, 2007). Traditionally, large campus-based lectures have offered little room for studentlecturer interaction due to challenges related to the number of students present. Consequently, the amount of time the lecturer can use on each student is limited. The willingness of students to raise their hand and participate in plenary discussions is also an obstacle – since many students are scared to speak up in public (Caldwell, 2007). For these reasons, lectures in higher education have been criticized for being student-passive and in conflict with the idea of what is good teaching. Response systems, however, can be used to transform the learning environment from lecturercentered to student-active. Some of these systems use a handheld device with which the students can submit answers in writing or multiple-choice. There are also web-based response tools where the students can answer multiple-choice questions, write on shared online

50

Clicker Interventions in Large Lectures in Higher Education

1239

screens, or express themselves in other modalities than writing. These technologies provide opportunities for both the lecturer and the students to keep track of their progression and adjust focus, teaching methods, and study strategies (Schell, Lukoff, & Mazur, 2013). The emphasis of this chapter is on clicker interventions as an instructional design using a student response system to facilitate peer discussions, mediate studentlecturer interaction, and provide both students and lecturer with formative feedback during a lecture. We start by giving a short introduction of the two most common ways of using student response systems to conduct clicker interventions in lectures. We then present a short review of clicker interventions in the context of lectures in higher education. Finally, we discuss how clicker interventions can serve as a catalyst for formative feedback to both students and lecturer with reference to empirical research. This is done from a sociocultural perspective with an emphasis on dialogical interaction.

Clicker Interventions: The Classic and the Peer Instruction Approach Student response systems allow large student groups to respond individually to multiple-choice questions using a wireless remote control, called a “clicker.” The student answers can be visualized in a histogram on a large screen for the students and lecturer to see. The lecturer can also store the student answers for later use. The two most common ways of using SRS formatively are what Nielsen, Hansen, and Stav (2016) refer to as the “classic” approach and the “peer instruction” approach. In both cases, clicker interventions are conducted as parts of a lecture, typically following a mini-lecture on a certain topic. In “peer instruction,” building on the work of Mazur (1997), students are asked a multiple-choice question that they answer individually before they discuss their answer with the students seated next to them and then re-answer the same question. In the “classic” approach, students discuss with their peers before answering individually. Some studies have also used the same structure without peer discussions (Campbell & Mayer, 2009; Mayer et al., 2009; Shapiro & Gordon, 2012, 2013; Shapiro et al., 2017). The lecturer usually follows up on the student answers either by providing her explanations or engaging in a plenary discussion with the students.

Lectures and Clicker Interventions In its traditional form, the campus-based lecture in higher education has been about cultural preservation and knowledge distribution (Friesen, 2011). This has been challenged by several developments. Since the invention of the printing press, the production of teaching materials has gradually taken over much of the traditional lecture’s function as a way to distributing knowledge. Currently, the technological development has also made it possible to create lectures digitally in combination with different modalities, enabling students to watch instructional videos where and

1240

K. Egelandsdal et al.

whenever it suits them – challenging the campus-based lecture which is less flexible when it comes to time and place. The shift from a transference view of learning to a constructivist view of learning as the dominating paradigm has also come to challenge the lecture as a studentpassive way of teaching. According to constructivist and sociocultural perspectives, understanding is not about the transference of information from a sender to a receiver but is “conceptualized as a process of active construction wherein learners drew on prior knowledge and experiences – both individual and sociocultural – as they built new understandings” (Cochran-Smith & Villegas, 2015, p. 10). Empirical findings support this view of learning by showing that student activity and feedback situations promote student learning (Black & Wiliam, 1998; Evans, 2013; Hattie, 2009; Hattie & Timperley, 2007; Prince, 2004) and that student-active teaching is generally more effective in enhancing student achievement than lecturing (Deslauriers, Schelew, & Wieman, 2011; Hake, 1998; Hrepic, Zollman, & Rebello, 2007; Knight & Wood, 2005; Prince, 2004; Yoder & Hochevar, 2005). In addition, the human attention span and short-term memory are too limited to process and store most of the information from a long lecture (Risko, Anderson, Sarwal, Engelhardt, & Kingstone, 2012). Nevertheless, the campus-based lecture does have a potential for student involvement and interaction with the lecturer. In a lecture, students can ask questions, voice their own ideas, discuss with their fellow students, and reflect on the subject matter and their own understanding under the guidance of an expert. The lecturer can improvise and make changes in the teaching along the way based on the interaction with the students. Traditionally, however, student-lecturer interactions have been difficult to facilitate in large lecture halls with many students present. Although the lecturer is able to involve some students, many students find it socially uncomfortable to speak loud in a large assembly. The few who dare to participate are not necessarily representative of the student group. This can give the lecturer a skewed picture of the students’ understanding and ideas. The use of clicker interventions, however, can counter some of these challenges. For one, clicker interventions have been found to increase student attention (Blood, 2012; Cain, Black, & Rohr, 2009; Rush et al., 2010; Sun, 2014), and the majority of studies show that clicker interventions also have a positive effect on student learning (Chien, Chang, & Chang, 2016). Among explanations for why such activities have shown to have a positive effect on student performance are the testing effect, the possibilities for feedback, and opportunities for students to explain their thinking to each other when the system is used in combination with peer discussions (see Chien et al., 2016 for a more detailed review). Clicker interventions have also been found to increase student attendance, engagement, and preparation (Boscardin & Penuel, 2012; Kay & LeSage, 2009; Keough, 2012; Krumsvik & Ludvigsen, 2012; Lantz, 2010). Researchers from the research group Digital learning Communities at the University of Bergen – including the authors of this chapter – have also studied clicker interventions using a framework of formative assessment/feedback (Egelandsdal & Krumsvik, 2017a, 2019; Krumsvik, 2012; Krumsvik & Ludvigsen, 2012;

50

Clicker Interventions in Large Lectures in Higher Education

1241

Ludvigsen, Krumsvik, & Furnes, 2015). Apart from these studies, several studies on clickers present relevant findings without explicitly using the concepts “feedback” and “formative assessment.” In the next sections, we build on our previous work and review clicker interventions using a revised framework of formative feedback developed by Egelandsdal and Krumsvik (2017a).

Formative Feedback and Clicker Interventions: A Model for Discussion Feedback is the core of a formative assessment practice and can be an important factor in supporting students’ learning process (Black & Wiliam, 1998; Hattie & Timperley, 2007). According to Black and Wiliam (2009, p. 10), formative practices depend upon “the creation of, and capitalization upon, ‘moments of contingency.’” Such moments are understood as events that raise students or lecturer’s awareness of the students’ understanding of the subject. In this chapter, formative feedback is understood as “an event where something external informs the student [or lecturer] about her [the student’s] understanding or skills” (Egelandsdal & Krumsvik, 2017a, p. 57) following an action. Hence, formative feedback is not simply understood as a message transferred by a sender to a receiver but as an experience that arises when students and lecturers undergo the consequences of their actions (Dewey, 1997). The use of clicker interventions can promote such events in the lecture hall by creating situations in which students engage with the subject and receive feedback on their understanding resulting in more informed decisions in their studying and the lecturer receives feedback on students’ understanding to make informed decisions about her teaching. As illustrated in Fig. 1, there are three potential feedback sources/situations during a clicker intervention: (1) use of clicker questions and answers; (2) peer

Feedback from clicker interventions 3 potential feedback sources: (1) Clicker questions and answers (2) Peer discussions (3) The follow up phase Feedback supporting student selfassessment

Feedback enhancing student understanding of the subject matter

Feedback to the teacher

Student use of feedback supporting their self-assessment

Fig. 1 Feedback and feedback sources in clicker interventions (a revised model from Egelandsdal & Krumsvik 2017a)

1242

K. Egelandsdal et al.

discussion; and (3) follow-up of the clicker answers. From these three situations, students can potentially receive feedback that (a) supports their self-assessment (metacognition) by raising their awareness of their understanding. This feedback can be used by the students to study more purposefully. (b) The students can also receive feedback that enhances their understanding of the subject matter. (c) The lecturer can potentially receive feedback on the students’ understanding from the clicker answers and the plenary discussion in the follow-up phase that can be used to adapt teaching both in situ and in the planning of future teaching. In the following, we will use this model of feedback types and feedback sources to structure our discussion on clicker interventions. First, we present the different kinds of feedback students and lecturers can experience from the interventions and how students and lecturers (can) use this feedback. This is done in four subsections: (1) “Feedback Supporting Students’ Self-Assessment,” (2) “Students’ Use of Feedback Supporting Their Self-Assessment,” (3) “Feedback Enhancing Students’ Understanding of the Subject Matter,” and (4) “Feedback to the Lecturer from Clicker Interventions.” Secondly, we discuss the three different feedback sources – clicker questions, peer discussions, and lecturer follow-up – in relation to the empirical findings on the two kinds of feedback for the students. Finally, we summarize our discussion and conclude on whether clicker interventions can contribute to transform large plenary lectures in higher education.

Feedback Supporting Students’ Self-Assessment The first kind of feedback, feedback supporting students’ self-assessment, involves situations that make students more aware of their understanding. Such awareness is important for students to choose a productive focus and devote their efforts purposefully when studying. Since students differ in their abilities as self-regulated learners, how they adapt their focus and effort will also differ (Nicol & MacfarlaneDick, 2006). Studies have shown that creating situations that raise students’ awareness of their understanding can improve student performance and help students selfregulate (Hattie & Timperley, 2007), particularly in the case of low-achieving students (Black & Wiliam, 1998). In particular, low competence can to lead to an inflated self-assessment (Kruger & Dunning, 1999), and low-achieving students have been found to overestimate their understanding of the content (Isaacson & Fujita, 2006). This highlights the importance of creating situations in instruction where students can articulate and experience how they understand a topic. For feedback to be effective in supporting students’ self-regulation, it is commonly agreed that it should raise the students’ awareness of the learning intentions (feed up), their understanding of the subject (feed back), and how to improve (feed forward) (Black & Wiliam, 2009; Hattie & Timperley, 2007). Through clicker interventions, such experiences can arise from being asked a clicker question, discussing questions and answers with peers, and engaging in the plenary discussion and listening to the lecturer in the follow-up phase. Some studies have indeed found that clicker interventions can make the students more aware of

50

Clicker Interventions in Large Lectures in Higher Education

1243

their understanding (Egelandsdal & Krumsvik, 2017a; Krumsvik & Ludvigsen, 2012; Ludvigsen et al., 2015). Egelandsdal and Krumsvik (2017a) found that the clicker interventions particularly raised the students’ awareness of their understanding of the content (feed back), what is important to learn in the course (feed up), and what they should focus on further (feed forward).

Students’ Use of Feedback Supporting Their Self-Assessment For students to fully benefit from feedback supporting their self-assessment requires that they actively use this information in their coursework in addition to the lecture; i.e., there is little use in knowing what is required of you, how well you are doing, and how to improve if you do not act on this information. Studies on feedback have shown that there is often a gap between how feedback is received and how it is used, referred to as “the feedback gap” (Evans, 2013; Jonsson, 2013). Only a couple of studies have investigated how students use feedback from clicker interventions in their coursework. In one study, Ludvigsen et al. (2015) found that students employed the feedback from the interventions in various ways in their coursework. Based on six interviews, they found that students used their experiences from the clicker interventions to identify difficult topics for further studying, to discuss difficult concepts with each other, and to adjust the focus of their reading. One student also claimed that the clicker interventions had transformed the way she was studying. Because of the interventions, she had started constructing questions for herself while studying. In another study, Egelandsdal and Krumsvik (2019), using student logs, found that only half of the students reported using feedback in their coursework. The students, who used the feedback, used it (1) to adjust their focus when studying; (2) to discuss various concepts with their peers; or (3) to use the clicker questions when practicing. The other half stated that they did not use the feedback or did not answer the question, even though 86% of the students experienced that the clicker lectures made them more aware of their understanding than the lectures without clickers. This indicates that there is a gap between how many students experience feedback from clicker interventions and how they use it in their coursework. This “feedback gap” may be related to variations in the students’ study strategies. Previous studies have found that students often use feedback passively as an indicator of progress but lack the strategies to use the feedback actively in their coursework (Jonsson, 2013). In a review, Jonsson (2013) found that the students’ use of feedback depends on their perception of the information as well as the opportunity to use it in the near future. If the clicker interventions are not directly related to other organized course activities, the opportunities to use the feedback needed to be constructed by the students themselves. Hence, variation in the students’ studying strategies and capabilities is likely to affect their use of the feedback. Students’ orientation toward learning and performance is also likely to affect how feedback is used (Shute, 2008). Since all students at some point need to focus on exams, their experience of the clicker interventions’ relevance for this purpose will affect whether and how they use the feedback. Egelandsdal and Krumsvik (2017a) found that even

1244

K. Egelandsdal et al.

though 80% of the students experienced that clicker interventions had provided them with information on what was important to learn, only 47% experienced that the interventions provided them with information about what they needed to know to do well on their exam. This could be one explanation for why some students place less emphasis on using feedback from the clicker interventions in their coursework. In this case, a clear alignment between learning objectives, course activities, and exams might contribute to making the feedback more useful for the students (Biggs & Tang, 2011).

Feedback Enhancing Students’ Understanding of the Subject Matter The second kind of feedback regards situations that raise the students’ understanding of the subject matter. We assume that all three parts of a clicker intervention, i.e., questions about key topics and reflection on these topics, discussions with peers, or listening to the perspectives of others (students and lecturer), can contribute to developing a student’s content understanding. The use of questions can evoke a testing effect that help the students remember the content better in addition to reflection upon key topics, while peer discussions and the follow-up by the lecturer can help the students develop their understanding of the subject matter through co-creation of knowledge. In clicker studies this kind of feedback is usually measured by changes in student performance before and after the clicker interventions (Chien et al., 2016).

Feedback to the Lecturer from Clicker Interventions One of the benefits of using clickers is that the lecturer can quickly collect answers from all students present in the auditorium. Looking at the distribution of the student clicker answers and engaging in the plenary discussion in the follow-up phase can provide the lecturer with feedback on the students’ understanding, assuming that these sequences can help the lecturer to assess how well the students have understood various topics. This is important information for the lecturer because students’ pre-understanding has a big impact on how a lecture is experienced (Schwartz & Bransford, 1998). The understanding of the lecturer as an expert also differs greatly from the understanding of the students as novices (Hrepic et al., 2007). For this reason, clicker interventions aim to make the lecturer aware of the students’ understanding and the difficulty of various topics. Nevertheless, clicker studies have shown that the results sometimes misrepresent the students’ understanding of a topic (James & Willoughby, 2011; Knight, Wise, Rentsch, & Furtak, 2015; Wood, Galloway, Hardy, & Sinclair, 2014). Based on the recordings of peer discussion, these studies found that correct answers do not necessarily mean that the students have understood the topics well and that incorrect answers do not always mean that the students have little or no understanding. For instance, James and Willoughby (2011) found that the clicker statistics

50

Clicker Interventions in Large Lectures in Higher Education

1245

misrepresented the students’ actual understanding in 26% of the recorded conversations. Other studies that draw conclusions based on recordings of student discussions have found that students sometimes vote correctly, but for the wrong reasons (Knight et al., 2015), and that if a student votes correctly, it does not always imply that they understood the concept they were discussing (Wood et al., 2014). These findings illustrated that interpreting the multiple-choice answers is not a straightforward process. Although the clicker statistics provide the lecturer with a general overview of the difficulty of a topic, there are several nuances not captured by the clicker answers. It is therefore important for the lecturer to follow-up on the student answers at the end of the interventions to ask them to explain their reasoning or use other approaches to collect qualitative data on the students’ understanding. When it comes to the usefulness of feedback, it can, in theory, be used both synchronously to make adjustments in the current lecture (explain more or differently, ask new questions, etc.) or asynchronously in the planning of future lectures (use more time on difficult topics, make adjustments in how the material is presented, make new clicker questions, etc.) (Black & Wiliam, 2009). One example of asynchronous use is D’Inverno, Davis, and White (2003) who found through clicker answers that the students’ learning needs were at a more fundamental level than what had been assumed. This resulted in the lecturers changing their teaching based on these experiences. In another study, Kolikant, Drane, and Calkins (2010) followed and interviewed three university lecturers who used clickers in their classes. These lecturers stated that the clicker interventions helped them assess the students’ understanding and address misconceptions. One of the lecturers claimed that she had previously weighted different parts of the content equally but that her experiences with the clickers had made her allocate different amounts of time and attention to various topics based on the students’ needs. Anderson, Healy, Kole, and Bourne (2011) also found that the clicker interventions helped cut down on lecturing time because the lecturer could focus on the most relevant topics for the students’ current level of understanding. Egelandsdal and Krumsvik (2019) also found that lecturers experienced that the lecture became more structured because of the interventions and that in the planning of the lecture, they were forced to really focus on what was most important for the students to learn. Although lecturers do find the interventions useful, they do not necessarily use the information they collect. Egelandsdal and Krumsvik (2019) found that on one hand, all lecturers participating in their study experienced that the interventions raised their awareness of the students’ understanding. On the other hand, the lecturers were mostly focused on the immediate use of the feedback in the lecture and that the interventions made the students more involved. Only one of the lecturers also considered the interventions useful for the planning of future lectures and making changes in the reading list. A reason for this might be that the lecturers were using the clicker interventions for the first time. Becoming familiar with the technology, creating appropriate questions, and learning how to adjust their teaching based on the information from the lecture very likely take time (Boscardin & Penuel, 2012). Studies have found that the perceived advantages of the clicker interventions increase when the lecturers

1246

K. Egelandsdal et al.

become more experienced with using them (Draper & Brown, 2004; Kolikant, Drane, & Calkins, 2010). When it comes to negative remarks, lecturers relate this to loss of lecture time due to the interventions because of the time it takes to hand out the clickers; that the transition between the interventions and lecturing can create a noisy environment which takes time to subside; and that the interventions take more time from the lecturing than anticipated (Egelandsdal & Krumsvik, 2019). These remarks illustrate the major tradeoff when using clicker interventions, namely, that there will be less time for lecturing. However, if the lecturers experience that they are “teaching more by lecturing less” (Knight & Wood, 2005), this tradeoff might be well worth it. As illustrated by many studies (Deslauriers et al., 2011; Hake, 1998; Hrepic et al., 2007; Knight & Wood, 2005; Yoder & Hochevar, 2005), the amount of material covered in a lecture does not equal to what is learned by the students. As pointed by a lecturer, it is better to focus on a few important points than “pepper” the students with a lot of information where little is retained (Egelandsdal & Krumsvik, 2019). Considering that human short-term memory and attention span are limited when it comes to retaining information from a lecture (Risko et al., 2012) and that clicker interventions increase student attention (Blood, 2012; Cain et al., 2009; Rush et al., 2010; Sun, 2014), this is a valid point in itself. Other studies have also found that brief activities help student remember the content better (Prince, 2004), that the use of clicker questions enhances student retention (Campbell & Mayer, 2009; Mayer et al., 2009; Shapiro & Gordon, 2012, 2013), and that students are likely to understand more of the content if it is simple, explicitly stated, and reiterated multiple times (Hrepic et al., 2007). Hence, reducing the amount of material covered, slowing down the tempo, and introducing the use of questions and peer discussions might be acceptable from the point of view of student learning.

Clicker Questions The primary affordance of student response systems is that they allow for effectively collecting and visualizing student answers from large student groups, where all students present can participate. The student answers can be visualized in a histogram, giving a quick overview of the percentage of students who have chosen the different alternatives. There are, however, various ways of using clicker questions during a lecture, alone or in combination with other activities. Whether and how the student response systems contribute to self-assessment and co-construction of knowledge are dependent on this use. In a basic way, SRS can be used to ask a few factual questions during an otherwise traditional lecture. In this case, the change from a traditional lecture is modest but still considerable because the whole student group can participate, as opposed to only a few students raising their hand. It is, however, important that the kind of questions used aligns with the intentions behind the course. A study found that the sole use of factual questions can improve student retention but at the same time impede conceptual understanding because students become too oriented toward

50

Clicker Interventions in Large Lectures in Higher Education

1247

facts (Shapiro et al., 2017); hence, the questions need to be adapted to the kind of knowledge that the lecture aims to promote. Even though the clicker questions are multiple-choice, it is still possible to construct questions that require a deeper understanding, for instance, by using case questions where the students need to apply their subject understanding to solve cases (Ludvigsen et al., 2015). The clicker questions can also be used in combination with other modalities than text and speech such as video cases. The questions can also be more or less an authentic exploration of the students’ ideas or test questions aimed at testing the students’ understanding (Nystrand & Gamoran, 1997). Test questions would typically address only one meaning or one correct answer. The purpose of these questions is for the students to test their understanding and to raise the students and lecturer’s awareness of how well the students have understood the topic. Authentic questions, on the other hand, are not posed with a correct answer in mind but seek to explore the students’ opinions or ideas. Even though the lecturer needs to create the alternatives, it is still possible to create questions where the various options shed light on different ideas and questions where several answers are correct. Such questions can be used as a way of initiating a discussion about various perspectives on a particular topic. A limitation of multiple-choice questions is that both the questions and answers need to be constructed by the lecturer. For this reason they might be poorly adapted to a student’s own questions and ideas and less open to divergent perspectives on a topic not captured by the lecturer. One possibility for better capturing students’ perspectives is combining the use of clicker interventions with a “just-in-time teaching” approach (Novak & Patterson, 2010). This allows students to submit their ideas and questions to the lecturer before the lecture. In this way the lecturer can design the lecture to address the students’ needs and use the students’ own questions in the clicker interventions. A study by Schwartz and Bransford (1998) found that students can benefit considerably from a lecture if it is adapted to answer the questions with which the students themselves are struggling. Considering that the subject understanding of a lecturer and the subject understanding of students are considerably different (Hrepic et al., 2007), this can be useful for designing clicker questions that are better adapted to the students’ pre-understanding. Another possibility is to combine clicker interventions with the use of other kinds of response systems where the students can write their own ideas for the lecturer and peers to see. In combination with multiple-choice questions, such systems might give a more sophisticated picture of students’ questions and ideas and, thus, enable a more dialogical teaching. One example is the use of a shared online whiteboard to collect student explanations which made the intervention last 20–30 min – including peer discussions, writing the text posts, and a plenary discussion led by the lecturer (Ludvigsen & Krumsvik, Forthcoming). Hence, if two or three interventions are employed during a 90-min lecture, this represents a radical shift from lecturer monologue to student activity in the auditorium. Such a transformation of the lecture, however, places a great demand upon the quality of the interventions, because a lot of time is wasted if the interventions go poorly.

1248

K. Egelandsdal et al.

Clicker Question Enhancing Student Understanding Studies have found that the use of clicker questions can increase the students’ retention (Campbell & Mayer, 2009) and that the use of clicker questions in lectures improved students’ exam performance by one-third of a grade compared to lectures without the use of clickers and lectures without questions (Mayer et al., 2009). These findings can be related to the testing effect (Roediger & Karpicke, 2006) that has shown that the use of questioning in itself can improve student retention. Shapiro and Gordon (2012) found that clicker questions given in a psychology class improved performance on delayed exam questions by 10–13% and concluded, based on their controlled experiment and survey, that interventions invoked the testing effect. In another study, Shapiro and Gordon (2013) found that the use of clicker questions also promoted significantly higher performance on test questions than did repetition of the same material.

Clicker Questions Supporting Student Self-Assessment The mere use of question can also contribute to supporting students’ self-assessment by making the students reflect upon their own understanding in relation to key questions related to the topic of the lecture. Egelandsdal and Krumsvik (2017a) found that students experienced that clicker questions gave them an idea of what was important to learn in the subject and what they had not or misunderstood. The students also experienced that questions where several answers were correct made them more aware of different nuances in the subject and facilitated better discussions. In isolation, the mere use of questions can contribute to raising the students awareness of their own understanding (feed back) and what is important to learn (feed up). It is, however, less likely that use of questions can provide students with feed forward on how to improve without the support of peer discussions or follow-up by the lecturer beyond highlighting the correct answer(s).

Peer Discussions From a sociocultural perspective, student learning requires active participation in dialogical activities. Dysthe (2011) refers to this as a dialogical way of thinking which includes activities that promote different perspectives being set against each other. According to Bakhtin (2010), it is in the tension between voices that knowledge is created. To create such tension, the interventions need to be well planned and adapted to this purpose. If the aim of the intervention is to promote student reflection and deeper learning, this needs to be considered when constructing the clicker questions and facilitating the student activities. In a traditional educational context, we often see that students are asked to find the correct answers instead of discussing questions that are open and encourage true dialogue. When the questions do not stimulate discussion of different perspectives, but instead encourage surface learning

50

Clicker Interventions in Large Lectures in Higher Education

1249

of facts and definitions, this hinders the students from developing a dialogical way of thinking (Dysthe, 2011; Samuelsson & Ness, Forthcoming). It can, however, be quite demanding for students to be open to others’ opinions and views. According to Bakhtin (2010), an alien perspective is presented in a dialogue, and this new perspective brings in new thoughts and ideas. Further, it is not enough to bring together different perspectives; they also need to challenge each other. So peer discussions are both about the need for students to co-construct their understandings in collaboration and to see each other as co-creators of knowledge. This requires that the students are open to each other’s perspectives. A study of multidisciplinary groups engaged in innovative knowledge development found that explicit exposure to different perspectives – with group members disagreeing and challenging each other’s view – enabled co-construction of knowledge and the emergence of new perspectives (Ness & Søreide, 2014). In these groups, some relational conditions were necessary: curiosity, to ensure that the group members were listening to each other and saw each other as resources with a variety of knowledge on a subject; openness, to ensure that there also was present a certain interest in each other; and respect, to ensure a constructive climate in the groups in which it was safe to disagree without having relational conflict (Ness & Riese, 2015). This illustrates that there might be an additional need for the lecturer to focus on relational conditions with the students to ensure that the students learn to view other perspectives as resources and not a “threat” to a predefined “correct” answer. Most of the research on how peer discussions in clicker interventions can support student learning uses pre- and posttests or self-reported data. Only a few studies have used audio recordings or video of student discussions to examine the processes occurring when they are played out in an authentic setting of a lecture (Ludvigsen & Krumsvik, forthcoming). Such studies are important for providing directions for practice because there might be a discrepancy between how students perceive and describe the processes occurring in a discussion and how the discussions actually play out (Nielsen et al., 2016). Studies that use recordings of peer discussion as data illustrate that small changes in the design of learning activities had implications for the quality of student dialogue (Knight et al., 2015; Knight, Wise, & Southard, 2013; McDonough & Foote, 2015; Nielsen et al.). One example is a study by McDonough and Foote (2015) who found that when the students were asked to reach an agreement by sharing clickers, they used more arguments than when they were asked to give individual answers to questions. A study by Knight et al. (2015) found that groups used more arguments when they were explicitly told to argue and justify claims than in cases where they were told to discuss only. An initial individual thinking time before voting also increased the use of arguments (Nielsen et al.). Knight et al. (2013) and James and Willoughby (2011) found that the quality of the peer discussion did not necessarily depend on the cognitive level of the questions posed, and Knight et al. (2015) found that the students tended to use more reasoning on assessing the alternatives in low-order multiple-choice questions compared to when they discussed higher-order questions. The authors suggested that this was

1250

K. Egelandsdal et al.

because students were more confident in engaging in the discussions when questions are fact-based than in higher-order questions.

Peer Discussions Enhancing Student Understanding When it comes to peer discussions, several studies have found that the number of students answering correctly increases when the same clicker question is re-answered after a discussion (Crouch & Mazur, 2001; Mazur, 1997; Rao & DiCarlo, 2000; E. L. Smith, Rice, Woolforde, & Lopez-Zang, 2012; M. K. Smith et al., 2009; Vickrey, Rosploch, Rahmanian, Pilarz, & Stains, 2015). The average improvement varies between 8% and 30%. M. K. Smith et al. (2009) found that the number of students answering correctly also increases when the students are asked a new (isomorphic) question after the discussion, requiring approximately the same level of understanding as the first question, but is posed as a new case. The average improvement on these isomorphic questions was 21%. In a similar study, Egelandsdal and Krumsvik (2017b) found an average improvement of 12% when using isomorphic questions.

Peer Discussions Supporting Student Self-Assessment Egelandsdal and Krumsvik (2017a) found that only half of a group of 173 students experienced that peer discussions supported their self-monitoring. The students themselves explained these findings in focus groups by saying that the students sometimes had no one to discuss with and that the quality of the discussions varied. Variations in the quality of the discussions were related to the students not always having an opinion about the topic in question, that some students were not willing to discuss, that some students found it hard to discuss with strangers, and that the discussions were sometimes superficial – talking about what the correct answer is and not discussing why. Some students also experienced that the discussions were harder to engage in if they had not read the recommended literature before the lecture. The challenge of some students not having anyone to discuss with could be reduced by the lecturer more actively encouraging the students to participate in the discussions and making sure that no one was sitting alone. A follow-up survey in the same course, however, showed that only 6% did not participate in the peer discussions (N: 174) (Egelandsdal & Krumsvik, 2017b). Regarding the quality of the discussions, the same survey showed that 86% of the students answered they spent time justifying and explaining their answers in the discussions, while 8% experienced the discussions as superficial. The two studies were conducted in the same course over two semesters using the same kind of clicker questions. Thus, one could imagine that the students’ experience of the feedback was similar in both semesters. However, the findings indicated that the students did experience the peer discussions as more useful in the second

50

Clicker Interventions in Large Lectures in Higher Education

1251

study (Egelandsdal & Krumsvik, 2017b). The primary objective of this study was to measure the effect of peer discussions on student performance, which led to an increased use and focus on peer discussions. The use of re-voting and isomorphic questions might also have made it clearer for some of the students that they actually improved their performance because of the discussions. Hence, the focus on peer discussions in this study might have caused a kind of Hawthorne effect (Krumsvik, 2014) increasing either the quality of the discussions or the students’ experience of quality or both. Students who did find the peer discussions useful claimed that they helped them reflect upon their own understanding and become actively engaged with the content (Egelandsdal & Krumsvik, 2017a). Some students also claimed that discussions challenged them to express and visualize their understanding in a way that made a stronger impression than would have resulted from reflecting upon a question in silence (Egelandsdal & Krumsvik, 2019).

Lecturer Follow-Up Although a lecture can be made more dialogical by collecting and visualizing the students’ ideas with the use of response technology, the quality of the interventions also rests upon how well the lecturer follows up on the student answers. In light of Bakhtin (2010), this depends upon the level of interanimation of ideas, the extent to which the students’ ideas are compared and contrasted with each other and the ideas of the discipline. The lecturer might simply collect the student answers, but not engage them in a discussion of different perspectives. This process involves a low level of interanimation. On the other hand, the lecturer might relate the student ideas to one another and to the authoritative ideas of the discipline. This process involves a higher level of interanimation of ideas. Even though the use of multiple-choice questions restricts the students to answers created by the lecturer, the lecturer still gets an overview of which kind of understandings or misunderstandings the students choose. In addition, the lecturer can follow-up on the student answers by asking students to raise their hands and provide explanations for different alternatives – or use a qualitative response system to collect explanations in writing. This might even open for ideas that challenge the lecturers’ understanding of a subject, which makes the interventions both interesting and demanding.

Lecturer Follow-Up Enhancing Student Understanding Some studies have found that the combination of peer discussions with the lecturer’s follow-up enhances student performance more than just peer discussions alone (M. K. Smith, Wood, Krauter, & Knight, 2011; Zingaro & Porter, 2014). This is likely due to combination between the students’ own ideas and the authoritative ideas of the discourse that the lecturer can contribute with and elaborate on. Studies

1252

K. Egelandsdal et al.

have found that students appreciated the follow-up phase because they experienced that the lecturer adapted his explanations on the basis of the students’ answers, plus the fact that they got an expert explanation on all the multiple-choice alternatives (Egelandsdal & Krumsvik, 2017a, 2019). A study on science teaching by Scott, Eduardo, and Aguiar (2006) found that tension between authoritative and dialogical approaches supported the students’ meaning making. In line with sociocultural perspectives on learning, students’ previous experiences condition how they understand new topics; it is therefore important that they are able to make connections between their everyday views and the scientific view of a phenomenon. For this to happen, it is not enough to simply engage students’ own ideas about a phenomenon; students also need to be introduced to the scientific perspective – and have opportunities to speak the scientific language themselves. By creating tension between an authoritative and dialogical discourse, the lecturer as an expert can provide students with opportunities to use the discourse of the discipline in new situations, expanding its possibilities for application and constructing new ideas that are meaningful for them. As argued by Scott et al. (2006, p. 622), this requires the students “to engage in the dialogic process of exploring and working on ideas, with a high level of interanimation,” within the context of the discipline. If the lecturer succeeds in creating such situations, the use of student response systems can both transform the university lecture into being more student-active, dialogical, and interactive and at the same time preserve the lecturer as an authority who is responsible for teaching the content of a course.

Lecturer Follow-Up Supporting Student Self-Assessment Egelandsdal and Krumsvik (2017a) found that over 70% of the students experienced that the lecturer’s follow-up supported their self-assessment. Findings from focus group interviews and student logs show that the students found it particularly useful to become aware of different ways of thinking about a subject and possible misunderstandings and to receive feedback that they could relate to their own understanding (Egelandsdal & Krumsvik, 2017a, Forthcoming). It is interesting that both in their comments on the clicker questions and the follow-up phase, the students seem to appreciate being introduced to divergent perspectives and various correct answers to the same question. Such nuances might be beneficial for the students when reflecting on their understanding of the topics and experiencing that not all questions have one definitive answer.

Conclusion The research literature shows that clicker interventions can play an essential role in transforming the lecture from being student-passive to promoting a climate for active learning. Such interventions can contribute to transforming the lecture in

50

Clicker Interventions in Large Lectures in Higher Education

1253

different ways: as a space for students to reflect, which invites students to engage in self-assessment, and as a room for peers to share each other’s ideas and co-construct knowledge, as well as serving as a catalyst for lecturer and student interaction. As illustrated in the review of previous research, clicker interventions can have a positive impact on student attendance, motivation, attention, engagement, and preparation. They can also have a positive impact on student learning, provide feedback on the students’ understanding, and support peer discussion. Like any practice, however, the benefits depend on the implementation. At its worst, clicker interventions can be disturbing, draw student attention away from important questions, and provide students and lecturers with misleading feedback. At its best, it can create new opportunities for teaching large student groups, transforming the lecture from being an arena where students are listening to a lecturer talking to an arena for students to share ideas and challenge each other’s thinking under the guidance of an expert. The use of clicker interventions can facilitate student activities in the lecture hall and convey student answers and ideas to the lecturer. Hence, it can contribute to shifting the balance between lecturer and student activity. At the same time, it is important that this shift does not eradicate the authority of the lecturer. It is possible to change the auditorium into a place where students can discuss and voice their opinions the entire time. If these activities are not related to the content of the course, however, a traditional lecture without student activity is likely to be better. Thus, finding a balance between the students’ own ideas and the authoritative ideas of the course is essential. The most important contributions of the student response system are (1) the opportunity for interaction between the lecturer and the whole student group and (2) being a catalyst for peer discussions. The students can be engaged in discussions about important concepts and ideas that generate feedback on the students’ understanding. Such feedback can be used by the lecturer to adapt the lecture in situ and to make changes when planning future lectures. The student-lecturer interaction also enables the lecturer to bring the written word into life through dialogue with the students’ ideas and to relate the students’ perspectives to the authoritative perspectives of the discipline. The students are also able to test out their ideas and co-construct knowledge through the discussions. The degree to which the clicker interventions does transform the university lecture, however, depends on how and the extent to which it is used. An increase in student participation and interactivity leads to less lecturer monologue. This can be time well spent but places a demand on the lecturer to ensure that the quality of questions is high, that the follow-up of the student answers is good, and that she takes the process of evaluating her teaching seriously, in order to improve. Other factors that are likely to affect the usefulness of response technology in a lecture is how well the questions and lecture activities relate to other course activities, the literature, learning intentions, and summative assessments and how the students and lecturer use the information from the interventions between lectures.

1254

K. Egelandsdal et al.

References Anderson, L. S., Healy, A. F., Kole, J. A., & Bourne, L. E. (2011). Conserving time in the classroom: The clicker technique. The Quarterly Journal of Experimental Psychology, 64(8), 1457–1462. https://doi.org/10.1080/17470218.2011.593264 Bakhtin, M. M. (2010). The dialogic imagination: Four essays (C. Emerson, Trans.). Austin: University of Texas Press. Biggs, J., & Tang, C. (2011). Teaching for quality learning at university (4th ed.). Maidenhead: McGraw-Hill and Open University Press. Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139–144. Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. Blood, E. (2012). Student response Systems in the College Classroom: An investigation of shortterm, intermediate, and long-term recall of facts. Journal of Technology and Teacher Education, 20(1), 5–20. Boscardin, C., & Penuel, W. (2012). Exploring benefits of audience-response systems on learning: A review of the literature. Academic Psychiatry, 36(5), 401–407. Retrieved from :// WOS:000308454500013 http://psychiatryonline.org/data/Journals/AP/24913/401.pdf. https:// doi.org/10.1176/appi.ap.10080110 Cain, J., Black, E. P., & Rohr, J. (2009). An audience response system strategy to improve student motivation, attention, and feedback. American Journal of Pharmaceutical Education, 73(2). Retrieved from ://WOS:000265219700001). https://doi.org/10.5688/aj730221 Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-practice tips. CBE - Life Sciences Education, 6(1), 9–20. Campbell, J., & Mayer, R. E. (2009). Questioning as an instructional method: Does it affect learning from lectures? Applied Cognitive Psychology, 23(6), 747–759. Retrieved from :// WOS:000268971400001. http://onlinelibrary.wiley.com/doi/10.1002/acp.1513/abstract?systemMes sage=Wiley+Online+Library+will+be+disrupted+on+7+December+from+10%3A00-15% 3A00+BST+%2805%3A00-10%3A00+EDT%29+for+essential+maintenance. https://doi. org/10.1002/acp.1513 Chien, Y.-T., Chang, Y.-H., & Chang, C.-Y. (2016). Do we click in the right way? A meta-analytic review of clicker-integrated instruction. Educational Research Review, 17, 1–18. Retrieved from http://www.sciencedirect.com/science/article/pii/S1747938X15000500. https://doi.org/10.1016/j. edurev.2015.10.003 Cochran-Smith, M., & Villegas, A. M. (2015). Framing lecturer preparation research: An overview of the field. Part 1. Journal of Lecturer Education, 66(1), 7–20. https://doi.org/10.1177/ 0022487114549072 Crouch, C. H., & Mazur, E. (2001). Peer instruction: Ten years of experience and results. American Journal of Physics, 69(9), 970–977. https://doi.org/10.1119/1.1374249 D’Inverno, R., Davis, H., & White, S. (2003). Using a personal response system for promoting student interaction. Teaching mathematics and its applications, 22(4), 163–169. Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-enrollment physics class. Science Education International, 322(6031), 862–864. https://doi.org/10.1126/ science.1201783 Devlin, M., & Samarawickrema, G. (2010). The criteria of effective teaching in a changing higher education context. Higher Education Research & Development, 29(2), 111–124. https://doi.org/ 10.1080/07294360903244398 Dewey, J. (1997). Experience and education. New York: Touchstone. Draper, S. W., & Brown, M. I. (2004). Increasing interactivity in lectures using an electronic voting system. Journal of Computer Assisted Learning, 20(2), 81–94. Dysthe, O. (2011). Opportunity spaces for dialogic pedagogy in test-oriented schools: A case study of teaching and learning in high school. In J. White & M. Peters (Eds.), Bakhtinian pedagogy:

50

Clicker Interventions in Large Lectures in Higher Education

1255

Opportunities and challenges for research, policy and practice in education across the globe. New York, NY: Peter Lang Publishing Group. Egelandsdal, K., & Krumsvik, R. J. (2017a). Clickers and formative feedback at university lectures. Education and Information Technologies, 22(1), 55–74. Retrieved from. https://doi.org/ 10.1007/s10639-015-9437-x Egelandsdal, K., & Krumsvik, R. J. (2017b). Peer discussions and response technology: Short interventions, considerable gains. Nordic Journal of Digital Literacy, 12(01–02), 19–30. Retrieved from http://www.idunn.no/dk/2017/01-02/peer_discussions_and_response_technol ogy_short_interventio Egelandsdal, K., & Krumsvik, R. J. (2019). Clicker Interventions: Promoting Student Activity and Feedback at University Lectures. In: Tatnall A. (eds) Encyclopedia of Education and Information Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-60013-0 Evans, C. (2013). Making sense of assessment feedback in higher education. Review of Educational Research, 83(1), 70–120. https://doi.org/10.3102/0034654312474350 Friesen, N. (2011). The lecture as a transmedial pedagogical form: A historical analysis. Educational Researcher, 40(3), 95–102. https://doi.org/10.3102/0013189x11404603 Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64–74. Retrieved from http://scitation.aip.org/content/aapt/journal/ajp/66/1/10.1119/1. 18809. https://doi.org/10.1119/1.18809 Hattie, J. (2009). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. London, UK: Routledge. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. Hrepic, Z., Zollman, D. A., & Rebello, N. S. (2007). Comparing Students’ and Experts’ understanding of the content of a lecture. Journal of Science Education and Technology, 16(3), 213–224. https://doi.org/10.1007/s10956-007-9048-4 Isaacson, R. M., & Fujita, F. (2006). Metacognitive knowledge monitoring and self-regulated learning: Academic success and reflections on learning. Journal of Scholarship of Teaching and Learning, 6(1), 39–55. James, M. C., & Willoughby, S. (2011). Listening to student conversations during clicker questions: What you have not heard might surprise you! American Journal of Physics, 79(1), 123–132. https://doi.org/10.1119/1.3488097 Jonsson, A. (2013). Facilitating productive use of feedback in higher education. Active Learning in Higher Education, 14(1), 63–76. https://doi.org/10.1177/1469787412467125 Kay, R. H., & LeSage, A. (2009). Examining the benefits and challenges of using audience response systems: A review of the literature. Computers & Education, 53(3), 819–827. https://doi.org/ 10.1016/j.compedu.2009.05.001 Keough, S. M. (2012). Clickers in the classroom: A review and a replication. Journal of Management Education, 36(6), 822–847. https://doi.org/10.1177/1052562912454808 Knight, J. K., Wise, S. B., Rentsch, J., & Furtak, E. M. (2015). Cues matter: Learning assistants influence introductory biology student interactions during clicker-question discussions. CBE Life Sciences Education, 14(4), ar41. https://doi.org/10.1187/cbe.15-04-0093 Knight, J. K., Wise, S. B., & Southard, K. M. (2013). Understanding clicker discussions: Student reasoning and the impact of instructional cues. Cbe-Life Sciences Education, 12(4), 645–654. https://doi.org/10.1187/cbe.13-05-0090 Knight, J. K., & Wood, W. B. (2005). Teaching more by lecturing less. Cell Biology Education, 4(4), 298–310. Retrieved from ://MEDLINE:16341257. https://doi.org/10.1187/ 05-06-0082 Kolikant, Y. B.-D., Drane, D., & Calkins, S. (2010). “Clickers” as catalysts for transformation of teachers. College Teaching, 58(4), 127–135.

1256

K. Egelandsdal et al.

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134. https://doi.org/10.1037//0022-3514.77.6.1121 Krumsvik, R. J. (2012). Feedback clickers in plenary lectures: A new tool for formative assessment? In L. Rowan & C. Bigum (Eds.), Transformative approaches to new technologies and student diversity in futures oriented classrooms: Future proofing education (pp. 191–216). Dordrecht: Springer Netherlands. Krumsvik, R. J. (2014). Forskningsdesign og Kvalitative Metode. Bergen: Fagbokforlaget. Krumsvik, R. J., & Ludvigsen, K. (2012). Formative E-assessment in plenary lectures. Nordic Journal of Digital Literacy, 7(01). Lantz, M. E. (2010). The use of 'Clickers' in the classroom: Teaching innovation or merely an amusing novelty? Computers in Human Behavior, 26(4), 556–561. https://doi.org/10.1016/j. chb.2010.02.014 Ludvigsen, K., & Krumsvik, R. J. (Forthcoming). Behind the scenes: Bringing student voices to the lecture Ludvigsen, K., Krumsvik, R. J., & Furnes, B. (2015). Creating formative feedback spaces in large lectures. Computers & Education, 88(0), 48–63. https://doi.org/10.1016/j.compedu.2015.04.002 Mayer, R. E., Stull, A., DeLeeuw, K., Almeroth, K., Bimber, B., Chun, D., . . . Zhang, H. (2009). Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes. Contemporary Educational Psychology, 34(1), 51–57. https://doi.org/10.1016/j. cedpsych.2008.04.002 Mazur, E. (1997). Peer instruction: A user’s manual. Upper Saddle River, NJ: Prentice Hall. Mazur, E. (2009). Farewell, lecture? Science, 323(5910), 50–51. Retrieved from http://www. sciencemag.org/content/323/5910/50.short. https://doi.org/10.1126/science.1168927 McDonough, K., & Foote, J. A. (2015). The impact of individual and shared clicker use on students’ collaborative learning. Computers & Education, 86, 236–249. https://doi.org/ 10.1016/j.compedu.2015.08.009 Ness, I. J., & Riese, H. (2015). Openness, curiosity and respect: Underlying conditions for developing innovative knowledge and ideas between disciplines. Learning, Culture and Social Interaction, 6, 29–39. https://doi.org/10.1016/j.lcsi.2015.03.001 Ness, I. J., & Søreide, G. E. (2014). The room of opportunity: Understanding phases of creative knowledge processes in innovation. Journal of Workplace Learning, 26(8), 545–560. https:// doi.org/10.1108/JWL-10-2013-0077 Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. Nielsen, K. L., Hansen, G., & Stav, J. B. (2016). How the initial thinking period affects student argumentation during peer instruction: Students’ experiences versus observations. Studies in Higher Education, 41(1), 124–138. https://doi.org/10.1080/03075079.2014.915300 Novak, G., & Patterson, E. (2010). An introduction to just-in-time-teaching (JiTT). In S. Simkins & M. Maier (Eds.), Just-in-time teaching: Across the disciplines, across the academy. Sterling, VA: Stylus Publishing. Nystrand, M., & Gamoran, A. (1997). The big picture: Language and learning in hundreds of English lessons. In M. Nystrand (Ed.), Opening dialogue. New York: Lecturers College Press. Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), 223–231. Retrieved from ://WOS:000226629800009. Rao, S. P., & DiCarlo, S. E. (2000). Peer instruction improves performance on quizzes. Advances in Physiology Education, 24(1), 51–55. Risko, E. F., Anderson, N., Sarwal, A., Engelhardt, M., & Kingstone, A. (2012). Everyday attention: Variation in mind wandering and memory in a lecture. Applied Cognitive Psychology, 26(2), 234–242. Retrieved from. https://doi.org/10.1002/acp.1814

50

Clicker Interventions in Large Lectures in Higher Education

1257

Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory. Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181–210. https://doi.org/10.1111/j.1745-6916.2006.00012.x Rush, B. R., Hafen, M., Biller, D. S., Davis, E. G., Klimek, J. A., Kukanich, B., . . . White, B. J. (2010). The effect of differing audience response system question types on student attention in the veterinary medical classroom. Journal of Veterinary Medical Education, 37(2), 145–153. Retrieved from ://WOS:000279723700007. http://utpjournals.metapress.com/content/n942321151651876/? genre=article&id=doi%3a10.3138%2fjvme.37.2.145. https://doi.org/10.3138/jvme.37.2.145 Samuelsson, M. & Ness, I. J. (Forthcoming). How to turn democratic deliberations into productive processes of co-operation – a response to “Deliberating public policy with adolescents”. Sanderson, B. (2017). Brandon Sanderson discusses the past and future of the Stormlight archive/ interviewer: A. Moher. New York, NY: Barnes & Noble Sci-Fi and Fantasy blogg. Schell, J., Lukoff, B., & Mazur, E. (2013). Catalyzing learner engagement using cutting-edge classroom response systems in higher education. In C. Wankel (Ed.), Increasing student engagement and retention using classroom technologies: Classroom response systems and mediated discourse technologies (pp. 233–261). Bingley, UK: Emerald Group Publishing Limited. Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–522. https://doi.org/10.1207/s1532690xci1604_4 Scott, P. H., Eduardo, F. M., & Aguiar, O. G. (2006). The tension between authoritative and dialogic discourse: A fundamental characteristic of meaning making interactions in high school science lessons. Science Education, 90(4), 605–631. https://doi.org/10.1002/sce.20131 Shapiro, A. M., & Gordon, L. T. (2012). A controlled study of clicker-assisted memory enhancement in college classrooms. Applied Cognitive Psychology, 26(4), 635–643. Retrieved from ://WOS:000306401100017. http://onlinelibrary.wiley.com/doi/10.1002/acp.2843/ abstract?systemMessage=Wiley+Online+Library+will+be+disrupted+on+7+December+from +10%3A00-15%3A00+BST+%2805%3A00-10%3A00+EDT%29+for+essential+maintenanc e. https://doi.org/10.1002/acp.2843 Shapiro, A. M., & Gordon, L. T. (2013). Classroom clickers offer more than repetition: Converging evidence for the testing effect and confirmatory feedback in clicker-assisted learning. Journal of Teaching and Learning with Technology, 2(1), 15–30. Shapiro, A. M., Sims-Knight, J., O'Rielly, G. V., Capaldo, P., Pedlow, T., Gordon, L., & Monteiro, K. (2017). Clickers can promote fact retention but impede conceptual understanding. Computers in Education, 111(C), 44–59. https://doi.org/10.1016/j.compedu.2017.03.017 Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. Smith, E. L., Rice, K. L., Woolforde, L., & Lopez-Zang, D. (2012). Transforming engagement in learning through innovative technologies: Using an audience response system in nursing orientation. Journal of Continuing Education in Nursing, 43(3), 102–103. https://doi.org/ 10.3928/00220124-20120223-47 Smith, M. K., Wood, W. B., Adams, W. K., Wieman, C., Knight, J. K., Guild, N., & Su, T. T. (2009). Why peer discussion improves student performance on in-class concept q. Science, 323(5910), 122–124. https://doi.org/10.1126/science.1165919 Smith, M. K., Wood, W. B., Krauter, K., & Knight, J. K. (2011). Combining peer discussion with instructor explanation increases student learning from in-class concept questions. Cbe-Life Sciences Education, 10(1), 55–63. https://doi.org/10.1187/cbe.10-08-0101 Sun, J. C.-Y. (2014). Influence of polling technologies on student engagement: An analysis of student motivation, academic performance, and brainwave data. Computers & Education, 72(0), 80–89. Retrieved from http://www.sciencedirect.com/science/article/pii/S0360131513002959. https://doi.org/10.1016/j.compedu.2013.10.010 Vickrey, T., Rosploch, K., Rahmanian, R., Pilarz, M., & Stains, M. (2015). Research-based implementation of peer instruction: A literature review. Cbe-Life Sciences Education, 14(1). https://doi.org/10.1187/cbe.14-11-0198

1258

K. Egelandsdal et al.

Wieman, C. (2007). Why not try a scientific approach to science education? Change: The Magazine of Higher Learning, 39(5), 9–15. Retrieved from http://www.tandfonline.com/doi/abs/10.3200/ CHNG.39.5.9-15. https://doi.org/10.3200/CHNG.39.5.9-15 Wood, A. K., Galloway, R. K., Hardy, J., & Sinclair, C. M. (2014). Analyzing learning during peer instruction dialogues: A resource activation framework. Physical Review Special Topics – Physics Education Research, 10(2), 020107. Yoder, J. D., & Hochevar, C. M. (2005). Encouraging active learning can improve students’ performance on examinations. Teaching of Psychology, 32(2), 91–95. Retrieved from ://WOS:000228768600002. https://doi.org/10.1207/s15328023top3202_2 Zingaro, D., & Porter, L. (2014). Peer instruction in computing: The value of instructor intervention. Computers & Education, 71, 87–96. https://doi.org/10.1016/j.compedu.2013.09.015

Dr. Kjetil Egelandsdal is a researcher at the Centre for the Science of Learning and Technology (SLATE) and part of the research group Digital Learning Communities at the University of Bergen. His Ph.D. dissertation was on the use of student response systems to promote formative feedback at university lectures. Egelandsdal has extensive experience with the use of mixed methods research and research on ICT in education. In addition to studies in higher education, he is one of the researchers behind the large national study “SMILE,” which investigated the use of ICT in upper secondary school in Norway. His research interests are ICT and education, formative assessment, and philosophy of education. Kristine Ludvigsen is a Ph.D. candidate at the Department of Education and part of the research group Digital Learning Communities at the University of Bergen. Her research area is how technology can support student-active learning within the context of lectures and seminars in higher education. Dr. Ingunn Johanne Ness is an expert on learning, creativity, and innovation in the fields of education and business. She currently holds the position as a postdoc and cluster leader at SLATE, the Centre for the Science of Learning and Technology at the University of Bergen, Norway. Ness has a particular interest for the sociocultural approach to innovative knowledge development and works with one of the world’s leading environments on sociocultural theory, the OSAT group at the Department of Education, University of Oxford. Ness has done extensive empirical research on collaborative creativity in strategy and innovation contexts.

Transformative Learning in an Online Doctoral Programme: Autoethnography as a Pedagogical Method

51

Kyungmee Lee

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Tutor’s Autobiographical Writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Theoretical Framework: Transformative Learning and Autoethnography . . . . . . . . . . . . . . . . . . An Illustration: Autoethnography in an Online Doctoral Programme . . . . . . . . . . . . . . . . . . . . . . . . Building a Community and Facing a Disorienting Dilemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Planning an Autoethnography and Continuing Critical Reflection and Rational Dialogues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Writing an Autoethnography and Transforming Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Closing Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1260 1261 1265 1266 1268 1272 1275 1277 1278

Abstract

For many adults, doing a doctoral study at a distance can be a very challenging process. In that process, many adult students face a series of disorienting dilemmas. Those moments may trigger critical reflection and rational dialogue, which in turn may lead to planning different actions and developing new perspectives: effectively, transformative learning. However, given that transformative learning processes are often accompanied by negative emotional experiences, which can result in failures and dropouts, it is very important to provide online doctoral students with appropriate pedagogical support. This chapter introduces “autoethnography” as a pedagogical method, illustrating how it can be used to facilitate and support transformative learning experiences of online doctoral students (as well as the ones of online tutors). The chapter is built upon the author’s multi-year design-and-teaching experiences on one UK-based online doctoral programme in the field of educational research. In this focused case, 24 adult students in a single cohort are all working professionals in different K. Lee (*) Lancaster University, Lancaster, UK e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_150

1259

1260

K. Lee

educational contexts, pursuing their doctoral study as part-time distance students. Many of them have experienced disorienting dilemmas while participating in the very first module of their online doctoral programme. To support online doctoral students’ transformative learning, the author has used and taught a qualitative methodology of “autoethnography,” which allows researchers to articulate their personal experiences and emotions and use them to explore and interpret aspects of their own cultural practice and social relationships. The chapter also describes how using autoethnography as a pedagogical method has guided the author’s course (re-)design and teaching practices, which further enable the author to effectively transform her pedagogical perspectives. Keywords

Online doctoral programme · Transformative learning · Autoethnography · Disorienting dilemmas · Online tutor

Introduction New information is only a resource in the adult learning process. To become meaningful, learning requires that new information be incorporated by the learner into an already welldeveloped symbolic frame of reference, an active process involving thought, feelings, and disposition. The learner may also have to be helped to transform his or her frame of reference to fully understand the experience. (Mezirow, 1997, p. 10)

For most adults, with long-held beliefs on the self and their neighboring others and multiple responsibilities in their personal and professional lives, doing a doctoral study at a distance, as a part-time student, can be very challenging (see Lee, Choi, & Cho, 2019). In the new learning and living conditions, doctoral students often face a series of “disorienting dilemmas,” the moments when one’s expectations and experiences do not match, one’s everyday habits and new ways of being are conflicting, and one’s frame of reference is destabilized and questioned (Mezirow, 2000). Those moments may trigger critical reflection and rational dialogue among those students, through which they may be able to plan different actions and develop new perspectives: effectively, transformative learning (see Lee & Brett, 2015). However, such transformative learning processes are often accompanied by negative emotional experiences (e.g., a sense of anger, distress, doubts, denial), which can result in failures rather than effective learning in online doctoral programmes. Not only doctoral students but also academics in online doctoral programmes may experience different moments of a disorienting dilemma when their frame of reference about doctoral studies does not work and further crashes into the ones of their doctoral students. Most academics, teaching at universities offering doctoral programmes, have developed their pedagogical beliefs and expectations about doing a doctorate based on their own experiences of being (also, teaching) full-time doctoral students in more traditional face-to-face settings. Thus, becoming a good “tutor” in such peculiar, despite their growing popularity, online doctoral programmes also requires traditional

51

Transformative Learning in an Online Doctoral Programme:. . .

1261

academics to transform their long-held pedagogical beliefs and actions. Thus, the transformative learning process can be equally – both emotionally and cognitively – challenging for many academics responsible for supporting their doctoral students with a great diversity of needs, backgrounds, and learning, working, and living conditions. This chapter, proposes “autoethnography” as a pedagogical method for supporting both doctoral students and academics’ transformative learning processes. Autoethnography is a form of qualitative research that combines characteristics of autobiography and ethnography and aims to develop social, cultural, and political meanings of a researcher’s personal experiences through “autobiographical writing” (Chang, 2008). This form of research allows researchers to articulate their inner, often subconscious, thoughts and emotions, to use them to explore and interpret particular aspects of their cultural practices and social relationships, and to share their experiences and interpretations with others. That is, autoethnography is both a process and an outcome of the research; and a researcher is an author at the same time in the research (Adams, Holman Jones, & Ellis, 2015). The most critical component of autoethnography is to consider ways in which others may experience similar events. Thus, researchers need to compare and contrast their experiences and interpretations against the ones of others that are often accessible through reviewing previous literature and communicating with (interviewing) those in similar situations with the researchers. Such a methodological emphasis both on “the self” and “the others” in autoethnography can effectively serve as a pedagogical principle in online doctoral programmes to support students’ transformative learning processes of becoming a researcher. It can also facilitate academics’ transformative learning processes of becoming a good tutor by enabling the academics to better understand their students. This chapter is built upon the author’s own experiences of (re-)designing and teaching a research methodology module on one online doctoral programme in the United Kingdom, capturing the transformations in her pedagogical perspectives about the meaning of doing a doctorate and being a doctoral student. Thus, the chapter first presents the author’s autobiographical writing of her moment of the disorienting dilemma, which triggered her transformative learning experience. The chapter then introduces a theoretical and methodological base for the use of autoethnography as a pedagogical method and illustrates how autoethnography has been incorporated into the author’s module. This part will include actual voices of online doctoral students to offer a clearer sense of how the method has facilitated their perspective transformations during the first 6 months of their online doctoral studies.

A Tutor’s Autobiographical Writing Monday morning at 9 o’clock, on 18th of June 2018, I was sitting at my desk, staring at my computer screen, on which I had an email opened saying:

1262

K. Lee

Dear Administrator, The module evaluation for [the module] has now finished and the report for this module is attached to this email. If you have any questions about this evaluation, please contact the ISS Service Desk (ext. XXXX). Best regards, ISS I felt my heart beating faster. An enormous amount of stress was coming down onto my stomach, which was not a pleasant feeling at all. It took at least 5 min for me to click and open the attached pdf file entitled “[the module code]2018.” That stress! Who is going to understand this? Two of my colleagues, also good friends of mine, had been telling me that “you are an amazing tutor and a great researcher. You should be more confident.” One of them even told me: “I can’t understand why you are so worried about how students think about you. You can’t satisfy all of them. There will always be moaners. Don’t let them bother you!” It was when I received a module evaluation report last year. According to the report, 2 students had given a score 1, very poor, on the quality of my teaching with some very negative comments on my teaching approach. I knew that it was just 2 unhappy students out of 25. Most of the other students seemed happy and at least appreciative of my effort. The overall mean of student scores was 4.27 out of 5.0 with a standard deviation of 1.22, which is good as far as the university is concerned. However, the fact that two students hated my module, my teaching, and “me” made me extremely surprised and embarrassed at first. Over some time (I am not sure, maybe 2 weeks or a month), the initial surprise and embarrassment had grown into some sort of anger and a sense of disappointment. I had put so much effort into my teaching, hours after hours, discussed with each of those 25 students about their research project (module assignment) ideas, reviewed their draft and final proposals, and offered detailed feedback on their 5500-word drafts, and I even made myself available on weekends for Skype conversations with some of them! What did I do wrong? I certainly did not know how to stop this bothering me! I felt trapped in an inescapable “what if” loop: just like a black hole. What did I eventually do to escape from the hole and get rid of those negative emotions? Nothing really; the new semester started, and soon, I became extremely busy with teaching another module and gradually, day by day, forgot about the previous module and the two anonymous students (I still do not know who they are). Well, at least, I thought I had forgotten. Another few months passed by quickly, and it was again the time to set up a Moodle site for the same module. As soon as I thought about the module, all those negative emotions, all of a sudden, came back to me as strong as they were when I first read the two anonymous students’ aggressive comments. I was in a dilemma:

51

Transformative Learning in an Online Doctoral Programme:. . .

1263

What am I going to do now? Am I going to be okay to do the module in the same way I did last year? Come on, 4.27 is not too bad. Only the two. . . but, am I going to be okay to have another pair of students hating my module? Definitely, no. Okay. . . now I need to do something differently. I need to make the module better. But how?

I brewed coffee and set down. On an empty page of my notebook, I put a huge question mark: “what did I do wrong last year?” After a couple of hours, the page was just full of meaningless scribbles. I had to move further back: “how did I end up teaching on the programme? What and how have I done so far on the programme?” In September 2015, I was a doctoral student and then became a tutor on this doctoral programme overnight (I submitted my final thesis to my previous university, only a day before I started to teach here). I was extremely lucky to get this lectureship immediately after my PhD. Being overjoyed, I started designing the module enthusiastically, which I taught for the first time in January 2016. I did my doctoral study at a prestigious university in North America with a group of world-leading academics (some were internationally well-known scholars). As a full-time PhD student whose study was fully funded by the university, I was privileged to explore multiple theoretical and methodological approaches to educational research within relatively broad disciplinary boundaries of “curriculum, teaching, and learning” without worrying about performing any other social and professional responsibilities. I learned, or at least explored, a dozen different research methodologies before choosing the one I used in my thesis. This was mainly my push out of my curiosity rather than my programme’s; but it is important to note that the programme enabled me to do so by offering a large number of research methodology courses as well as, more fundamentally, by offering me the fully funded studentship. Sitting in a classroom as 1 of 15 full-time doctoral students, who were all eager to learn more from (and often impress) their “professors” (well-known scholars in their topic areas) with full authority over the classes, I never thought about questioning any aspects of the classes or their teaching approaches. Whatever was given to me, either as learning materials or as course assignments, I was rather grateful; I almost killed myself to meet all those expectations and deadlines set by each of the academics who taught me during my doctoral study. If I found any of the given materials or tasks challenging, then it was always my fault (I thought.) for not being intelligent enough or not trying hard enough – it was never any of those professors’ fault, of course. There was only one way in which I could become a good doctoral student – read more and write more. Asking for additional support or for extensions to given deadlines was considered incompetent (and unacceptable) among those fulltime doctoral students; thus, I never did it. Eventually, I grew up as an educational researcher, equipped with multiple qualitative research methodologies, whose research interest is in better understanding and supporting adults’ distance learning – more specifically, educational professionals’ online learning (see Lee & Brett, 2015; Lee, 2018). Given that I knew

1264

K. Lee

both doctoral studies and adult distance learning, teaching on the current online doctoral programme seemed straightforward. I designed the module with very structured tasks and specific deadlines, so as to be clear to distance students who often find ill-structured learning tasks and environments challenging. I, as an educational researcher with a more favorable view on qualitative research, then put a lot of effort to help the doctoral students learn what I knew and develop the same view as mine. Well, I did the same as my professors did, which worked well for me and my doctoral student colleagues. I was completely confident, thinking “if students follow the module structure and complete all tasks, then they should be able to conduct a good qualitative study,” but only until the day that I received the disheartening reviews: one of them said, “If you are not interested in what the tutor is forcing you to do, this module is completely a waste of your time!” Fast-forwarded again to the day that I reflected on the negative reviews, staring at the page full of meaningless scribbles; I went through each of the 25 students I taught in the previous year – I could not help asking myself who those two were! Suddenly, I realized that most of them were not like how I was as a doctoral student. This group of part-time online doctoral students tends to be always pressed for time and resources; openly shows their frustrations or dissatisfactions toward any given tasks if they were different from their expectations; does not hesitate to contact me to share their concerns and ask for additional support or time; and does not see themselves as a novice and me as an expert. Rather, these experienced educational professionals (including a reasonable number of university lecturers) see themselves as an expert in what they are doing and what they know; and each has their own view and knowledge on educational research as strong as (if not stronger than) mine. This group tends to be pragmatic about their doctoral study as well – very goal-oriented and task-oriented: often focusing on passing modules, finishing the PhD, and earning a doctorate, rather than learning new knowledge and becoming a researcher. Yes! They were not like how I was and different from how I expected them to be. If not, who were they? All of a sudden, I remembered what one of my interviewees in my thesis project (Lee, 2015), an experienced distance teacher, said: The first thing is to get to know the student as a “person” as best you can. Some students are struggling, they have to struggle to get the time. There are single mothers with children and earning for a living. And sometimes, they need some recognition from us just how hard it is to become a student and to fulfil those requirements [...] In order to really begin the teaching process, there is a point of contact where you have to recognize another human being on the other end of the relationship [...] I am a distance learning person so I try to deal with students individually as a human to see what their resources are, what their experiences are, how they can use the experiences to fulfil the requirements in the course. (Lee, 2015, emphasis added)

I opened a new word document, starting to redesign my module and thinking about how I can get to know those students as a person as best I can in my research methodology module. I remembered that I had taken a course entitled “auto-biographies in educational research,” from which I learned about autoethnography. I strongly felt that there should be something very useful in doing and teaching

51

Transformative Learning in an Online Doctoral Programme:. . .

1265

autoethnography. A week later, I had uploaded a completely new version of the module handbook online, which said: “Among many different methodologies used for educational research, in this module, we are going to focus on autoethnography.” This short anecdote describes the critical moments of my own transformative learning process triggered by receiving negative student feedback when I faced a “disorienting dilemma” and I started “critical reflection.” By employing autoethnography in my teaching, I created an opportunity to have “rational dialogue” with my doctoral students, which eventually led to the full “transformation” of my pedagogical beliefs and practices (see the next section for Mezirow’s 2000 transformative learning process).

A Theoretical Framework: Transformative Learning and Autoethnography Mezirow (1991, 2000) illustrates transformative learning as a process that individuals experience radical changes in their perspectives (or frames of reference) through critical reflection and rational dialogue. There are, at least, two different approaches to examining transformative learning: each focuses on learning outcome and the learning process. Based on Mezirow’s theorization, Yorks and Kasl (2006) define outcome of transformative learning as: “a wholistic change in how a person both affectively experiences and conceptually frames his or her experience of the world when pursuing learning that is personally developmental, socially controversial, or requires personal or social healing” (p. 46). That is, the outcome of transformative learning can include all cognitive, affective, and behavioral changes, which fully shift ways in which the person sees, feels, and interacts with others in society. These shifts are usually expected and considered to be positive and developmental in transformative learning (Stevens-Long et al., 2012). A learning process that leads to meaningful perspective transformations can be condensed into four main phases: (1) a disorienting dilemma which causes the learner to experience a radical mismatch between their own assumptions and the particular situations, (2) critical reflection including self-examination of the experience and a critical assessment of their own assumptions, (3) rational dialogue which involves sharing the experience with others and exploring alternative approaches to the situations, and (4) planning a different action, acquiring knowledge for implementing the plan, and reintegrating new perspectives into one’s life (Herbers, 1998, as cited in Glisczinski, 2007; Mezirow, 2000). Mezirow (2003) further suggests four characteristics of rational dialogue that support transformative learning: (1) representing the self to others, (2) sharing different perspectives, (3) assessing various values, and (4) reconstructing new beliefs. In this sense, the nature of dialogue required for transformative learning needs to be distinguished from the common characteristics of discussion and interaction in social constructivist learning theories that aim to share knowledge and construct new knowledge (e.g., Hannafin, 1989; Woo & Reaves, 2007).

1266

K. Lee

An important question is, to the author then, how to facilitate and support doctoral students’ transformative learning process in such a research methodology module in an online doctoral programme. This is where “autoethnography” effectively comes into play. Autoethnography is a qualitative research attempt to collect stories about the self, to understand the shared aspects of general culture and the cultural practices embedded and represented in those self-narratives (Chang, 2008). By using autobiographical stories and self-reflection on those stories as main data sources, researchers can explore and access their complex inner thoughts and emotions relevant to those stories and, thus, develop a more comprehensive understanding of social phenomena (Adams et al., 2015). Given that autoethnography often embarks from a researcher’s narrative exploration of their transformative moments (i.e., epiphanies) and it often results in other epiphanies through the inquiry processes, there is a strong parallel between autoethnography and transformative learning. At the same time, autoethnography shows researchers in “the process of figuring out what to do, how to live, and the meaning of their struggles” (Bochner & Ellis, 2006, p. 111), and so, ultimately, strives for social justice and to make life better (Adams et al., 2015, p. 2). Adams et al. (2015) describe general principles of doing autoethnography: (1) autoethnographers foreground personal experiences (often focusing on sadness and discomfort) in their research and writing; (2) autoethnographers illustrate the sense-making processes of their experiences; (3) autoethnographers use and show reflexivity to turn back to their social identities and relationships in order to consider how they influence their sense-making processes; (4) autoethnographers offer insider knowledge of the cultural phenomenon by researching and writing from the lived, inside moments of their experiences; (5) autoethnographers describe and critique cultural norms and practices; and (6) autoethnographers seek reciprocal responses from audiences. The unique methodological approach of autoethnography can effectively serve as a pedagogical tool for transformative learning, which will be illustrated in the following section.

An Illustration: Autoethnography in an Online Doctoral Programme The online doctoral programme focused in this chapter consists of two academic phases: in Part 1, approximately 25 doctoral students, who are all in-service educational professionals, enter the programme at the same time and take 6 online modules together for the first 2 years. This cohort-based social learning process is effectively facilitated by a range of collaborative activities (e.g., group discussions, peer reviews) and by annual residential meetings, during which all cohort members come to campus and participate in face-to-face sessions. Subsequently, students move to Part 2, in which they independently work on their thesis project with some supervisory guidance for a period of 2–3 years. The author’s research methodology module is the very first module of the programme, which lasts for 20 weeks. The purpose of the module is to provide doctoral students with a solid understanding

Transformative Learning in an Online Doctoral Programme:. . .

51

1267

of how to plan, conduct, and evaluate educational research. The module is structured around three different learning phases: (i) Phase I (10 weeks), during which students independently read suggested materials and participate in online group discussion guided by the tutor while planning their independent module project (i.e., autoethnography); (ii) Phase II (6 weeks including a week-long residential meeting), during which students conduct their autoethnography and write and submit their draft assignment; and (iii) Phase III (4 weeks), during which students review two of their peers’ drafts, revise their draft based on tutor and peer feedback, and evaluate their learning. The major assignment of the module is to plan, conduct, and write a 5500-word autoethnography on a specific topic chosen by each student and supported by the module tutor, which enables students to research issues that arise out of their personal experiences. The assignment is divided into several sub-tasks – a belief description of relevant sub-task submission deadlines and available support is provided in the module handbook as follows: Task submission deadlines: Your autoethnography will be supported and assisted by your module tutor and other members in your cohort throughout the module period. To work effectively together, keeping the following submission deadlines is strongly encouraged: • • • • • •

Week 1: a visual self-introduction that includes your research interests Week 5: a one-page research proposal for peer-tutor comments Week 10: a final research proposal for peer-tutor comments and approval Week 16: a draft assignment for peer-tutor comments Week 18: two peer reviews Week 20: a final assignment for assessment

In the last academic year, throughout January 2018 till June 2018, 24 doctoral students completed the author’s redesigned module 1. While teaching the module, the author wrote detailed field notes each week to record how autoethnography supports and triggers those doctoral students’ transformative learning. These field notes provide basic material for the author’s narratives in this chapter. The module artifacts, all objects produced by module participants and stored on the Moodle site as a result of their engagement in module activities (e.g., discussions, assignments), are also collected and used in this chapter to elaborate the author’s points. A formal ethics review was approved for collecting the data stored on the Moodle site, and individual informed consent for using the data for research purposes was obtained from module participants during the residential week. The chapter also includes some of the author’s insights gained through her interactions (both face-to-face

1268

K. Lee

conversations and email exchanges) with the students and other tutors. In this case, it is avoided to use direct quotes from those interactions. In order to maintain the author’s ethical position, this chapter employs a notion of “relational ethics” (read Ellis, 2007 about relational ethics in research with intimate others). As mentioned above, the author has strived to get to know her students as a person and to develop a positive relationship with her students throughout the module period. Even after the module, the author keeps a strong sense of bonding and caring toward all students. Thus, when writing this chapter, the author has tried to put herself in the shoes of each student, imagining how each of them would read this manuscript. Any data (stories/voices) and claims, making the author unsure or worried even marginally, have been deleted and excluded in the chapter. Also, a draft has been shared with those participants whose voices are included, and any concerns and questions raised by the participants have been seriously considered and carefully addressed. The author does believe that nothing is worth more than maintaining valuable relationships with student participants. From the next paragraph, to better illustrate doctoral student and the tutor’s transformative learning experiences in the module, the author will shift her position from an author (a researcher) to an online tutor and speak in the first person.

Building a Community and Facing a Disorienting Dilemma The module began with students’ visual self-introduction posts on the module Moodle site. Each student included two images in their posts: One represented their professional role and context, answering the question “what do you do, and what connection does it have with your research interests and doing a PhD study in this programme?” The other photo represented their learning context and strategies, answering the question “how, where, and when do you learn and study, and what learning strategies/methods are you planning to use during this module or programme of your study?” Introducing the self in a written post is a common practice in most online modules. However, encouraging students to select and post two images best represents who they are as a professional, and a learner seemed to provide them with an opportunity to take time to think about the self and to carefully carve their first post into something more meaningful. Figure 1 below is a collection of some images representing students’ learning contexts from the posts (the images that do not directly expose the students’ identity were purposefully selected to be presented here). That visual part of those posts effectively facilitated and mediated interactions among the doctoral students who had never met before and just got together in the virtual space, by increasing a perceived sense of authenticity of each other. The cohort quickly engaged in the dialogues about the self and the others. There were lots of expressions of surprise and amusement such as “we picked the same photos!” and “I am at the other end of you!” A significant level of uncertainty about the others’

51

Transformative Learning in an Online Doctoral Programme:. . .

1269

Fig. 1 A collage of students’ photos from Week 1

social presence seemed to become much alleviated by interacting through those carefully selected and presented visual artifacts. Before learning about autoethnography and thinking about the self as a subject, from which research projects and data would be drawn, it seemed to be a good exercise to draw some meaningful boundaries about the self and the others. From the tutor’s perspective, too, reading (or seeing) those posts enabled me to feel each student more like a person separated, or stood out, from the cohort, which I had previously regarded as a single (and rather homogeneous) student group. During the first 4 weeks, there were some foundational readings about research paradigms of positivism and constructivism: each is commonly linked to quantitative research methodologies and qualitative research methodologies. Students were encouraged to think about different philosophical or epistemological ideas and assumptions that comprise the two research paradigms while reading suggested chapters from a module textbook (Cohen, Manion, & Morrison, 2011), which most students found very challenging to read through. Most students, before the module, had neither been engaged in those “paradigm wars” type of debates (see Johnson & Onwuegbuzie, 2004) nor used qualitative research methodologies. Because of the complex ideas and a large number of research “jargons” in the suggested chapters, students tended to be frustrated and anxious during the first couple of weeks, questioning their capability of and readiness for doing a PhD. This was the first moment in the module when most students faced significant disorienting dilemmas both as a researcher and a student. I, then, asked them to write a post: The first two chapters in (Cohen, Manion, & Morrison, 2011) should be quite intensive reading to all. If you feel like you cannot understand anything, it is all fine. Do not worry about it at all. Online group discussions in the second week will be certainly helpful; and then, as you progress in this programme, you will gradually get used to and understand all those concepts [. . .] For now, please bring one paragraph, which you have particularly found challenging to understand from the chapters [. . .] Try to articulate why it is difficult to understand and how you understood the paragraph.

There were a lot of posts including rather candid emotional reactions and “shocks” toward the reading assignment. Those posts provided a good starting point for the

1270

K. Lee

cohort to share the moments of the disorienting dilemma with each other. What was interesting here is that this sharing practice seemed to be legitimated and approved by my precedent acknowledgment of the challenging nature of the texts: in fact, many students referred to me when they expressed their frustrations like “as [my name] mentioned,” “[my name] was right,” and “tutor helpfully pointed out.” The discussion topic asking everyone to say something they could not understand, instead of saying what they know, also effectively set up the supportive atmosphere that each student could reveal their initial resistance to the new ideas suggested by the constructivist research paradigm. This was another common example of the disorienting dilemma faced by many students whose long-held, more conventional, objectivist beliefs about research were strongly challenged. For example, Bill picked up the following paragraph from the textbook: Where positivism is less successful, however, is in its application to the study of human behaviour where the immense complexity of human nature and the elusive and intangible quality of social phenomena contrast strikingly with the order and regularity of the natural world. This point is nowhere more apparent than in the contexts of classroom and school where the problems of teaching, learning and human interaction present the positivistic researcher with a mammoth challenge. (Cohen, Manion, & Morrison, 2011, p.7)

In response to the authors, Bill continued in his post: I feel this paragraph undermines the virtues of the scientific approach. Implicit (within the paragraph and elsewhere in the text) is the characterisation of science as quantitative data collection only, and therefore, less effective than subjective approaches when attempting to capture the complexities of human behaviour [. . . however,] objective scientific methods in educational research using both qualitative and quantitative data in a sound methodological manner which can be used to test theories and explain patterns of data. Most importantly, the empirical data collected allows for self-correcting through repetition by other researchers. In short, my difficulty with the paragraph above is that the authors are ambiguous when suggesting positivism is not the best approach to the study of human behaviour. . . Is subjectivism the most appropriate research approach to the study of human behaviour?

Regardless of the accuracy of Bill’s understanding of the authors’ position, it was clear that the authors’ preference for the subjectivism to the objectivism had made Bill uncomfortable, which enabled him to continue his “critical reflection” on his uncomfortable feelings toward the authors’ position. Bill’s post created intensive “rational dialogues” among the cohort as well. Through being engaged in the dialogues, putting themselves with the authors and the other cohort members, each student’s philosophical position was made clearer. In this context, most students were not hesitant to admit that they were struggling not only to obtain new information but also to make sense of new perspectives, which were far apart from their established understandings of good research. It was also evident that many students found it very challenging to set new daily routines and lifestyles that effectively incorporate required learning activities in their everyday lives without damaging their relationships with other members in their lives: “the learner may also have to be

51

Transformative Learning in an Online Doctoral Programme:. . .

1271

helped to transform his or her frame of reference to fully understand the experience” (Mezirow, 1997, p. 10, emphasis added). It was meaningful, from tutor’s perspective as well, to listen to students’ voices and to get a sense of “who is going to find doing an autoethnography more challenging” since autoethnography can be perceived as an “awfully” constructivist research methodology (borrowing another student Anne’s expression)! Each student’s post also revealed who each student was as a researcher, where each student was in terms of their developmental stage of becoming a researcher, and what each student would need from me to be able to conduct a good autoethnography. I was anxious about potential resistance among some students who were not interested in what I would, perceivably, “force” them to do: that is, doing an autoethnography. However, these worries, together with a clear understanding of where the potential resistance would come from, made me put extra careful attention to almost “selling” the assignment to the students rather than “forcing” them to do it. I produced an hour-long video lecture, explaining different research paradigms and the one underpinning autoethnography. While very gently correcting some of the misconceptions in students’ previous posts, I promoted the value of “trying out” this radically subjectivist research methodology for developing a deeper understanding of educational research. I also talked about the values of the effective utilization of “the self” as a research subject as well as a research tool, in terms of making real changes in their personal and professional contexts. After posting this lecture, I received emails from several students expressing their thanks for posting the lecture and telling me how helpful my lecture was for them to understand different research positions. Sophia reflected on this in her self-reflection post at the end of the module: The module provided me with the learning opportunity to experience fundamental components of research, namely ontology, epistemology, methodology, methods, and data. I learned research methods before and attended various courses on research methodology, but I did not understand these concepts. After taking this module, these concepts became crystal clear to me, and I will be always grateful to the module tutor for her concise explanations and suggested reading materials.

It was another interesting moment for me to be able to challenge my own pedagogical belief about “teaching” doctoral students. I had strongly believed (not only me, this is something argued a lot in teaching committees and programme meetings by other tutors as well) that doctoral students are independent adult learners, and so we are “not” teaching them. In a broader context of doctoral education as well, there are common understandings about what doctoral-level of study should look like and what doctoral students should be able to do. I, as a doctoral student as well, had never expected any of my professors to kindly give me a lecture and explain any academic concepts even when I was struggling to understand those concepts: those intellectual struggles were, in fact, considered as rather noble and honorable tasks for doctoral students. However, supporting students’ conceptual development seemed to be one of the most important factors that facilitated their transformative learning, by

1272

K. Lee

removing the negative emotions that hindered their progress. And some actually expressed their excitement of “trying out” autoethnography, despite their uncomfortable feelings toward qualitative research methodologies and subjectivist/constructivist research paradigm.

Planning an Autoethnography and Continuing Critical Reflection and Rational Dialogues In Week 4, after reading suggested materials about autoethnography, students were asked to start thinking about their own autoethnography as follows: Let’s start brainstorming now! What are you going to do about the module assignment? Is the notion of autoethnography still messy in your head? If then, you can start with your very personal experiences. Tell us one of your stories (related to learning and teaching in general) that have made you feel uncomfortable, uneasy, difficult, unclear, etc.; so you feel like it is worth spending some time and effort on unpacking and better understanding the story (also, making yourself feel better). Where you are/were/have been in that story and who else is in that story? Also, tell us why it may be worthwhile for others to listen to the story.

At the end of the week, a total of 24 stories were posted in the discussion, all including each student’s memorable moment of a disorienting dilemma (i.e., epiphany). Among them, two posts particularly made me think “I cannot wait to read this autoethnography!” with genuine emotional reactions to the students. Margaret posted: It happened in April 2017. It was the first [module name] class for first-year students. When my lesson was about to end, two female students criticized my teaching style in front of the other students and claimed that they did not understand the lesson. I was deeply shocked by their behaviour because [. . .] I have been teaching [. . .] students since 1993 and it was the first time I was verbally abused by the students. Whenever I teach students, I teach not only with my mouth, but also with my heart as I want them to have the best learning experiences. After this incident, I had even considered changing the job although I am passionate about teaching. Therefore, I want to understand why these first-year students misbehaved in week 1 of the semester. Through this understanding, I (teachers) can develop strategies to handle these problems in the classrooms and to help students in taking responsibility for their behaviours and actions.

Here is Abigail: Towards the end of a very, very busy term of teaching and managing many other demands [. . .] two students on an online Master’s programme individually send me (as Course Director) emails challenging the need to re-submit elements of formative assessment. The content of the emails is similar and the tone and language highly critical and accusatory towards the module tutors. Whilst I am open to receive feedback from students on their concerns, I found the nature of these incidents both challenging and somewhat unusual. They provoked a strong reaction in me, distracted me from my teaching and family activities

51

Transformative Learning in an Online Doctoral Programme:. . .

1273

in that week and left me feeling even more ‘overloaded’ [. . .] I would like to use analytic autoethnography to explore these incidents for insight that could inform other possibilities for managing student/tutor communications and expectations regarding assessment feedback in an asynchronous, 24/7/365, online (versus face-to-face) environment.

I was, too, an online tutor who got hurt by students’ disheartening comments after me trying hard to be helpful in the asynchronous, 24/7/365, online environment! Reading those stories made me somehow relieved, feeling like I was finally understood and emotionally supported by others (ironically by students!). Even my colleagues, my close friends, were not able to make me feel that way. Having the two students who suffered from very similar incidents and likely to appreciate the challenging nature of being a good tutor was very healing. Interestingly, soon after the initial posts, the entire cohort started sharing their own, very similar, stories about getting feedback from students. They all very sympathetically talked about the increasing pressure on tutors (and teachers) to satisfy their students. The conversation naturally developed into the next phases of transformative learning (i.e., critical reflection and rational dialogue): for example, the cohort discussed the neoliberal culture within the current university system and its impacts on the tutors’ working and living conditions. Besides those 2, 22 more stories made me get to know each of them as a person and carefully, but legitimately, invited me to see and talk about their personal, offline, spheres. For example, Julia says: I am currently feeling pressure just now, in part with starting this course, juggling ‘real’ life but I’m consumed with emotions related to a situation where I feel I have been ostracised by my ‘Rector’ (leader) at work. He has taken credit for my ideas and then pushed me aside. To make it worse it is two men who have taken over the implementation of my idea (so much for getting women into STEM!). This feeling has been growing gradually since he arrived in the post a year and a half ago and it has now escalated as a lot of information has come to light in recent weeks. I feel very unsettled, confused and emotional which I believe is an excellent catalyst for an autoethnographic study however I’m not sure how I could approach/include my colleagues without making my working life impossible. It might be a cathartic process but I would be unable to make this work public which I assume defeats the purpose. Funnily enough, I’ve never met any of you, but this feels like a safe space for me to share!

That is, to make her personal story into a research project, I (and the cohort) was invited by Julia to critically reflect on her experiences and situations and further explore ways in which she could see and approach the story differently, or more effectively. Such rational dialogues, opened up by the Week 4 discussions, continued throughout the whole module period and further elaborated by the series of sub-tasks including submitting a one-page research proposal (Week 5) and a final research proposal (Week 10). In a broad sense, despite different levels of engagement in those subsequent dialogues among students, the strong sense of community built upon the fact that they got to know each other seemed to facilitate and support their transformative learning processes of becoming both an online doctoral student and an autoethnographer researching their own educational practices. Students seemed to

1274

K. Lee

go through two interconnected sets of rational dialogues while planning and conducting their autoethnography: one in the online learning space with me and other cohort members and the other in their personal and/or professional environments where the research problems occurred and where the new action plans needed to be implemented. One of the important methodological characteristics of autoethnography is its focus on collecting data from a researcher’s memories and reflections on critical moments; and there are always others that exist in those memories since the researcher, as a social being, interacts with other social beings in one’s everyday life. Thus, autoethnographers, in their data collection and analysis processes, naturally have rational dialogues with those important others, who can help them to create more accurate “collective” memories or broaden their perspectives by adding different interpretations or reactions to the recalled events. Autoethnographers also invite different people, who are not necessarily their friends or acquaintances, to rational dialogues, through which participants share different perspectives, assess various values, and reconstruct new beliefs (Mezirow, 2003). For example, Gina wrote a brief description of her methodological choices as: Autoethnography is a means by which the author to “[illuminate] multiple layers of consciousness and understanding, explicitly linking the personal to the cultural” (Campbell, 2015). The focus of this research is my personal experience and perceptions, situated within my upbringing and career path. Therefore, I am necessarily linked to the research including the inherent bias associated with the outcomes of my observations and analysis. RQ1 deals with the factors (e.g. gender, cultural, class, racial) that influenced my trajectory into social science. To answer this question, I plan to first conduct one-on-one interviews with my parents. I will also conduct a joint interview in hopes of creating additional data based on the synergies and dynamics between my parents. The data collected will be coded to identify themes related to the research question. My primary concern is to encourage my parents’ honesty in being the primary data points for this autoethnography.

Another student, Jennifer, whose study was to examine her own experiences parenting her teenage daughter with the excessive use of social media, wrote in her final proposal: As autoethnography is very much based on my experiences, feeling and thoughts I plan to discuss my research with my partner to aim for as accurate an account of information as possible. He, along with my daughter, will also be my guide to what information to share and what should remain private. To enhance my research further I intend to send a (parent & teenager) online questionnaire to volunteers to investigate their perceptions and opinions around social media. This questionnaire will help identify RQ1 and RQ2 [. . .] if the questionnaires do not deliver any useful data I might amend this to be a chat with the parent only.

Through these rational dialogues including ones that happened within the module space, students gradually deepened their understanding of the focused social and cultural events as well as the self and the others in those events, which eventually led to meaningful changes in their frame of reference. It is also worth to note that

51

Transformative Learning in an Online Doctoral Programme:. . .

1275

students tended to effectively use others’ voices, ones not only different from theirs but also similar to theirs, to validate their autoethnographic conclusions and such validation seemed to make them feel supported and healed – just like myself used some of my students’ voices to heal my wound created by those two students from the previous year.

Writing an Autoethnography and Transforming Perspectives The last stage of the transformative learning process is to plan a different action, acquire knowledge for implementing the plan, and reintegrate new perspectives into one’s life. The last stage of autoethnography is to write and share one’s autoethnography (i.e., research report) with others in order to achieve the ultimate aim of the autoethnography, which is to figure out what to do and how to live, and, in doing so, make one’s (and others’) life better. In Week 16, 24 draft autoethnographies had arrived (or uploaded) on the Moodle site. One of the programme requirements, applied into all six modules, is that students participate in a peerreview process, in which each student reviews and provides comments on two of their peers’ drafts. That is, in this module, a submitted draft autoethnography was reviewed by three people (two peer students and a module tutor). From a tutor’s perspective, this is the most challenging couple of weeks in the module period that I need to read all of those submitted drafts (each 5500 words long) and to provide both annotated comments on the draft files and general feedback on the separate feedback sheet. However, I somehow felt excited this time – while downloading each draft file from the Moodle site and opening it on my laptop, I felt that I was invited to another meaningful dialogue with each student. Methodological characteristics of autoethnography allow each autoethnographer (as a storyteller and a researcher at the same time) to have a degree of freedom to explore and express the self and one’s boundaries in relation to their neighboring others while sending their messages to an academic audience still relatively in a conventional format of research presentation. At the other end, the addressees are also given freedom to read, interpret, and respond to the autoethnographic texts in which multiple valid perspectives coexist (including perspectives of the author, neighboring others, and other researchers, drawn from literature). As a result, I enjoyed this rather daunting task! Reading each draft made me feel like I was listening to each student’s transformative life stories as an authentic person (tutor, designer, librarian, childcarer, mom, woman, student, etc.). Each draft included each student’s research (or learning) “process of figuring out what to do, how to live, and the meaning of their struggles” (Bochner & Ellis, 2006, p. 111). Thus, while reviewing each draft, I was exposed to multiple perspectives coexisting in each text, and I too added mine into the text by commenting on the margin of the draft, which turned the daunting marking task into having a meaningful dialogue that involves reciprocal acts of listening and speaking. I also suggested

1276

K. Lee

different ways in which they can analyze their data and discuss their findings to help them deepen their thoughts and change their perspectives. Through the dialogues, I was also able to construct and reconstruct new perspectives beyond just getting to know each student as a person. It was obvious that I am not the only one, who enjoyed being engaged in those dialogues. Students’ self-reflection posts submitted at the end of the module show the perceived value of the peer-review processes as Sara and Mike wrote: I was, to be honest, a little confused and worried about carrying out research adopting autoethnography, but whilst working in this module, I have noted the importance of peer reviews, in both offering and receiving. The reviews, reading others papers highlights how a methodology can take many forms and interpretations and in your approach, you can adapt to meet your goals in researching the topic. I have learnt a lot from peer and tutor comments, to begin to focus my thoughts again, as I had been worried about studying and taking the PhD. (Sara) The autoethnographic experience is something that I found really useful and actually enjoyable, and the process of our first peer review (of many I’m sure) although uncomfortable, was a nudge towards betterment. This experience will drive me on to be a muchimproved researcher and author—so thank you very, very much to Sayed and Sheelagh for their time and thoughtful comments. (Mike)

The depth of students’ engagement in those dialogues mediated by the reviewing processes of each other’s drafts was represented in the quality of comments they exchanged during the 2 weeks. The nature of the provided comments, including the tutor’s comments, was also fundamentally different from more common academic review comments. The comments themselves seemed more dialogical than being evaluative. I recall talking to one of my colleagues about the difficulty in evaluating students’ autoethnography and granting a numeric value, mark, to each as if it is just a research report: “all submissions include the author’s personal stories, meaningful reflections on the stories and rich evidence of perspective transformations—how can we say this is 60 and that is 70?” At the end of the module, all students posted their self-reflections online, and most students clearly articulated their transformative learning outcomes. Most students seemed to experience a certain degree of changes in their self-perceptions and their research paradigm. For example, in Peter’s case, his perception of himself as a supervisor, a colleague, and an academic was all transformed as he said: At the start of this module, despite having supervised Masters students for many years, I was anxious that my understanding of research philosophies and methodologies was too superficial. I have always found some of the terminologies of research quite obscure. I was also concerned that I would be exposing this uncertainty to new colleagues. The first reading was hard. . . I wasn’t alone in this feeling so that was great. I wondered why the tutor would put the hardest reading first. It did not take me long to see why. By tackling Cohen with the determination of someone new to the doctoral programme I did not give up and it eventually made sense. That was a great boost. As we began to exchange our understanding, views and opinions and dialogue grew the value of being part of an online collaborative learning community became even more apparent. It was for me the reality behind the rhetoric. Then

51

Transformative Learning in an Online Doctoral Programme:. . .

1277

there was autoethnography! As a research method, many of us questioned why. I understand that without this challenge I would have slipped very quickly into the traditional methods of how I research. Asking me to use this methodology challenged my research pre-conceptions in a very positive way. I will in future be open to new methodologies. Have I grown academically as a consequence of this module? The answer is a definite yes. I can see already how I have changed as an academic whether it is through my supervision of dissertations or the application of my new found understanding of my own research. (Peter) Here is Alice whose research paradigm was transformed (or at least, expanded): The truth. This module for me from the outset has been intellectually inspiring. . . In reflecting upon the module, I feel the first week’s task, to identify a difficult paragraph and explain why it was hard to understand, was the most challenging and important for me. I really didn’t understand Kierkegaard’s explanation of ‘truth’ (Cohen, Manion, & Morrison, 2011, p. 14), and I desperately wanted to just pick something easier to explain; but I kept being drawn back to it. I had considered ‘truth’ to be something you could absolutely prove, and Kierkegaard opened the notion of ‘subjective truth’ as a reality for me. Indeed, coming from a natural science background, I had never really considered qualitative methodologies to be of much worth (my subjective truth), but Kierkegaard enabled me to consider other ‘truths’ were possibly true. This was quite a light bulb moment for me; and one which really inspired me to read other philosophical text (which I can honestly say, had never really interested me much before). Therefore, what I have gained from this module is a real interest in philosophy and a new-found appreciation of qualitative research. Paul’s post, as rightly pointed out that I “advised”, rather than “forced”, made me laugh: No longer a Positivist! I really enjoyed this module. The self-directed and peer-to-peer learning suited my learning style quite well. I am delighted to report that my knowledge of research studies has vastly improved. The residential provided an opportunity to share concerns and words of support which was a nice experience. The fact that we were ‘advised’ to choose an autoethnographical methodology really opened my eyes. Prior to this study, I would have considered myself a positivist, now I’m not so sure! The subjective is so much more interesting than the objective.

Closing Remarks All in all, it is clear to me that most students in the module went through (different versions of) transformative learning processes. How each of them had been as a teacher, a professional, a researcher, and a person was examined, questioned, challenged, and changed by a series of pedagogical activities planned around the task of learning, planning, and doing an autoethnography. It was a great privilege to be able to observe, support, and contribute to the students’ transformative learning experiences as a module tutor. What I experienced as a tutor on the online doctoral programme was nothing like what I experienced as a doctoral student or what I expected as a novice tutor in the previous year. Through teaching this module using autoethnography as a pedagogical tool, I learned so much about this new group of doctoral students that I did not really know before. As a result, my original perspectives on the meanings of “PhD,” “PhD study,” and “PhD student” – which were established based on my own doctoral experiences – have all been challenged and

1278

K. Lee

shifted. Those perspective transformations have made me become a different tutor with new pedagogical beliefs and practices. Going back to that Monday morning, 18th of June, again: I finally opened the pdf file, feeling extremely nervous and stressed. However, there was nothing negative this time, and all numbers were near to perfect including an evaluation score for the helpfulness of the tutor, which was 5.0 out of 5.0! Reading through all of the positive and considerate comments from students also made me so warm, relieved, and satisfied. What made me feel truly satisfied was not those numbers in the pdf file but the feeling of being connected to each student and the fact that I know each of them as a person. This chapter introduce how “autoethnography” can be used as a pedagogical tool to support and facilitate students’ transformative learning experiences in online doctoral education contexts. In particular, the chapter provide a detailed account of how rational dialogues – not only between the tutor and each student but also among students themselves – can be initiated and facilitated at different stages of planning, conducting, and reviewing an autoethnography. Having rational dialogues in a supportive community is an essential element of transformative learning (see more examples in Lee & Brett, 2015). This is with a hope that other online tutors in higher education settings can find this pedagogical method useful in their teaching practices, have more satisfying interactions with their students, and further experience meaningful changes in their pedagogical perspectives and practices. I believe that even in other pedagogical contexts, where it is not feasible for students to conduct and write an entire piece of autoethnography, general methodological principles of doing an autoethnography can still be effectively and creatively integrated into tutor’s teaching and module design.

References Adams, T. E., Holman Jones, S., & Ellis, C. (2015). Autoethnography: Understanding qualitative research. New York, NY: Oxford University Press. Bochner, A. P., & Ellis, C. (2006). Autoethnography. In G. J. Shepherd, J. S. John, & T. Striphas (Eds.), Communication as. . .: Perspectives on theory (pp. 110–122). Thousand Oaks, CA: Sage Publications. Campbell, K. (2015). The feminist instructional designer: An autoethnography. In: B. Hokanson, G. Clinton, & M. Tracey (Eds.), The design of learning experience. Cham: Springer. Chang, H. (2008). Autoethnography as a method. Abingdon, England: Taylor & Francis. Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed.). Abingdon, England: Taylor & Francis. Ellis, C. (2007). Telling secrets, revealing lives: Relational ethics in research with intimate others. Qualitative Inquiry, 13(1), 3–29. Glisczinski, D. J. (2007). Transformative higher education: A meaningful degree of understanding. Journal of Transformative Education, 5(4), 317–328. Hannafin, M. J. (1989). Interaction strategies and emerging instructional technologies: Psychological perspectives. Canadian Journal of Educational Communication, 18, 167–179. Herbers, M. S. (1998). Perspective transformation in preservice teachers. Unpublished doctoral dissertation, University of Memphis, Tennessee.

51

Transformative Learning in an Online Doctoral Programme:. . .

1279

Johnson, B. R., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. Lee, K. (2015). Discourses and realities of online higher education: A history of [discourses of] online education in Canada’s Open University (Doctoral dissertation, University of Toronto). Lee, K. (2018). Everyone already has their community beyond the screen: Reconceptualizing online learning and expanding boundaries. Educational Technology Research & Development, 66, 1255–1268. Lee, K., & Brett, C. (2015). Dialogic understanding of teachers’ online transformative learning: A qualitative case study of teacher discussions in a graduate-level online course. Teaching and Teacher Education, 46, 72–83. Lee, K., Choi, H., & Cho, Y. H. (2019). Becoming a competent self: A developmental process of adult distance learning. The Internet and Higher Education, 41, 25–33. Mezirow, J. (1991). Transformative dimensions in adult learning. San Francisco, CA: Jossey-Bass. Mezirow, J. (2000). Learning as transformation. Critical perspectives on a theory in progress. San Francisco, CA: Jossey-Bass. Mezirow, J. (2003). Transformative learning as discourse. Journal of Transformative Education, 1, 58–63. Mezirow, J. (1997). Transformative learning: Theory to practice. New directions for adult and continuing education, 1997(74), 5–12. Stevens-Long, J., Schapiro, S. A., & McClintock, C. (2012). Passionate scholars: Transformative learning in doctoral education. Adult Education Quarterly, 62(2), 180–198. Woo, Y., & Reeves, T. C. (2007). Meaningful interaction in web-based learning: A social constructivist interpretation. Internet and Higher Education, 10(1), 15–25. Yorks, L., & Kasl, E. (2006). I know more than I can say: A taxonomy for using expressive ways of knowing to foster transformative learning. Journal of Transformative Education, 4(1), 43–64.

Operationalizing Transformative Learning: A Case Study Demonstrating Replicability and Scaling

52

Jeff King and Brenton Wimmer

Contents Introduction – Operationalizing Transformative Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transformative Learning in Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . UCO’s Student Transformative Learning Record . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . History of TL at UCO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Building and Implementing STLR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Training, Technology, and How STLR Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tracking and Assessing STLR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Researching STLR’s Impact as Institution-Wide TL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Impact on Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Impact on Faculty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . STLR’s Impact at UCO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Spreading TL in Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusions, Limitations, and Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1282 1285 1286 1289 1290 1295 1297 1298 1300 1303 1305 1306 1308 1310

Abstract

The University of Central Oklahoma’s (UCO’s) Student Transformative Learning Record (STLR) initiative is an institution-wide implementation of transformative learning (TL) in both the curriculum and the cocurriculum. Most of the institution’s faculty were unfamiliar with TL when it became a focus of the university’s mission. Consequently, UCO had to train faculty in instructional practice meant to prompt for TL. This chapter examines STLR’s impact on student success and beyond-disciplinary learning. Also examined is the impact of faculty’s adoption of STLR on their pedagogy/andragogy and upon their conceptions of the teaching/learning enterprise. The multiparadigmatic research approach advocated by Taylor, Taylor, and Luitel (International Handbook of Science Education. J. King (*) · B. Wimmer Center for Excellence in Transformative Teaching and Learning, University of Central Oklahoma, Edmond, OK, USA e-mail: [email protected]; [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_151

1281

1282

J. King and B. Wimmer

Dordrecht, The Netherlands: Springer, 2012) serves well in conjunction with the transformative framework described by Creswell and Plano Clark (Designing and conduction mixed methods research. Los Angeles, CA: Sage, 2011) for this inquiry. An overview of the STLR process is provided for context. That discussion includes a description of STLR’s beyond- or trans-disciplinary focus areas, called “tenets,” and how curricular and cocurricular learning activities align with them to help scaffold students’ growth toward transformative understandings. Quantitative results illustrate significant association between STLR and student retention as well as academic performance across all 5 years to date of the initiative. Multiparadigmatic analysis, while still preliminary, supports faculty and student transformation also associating with STLR. Recommendations for further research are presented. Keywords

Transformative learning · Institutional change · Faculty development · Retention improvement · Theory of change

Introduction – Operationalizing Transformative Learning Transformative learning (TL) has formal roots going back to the mid-1970s with the early work of Jack Mezirow, widely considered the foundational theorist of transformative learning (e.g., ▶ Chap. 49, “Are Students and Faculty Ready for Transformative Learning?”; Hoggan, Mälkki, & Finnegan 2017; Kitchenham 2008, 2015; Swartz & Sprow 2010). The premises of TL, however, trace back through Paulo Freire (1970) and Jürgen Habermas (whose contributions informing TL theory have been acknowledged by Mezirow 2000), as well as John Dewey. Fisher-Yashida et al. (2009) even suggest that TL and transformative education “have been present in probably every culture since at least the beginning of recorded history” (p. 3). But where is higher (tertiary) education more than 40 years after Mezirow proposed a conceptual framework for TL? Has higher education acknowledged a need for students and faculty to have transformative experiences, and if so, how do we leverage transformative growth to greatest positive effect for students, faculty, and the institution? Is there proof that TL really impacts faculty’s conception and practice of teaching, and students’ experience of learning? This chapter examines one institution’s operationalization of TL via its Student Transformative Learning Record (STLR) process. UCO began designing STLR in February 2012, launched a small pilot in fall 2014, and proceeded to campus-wide rollout starting in fall 2015. STLR has been in place at UCO continuously since then, spreading to more classes and cocurricular activities every year. Now, more than 70% of full-time faculty are trained in the STLR process. Because STLR activity occurs in both the curriculum and cocurriculum, STLR is not an opt-in/opt-out process – STLR-associated assignments in a class mean that

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1283

students engage with STLR when they take “STLR-tagged” classes. Those classes are common, including core courses as well as disciplinary major courses and departmental capstone courses. With more than 80% of UCO’s students commuting to campus, in-class engagement is critically important because commuters often do not have time to participate in student activities compared to residential students. However, extra- and co-curricular activities are also associated with STLR, as over 150 such activities now take place each semester. UCO’s operationalization of STLR is institution-wide. In addition, STLR has spread to other institutions and launched ancillary operations such as the transformative learning international collaborative (see below). This chapter also examines whether UCO’s STLR approach to TL as an institution-wide initiative is changing teaching and learning at the university. If these changes are positive – and evidence indicates they are – then what UCO is doing with STLR across its entire institution prompts the question of whether the same can be done across Higher Education writ large. The literature on TL is replete with investigations of discrete approaches to inculcate TL, but there is a dearth of research about TL as practiced institutionwide at large universities. What one finds in searching for how TL is actually implemented in higher education is research conducted to determine if certain limited projects (compared to teaching and learning across the entire university) or focused areas of engagement are efficacious when initiatives involve curricular work. Cocurricular engagement to prompt TL may be more broadly based (see item “g” below), but cocurricular activities often do not include significant faculty involvement compared to the total number of faculty at the institution. A few examples of research about TL in higher education include: (a) TL for sustainability (Bell 2016; Palma & Pedrozo 2019; Taylor 2015; Taylor & Luitel 2019) (b) TL via study abroad (Seifen, Rodriguez, & Johnson 2019; Walters, Charles, & Bingham 2017) – this approach to prompting transformative experiences does not widely apply to US undergraduate students, fewer than 10% of whom have the chance to experience study abroad (Witherell & Clayton 2014) (c) Service Learning to prompt individual TL (Greene 2013; Hullender, Hinck, Wood-Nartker, Burton, & Bowlby 2015; Kiely 2005) (d) Civic engagement to prompt individual TL (McCunney 2015) (e) TL within a graduate education program (Lebsack 2016) (f) TL student experiences around conceptual understandings of natural selection (Petersen 2016) (g) TL to shift students’ attitudes toward physics (Mistades, delos Reyes, & Scheiter 2011) (h) TL within Student Affairs implementation (Fried & Associates 2012) One reason for the paucity of research about TL as practiced institutionally may be that a few such institutions exist. Given the existence of a robust, theoretical, and research base about the benefits of transformative learning for adults (e.g., Hoggan

1284

J. King and B. Wimmer

2016; Mezirow 1990; Taylor & Snyder 2012), should not there be more universities at which TL is widely practiced? UCO’s own experience in moving toward broad TL implementation included a focus on helping faculty understand and value the benefits of instructional practice that prompts student critical reflection related to their own development as humans, workers, creators, and community members. Another reason institution-wide TL may not exist more broadly in higher education is faculty resistance and/or faculty fatigue from what they perceive to be “flavor of the month” initiatives that seem to fade quickly. Much literature addresses faculty resistance to change in general; therefore, it is logical to expect faculty would resist learning how to scaffold student work and reflection toward transformative selfunderstandings. Tagg (2012) summarizes well the challenges in moving faculty to new pedagogies and andragogies. Among others, he quotes Derek Bok (2006), Harvard’s president emeritus: No faculty ever forced its leaders out for failing to act vigorously enough to improve the prevailing methods of education. On the contrary, faculties are more likely to resist any determined effort to examine their work and question familiar ways of teaching and learning. (p. 334)

UCO’s research of the impact its implementation of TL has had on faculty and students has itself been transformative as well as challenging. The research has called for examination of such processes as narrative writing and metaphorical logic in addition to other forms of quantitative and qualitative analysis when investigating TL (Barabasch 2018; Brooks & Clark 2001; Hoggan 2014; Howie & Bagnall 2013). Because such research approaches can often better accommodate the interpretive demands created by critical reflection, for instance, they can yield findings that translate to theoretical and practitioner perspectives. Taylor et al. (2012) indicate that transformative research’s multiparadigmatic approach can be well suited to help transform educational policy and practice. Making the case that STLR’s processes, tools, infrastructure, training, and messaging have largely succeeded as an institutional approach to prompt something as personal as transformation is not without difficulties, even within multiparadigmatic mixed-method research designs. UCO’s quantitative analyses have admittedly employed proxy measurements. They include measures such as retention and academic achievement (represented by the Cumulative Grade Point average). While recognizing the need for good quantitative research, the institution’s examination of STLR’s effectiveness was from the outset predicated on the need for qualitative analysis when assessing TL. For instance, Fullerton (2010) contends that TL’s constructivist nature requires qualitative methodological research, and Patterson et al. (2015) indicate that TL’s “open-inquiry, multi-modal nature” (p. 303) requires qualitative analysis (in their case, grounded theory). UCO’s findings have also been informed by interpretivism, criticalism, and postmodernism (Taylor 2015) as lenses through which researchers have made sense of faculty’s and students’ transformative experiences.

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1285

Transformative Learning in Higher Education While Jack Mezirow (e.g., 2006) and other TL theorists (e.g., Cranton 2002; Dirkx 1997; Taylor 1998) have their own definitions for TL, Illeris is concise: “learning which implies change in the identity of the learner” (2014 p. 40). However, when speaking about TL as a theoretical construct to inform and support TL practitioners, it is difficult to prescribe instructional strategies exclusively defined as promoting TL. Cranton states, “There are no particular teaching methods that guarantee transformative learning” (2002 p. 66), and Cranton and Taylor (2012) explore the challenge of defining the actions and activities that produce transformative outcomes. Their and others’ observations wrestle conceptually with the idea of being able to see TL in action, much less defining what “good” or “bad” TL teaching practice is. An added difficulty arises when TL is used inaccurately to represent teaching and learning activities that have tenuous or no connections to TL’s conceptual underpinnings. Hoggan (2016) raises the issue: “. . . [TL] is increasingly being used to refer to almost any instance of learning” (p. 57). He then offers this suggestion concerning scholarly work around TL: It would be simplistic to conceive of learning in a binary fashion as either transformative or not transformative. Nevertheless, scholars should take care to justify why and the extent to which learning is transformative. (p. 72)

Finnegan (2014) also remarks on this phenomenon, saying TL has perhaps become a “capacious metaphor.” He speaks not only of the wide range of activities sometimes placed under the TL banner, but also the educational and political agendas that “jostle up alongside each other under the illusion that, at least in some tangential way, we all mean the same thing when we speak of transformation” (p. 1). Against such a background, TL can be a slippery construct to identify in vivo. That slipperiness creates design and implementation challenges. It also means TL may look quite different at different times in widely divergent settings, even though such differences need not mitigate the efficacy of TL. Accurately identifying how widely TL has spread within higher education is, therefore, problematical. Kasworm and Bowles (2012) attempted to codify at least some characteristics of exemplary TL-focused teaching and learning in higher education by assaying various institutions that identify as TL-focused based on goals and mission statements. UCO was among the institutions Kasworm and Bowles considered, though at the time of their investigation UCO had not yet developed and implemented its STLR process. Kasworm and Bowles’s list illustrates that successful TL can be seen in different instances as being discretely represented by “critical thinking and reflection” or “experiential and active learning” (2012 p. 400). Both instructional approaches can be exemplary in prompting TL, but that does not exclude the possibility that a teacher without a background in TL could be using either strategy in isolation and achieving good outcomes.

1286

J. King and B. Wimmer

Duerr et al. (2003) surveyed 111 institutions (mostly in North America) to address the “Where?” and “How much?” regarding TL. Though their research was exploratory, of a pilot nature, and not a study quantifying TL across higher education, their examination of narrative responses illustrated the difficulty of defining TL as a precursor to counting the places it happens: “It became apparent that there are as many ways of describing transformative learning as there are people involved in it” (p. 185). Further, their research indicated TL was not widespread: “Questionnaire responses and interviews indicated that although there is great interest in bringing transformative/spiritual elements into higher education, this movement still exists primarily among individual faculty within classrooms rather than as a departmental or institutional strategy” (p. 177). Similarly, Cheney (2010) remarks on the difficulties encountered when trying to identify strategies that prompt TL. Researchers quoted above, and others (e.g., Fried 2007; Keeling 2004 & 2006), nonetheless are clear that TL benefits higher education instructional practice and student success. For instance, Taylor and Marienau (2016) say that TL “emphasizes enabling adults to more fully experience the benefits of autonomy, rationality, and voice” (p. 273). Tagg (2019) considers what should happen in higher education to correct course toward a saner and more successful means of helping students succeed and prepare for life after college, including educative processes that enable transformative personal development. Within this context, UCO’s STLR process to operationalize TL across the entire institution may seem an audacious undertaking, even given its potential benefits. Cebrián et al. (2015) remark on the scope of such an undertaking: “Organisational support and leadership, quality assurance processes, professional development and creating reward structures are necessary strategies towards academic staff engagement in this agenda” (p. 79). Paphitis and Kelland (2016) say success in this type of venture requires an epistemological shift in institutional culture to emplace and maintain what Maloney and Kim (2019) characterize as sustainable and resilient institutional change. For STLR, faculty adoption had to be nurtured. Training and infrastructure supporting TL had to be designed and implemented. Rubrics and tracking to assess students’ progression toward transformative realizations had to be devised, and tools and processes to track and document students’ progress had to be developed. Is it working? Is STLR as a means of operationalizing TL actually prompting student shifts in perspective? Does STLR associate with other aspects of student success? See Fig. 1 for UCO’s Theory of Change model for STLR:

UCO’s Student Transformative Learning Record STLR is the way UCO operationalizes transformative learning. STLR’s processes, tools and technology, infrastructure, messaging and branding, training, tracking, and research and assessment enable the institution to intentionally design into the curriculum and the cocurriculum assignments, activities, and environments likely to prompt transformative realizations. STLR also generates an evidence-based

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

Fig. 1 STLR theory of change model

1287

1288

J. King and B. Wimmer

Fig. 2 STLR tenets metro map

record via authentic assessment using vetted, high-quality rubrics as applied by trained faculty and staff. What UCO means when it says it is “doing TL” as operationalized via STLR is that TL: (1) develops students’ beyond-disciplinary skills (see Fig. 2), and (2) expands students’ perspectives of their relationships to self, others, community, and environment. Several years before the launch of STLR, UCO devised a structure to encompass the broad areas of student development in which TL would occur. In addition to TL happening in the curriculum (discipline knowledge), five additional areas were codified within which students develop transformatively: global and cultural competencies; health and wellness; leadership; research, creative, and scholarly activities; and service learning and civic engagement. These six areas comprised what UCO labeled its central six tenets of transformative learning. Numerous national and international employer surveys reveal that many employers do not think recent college graduates possess needed workplace skills upon graduation (e.g., CBI/Pearson 2017; Hart Research Associates 2013, 2015; Murphey 2006). Workforce-related beyond-disciplinary skills are among personal abilities and characteristics developed in UCO’s approach to transformative education. For example, UCO’s TL-focused process includes helping students develop global and cultural competencies, and employers indicate in surveys such as those

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1289

mentioned above the need for new hires to be able to work with people different from themselves. Employers desire new employees who can lead when the situation demands, something targeted for development in assignments and activities associated with STLR’s leadership tenet. In these examples and others, TL helps students develop the skills and mindsets desired by employers. The five tenets related to beyond-disciplinary learning, and the skills employers expect of graduates in each of those areas, are illustrated in Fig. 2. This “STLR Tenets Metro Map” was built from input provided by UCO’s STLR Employer Advisory Board (SEAB), with which it has been working since 2015. SEAB comprises hiring managers and human resources professionals from companies representing most of the major workforce sectors in the Oklahoma City metropolitan area. The second part of UCO’s operational definition of TL owes directly to Mezirow’s theory of perspective transformation as an underpinning for TL (Mezirow 1978). In this regard, UCO envisions the outcome of a transformative college education to be one in which graduates have developed the capacity to be productive, creative, ethical, and engaged citizens, as explicated in the UCO mission statement. In conceiving STLR, UCO hoped it would specifically benefit three categories of students: first-generation students, low-income students, and underrepresented (minority) students. These student subpopulations (“priority populations” or “atpromise students”) are relatively large at UCO, with the percentage of students in each subpopulation comprising between one-third and one-half of the entire student body. (Students can fall into more than one subpopulation.) While STLR was designed to help all students succeed, the size of these subpopulations meant STLR had to be as effective, if not more so, with such students. STLR at UCO needed to meet the added challenges these students often face in obtaining college degrees (Carter 2006; JED Foundation 2016; Verschelden 2017).

History of TL at UCO The groundwork that ultimately birthed STLR at UCO began in the late 1990s when multiple, discrete student success initiatives began emerging around campus, seemingly of their own accord. By 2005, many such activities were in place with more on the horizon. UCO leadership asked itself: “How do we organize and manage this?” University leadership sought some kind of conceptual organizer that might help drive future strategic planning and operational decisions. Around this time, the influential publications Leadership Reconsidered and Leadership Reconsidered 2 (Keeling 2004 & 2006) appeared, calling for higher education to adopt a holistic, transformative approach to student engagement that would unite academic learning and student development. UCO leadership recognized the excellent fit transformative learning provided as a concept for how the institution wanted to help students succeed. TL also provided the conceptual umbrella for the student success initiatives that had arisen organically. In late 2008, UCO formally centered itself around TL

1290

J. King and B. Wimmer

with the phrase “providing transformative education experiences to students” appearing in a revamped mission statement. At the same time, UCO adopted the aforementioned central six tenets. (Note that STLR pertains to the five as shown on the STLR Tenets Metro Map – Fig. 2 – with discipline knowledge, which is tracked via the academic transcript and student grades, being the sixth.) Formalizing the lexicon and the mission statement did not mean TL automatically became the guiding principle UCO faculty and staff used to engage with students. Indeed, the difficulty in clearly communicating what is meant by TL, as discussed above, created understandable challenges for operationalizing TL at the institution. UCO launched various initiatives, committees, and convenings across several years as it attempted to operationalize TL. In spite of all this intentionality and smallstep progress by the 2011–2012 academic year, UCO knew it still needed to be more successful implementing TL. Reconceiving the faculty development unit and function was recognized as a necessary step given TL’s inherent dependence on the kind of instructional strategies that may not necessarily be unique to TL, but which nonetheless are common in TL-focused classrooms (e.g., active learning, group work, constructivist-based approaches). Helping faculty develop these skills naturally involves the faculty development unit (i.e., the “teaching center”), so UCO administration realized it should widen and enhance its faculty professional development function. Institutional leadership also hoped a “re-booted” faculty development unit could enhance faculty motivation to more widely adopt TL strategies. Therefore, UCO reconstituted the existing Faculty Enhancement Center, rebranding it as the Center for Excellence in Transformative Teaching and Learning (CETTL). The new CETTL Executive Director began work at UCO in January 2012 to provide increased training and support for faculty around “doing TL,” and also began helping design a process to operationalize TL at UCO.

Building and Implementing STLR At the start of 2012, UCO needed a means to inculcate TL more widely throughout teaching and learning across the institution. This would ultimately require designing and implementing the processes, tools, infrastructure, training, technology, and messaging to support the institution’s attempt to more fully align the way it engaged students with its mission statement. In February 2012, UCO began designing STLR and planning for its implementation by developing a detailed guide that included benchmarks, assessments, and desired outcomes. Creating the document, which became the narrative for a grant application, required answering many questions. That meant people from units across campus had to be involved. UCO convened the STLR Project Team for this purpose in fall 2013. Some of the questions were: (a) What is the best way to put into place the vision for having students engage in the key transformative process of critical reflection (Mezirow 1990)?

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1291

(b) What is the best project management approach to build a system with standards to assess that reflection? (c) How should UCO identify the details and implement solutions for capturing this information? (d) What should the standards be for measuring student progress toward transformative understandings? (e) What is the best approach for accomplishing this with existing personnel and resources? (f) Will faculty and staff ultimately embrace the tools and processes implemented? Always in the background was the question, “Will this work?” It was a logical question: UCO’s environmental scan seeking other institutions whose TL operationalizations were evidence- and rubrics-based as well as institution-wide across both the curriculum and the cocurriculum did not reveal any US institutions with such a process. Some of the earliest tasks for the STLR Project Team included conceptualizing the broad assessment scheme for STLR and defining assessment standards. With existing UCO Central Tenets as the structure around which the university had built its TL approach, it became clear some means for assessing student developmental progress within the tenets would be necessary. The Association of American Colleges and Universities (AAC&U) released in fall of 2009 its VALUE rubrics (Valid Assessment of Learning in Undergraduate Education). The project developed 16 rubrics in areas such as creative thinking, teamwork, problem-solving, and critical thinking. Spread across the range of those rubrics there was much crossover with outcomes reasonably to be expected as part of student development within UCO’s Central Tenets. The VALUE rubrics (Rhodes 2009) provided key source material utilized by the STLR project team when it began developing rubrics for the five beyond-disciplinary tenets. The project team engaged additional faculty and staff for this important work in an iterative process to develop the first set of STLR rubrics. These rubrics became the yardstick for evidence-based assessment of students’ reflective artifacts produced in work related to one or more tenets in STLR-tagged assignments and activities. The rubrics enabled faculty and staff to assign STLR developmental achievement at one of three levels: exposure (lowest), integration (middle), and transformation (high). Meeting these challenges required input and expertise from constituencies across the entire campus. The information technology (IT) unit played a key and ongoing role, and it was IT’s assistant vice president who served as project manager for the team. Student Affairs appointed multiple representatives, as did academic affairs. Faculty were represented as well as UCO’s Office of Institutional Assessment. In addition, an ex-officio member of the Project Team was the Associate Vice President in Academic Affairs. The CETTL executive director led the project team. A key aspect of the project team’s structuring was sponsorship by three UCO vice presidents: VP of academic affairs and provost, VP of student affairs, and VP of information technology. This ensured a direct line of communication from the

1292

J. King and B. Wimmer

project team to the President’s Cabinet. Project team members, in keeping their supervisors apprised of the work of the STLR project team, enabled cabinet-level awareness of STLR’s progress. The STLR project team was the nexus for UCO’s in-house development of STLR. Some outside contracting, however, was necessary. The learning management system (LMS) vendor created a competency structure for gathering STLR data that cut laterally across the student experience (instead of collecting data vertically, as is typical in LMS hierarchical structures: class-to-program-to-college-to-institution). In a vertical structure, it can be difficult to extract data on a per-student basis that aggregates work both from classes within the hierarchy and from classes outside it, such as elective and general education classes, which are important opportunities for TL engagement. UCO also had to engage outside services to adapt its existing system for student ID card swipe-in as a means of tracking student attendance. Such swipers were already in place, for instance, in the Health and Wellness Center, where students had to swipe in with their UCO ID cards to use the facilities. However, STLR required that in addition to attendance, a swipe-in system also had to assign to the student the lowest level of achievement (“Exposure”) in the tenet(s) to which the event was tagged: mere attendance at an activity ensures nothing more than being exposed to concepts. STLR’s implementation moved into pilot faculty training in March 2014 followed by a pilot rollout to incoming students in fall 2014. Broad implementation for incoming classes began in fall 2015. Messaging to faculty was ongoing. The vice president for academic affairs/ provost continued his relentless advocacy about TL at UCO until he retired in June 2013, and the incoming VPAA/provost immediately took up that banner. The president, too, immediately became a regular advocate for STLR, whether to internal or external audiences. While messaging to faculty from upper administration was strong, consistent, and supportive, much more was necessary to convince faculty to accept and adopt STLR in their teaching practice. An existing resource was the promotion and tenure (P&T) policy. As a teaching-focused university, UCO’s P&T policy was weighted toward quality of teaching as evidence of success supporting promotion to tenure and for posttenure review. This provided one lever to message STLR to faculty because STLR would provide training and support in authentic assessment, prompting student critical reflection, use of rubrics, and other teaching strategies. Participating in STLR training and practice, then, could be evidence of professional development in teaching, a positive factor in one’s P&T dossier. It took time for department chairs, Deans, and P&T committees to become familiar with the idea of STLR training as a value-add to faculty teaching performance, and UCO struggled to communicate this effectively. It was, however, and still is, emphasized to newly hired faculty as part of their introduction to institutional culture upon hire, with STLR adoption held out as an expectation at UCO.

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1293

Soon it became apparent that the most important messaging to faculty about the STLR initiative came from their own peers. In a case study examining faculty and administration roles in the decision to add new majors at Ohio Wesleyan University, Gardner (2017) points out: “[T]here is no better faculty evangelist for a new idea than a fellow faculty member” (para. 12). UCO always worked from this assumption in bringing STLR to the classroom. It was necessary to identify faculty willing to advocate among peers for adopting STLR. Rogers’ theory of diffusion of innovations (2003) identifies innovators and early adopters as likely comprising 16% of a population who would be first to adopt innovation; this seemed to hold true among faculty at UCO. It was these early STLR proponents who became faculty evangelists and change agents (Clavert 2017) as well as providing a key insight UCO leveraged in spurring faculty adoption: Many faculty were already “doing STLR” in that they had always focused on holistic student development and were already doing things such as connecting UCO’s tenets to the student developmental process. In such cases, STLR finally provided those faculty a formal mechanism to track and recognize their students’ accomplishments in this area, and for UCO to recognize them for doing this work. These processes did not exist before STLR. UCO was slow to realize the power of faculty “already doing TL.” However, as faculty conversations about STLR grew in frequency, it was the innovators and early adopters who were the most successful advocates to their peers because they employed this line of reasoning (and others). In its role as an innovator with STLR, UCO did not have proof early on that the innovation helps students learn. Such proof is a strong faculty convincer strategy. FTI Consulting’s 2015 report describes it as the most powerful means of influencing faculty to adopt a new technique: “Faculty were most influenced and motivated to adopt innovative techniques if the techniques ensured that students learn” (p. 5). In the beginning, STLR was untested in this regard, and some faculty were skeptical it would work. However, UCO’s longtime success with undergraduate research as a way to help students learn supported the idea STLR would succeed because one of the STLR tenets was research, creative, and scholarly activities. In addition, students’ strongly positive experiences with STLR, and the promotion of student stories about those experiences, quickly provided in vivo corroboration important for many faculty. Retention data for the first entering class to experience STLR implemented broadly across campus (fall 2015) showed retention into the second year strongly correlated with STLR engagement. Analyses also revealed improved GPA associated with STLR. These results provided hard evidence to faculty on the issue of STLR efficacy. Even better results were revealed in the analysis of the second incoming class, and the strong association to improved retention held for the first incoming class into their junior year. These large-N, p < 0.005 analyses provided compelling talking points to faculty about STLR based on quantitative research.

1294

J. King and B. Wimmer

Messaging to students about STLR begins during outreach and recruitment. New student orientations cover STLR via STLR staff presentations that include student ambassadors interacting with new students at a “STLR Booth” where STLR promotional items are distributed. UCO quickly learned the value of promoting STLR to parents of students who enroll immediately after high school. While such students themselves were receptive during new student orientations, parents were very positively inclined toward STLR, sharing that they understood STLR’s role in helping raise the chances their children would be better prepared for good jobs upon graduation. Making STLR visible to students was also important, and the STLR operation invested in outdoor and indoor banners. When students recognize STLR’s colors, spotting a STLR-associated event across campus by seeing that color on an outdoor stand-up banner, they can communicate effectively that there is a STLR-tagged event happening or about to happen. Indoor stand-up banners serve the same purpose. Student awareness of outside-of-class STLR-associated events was very important. As STLR became inculcated at UCO, students embraced it to the degree they actively began seeking out STLR opportunities. Branding STLR events with tenet colors and icons helped students quickly recognize STLR-tagged activities. The same held for website and other electronic communications about STLR, which were also associated with tenet icons and colors. STLR’s tenet icons were developed by students in a design competition. Messaging to employers began with the August 2015 creation of the STLR Employer Advisory Board (SEAB; mentioned above). The solicitation to Oklahoma City metropolitan area employers identified three ways UCO was seeking help: (a) Provide valuable industry input into what employability skills industry needs in graduating students (b) Brainstorm ideas in the development of the STLR tool that documents these nondisciplinary skills (c) Participate in workplace assessment for measuring nondisciplinary/employability skills development and STLR transformation SEAB has been extremely helpful in guiding outreach to employers in general, and in providing advice about the STLR Snapshot (a “transcript” of student development within the five nondisciplinary tenets) and the STLR eportfolio. Board members have also conducted mock interviews of students who have produced STLR Snapshots and eportfolios. As mentioned earlier, SEAB provided the information needed to create the STLR Tenets Metro Map (Fig. 1). The board also provided feedback on the design, layout, and content of the STLR Snapshot and the templates for STLR eportfolios. One suggestion was to include a “rubrics abstract” on the back pages of the snapshot so that hiring managers seeing a snapshot for the first time could familiarize themselves quickly with STLR. Consequently, the STLR project team launched a work group to develop very brief characterizations of each badge level in each of the five beyond-disciplinary tenets using employer-friendly language.

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1295

Training, Technology, and How STLR Works Faculty associate one or more STLR tenets to an existing assignment by adding a reflective prompt to the assignment that elicits students’ consideration of their learning connected to the tenet and the assignment. Faculty grade the assignment as usual except for the student reflection, then use the STLR rubric(s) for the tenet (s) associated with the assignment to assess students’ growth toward transformative understandings. An example would be a statistics professor who had previously used a dataset that came with the textbook for a homework assignment on central limit theorem. The professor realizes he could associate global and cultural competencies to the assignment by having students use an OECD (Organization for Economic Cooperation and Development) database showing kilometers walked to the nearest drinkable water among subpopulations in certain regions of the world. Students do the assignment as usual but respond to a reflective prompt about conditions preventing accessible drinking water. The professor grades the assignment as he always has but then uses STLR’s global and cultural competencies (GCC) rubrics to assess student reflections. The grade the student earns for the assignment and the assessment for the GCC badge level attained – exposure, integration, or transformation – are independent, and STLR activity has no bearing on the assignment grade or vice versa. The above example illustrates that faculty must be supported in: (1) how to write good reflective prompts, (2) how to use STLR rubrics to assess students’ engagement with GCC concepts, and (3) how to build the STLR part of the assignment into the structure of the learning management system (LMS) course shell. Students then push the reflective artifact itself, the rubrics used by the professor to assess the artifact, and the badge-level rating assigned for the reflection into the STLR eportfolio. Notice that writing effective reflective prompts and using rubrics may be things some faculty have never done. In such cases, STLR training expands faculty’s overall instructional strategies toolkit. Staff attend training alongside faculty, working together in teams on case scenario activities. This helps break down silos existing within the institution. UCO found this to help: (1) energize across-discipline discussions about teaching, and (2) ameliorate some faculty’s preconceptions about cocurricular operations existing primarily as noneducational social activities. STLR training has helped some faculty previously holding pejorative opinions of student affairs work understand its vital role in the college experience. The training enables a student affairs staff member, for instance, to use STLR’s leadership rubrics to assess reflective artifacts required of student government officers upon conclusion of their term of service. In that case, staff would use the online STLR tracking system to request an LMS “course pseudo-shell.” They would then assess the students’ artifacts just as faculty do, and students would be enrolled into that pseudoshell as part of the process. This means that part of STLR training for both faculty and staff concerns how to use the online mechanism to request STLR-tagged activities that happen outside of

1296

J. King and B. Wimmer

class. Such activities could be student affairs events, student projects outside of class led by faculty mentors (similar to the undergraduate research model), peer health mentors’ work across a term under the direction of Health & Wellness Center staff, and so on. Training is also the place where the “why” for STLR helps convince faculty of STLR’s worthiness. Faculty realization about STLR as a way to more frequently spur transformative moments for students, as well as faculty realization that STLR can capture and record in an evidence-based fashion the good work both faculty and students are doing, is a motivator for adopting STLR. Faculty and staff receive stipends to attend STLR training, which is comprised of two 3-hour modules. Module I includes basics about Transformative Learning, STLR’s structure, and a look at STLR data. Module II addresses STLR assessment and individualized planning for STLR-tagged assignments. Module II also includes information about using the tools in the course shell to accomplish STLR work. The break between the first and second segments allows for hands-on “homework” that prepares trainees for the second session, in which they set up their own STLR assignments in a course shell (or pseudoshell) for an upcoming term. As of September 30, 2018, UCO had trained 321 full-time faculty, 158 adjunct faculty, and 188 staff. UCO’s Technology Resource Center, a division within IT, created brief online videos as just-in-time help for faculty and staff regarding STLR operation in the LMS course shell. As with any introduction of broad-based initiatives involving new pedagogy/andragogy and new technology, STLR staff and Technology Resource Center personnel address one-off requests for assistance. Based on feedback from surveys administered after all faculty/staff training sessions, STLR staff improved the training by off-loading some of the technical training to the online video format to free more time to spend on training to create good reflective prompts. The training manual is updated regularly based on user feedback and whenever new features of STLR come online (such as the introduction of the STLR Snapshot in fall 2017). STLR-tagged activities that do not require production of artifacts for assessment are auto-assigned at the lowest badge level (see explanation above). Setting up the activity in the online system enables students to swipe into the activity using their student ID cards, and the system automatically assigns them exposure-level badge assessment in the associated tenet in a course named, for example, “Asian Moon Festival Fall 2018” (there is an actual naming protocol; what is described is for illustration purposes). The system links the student’s data from the card swipe into the “course” pseudoshell. The card-swipe technology already existed on UCO’s campus, as it does on many campuses (Campus Safety 2012), so the institution worked with the vendor to adapt the system to STLR, including use of portable “swipe sleds” on iPads or iPods. The system therefore allows the mobility necessary for activities that could happen anywhere on campus, and sometimes off campus. Such functionality includes the ability to download and store swipe-in data on the devices when immediate upload to

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1297

UCO’s system is unavailable (e.g., a remote location with poor wi-fi coverage) – when back on campus, the data are easily uploaded. It was important to minimize technology hurdles for faculty and staff to “do STLR.” IT personnel who were part of the STLR project team worked with the LMS vendor to streamline the number of steps in the course shell necessary for STLR assessments. Also, the user interface for faculty badge-level assessment of STLR artifacts was designed for fast input. UCO already had in place a third-party solution for delivering electronic versions of academic transcripts. Working with that vendor enabled the addition of the STLR Snapshot for electronic delivery. Student training occurs as part of the introduction of STLR in all sections of UCO’s freshman seminar class, “Success Central,” and in all sections of the “Healthy Life Skills” class, a requirement in the university core. The STLR-tagged class assignments in these classes ensure entering students are exposed to STLR’s purpose and benefits, and to the STLR Snapshot, eportfolio, and mobile student app. Transfer students are introduced to STLR via the online video for students explaining STLR. Near the end of students’ UCO careers, capstones are logical spots for the production of eportfolios as an artifact prompting reflective evaluation of students’ college educations and their preparation for the workforce, community, and relationships. Having their STLR Snapshot available in capstone classes helps students see the arc of their own holistic development. This can help them express more expansive understandings in reflections done later in college compared to their first year on campus. One component of students’ introduction to STLR is tangentially connected to being “trained in STLR” but is better described as information-sharing about STLR’s benefits for students and graduates. Included among such information is the fact that achievement of the transformation-level badge carries with it the award of a STLR honor cord in the color associated with the tenet in which the transformation-level badge has been earned. At a special cording ceremony held each semester the morning before UCO’s regular graduation ceremonies begin, the faculty or staff member who assessed the student at the transformation level presents the cord. (If the faculty or staff member cannot attend, the president or provost, who attends and speaks at the ceremony, stands in for that purpose.) Students wear their STLR honor cords at UCO’s regular graduation ceremonies.

Tracking and Assessing STLR With STLR a new, large-scale, and cross-campus initiative requiring resources at appropriate levels to support the institutional mission of providing transformative educational experiences for all students, it was important from the beginning to assess STLR efficacy in all possible ways to examine all possible impacts. This meant designing both quantitative and qualitative assessments. It meant assessing

1298

J. King and B. Wimmer

not only impact on students but also impact on faculty and on the institution as well as other communities the institution serves. For that reason, a number of metrics were identified along with the resources to conduct the assessments. The nexus of STLR assessment is the STLR assistant director for assessment. He works closely with an institutional research analyst in the institutional research (IR) unit. This duo has been effective in deriving persistence and academic improvement and impacts each term associated with STLR. There are STLR-related questions on the graduating student survey (GSS) that factor into the assessment mix, as do STLR-focused questions on UCO’s end-ofterm student surveys of instructors and classes when STLR assignments are part of the class. Faculty and staff in all sections of STLR training are surveyed about the training experience, and STLR assessment rollups become part of larger institutional reports (e.g., institutional and programmatic accreditation reports and reports to Boards of Regents). In addition, fidelity to STLR rubrics is tracked when the STLR assistant director for assessment spot-checks during each semester the ratings faculty have given when they assess STLR student artifacts. There is another inter-rater reliability assessment performed periodically when the STLR-trained representatives on the TL Steering Committee participate in “ratings work sessions” during certain meetings. Every term, the data for the number of badge-level achievements in each tenet are also collected by colleges at the institution. Other kinds of quantitative data help inform STLR’s operation. Predictive analytics data are processed on an ongoing basis, with the STLR assistant director for assessment and UCO IR interpreting those results. Qualitative data collection occurs via one-on-one interviews with faculty, staff, and students, as well as via focus group interviews. To date, STLR student workers help transcribe those interviews for analysis by the STLR assistant director for assessment. (A more detailed discussion of qualitative analyses follows in the section immediately below.) The CETTL executive director presents a quarterly STLR summary report to the president’s cabinet. For the duration of the Title III grant helping support STLR, annual performance reports are not only for internal consumption but also for the US Department of Education. In addition, the grant’s external evaluator conducts annual site visits with following reports.

Researching STLR’s Impact as Institution-Wide TL The narrative above describes one institution’s approach to implementing TL across the entire campus. As such, UCO’s STLR provides a unique opportunity for TL research in higher education because STLR: (a) Is an institution-wide approach to implementing TL, a rarity in higher education (b) Provides a longitudinal view, relatively speaking, because it has been in operation for 5 years (Snyder 2008 calls for more longitudinal research in TL)

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1299

(c) Employs a process for collecting students’ reflective artifacts that avoids the problem of asking research participants to recall moments of transformation, a weakness in much research about TL – see Taylor’s 2017 comments about an “over-reliance on retrospective interviews” (p. 79) as a research technique concerning subjects’ critical reflections; in contrast, STLR students provide their reflective statements in connection with, or shortly after, the assignment or activity prompting the reflection (d) Operates within an existing framework of five areas in which faculty and staff work to scaffold and facilitate student transformative realizations (UCO’s STLR tenets; see Fig. 2); several tenets represent areas of student experience that have been selectively examined regarding transformative effect (see the Introduction) (e) Involves hundreds of faculty and staff, and thousands of students, thereby enabling various large-N quantitative analyses (f) Provides an opportunity to interpret students’ responses to reflective prompts, and faculty’s and students’ responses in one-on-one and focus group interviews, through the lenses of interpretivism, criticalism, and postmodernism, thereby further testing a multiparadigmatic approach for transformative research, as has Alsulami (2019) (g) Involves a student population comprised of large percentages of underrepresented, low-income, and first-generation subpopulations, offering opportunity to check for critical reflectivity of researchers engaging in work meant to support mutually empowering relationships (Taylor 2014) in a problem-posing approach during research interviews (Freire 1970; Mertens 2007, as “raising questions about the assumptions that underlie research,” p. 224) (h) Provides an opportunity to contribute to such research and attendant literature in a way that starts to realize the transformative potential of qualitative research (Taylor 2014) The general approach regarding UCO’s STLR research fits within Creswell and Plano Clark’s transformative framework within mixed-methods design (2011 p. 70) (Fig. 3): The research being carried out at UCO to determine whether STLR is succeeding as an institution-wide implementation to prompt TL experiences among students and faculty benefits from existing research about TL done at smaller scale in higher education (examples given in Introduction). However, it extends beyond that in important ways quantitatively due to the large-N analyses possible with STLR. Similarly, the hundreds of STLR-trained faculty comprise a cohort much larger

Fig. 3 STLR research as occurring within the transformative framework (Creswell & Plano Clark 2011 p. 70)

1300

J. King and B. Wimmer

than the single faculty member who leads a transformative class assignment or service learning project or study abroad experience – all examples of much more common TL projects reported in the research (Duerr, Zajonc, & Dana 2003).

Impact on Students UCO has gathered data about STLR’s correlations with retention and GPA since its inception. Other kinds of quantitative data analyses and tracking include the number and percentage of assignments/activities faculty and staff associate with each tenet and the number of badge-level assessments in each tenet. This information is gathered each year and reported for internal and external audiences. This process allows various assessments of STLR’s impact on students, on the implementation of STLR, on STLR’s spread across the institution, and so on. Two examples across various reporting periods for the aforementioned correlations to retention and GPA improvements are illustrated in Figs. 4 and 5. In Fig. 4, retention gains associating with STLR engagement are shown across four successive cohorts of incoming firsttime/full-time students, with the gray (left-most) bars indicating retention for students who did not engage with STLR. The silver (middle) bars indicate retention for students whose only STLR engagement was via ID card swipe-in at STLR-tagged events (and for which they were automatically assigned only the lowest level of achievement, “Exposure”). The blue (right-most) bars indicate retention for students whose STLR engagements had them produce reflective artifacts that were then assessed by faculty or staff.

Fig. 4 Example of STLR association with improved retention

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1301

Fig. 5 Example of STLR association with improved GPA

In Fig. 5, Grade Point Average (GPA) gains associated with STLR engagement are shown for fall and spring semesters of the Cohort 3 and Cohort 4 populations. Again, gray (left-most) bars represent average GPA among students not engaging with STLR that semester. The silver (middle) bars represent students engaged with STLR only via ID card swipe-in for attendance at activities for which they received automatic assessment at only the “Exposure” level. The blue (right-most) bars represent students whose engagement with STLR was via assignment or activity that required them to produce a reflective artifact that was then assessed using STLR rubrics. UCO has been enormously pleased with these positive results associated with STLR. Of particular note is that STLR’s associated lift in persistence for priority population students (low-income, first-generation, underrepresented) is nearly as strong as for nonpriority students. Tinto (2012), for instance, notes the retention/ completion gap between high- and low-income students, and the Education Trust reports that: Even with the improvements seen this last decade, underrepresented students still are not graduating at the same rate as white students were 10 years ago. A 14-percentage-point gap in completion remains between underrepresented and white students. (Education Trust, 2015)

STLR’s association to retention lifts among priority population STEM majors exceeds 20%. This is particularly important given both UCO’s and the National Science Foundation’s focus on graduating more STEM majors among priority populations.

1302

J. King and B. Wimmer

These very positive quantitative analysis results have ticked upward across the years, most likely a function of STLR’s broadening spread across campus as more faculty become trained and more STLR assignments and activities become available to students. The growth each semester in the number of graduates who have achieved the transformation-level assessment in one or more of the STLR tenets corroborates ever-widening STLR engagement at UCO. As heartening as results from quantitative analyses continue to be, the core of TL at university is the internal shift (Illeris 2014; Mezirow 1990) in students and faculty/ staff. Qualitative analysis is requisite to make that determination. UCO has used a multiparadigmatic qualitative analysis approach (Alsulami 2019; Taylor, Taylor, & Luitel 2012) in interpreting faculty and student narratives about their STLR engagement experiences. Qualitative research began shortly after initial quantitative analyses but has accelerated in recent years with more one-on-one and small-group interviews with students and faculty/staff, including the research done by Brunstein and King (2018). From the outset of qualitative research about STLR’s effects, UCO took the approach that TL’s very nature requires a postpositivist (Taylor 2015) research methodology that does not presuppose a thesis to be tested. Rather, UCO’s STLR researchers align strongly with Featherston and Kelly (2007), who said, “. . . we tried to allow our analysis to emerge from the data we collected, to reflect the picture of students’ experiences captured” (p. 266). STLR’s positive impact on students is seen not just in numbers; it comes through in students’ own words, as below concerning personal transformation resulting from participating in an outside-of-class STLR student project (the following quotes do not include student or faculty identifying information per the STLR project’s Institutional Review Board guidelines): . . . working with this idea so close to home of . . . violence used as a tool like social control, when I think about the art and music that I create [now], I’m less concerned about making something that’s a representation of myself, about my own personal emotion. I want to be the kind of artist and consume the kind of media and art that’s more important than the individual. I’ve been thinking more about what I can do as a student, an artist, and musician about these much more important topics than a self-indulgent kind of art. (UCO undergraduate student 2015)

A different undergraduate said of her STLR experience: These are the most valuable experiences that I have from my college career. This [STLR] project remained with me because it contributed to a bigger picture. There was a real audience and a real need. I was actually helping others, and I found that’s the best way for me to learn. (UCO graduating student, spring 2016)

Because higher education in general is interested in improving student retention and academic success (Biggs & Tang, 2011; Crosling, Heagney, & Thomas, 2009; DeAngelo, Franke, Hurtado, Pryor, & Tran 2011; Einolander & Vanharanta 2015; McGuire 2015; O’Keefe 2013), STLR’s efficacy is of keen interest in the academy.

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1303

UCO’s research about STLR’s connection to student success is ongoing, now informed by thousands of student reflections about their STLR engagements.

Impact on Faculty Over the past 3 years, UCO has been investigating STLR’s impact on faculty professional development and institutional adoption in addition to student success. Findings from preliminary data analyses have illustrated multiple factors at play, including: (a) Participation in STLR increases student and faculty engagement. (b) STLR employs multiple high-impact practices (Kuh 2008), including First Year Experience, Service Learning, Diversity/Global Learning, Undergraduate Research, Collaborative Projects, and ePortfolios. (c) STLR helps deepen student learning (see Marton & Säljö 1976) via critical reflection (CR, as posited by Brookfield’s distinction between CR and mere reflection, 2009). (d) STLR helps to foster student development. (e) STLR is woven into and reinforces the campus culture. UCO conducted a survey of full-time faculty/staff in Summer 2018 and found that over 58% of faculty reported modifying or planning an entirely new activity to STLR-tag. In addition, over 78% reported that being involved with STLR impacted their teaching or how they interacted with students. This is important because it suggests that STLR is changing faculty pedagogical practices. Interviews with faculty and students are key to the mixed-methods methodology that STLR is employing (Creswell & Plano Clark 2011). While this research is still ongoing, some themes have begun to emerge during preliminary analyses. One theme suggests that while many faculty might simply modify an existing assignment, they report that they are being more intentional with their teaching practice and how they approach classroom topics with their students. In fact, participating with STLR appears to be reinvigorating faculty’s passion for teaching – allowing them to see themselves as stewards of the future. By assessing learning impact, faculty are able to place value on the effect their teaching is having on their students. Another emerging theme relates to improvement of pedagogical practice, which helps corroborate the survey findings mentioned above. For some faculty, participating in STLR has allowed them the space to critically examine the learning outcomes in assignments and lessons. This process of examination appears to be resulting in more intentional and pedagogically creative teaching practices that attempt to increase student interest and engagement. Some of the faculty interviewed attribute these shifts in their pedagogy to not only engagement with the STLR program, but also to UCO’s robust faculty professional development programs that support and complement STLR activity. The Center for Excellence in Transformative Teaching and Learning and its twenty-first century pedagogy institute

1304

J. King and B. Wimmer

programming, for instance, reinforce STLR’s training in authentic assessment of learning outcomes and developing effective prompts for student critical reflection. With 70% of full-time faculty trained to date, and with new faculty training and orientation “STLRizing” incoming faculty, UCO has made remarkable progress in advancing a new pedagogical technique among its faculty. This in itself is a considerable impact on faculty. However, while the ability to refer to STLR training as evidence of professional development in P&T dossiers and the stipend received to do STLR training might be extrinsic motivators to adopt STLR, internal motivation must be developed and maintained (Ryan & Deci 2000) for faculty to continue “doing STLR.” As discussed above in the section about messaging STLR to faculty, a strong internal motivator discovered was some faculty’s realization that STLR allowed them to track and assess what they were already doing with students as these conscientious and dedicated faculty worked to make the learning in their classrooms be about more than just the content of the course. Some faculty in the College of Fine Arts and Design, for instance, indicated that the very nature of their one-on-one work with students to help them develop as artists is intrinsically transformative for the students. Before STLR, though, a mechanism did not exist to track that transformation other than the discipline’s metrics for artistic development of skill, vision, and creativity. As in the case of the student artist-musician quoted above, faculty responsible for prompting such a transformation in perspective deserve to know of this impact on their students. Before STLR, how would the faculty member know this transformation happened and when it occurred? With STLR, the student’s words are recorded in the STLR learning artifact, something the faculty member can see or hear as well as assess in the process of valuing such feedback that would otherwise not have been collected. But how widespread among faculty during STLR’s introduction was the perception that STLR finally provided a means for faculty to document such changes in students? To what degree is making explicit the change in students’ perceptions of themselves and their relationships a unique value-add of STLR in faculty’s perceptions? These are questions currently being investigated. Research suggests autonomous motivation is the greatest predictor of faculty use of effective teaching strategies focused on clarity, higher order learning, reflective and integrative learning, and collaborative learning (Stupnisky, BrckaLorenz, Yuhas, & Guay 2018). With one of those strategies focused on a key aspect of how TL works – reflective and integrative learning – then it may be the case that STLR, in turn, supports faculty autonomous motivation. The words of a faculty member serving as a department chair speak to the understanding of STLR as a means to assess beyond-disciplinary learning and its efficacy in doing so: STLR has given us new ways to interact with students. This emphasis on Transformative Learning has helped me broaden my scope to think about, ‘I’m not just teaching chemistry, but I’m really trying to prepare students to be lifelong learners, to be successful in a number of different areas,’ and I’m assessing their progress using the STLR rubrics. (Department Chair, College of Mathematics and Science 2015)

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1305

STLR as a mechanism for helping students build a metacognition about their own development is a point made by a college of education faculty member (2018) concerning the impact STLR has had on her and her teaching: “The STLR process keeps me wanting to design opportunities leading to awareness of changes in [students’] everyday lives.” Another STLR impact on faculty concerns how they conceive the tenets’ association to class content in order to prompt transformation: I hadn’t thought about that, like I wasn’t thinking until I taught at Central here, thinking of tying it to the tenets for me, makes it more possible to dig with the reflection questions [. . .] ‘How does this make me question your assumptions about sustainability when we did the poverty simulation or the community garden project?’ (Science Professor 2017)

Recently compiled results from a survey of STLR-trained faculty reveal some key ways faculty attribute benefit to “doing STLR.” On a 5-point Likert scale ranging from “Strongly disagree” to “Strongly agree,” the percentages agreeing or strongly agreeing with various statements were as follows: (a) I acquired new teaching techniques, 60.87%. (b) I have used one or more of the new teaching strategies to design my transformative learning activities, 67.40%. (c) I feel that my teaching is improving, 73.91%. (d) I felt my students learned more with transformative learning, 80.43%. (e) I felt my students learned more and have evidence to support this claim, 60.87%. (f) I feel my students are learning at deeper levels with transformative learning, 76.09%. One can argue that UCO’s operationalization of TL via STLR has had a considerable impact on teaching and learning at the institution and on faculty’s own development as teacher-scholars. The institution continues to mine narrative and other sources of information to substantiate perspective shifts among faculty and to engage in critical reflexity (Brookfield 2009) in interpreting its findings. STLR’s potential impact on a faculty member’s perspective shift in what is possible to know about teaching and learning might be exemplified in the words of a college of business professor: Just the transformation in the students. I mean, this is what you live for, you know?- And to see that you can actually flag it and demonstrate it, note it, get your hands on it. It was amazing to me . . . (2019)

STLR’s Impact at UCO STLR’s implementation has benefitted the institution in multiple ways: (a) Transforming students’ lives in a mindful, intentional manner that provides tools, processes, and support such that more transformation occurs compared to leaving this important aspect of a college education to chance.

1306

J. King and B. Wimmer

(b) Providing training, tools, and processes that support faculty and staff in helping develop students’ beyond-disciplinary skills and mindsets. (c) Operationalizing the mission around which the institution has organized itself. (d) Attracting faculty eager to teach at an institution with something like STLR in place – department chairs have shared information on multiple occasions that STLR was the reason a new faculty members accepted UCO’s offer instead of other, higher-paying, offers from other institutions – chairs in some departments now regularly bring new faculty candidates to CETTL to learn about STLR as part of campus visits (some visiting candidates have read in detail about STLR before their visits and have potential STLR projects in mind that they discuss with the CETTL executive director as part of their visit). (e) Providing tangible evidence of tools and processes that substantiate how and to what degree UCO is meeting its stated mission (UCO’s most recent institutional accreditation mid-cycle review mentioned STLR 250 times, and STLR has been valuable in programmatic accreditations, as well). (f) Positioning the UCO education with a unique value-add – some incoming students have reported STLR as the reason for their decision to attend UCO instead of other universities in the region offering similar degrees. Additional credence that STLR has been successful as a mechanism to help UCO operationalize TL was lent in the findings of a postdoctoral, mixed-methods research study conducted in fall 2017 about STLR’s operationalization of TL and critical reflection as the means of inculcating a sustainability mindset in students: What UCO has implemented . . . goes beyond the focus on transformation at the individual student level typically found in higher education due to its institutional-level administrative management to push TL forward as an operational process for how the entire campus functions to fulfill its mission. (Brunstein & King 2018 p. 162)

Spreading TL in Higher Education As an operationalization of TL, interest in STLR is widening in higher education, with the spread of STLR to other institutions perhaps the most tangible manifestation. Reasons STLR attracts the attention of other institutions (as shared by these institutions) include retention improvement, interest in STLR-like processes as a quality improvement project robust enough to consider as a quality initiative for institutional accreditation, and because such institutions are seeking proven practices for graduating employment-ready citizens motivated and equipped to begin contributing immediately to the social good. These reasons presuppose the institution’s recognition that TL can be an answer to vexing challenges of student engagement, retention, and graduate preparation for the workforce as well as more expansive and inclusive relationships, and for reenergizing the teaching/learning enterprise away from the outdated and outmoded

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1307

teaching paradigm toward the learning paradigm (Barr & Tagg 1995). Such an understanding of TL and interest in its practice is spreading in higher education (Jones 2017; Schols 2012). In one institution’s case, a provincial government mandate for postsecondary institutions to improve its preparation of graduates to enter the workforce precipitated investigating and adopting their version of STLR. Another institution’s adaptation of STLR was the focus of an accreditation quality initiative. A different institution sees STLR as a proven process that can address the challenge of how to track, assess, and document its students’ development of leadership skills. Yet another university’s adaptation meshed perfectly with its mission of outreach and inclusion regarding an indigenous population. STLR has also impacted higher education more broadly in the form of the Transformative Learning International Collaborative (TLIC), an ever-growing number of institutions interested in: (1) helping their faculty teach in TL-focused ways, and (2) ensuring their graduates are capable and motivated to immediately begin contributing to the social good upon graduation. TLIC began when a few international institutions, as a result of environmental scans about universities that had operationalized TL, found and contacted UCO, sometimes offering memoranda of understanding for faculty exchanges, UCO trainers visiting their campuses, and so on. TLIC grew from that beginning and now counts as members in 16 institutions in 11 countries outside the USA, including Iraq, Brazil, the U.K., South Africa, Australia, Ireland, Kenya, Malaysia, Canada, New Zealand, and Nepal. This international collaboration around TL indicates wide recognition of TL’s efficacy for twenty-first century postsecondary teaching and learning. All of this, however, regards STLR as but one means to spread TL in higher education. There are many more underway, and these need to be encouraged. (For example, there are many TL-for-sustainability initiatives and investigations, some of which are mentioned in the introduction.) Whereas STLR, with its tenets as conceptual organizers for development of students in a number of categories, might be seen as a broad-based approach to TL, in general, “working up” to TL is another approach to bring faculties and institutions to TL. One example is the use of high-impact practices such as undergraduate research “to stimulate transformative learning, where students change their perspective on learning and the purpose of higher education, as well as advance their self-perception from being students towards becoming researchers” (Wallin 2017 p. 315). Other mechanisms of spreading TL in higher education include conferences and journals for faculty/staff professional development and the advancement of teaching/learning. The International Transformative Learning Conference and the Transformative Learning Conference plus the Journal of Transformative Education and the Journal of Transformative Learning are examples of conferences and journals explicitly targeted at TL; but discussions about TL also take place in disciplinary, STEM, and education conferences and journals, (e.g., Brunstein & King 2018; Mistades, delos Reyes, & Scheiter 2011; Paredes 2018; Rahmawati & Taylor 2013).

1308

J. King and B. Wimmer

Many such conferences and journals have familiarized themselves with the concept of the scholarship of teaching and learning (SoTL) (Boyer 1990; Hutchings & Shulman 1999; Hutchings, Huber, & Ciccone 2011; Poole & Simmons 2013), which is a commodious and proper frame for TL: “SoTL represents an opportunity to support transformative education in daily pedagogical practices and scholarly teaching” (Gilpin & Liston 2009). The SoTL movement in higher education can offer an excellent opportunity to spread TL.

Conclusions, Limitations, and Future Directions Both quantitative and qualitative data presented in this case study of UCO’s STLR as an approach to operationalizing transformative learning support its success as an institutional process to improve teaching and learning and to inculcate TL in the curriculum and the cocurriculum. In this regard, STLR stands as an exception to Duerr, Zajonc, and Dana’s (2003) research findings that institution-wide TL in higher education is either rare or nonexistent, and UCO’s own search in 2012 for such institutions revealed the same results nearly a decade later. However, very recent instances of other institutions that have adopted/adapted STLR include several that are moving into institution-wide implementation. UCO’s theory of change model (Fig. 1) has largely been borne out through the short-term outcomes phase. Aspects of some of the long-term outcomes are also in place. The model’s ultimate impact – more TL in higher education along with the resultant personal and societal benefits – remains aspirational at this point. A primary question remains: “Why is STLR succeeding?” Even given UCO’s research and early findings, more investigations need to be undertaken, perhaps via additional doctoral and postdoctoral research by scholars from other institutions. While preliminary findings are starting to emerge, robust analyses are needed to support additional spread of STLR-like approaches at other institutions. UCO’s design, launch, implementation, and assessment of STLR include other limitations. Retention and GPA increases associating with STLR are correlational, and not yet proven causal. Ongoing mixed-methods research is beginning to reveal potential reasons for the association, but the research is still early and is subject to all the intervening variables inherent in social science investigations. Given this, faculty and student interviews, transcriptions, and analyses continue in order to increase the robustness of findings that will be published in the future. Other limitations encountered have been technological, though these have largely been overcome. For other institutions considering adoption of broad operationalization of TL, however, minimizing this limitation is important. Fortunately, as more institutions have adopted/adapted STLR, the variety of learning management systems (LMSs) and the solutions devised for STLR to work in those LMSs have greatly expanded to include four different LMSs which among them maintain the greatest number of postsecondary clients.

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1309

The technological limitation encountered perfectly illustrates a broader limitation: UCO had to devise solutions for many aspects of how to make STLR function well. This meant there was little found in the literature to guide decisions. In the absence of such guidance, STLR’s design and implementation unquestionably included less-than-optimal solutions. Indeed, as other institutions have brought STLR to their own campuses, they have sometimes found better solutions, at least in the context of their own campus resources and cultures. The voluntary nature of STLR engagement by faculty remains a limitation. On the one hand, if not all faculty elect to be STLR trained and then include STLRtagged assignments in their classes, STLR’s positive impacts on students are limited. On the other hand, mandated faculty participation may well have doomed STLR’s acceptance and spread. At least in UCO’s institutional culture, organic adoption by faculty has seemed wise even if there remain many course sections that do not include STLR-tagged assignments that generate student critical reflections. Trendline data seem to indicate that over time, more faculty will choose STLR. Perhaps one limitation to faster faculty adoption has been that not all UCO colleges have yet formalized how to count STLR activity among promotion and tenure (P&T) considerations. Future work should include activity to codify UCO’s STLR implementation processes, making easily available to other institutions the information needed to replicate STLR and/or adapt it as needed. Materials providing this direction should include best practice in STLR and STLR-like implementation, at UCO and other institutions. UCO has begun this already, providing such information to institutions interested in adopting/adapting STLR. Work to identify the institutional characteristics and student demographics that may or may not predispose an institution to successful TL on its campus(es) would also be of benefit. While STLR is proving a successful model in the case study described here, further research to determine whether any uniquenesses at UCO would militate against successful STLR at other colleges and universities is appropriate. Ultimately, assessment of transformative learning is a tectonic shift in the way postsecondary education conceives of measuring student learning. While the academic transcript provides a view of student learning via the A–F grading scale and its supposed indication of competencies acquired, assessment based on STLR rubrics provides an alternative view of student growth by looking at the impact of learning experiences on the students themselves. As faculty have shaped activities that challenge students’ perspectives and worldviews, UCO has faced the challenge of expanding the existing research and literature in higher education, transformative learning, and the role of student and faculty emotions when engaging in TL. The scope and range of what this institution has accomplished in making TL happen on its campus offer hope that the formidable challenges of bringing about an institution-wide shift to TL across a relatively short time span is not only possible, but worthy and rewarding.

1310

J. King and B. Wimmer

References Alsulami, N. M. (2019). Transforming Saudi educators’ professional practices. In P. C. Taylor & B. C. Luitel (Eds.), Research as transformative learning for sustainable futures: Glocal voices and visions. Leiden, The Netherlands: Brill | Sense. Barabasch, A. (2018). The narrative approach in research and its use for policy advice. International Journal of Lifelong Education, 37(4), 468–481. Barr, R. B., & Tagg, J. (1995). From teaching to learning: A new paradigm for undergraduate education. Change, 27(6), 13–25. Bell, D. V. J. (2016). Twenty-first century education: Transformative education for sustainability and responsible citizenship. Journal of Teacher Education for Sustainability, 18(1), 48–56. Biggs, J., & Tang, C. (2011). Teaching for quality learning at university (4th ed.). New York, NY: Open University Press. Bok, D. (2006). Our underachieving colleges: A candid look at how much students learn and why they should be learning more. Princeton, NJ: Princeton University Press. Boyer, E. (1990). Scholarship reconsidered: Priorities of the professoriate. Princeton, NJ: Carnegie Foundation for the Advancement of Teaching. Brookfield, S. (2009). The concept of critical reflection: Promises and contradictions. European Journal of Social Work, 12(3), 293–304. Brooks, A., & Clark, C. (2001). Narrative dimensions of transformative learning. Adult Education Research Conference. East Lansing, MI. http://newprairiepress.org/aerc/2001/papers/12 Brunstein, J., & King, J. (2018). Organizing reflection to address collective dilemmas: Engaging students and professors with sustainable development in higher education. Journal of Cleaner Production, 203, 153–163. Campus Safety. (2012, April 12). ID cards aren’t just for access control anymore. Framingham, MA: EH Publishing. Retrieved July 10, 2018, from https://www.campussafetymagazine.com/ news/college-campus-id-cards-aren-t-just-for-access-control-any-more/ Carter, D. F. (2006). Key issues in the persistence of underrepresented minority students. New Directions for Institutional Research, 130, 33–46. CBI/Pearson. (2017). Helping the U.K. thrive: CBI/Pearson education and skills survey 2017. London, England: Pearson. Retrieved June 14, 2018, from http://www.cbi.org.uk/index.cfm/_ api/render/file/?method¼inline&fileID¼DB1A9FE5-5459-4AA2-8B44798DD5B15E77 Cebrián, G., Grace, M., & Humphris, D. (2015). Academic staff engagement in education for sustainable development. Journal of Cleaner Production, 106, 79–86. Cheney, R. S. (2010, September). Empirical measurement of perspective transformation, 1999– 2009. Paper presented at the Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education. East Lansing, MI. Clavert, M. (2017). Academics’ transformative learning at the interfaces of pedagogical and discipline-specific communities (Doctoral dissertation). Retrieved July 6, 2018, from https:// helda.helsinki.fi/bitstream/handle/10138/229317/Academic.pdf?sequence¼1 Cranton, P. (2002). Teaching for transformation. New Directions for Adult and Continuing Education, 93, 63–71. Cranton, P., & Taylor, E. W. (2012). Transformative learning theory: Seeking a more unified theory. In E. W. Taylor, P. Cranton, & Associates (Eds.), The handbook of transformative learning: Theory, research, and practice (pp. 3–20). San Francisco, CA: Jossey-Bass. Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conduction mixed methods research (2nd ed.). Los Angeles, CA: Sage. Crosling, G., Heagney, M., & Thomas, L. (2009). Improving student retention in higher education: Improving teaching and learning. Australian Universities’ Review, 51(2), 9–18. DeAngelo, L., Franke, R., Hurtado, S., Pryor, J. H., & Tran, S. (2011). Completing college: Assessing graduation rates at four-year institutions. Los Angeles, CA: Higher Education Research Institute, UCLA.

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1311

Dirkx, J. M. (1997). Nurturing soul in adult learning. In P. Cranton (Ed.), Transformative learning in action: Insights from practice (New directions for adult and continuing education) (Vol. 74, pp. 79–88). San Francisco, CA: Jossey-Bass. Duerr, M., Zajonc, A., & Dana, D. (2003). Survey of transformative and spiritual dimensions of higher education. Journal of Transformative Education, 1(3), 177–211. Education Trust. (2015, December 2). Rising tide: Do college grad rate gains benefit all students? Washington, DC: The Education Trust. Retrieved June 17, 2018, from https://edtrust.org/ resource/rising-tide/ Einolander, J., & Vanharanta, H. (2015, July). Assessment of student retention using the Evolute approach, an overview. Paper presented at the 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015) and the Affiliated Conferences, AHFE 2015. Las Vegas, NV. Featherston, B., & Kelly, R. (2007). Conflict resolution and transformative pedagogy: A grounded theory research project in learning in higher education. Journal of Transformative Education, 5(3), 262–285. Finnegan, F. (2014). Embodied experience, transformative learning and social change: Notes on a theory of social learning. Paper presented at ESREA (European Society for Research on the Education of Adults). Interrogating Transformative Processes in Learning and Education Network. Athens, Greece. Retrieved June 1, 2018, from https://www.maynoothuniversity.ie/sites/ default/files/assets/document/Finnegan%20Paper%20on%20Transformative%20learning.docx Fisher-Yoshida, B., Geller, K. D., & Schapiro, S. A. (Eds.). (2009). Innovations in transformative learning: Space, culture, and the arts. New York: Peter Lang Publishing. Freire, P. (1970). Pedagogy of the oppressed. New York, NY: Herder & Herder. Fried, J. (2007). Higher education’s new playbook: Learning reconsidered. About Campus, 12(1), 2–7. Fried, J., & Associates. (2012). Transformative learning through engagement: Student affairs practice as experiential pedagogy. Sterling, VA: Stylus Publishing. FTI Consulting. (2015). U.S. postsecondary faculty in 2015: Diversity in people, goals and methods, but focused on students. Washington, DC: FTI. Fullerton, J.R. (2010). Transformative learning in college students: A mixed methods study. Digital commons. University of Nebraska – Lincoln. Retrieved on July 24, 2017, from the Website: http://digitalcommons.unl.edu/cehsdiss/65 Gardner, L. (2017, June 18). The subtle art of gaining faculty buy-in: Building consensus requires finesse, strategy, and a little psychology. The Chronicle of Higher Education, 63(39). Retrieved June 1, 2018, from https://www.chronicle.com/article/The-Subtle-Art-of-Gaining/ 240373 Gilpin, L. S., & Liston, D. (2009). Transformative education in the scholarship of teaching and learning: An analysis of SoTL literature. International Journal for the Scholarship of Teaching and Learning, 3(2). Article 11. Available at: https://doi.org/10.20429/ijsotl.2009.030211 Greene, R. W. (2013). An examination of the efficacy of service learning in higher education to promote or hinder individual transformative learning at a Jesuit institution (Doctoral dissertation). Retrieved from https://search.proquest.com/docview/1466566988/?pq-origsite¼primo Hart Research Associates. (2013). It takes more than a major: Employer priorities for college learning and student success. Washington, DC: Hart Research Associates. Retrieved June 14, 2018, from https://www.aacu.org/sites/default/files/files/LEAP/2013_EmployerSurvey.pdf Hart Research Associates. (2015). Falling short? College learning and career success. Washington, DC: Hart Research Associates. Retrieved June 14, 2018, from https://www.aacu.org/sites/ default/files/files/LEAP/2015employerstudentsurvey.pdf Hoggan, C. (2014). Transformative learning through conceptual metaphors. Adult Learning, 25(4), 134–141. Hoggan, C. (2016). Transformative learning as a metatheory: Definition, criteria, and typology. Adult Education Quarterly, 66(1), 57–75.

1312

J. King and B. Wimmer

Hoggan, C., Mälkki, K., & Finnegan, F. (2017). Developing the theory of perspective transformation: Continuity, intersubjectivity, and emancipatory praxis. Adult Education Quarterly, 67(1), 48–64. Howie, P., & Bagnall, R. (2013). A beautiful metaphor: Transformative learning theory. International Journal of Lifelong Education, 31(6), 816–836. Hullender, R., Hinck, S., Wood-Nartker, J., Burton, T., & Bowlby, S. (2015). Evidences of transformative learning in service-learning reflections. Journal of the Scholarship of Teaching and Learning, 15(4), 58–82. Hutchings, P., Huber, M., & Ciccone, A. (2011). The scholarship of teaching and learning reconsidered. San Francisco, CA: Jossey-Bass. Hutchings, P., & Shulman, L. E. (1999). The scholarship of teaching: New elaborations, new developments. Change, 31(5), 10–15. Illeris, K. (2014). Transformative learning and identity. New York, NY: Routledge. JED Foundation. (2016, January 13). New partnership to support mental health of college students of color. Retrieved June 10, 2018, from https://www.jedfoundation.org/Steve-Fund-JEDAnnouncement/ Jones, P. (2017, April). Transformative learning in higher education: A pedagogy for individual and social change. Paper presented at Conference on Teaching Excellence: Effective teaching methods to facilitate student deep learning. Beijing, China. Kasworm, C. E., & Bowles, T. A. (2012). Fostering transformative learning in higher education settings. In E. W. Taylor, P. Cranton, & Associates (Eds.), The handbook of transformative learning: Theory, research, and practice (pp. 388–407). San Francisco, CA: Jossey-Bass. Keeling, R. P. (Ed.). (2004). Learning reconsidered: A campus-wide focus on the student experience. Washington, DC: NASPA: Student Affairs Administrators in Higher Education and American College Personnel Association. Keeling, R. P. (Ed.). (2006). Learning reconsidered 2: Implementing a campus-wide focus on the student experience. Washington, DC: American College Personnel Association; Association of College and University Housing Officers–International; Association of College Unions International; National Association for Campus Activities; NACADA: The Global Community for Academic Advising; National Association of Student Personnel Administrators; NIRSA: Leaders in Collegiate Recreation. Kiely, R. (2005). A transformative learning model for service-learning: A longitudinal case study. Michigan Journal of Community Service Learning, 12(1), 5–22. Kitchenham, A. (2008). The evolution of John Mezirow’s transformative learning theory. Journal of Transformative Education, 6(2), 104–123. Kitchenham, A. D. (2015). Transformative learning in the academy: Good aspects and missing elements. Journal of Transformative Learning, 3(1), 13–17. Kuh, G. D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter. Washington, DC: Association of American Colleges and Universities. Lebsack, D. D. (2016). An autoethnographic study of transformative learning in the doctor of education program (Doctoral dissertation, College of Education, Oral Roberts University). Retrieved from https://search.proquest.com/docview/1951780357/?pq-origsite¼primo Maloney, E. J., & Kim, J. (2019, October 30). How universities can avoid learning innovation theater: Unpacking the impact of institutionwide initiatives to advance teaching and learning. Inside Higher Ed. Retrieved October 31, 2019, from https://www.insidehighered.com/digital-learning/blogs/tech nology-and-learning/how-universities-can-avoid-learning-innovation?utm_source¼Inside+Higher +Ed&utm_campaign¼781fdea87f-InsideDigitalLearning_COPY_01&utm_medium¼email&utm_ term¼0_1fcbc04421-781fdea87f-197447533&mc_cid¼781fdea87f&mc_eid¼2653b70fdf Marton, F., & Säljö, R. (1976). On qualitative differences in learning. 1 – Outcome and process. British Journal of Educational Psychology, 46, 4–11. McCunney, W. D. (2015). Striving for the magis: An ethnographic case study of transformative learning and sustained civic engagement at a Jesuit university (Doctoral dissertation). Retrieved from https://search.proquest.com/docview/1696781656/?pq-origsite¼primo

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1313

McGuire, S. Y. (2015). Teach students how to learn: Strategies you can incorporate into any course to improve student metacognition, study skills, and motivation. Sterling, VA: Stylus Publishing. Mertens, D. M. (2007). Transformative paradigm: Mixed methods and social justice. Journal of Mixed Methods Research, 1(3), 212–225. Mezirow, J. (1978). Perspective transformation. Adult Education Quarterly, 28(2), 100–110. Mezirow, J. (1990). How critical reflection triggers transformative learning. In J. Mezirow & Associates (Eds.), Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning (pp. 1–20). San Francisco, CA: Jossey-Bass. Mezirow, J. (2000). Learning to think like an adult: Core concepts of transformation theory. In J. Mezirow & Associates (Eds.), Learning as transformation: Critical perspectives on a theory in progress (pp. 3–33). San Francisco, CA: Jossey-Bass. Mezirow, J. (2006). An overview on transformative learning. In P. Sutherland & J. Crowther (Eds.), Lifelong learning: Concepts and contexts (pp. 24–38). New York, NY: Routledge. Mistades, V., delos Reyes, R., & Scheiter, J. (2011). Transformative learning: Shifts in students’ attitudes towards physics measured with the Colorado Learning Attitudes about Science Survey. International Journal of Humanities and Social Science, 1(7), 45–52. Murphey, M. (2006). Leadership IQ study: Why new hires fail. Public Management, 88(2), 33–34. O’Keefe, P. (2013). A sense of belonging: Improving student retention. College Student Journal, 47(4), 605–613. Palma, L. C., & Pedrozo, E. A. (2019). Transformation for sustainability and its promoting elements in educational institutions: A case study in an institution focused on transformative learning. Organizac¸ ões & Sociedade, 26(89), 359–382. https://doi.org/10.1590/1984-9260898 Paphitis, S. A., & Kelland, L. (2016). The university as a site for transformation: Developing civicminded graduates at South African institutions through an epistemic shift in institutional culture. Education as Change, 20(2), 184–203. Paredes, S. G. (2018). Innovating science teaching with a transformative learning model. Journal of Education for Teaching: International research and pedagogy, 44(1), 107–111. Patterson, B. A. B., Munoz, L., Abrams, L., & Bass, C. (2015). Transformative learning: A case for using grounded theory as an assessment analytic. Teaching Theology and Religion, 18(4), 303–325. Petersen, M. R. (2016). The living dead: Transformative experiences in modelling natural selection. Journal of Biological Education, 51(3), 237–246. Poole, G., & Simmons, N. (2013). The contributions of the scholarship of teaching and learning to quality enhancement in Canada. In G. Gordon & R. Land (Eds.), Quality enhancement in higher education: International perspectives. London, England: Routledge. Rahmawati, Y., & Taylor, P.C. (2013, April). ‘Open the black box of culture and religion’: A transformative journey of a science educator in revealing teaching identity. Paper presented at the 4th Asian Conference on Arts & Humanities (ACAH). Osaka, Japan. Rhodes, T. (2009). Assessing outcomes and improving achievement: Tips and tools for using the rubrics. Washington, DC: Association of American Colleges and Universities. Rogers, E. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press. Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. Schols, M. (2012). Examining and understanding transformative learning to foster technology professional development in higher education. International Journal of Emerging Technologies in Learning, 7(1), 42–49. Seifen, T., Rodriguez, Y., & Johnson, R. (2019). Growth and meaning through study abroad: Enhanced perspectives with mixed methods. Journal of Transformative Learning, 6(1), 6–21. Snyder, C. (2008). Grabbing hold of a moving target: Identifying and measuring the transformative learning process. Journal of Transformative Education, 6(3), 159–181. Stupnisky, R. H., BrckaLorenz, A., Yuhas, B., & Guay, F. (2018). Faculty members’ motivation for teaching and best practices: Testing a model based on self-determination theory across institution types. Contemporary Educational Psychology, 53, 15–26.

1314

J. King and B. Wimmer

Swartz, A. L. & Sprow, K. (2010). Is complexity science embedded in transformative learning? Adult Education Research Conference, Sacramento, CA. http://newprairiepress.org/aerc/2010/ papers/73 Tagg, J. (2012). Why does the faculty resist change? Change: The Magazine of Higher learning, 44(1), 6–15. Tagg, J. (2019). The instruction myth: Why higher education is hard to change, and how to change it. New Brunswick, NJ: Rutgers University Press. Taylor, E. W. (1998). The theory and practice of transformative learning: A critical review (Information series no. 374). Washington, DC: Office of Educational Research and Development. (ERIC Document Reproduction Service No. ED 423 422). Taylor, E. W. (2017). Critical reflection and transformative learning: A critical review. PAACE Journal of Lifelong Learning, 26, 77–95. Taylor, E. W., & Snyder, M. J. (2012). A critical review of research on transformative learning theory, 2006–2010. In E. W. Taylor, P. Cranton, & Associates (Eds.), The handbook of transformative learning: Theory, research, and practice (pp. 37–55). San Francisco, CA: Jossey-Bass. Taylor, K., & Marienau, C. (2016). Facilitating learning with the adult brain in mind: A conceptual and practical guide. San Francisco, CA: Jossey-Bass. Taylor, P. C. (2014). Contemporary qualitative research: Toward in integral research perspective. In N. G. Lederman & S. K. Abell (Eds.), Handbook of research on science education (Vol. II, pp. 38–54). New York, NY: Routledge. Taylor, P. C. (2015). Transformative science education. In R. Gunstone (Ed.), Encyclopedia of science education (pp. 1079–1082). Dordrecht, The Netherlands. https://doi.org/10.1007/97894-007-6165-0_212-2 Taylor, P. C., & Luitel, B. C. (Eds.). (2019). Research as transformative learning for sustainable futures: Glocal voices and visions. Leiden, The Netherlands: Brill | Sense. Taylor, P. C., Taylor, E., & Luitel, B. C. (2012). Multi-paradigmatic transformative research as/for teacher education: An integral perspective. In K. Tobin, B. Fraser, & C. McRobbie (Eds.), International handbook of science education (pp. 373–387). Dordrecht, The Netherlands: Springer. Tinto, V. (2012). Completing college: Rethinking institutional action. Chicago, IL: University of Chicago Press. Verschelden, C. (2017). Bandwidth recovery: Helping students reclaim cognitive resources lost to poverty, racism, and social marginalization. Sterling, VA: Stylus Publishing. Wallin, P. (2017). The potential of complex challenges in undergraduate research to stimulate transformative learning. Nordic Journal of STEM Education, 1(1), 307–318. Walters, C., Charles, J., & Bingham, S. (2017). Impact of short-term study abroad experiences on transformative learning: A comparison of programs at 6 weeks. Journal of Transformative Education, 15(2), 103–121. Witherell, S. & Clayton, E. (2014). Open doors 2014 Report. Institute of International Education. Retrieved from https://www.iie.org/en/Why-IIE/Announcements/2014/11/2014-11-17-OpenDoors-Data

Jeff King, Ed.D., is Executive Director of the Center for Excellence in Transformative Teaching and Learning at the University of Central Oklahoma and Project Director for the institution’s US Department of Education Title III Strengthening Institutions Program grant. His research and application interests have long focused on what faculty can do to help students learn, to be motivated toward deep learning strategies, and to persist in their educations. A decades-long study of educational beliefs held by faculty and students and how to change the limiting beliefs has found its way into his role at UCO. His work over the years both as college faculty member and in faculty professional development matches passion to position in helping students learn. His Ed.D. in Higher Education with a cognate in Adult and Continuing Education is from the University of North Texas. He has been

52

Operationalizing Transformative Learning: A Case Study Demonstrating. . .

1315

invited as plenary speaker at national and international convenings ranging from the Innovation in Assessment and Credentials Summit in Sydney, Australia, to the World Academic Summit at University of California-Berkeley, to the IMS Global Learning Consortium Digital Credentials Summit at Arizona State University, and to the Universal Design and Transformation in Higher Education Congress in Dublin, Ireland. Brenton Wimmer, Ph.D., is the Assistant Director of transformative learning assessment at the University of Central Oklahoma and also teaches first-year experience courses part-time. In his primary role, he works with the Student Transformative Learning Record (STLR) in the Center for Excellence in Transformative Teaching and Learning (CETTL) where he assists with faculty development and assessment, and coordinates all research activities with the Title III Department of Education grant. His research interests include helping faculty to create learning environments that are conducive for TL experiences and to more deeply understand how emotions operate in a transformative learning experience when one is faced with a disorienting dilemma. He is particularly interested in how TL can be leveraged for student success and the public good. Brenton holds a B.S. in Business Administration from Oklahoma State University, an M.Ed. in Adult and Higher Education, and a Ph.D. in Educational Leadership and Policy Studies from the University of Oklahoma.

Transformative Learning and the Affordance of Flexible Habits of Mind

53

Michelle L. Maiese

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enactivism, Meaning-Making, and Habits of Mind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Flexible Habits, Growth, and Self-Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Affordances and the Development of Flexible Habits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pedagogy that Affords Habits of Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1318 1319 1322 1325 1329 1335 1335

Abstract

Transformative learning, as Jack Mezirow understands it, goes beyond the transmission of factual knowledge; it is the process by which we alter problematic frames of reference, which include habits of mind, meaning perspectives, assumptions, and expectations. In an effort to deepen our understanding of the psychological dynamics and pedagogical practices that enable transformative learning, this chapter draws on insights from Evan Thompson’s (Mind in life: Biology, phenomenology, and the sciences of the mind. Belknap Press, Cambridge, MA, 2007) enactivist account of cognition, John Dewey’s (Human nature and conduct: an introduction to social psychology. Henry Holt and Company, New York, NY, 1922) notion of “flexible habit,” and J. J. Gibson’s (The ecological approach to visual perception. Houghton Mifflin, London, England, 1979) notion of “affordance.” Because transformative learning involves significant changes in a subject’s forms of knowing and patterns of interpretation, it centers on the development of flexible habits of mind such as openness, empathy, curiosity, and imagination. However, transformation is not M. L. Maiese (*) Department of Philosophy, Emmanuel College, Boston, MA, USA e-mail: [email protected] © This is a U.S. Government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_153

1317

1318

M. L. Maiese

simply a matter of change within the individual. Habits are formed via interaction with social environments, which encourage particular modes of engagement and solicit various patterns of thinking, feeling, and behaving. Transformative learning therefore depends on educational environments and pedagogies that afford the development of flexible habits and promote self-growth. Pedagogical practices that tap into students’ lived experiences or incorporate creative expression deserve further consideration. Keywords

Affordances · Enactivism · Expressive arts · Habits · Embodied learning

Introduction Transformative learning, as Jack Mezirow understands it, goes beyond the transmission of factual knowledge; it is the process by which we alter problematic frames of reference, which include habits of mind, meaning perspectives, assumptions, and expectations. Building on this idea, O’Sullivan, Morrell, and O’Connor (2002) maintain that “transformative learning involves experiencing a deep, structural shift in the basic premises of thought, feelings, and actions” so that one’s way of being in the world shifts dramatically (18). Dirkx and Mezirow (2006) further emphasize that this kind of “deep” learning challenges existing, taken-for-granted assumptions, including notions regarding what learning is about (Dirkx & Mezirow, 2006, p. 126). Once such modification to a subject’s perspective has occurred, new insight that previously was inaccessible to the subject becomes available, and there is a kind of “psychic reorientation” that alters the “whole of the phenomenal framework in and through which the individual receives, classifies, channels, and responds to her experiences” (Ruth, 1973, p. 291). In an effort to deepen our understanding of the psychological dynamics and pedagogical practices that enable transformative learning, this paper draws on insights from Evan Thompson’s (2007) enactivist account of cognition, John Dewey’s (1922) notion of “flexible habit,” and J. J. Gibson’s (1979) notion of “affordance.” According to Thompson’s enactivist account of cognition, sensemaking is the process whereby a living organism actively constructs meaning and thereby shapes its world into a meaningful domain. This enactivist account meshes well with Mezirow’s constructivist view of learning and what he terms “meaning-making.” Adaptive sense-making requires that subjects both (a) learn from past experience and thereby develop relatively stable skills and patterns of bodily know-how and (b) remain flexible enough to navigate novel situations and engage effectively with an ever-changing world. To unpack this dual demand for stability and flexibility, this chapter looks to Dewey’s notion of “flexible habit” and maintains that transformative learning centers on the development of flexible habits of mind such as openness, empathy, curiosity, and imagination. However, transformation is not simply a matter of change within

53

Transformative Learning and the Affordance of Flexible Habits of Mind

1319

the individual. Habits are formed via interaction with social environments, which encourage particular modes of engagement and solicit various patterns of thinking, feeling, and behaving. Transformative learning requires pedagogical settings, practices, and processes that afford the development of flexible habits which allow subjects to remain open to change and pave the way for deep learning and selfdevelopment. The affordance of these new behavioral and interpretive possibilities can be understood in terms of self-growth that allows subjects to change their lives. While this chapter does discuss examples of pedagogical practices that might facilitate self-growth, its orientation is primarily philosophical. Its central theoretical claims remain rather speculative until they undergo further empirical investigation.

Enactivism, Meaning-Making, and Habits of Mind According to the enactivist account of the mind presented by theorists such as Weber and Varela (2002) and Thompson (2007), mentality is bound up with biology, and cognition is rooted in the structural dynamics associated with metabolism, adaptive self-regulation, and self-maintenance. Central to this view is the notion of sensemaking, which Thompson describes as a fully embodied and relational process whereby living organisms enact meaning: that is, they do not simply passively receive and process stimuli from a pre-given external world but rather actively make sense of their surroundings. Physical and chemical phenomena take on meaning and significance only to the extent that they relate positively or negatively to the “norm of the maintenance of the organism’s integrity” (Thompson, 2007, p. 70). A norm of maintenance can be understood as an organism’s optimal conditions of activity and its proper manner of realizing equilibrium within its environment. By defining itself and distinguishing between self and world, “the organism creates a perspective which changes the world from a neutral place to an Umwelt that always means something in relation to the organism” (Weber & Varela, 2002, p. 118). Thus, what Thompson calls “sense-making” can be understood as the process whereby living organisms interpret environmental stimuli in terms of their “vital significance.” At a basic level, vital significance is a matter of which aspects of the surrounding world contribute to or detract from survival; however, among more sophisticated creatures like us, vital significance goes beyond biological selfmaintenance and concerns “faring well” in a particular sociocultural setting (e.g., making friends, pursuing a career, and attaining some sort of social status). The basic idea is that as striving, living beings, we are not indifferent toward our own existence but rather endeavor to stay alive and fare well, and things in the surrounding world take on significance and value only in relation to our projects, interests, and concerns. Thus, there is good reason to think that sensemaking processes are simultaneously both cognitive and deeply affective (Colombetti, 2014). This enactivist conception of cognition, which emphasizes its active and relational nature, certainly resonates with Mezirow’s constructivist view of learning and

1320

M. L. Maiese

what he terms “meaning-making.” Like enactivists, Mezirow (1990, 1996, 1997, 2009) emphasizes that knowledge is not simply “out there” to be discovered but rather is created via an individual’s interpretations and reinterpretations. What he calls “frames of reference” selectively shape and delimit our perception, cognition, and feelings and come to serve as the point of view from which we construe meaning (Mezirow, 2009, p. 92). Both this process of “meaning-making” and what enactivist theorists call “sense-making” alike can be understood as the activity whereby we shape coherent meaning out of the raw material provided by inner and outer experience. If this characterization is accurate, then revised forms of meaningmaking involve a shift not only in what we know but also how we know, i.e., a shift in the underlying form of one’s way of knowing or constructing meaning (Kegan, 2000). But just what is a form of knowing, and what do we do when we learn something new or unlearn something old? In his description of “frames of reference,” Mezirow points to mindsets, meaning perspectives, and habits of mind. These habitual ways of thinking, feeling, and acting selectively shape and delimit our perception, cognition, and feelings and predispose us to particular intentions, beliefs, and expectations. These habits include linguistic frames, ideological orientations, values, religious world views, characteristic interpretations, and even learning styles. Together, this cluster of meaning schemes constitutes a point of view and acts as a set of implicit rules for interpreting all that we encounter. According to Mezirow (2009), transformative learning centrally involves self-reflection and the critical assessment of these habits of mind so as to bring about a shift in one’s meaning perspectives. But can more be said about the psychological dynamics involved in the transformation process? To elaborate on the notions of “habit” and habit transformation, this chapter looks to enactivism as well as John Dewey’s pragmatist account. Enactivist theorists have characterized habit largely in terms of repetitive behavior and patterns of sensorimotor engagement. Di Paolo (2005), for example, describes a kind of self-sustaining, self-generating dynamic form in animal behavior and in neural and bodily activity that is reflected in postural habits, perceptual invariants, and organized action. Likewise, Froese and Di Paolo (2011) hold that cognition involves “the adaptive preservation of a dynamical network of autonomous sensorimotor structures sustained by continuous interactions with the environment” (p. 18). Among human animals that are capable of coordinated and complex movement sequences, recurring modes of engagement and response begin to develop, and bodily dynamics come to exhibit certain characteristic patterns. Over time, different elements of the musculoskeletal system become “entrained” and exhibit highly integrated configurations that depend on both external and internal constraints. These organized structures (i.e., habits) encompass parts of the nervous system as well as physiological and structural systems of the body. Through the formation of habits, a subject acquires bodily know-how and a specific style of being or temperament. This particular manner of engaging with others and with the world “emerges from the body’s capacities, from habituated expressive postures, and ways of feeling, thinking, acting, and responding to others”

53

Transformative Learning and the Affordance of Flexible Habits of Mind

1321

(Käll & Zeiler, 2014, p. 112). Over time, these characteristic patterns of behavior and response become more engrained and play a significant role in shaping an animal’s customary manner of engagement, its movement repertoire, and its range of skills. Bodily habits thereby play a central role in our conduct insofar as they represent “our typical and cultivated ways of integrating and interacting with the environment” (Cuffari, 2011, p. 537). However, habits encompass not simply bodily behavior but also emotional and intellectual attitudes, meaning perspectives, and frames of reference. In addition to characteristic patterns of movement, a subject develops characteristic ways of attending to and interpreting the surrounding world. Such patterns come to constitute a subject’s unique temperament and the “form or structure of comportment” (Thompson, 2007, p. 80) whereby she gauges the relevance of objects, actions, and events. These “habits of mind” encompass schemas for making sense of one’s situation and environment and include, for example, a tendency to notice particular features of people and events while ignoring others; to deem particular considerations as morally significant while regarding others as irrelevant; to ascribe status and authority to some people while discounting the views of others; and to trust some sources of evidence while remaining suspicious of others. These schemas can be understood as dispositions to see things a certain way or as “scripts” for interaction with each other and with our environment (Haslanger, 2012, p. 415). Along these lines, Mezirow (2009) describes habits of mind as broad, abstract, orienting, habitual ways of thinking, feeling, and acting. Among neurobiologically complex creatures like us, these integrated patterns of attention become quite extensive and sophisticated, giving rise to what Mezirow terms “frames of reference.” Importantly, these patterns are not fixed or static but rather loosely assembled (Colombetti, 2014) and susceptible to ongoing modification as a result of continued learning and development. This means that there is always the potential to shift one’s modes of engagement and develop new habits of movement, thought, and feeling. Thus, habits should not be understood as rigid or mechanical responses but rather in terms of a situation-sensitive, flexible, and adjustable ability to engage with the world (Standal & Aggerholm, 2016, pp. 272–273). This embodied sensitivity to one’s surroundings is the basis for bodily attunement and provides for a range of behavioral and interpretive possibilities. Along these lines, Wheeler (2005) describes engagement with the world as an ongoing adaptive process with continuous actionoriented perception. A creature displays “online intelligence” when it produces “a suite of fluid and flexible real-time responses to incoming sensory stimuli” (Wheeler, 2005). This sort of bodily intelligence involves a feeling of contextual familiarity and a pre-reflective sense of one’s own body as the “possessor of certain capacities for action” (Krueger, 2009, p. 40). Habits and skills help to anchor a subject in the world, allow her to gauge what sorts of response a situation calls for, and enable her to carry out fine-grained adjustments in order to meet the demands of her current circumstances.

1322

M. L. Maiese

Flexible Habits, Growth, and Self-Transformation According to John Dewey, developing flexible habits is central to self-growth. He characterizes education as “the reconstruction or reorganization of experience which adds to the meaning of experience, and which increases ability to direct the course of subsequent experience” (Dewey, 1916). This ability to shift how one organizes experience and to direct the course of later experience requires that people be capable of fluid and flexible action and interpretation so that they can adjust to the peculiarities of their present situation. Thus, the sort of flexible habits he has in mind should be distinguished from mere routine. While some habits do involve a great degree of mechanical repetition, it is possible for habits to become more varied and adaptable, so that one can fluidly take advantage of the wide range of possibilities offered by the environment (Levine, 2012, p. 264). Dewey’s notion of flexible habit begins to make sense of how individuals are able to achieve continual openness even as they retain significant stability. Elaborating on this idea, Proctor (2016) rightly notes that habits have a twofold temporal structure that encompasses both spontaneity and sedimentation. One the one hand, habits bring us into immediate relation with the world and provide us with possibilities for perceiving, engaging with, and responding to our surroundings. Because what we have learned in the past has become engrained in the body via repetition, these ways of engaging with the world are “on tap” and ready to use in an instant in future endeavors (Proctor, 2016, p. 254). This is in part because habits are informed by practical sense, or a “feel for the game,” rather than being dependent upon any sort of explicit rule (Levine, 2012, p. 267), and involve a special sensitivity or accessibility to particular aspects of the environment. This bodily know-how provides for a sense of familiarity and ease and allows us to engage effectively as purposeful agents in multiple contexts without having to deliberate or reflect; and this, in turn, opens up mental and practical “space” for us to think about other things and potentially acquire new skills and insights. Moreover, there are many instances in which antecedent intentions or deliberate reasoning would render bodily movements clumsy and awkward. In order for an individual to engage flexibly and fluidly and take advantage of the wide range of possibilities offered by the environment, her reflective consciousness must offload many of its tasks to automatic, yet rationally intelligible, bodily habits and skills (Levine, 2012, p. 264). Thus, adaptive habits pave the way for practical wisdom and allow us to act in a thoughtful and productive way. The ability to learn from prior experience, together with the establishment of patterns of attention and response that have proved effective in the past, helps subjects to adapt and fare well in their surroundings. On the other hand, due to their repetitive nature, sedimented habits have the potential to close off behavioral possibilities. In many cases, habits operate as “acquired sensori-motor coordinations that are automatically exercised in response to certain types of circumstances” (Levine, 2012, p. 262). Once habits form and become engrained, they may inhibit spontaneous action and make it difficult for subjects to think and behave differently or adjust to new circumstances. This means that even as habits make it possible to have an open future, their formation also has

53

Transformative Learning and the Affordance of Flexible Habits of Mind

1323

the potential to block the possibility of an open future. Cuffari (2011) describes this twofold structure in terms of an ambiguity or tension between stability and plasticity. Habits are developed through the sedimentation of experience into knowledge and know-how that enables meaningful and intelligible being-in-the-world and allows for spontaneous action. However, as life continues to unfold, cognitive plasticity is required in order for individuals to adapt to ongoing environmental changes. Thus, there is a sense in which habits walk the middle way between sedimentation and spontaneity (and between stability and plasticity); and because they anchor individuals in their past experiences, there is a danger that if they become overly rigid, they will limit a person’s ability to adapt and grow. Thus, the development of flexible habits is central to learning and self-development: “to learn every day, we must be ready to be different every day” (Cuffari, 2011, p. 544). No doubt such change is existentially frightening and may be experienced as a threat to one’s self, and this is because there is a sense in which habits constitute the self. Dewey (1922) maintains that habits form our effective desires, furnish us with working capacities, and rule our thoughts to such a great extent that “we are the habit.” Note that this understanding of habit, as intimately part of the self, resonates with the enactivist account sketched earlier in this chapter. As noted previously, habits can be understood as integrated patterns of engagement, response, and attention. Among neurobiologically complex creatures like us, these integrated patterns become quite extensive, giving rise to a characteristically human, personal form of life, or what might be deemed a self. The idea that the self can be understood as a particular form or structure builds on the enactivist idea that the identity of a living system consists in its dynamic, autonomous organization (i.e., its selfgenerating and self-maintaining internal organization). At a basic biological level, the living system’s individuality is a matter of “continuously regenerating itself and its boundary” and thereby demarcating itself from its surroundings as a unified and integrated system (Barandiaran, Di Paolo, & Rohde, 2009, p. 7). However, selforganization occurs not only at the level of metabolic self-maintenance but also via “interactively self-sustaining sensorimotor schemes” or a “sensorimotor repertoire” (Buhrmann & Di Paolo, 2015). Over time, orderly pattern and structure appear, various elements of the musculoskeletal system become “entrained,” and the whole human body behaves as a “pattern-forming, self-organized system governed by nonlinear dynamical laws” (Kelso, 1995, p. 6). Living bodily dynamics – including brain activity, heart rate, metabolic processes, circulation, respiratory processes, sensorimotor processes, etc. – begin to exhibit certain characteristic patterns. The brain and body are interdependent and mutually regulating, and as the animal interacts with the environment, a global pattern of distributed, coherent bodily activity comes to govern its sense-making activities. In addition to characteristic patterns of movement, a subject develops characteristic ways of attending to and interpreting the surrounding world. As part of a greater network, particular habits may depend on other behaviors and habitual expressions as conditions for their exercise. Interaction between various habits increases their number and variety and also tends to link them together, so that the sensorimotor agent is “individuated as a complex network of interdependent

1324

M. L. Maiese

[sensorimotor] schemes, each helping to sustain the others by avoiding both decay and over-rigidity” (Buhrmann & Di Paolo, 2015). What we call “the self” then can be understood as a particular form or structure, i.e., a set of habits, the formation of which is crucial for sense-making and adaptive agency (Maiese, 2015). “Bad habits,” as John Dewey describes them, are ones “so severed from reason that they are opposed to the conclusions of conscious deliberation and decision” (1916). That is, reason could not endorse them since they detract from a subject’s overall well-being. However, sometimes emergent patterns of organization that were adaptive in the short-term because they helped the subject to cope with her surroundings (e.g., smoking as a way to alleviate feelings of stress or cut-throat competition as a way to get ahead in the workplace) ultimately prove to be maladaptive and harmful in the long-term. Note that what is harmful about such habits, in part, is that they put an end to plasticity, so that they possess us instead of our possessing them (Dewey, 1916). At the extreme, inflexible habits can become compulsions or addictions, which include both rigid behaviors as well as intransigent thinking patterns. As particular habits become more engrained and an individual gets “locked into” particular behaviors and mindsets, the continuation of these dominant patterns of movement and interpretation can become goals in themselves (Froese & Di Paolo, 2011, p. 19). Because habits are developed via repetitive exercise and because they bring the past to life by acting out already-existing tendencies, sedimented habits can become increasingly difficult to transform. Käll and Zeiler (2014) give the example of Ed, who repeatedly avoids social interactions, so that this mode of non-interaction and disengagement becomes an integrated part of his habitual ways of behaving (113). There is a sense in which this sedimented bodily way of being-in-the-world puts restrictions on Ed in terms of what kinds of behaviors he is likely to display. While some actions will come easily to Ed, others will appear to be relatively closed off, so that “future possibilities are transformed into more or less likely probabilities” (113). While it is possible for people to behave unexpectedly, inflexible habits render certain kinds of actions highly unlikely or even seemingly impossible. When someone gets stuck in a rut or feels stuck, it is because overly rigid habits of mind have begun to inhibit spontaneous action. This makes it difficult for them to engage effectively with relevant action possibilities or modify their responses so that they better fit the situation. Because the stagnation of habits puts an end to self-development and makes it difficult for individuals to modify their frames of reference, overly rigid habits pose obstacles to significant transformation. However, as proponents of transformative learning are eager to note, significant change certainly is possible. In many cases, such change is gradual, and the alteration to a subject’s patterns of attention, sensitivity, and response is relatively subtle. But in some cases, there is dramatic change that occurs more rapidly, habits and concerns reconfigure, and subjects undergo changes to bodily comportment and a pronounced cognitive-affective reorientation that changes how they habitually engage with their surroundings. In moments of transformative learning, such alteration involves significant changes to the “dynamic architecture” (Kegan, 2000) of subjects’ forms of knowing and patterns of bodily attunement and a profound shift in

53

Transformative Learning and the Affordance of Flexible Habits of Mind

1325

how they gauge significance. This sort of modulation in habits of mind changes how subjects enact meaning via continuous reciprocal interaction with their environment (Thompson, 2007, p. 79) and dramatically alters their sense of what is relevant and important. Thus, transformative learning constitutes a deep form of change, one which alters learners’ very mode of being. What is modified is the self’s form or structure (its principle of organization) (Maiese, 2015), which this paper has argued should be understood in terms of the entrainment of various bodily dynamics that correspond to the formation of habits. The upshot is that the “form” that is transformed during the learning process (Kegan, 2000) is a subject’s form of life, i.e., her habits of mind; and because significant transformation of one’s habits qualifies as selftransformation and involves shifts in the neurobiological dynamics of a subject’s living body, there is a very real sense in which such learning changes our lives. But far from extinguishing the self, such transformation is central to self-growth. In order to fare well, adapt to our ever-fluctuating inner and outer worlds, and behave in a situationally appropriate manner, we need to remain open to change and continually modify our modes of engagement.

Affordances and the Development of Flexible Habits Transformation does not occur simply via change within the individual, however. Dewey describes habits as “interactions of elements contributed by the make-up of an individual with elements supplied by the outdoor world” (1922); and he describes experience as a transaction between an individual and the environment, which consists in an interaction between (a) the objective conditions that constitute the environment and (b) the internal conditions of the subject, including her needs, desires, purposes, and capacities (1916). I already have noted that this conceptualization of habit and experience as transactional and relational is a precursor to the view of cognition articulated by proponents of enactivism. On this view, cognitive processes are partly a matter of what is out there in the world and also partly a matter of what conscious subjects bring to the encounter. This constructivist, relational view of habits and meaning-making can be unpacked further via Gibson’s notion of “affordances.” According to Gibson (1979), “the affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill” (127). Krueger (2014) further describes affordances as “action possibilities in a perceiver’s environment that are specified relationally, that is, both by (i) the particular structural features of the environment and things in it, as well as (ii) the repertoire of sensorimotor capacities the perceiver employs to detect and respond to these structural features” (2). The basic idea is that the environment dynamically offers various possibilities for interaction and engagement (Chemero, 2009; Gibson, 1979) but only in relation to an organism with particular capacities. The landscape (or “total ensemble”) of available affordances is comprised of “the entire set of affordances that are available, in a given environment at a given time” to organisms

1326

M. L. Maiese

occupying a particular material and social world (Ramstead, Veissiere, & Kirmayer, 2016). Typically, an environment will afford an extremely wide range of behavioral possibilities, many of which will not be actualized. This is because many affordances that are offered by the environment will be irrelevant to the agent insofar as they have no bearing on the situation or on individual’s concerns or interests at the time, while others will stand out on the horizon as potentially relevant. For an affordance to have relevance is for it to “solicit” the individual and beckon certain forms of perceptual-emotional appraisal and bodily engagement (Ramstead et al., 2016, pp. 4–5). An affordance becomes a solicitation, Rietveld and Kiverstein (2014) maintain, “when it is relevant to our dynamically changing concerns,” takes on a “demand character,” and becomes manifest at the bodily level in a state of “action readiness.” Along similar lines, Dewey (1922) characterizes habits as “demands for certain kinds of activity.” What sorts of affordances a context provides, and which become solicitations or demands, depends partly on the surrounding environment and partly on a particular agent’s skills, needs, and concerns. Over time, human beings acquire characteristic, stereotypical ways of doing, thinking, and feeling, and I have suggested that these repeated solicitations can be understood as habits. Together, biology, developmental factors, and environmental influences help to shape a subject’s neurobiological patterns, interpretive tendencies, characteristic bodily responses, and habits of mind. Via learning and socialization, an individual develops specific behavioral tendencies and becomes selectively attuned to particular features of her surroundings. As she interacts with her environment and modifies her behavior in accordance with the possibilities and demands of her surroundings, her habitual modes of engaging with available affordances are formed and modulated. This means that the social settings and institutions that people inhabit play an important role in the formation of habits. As Di Paolo (2005) notes, “cultural interaction provides the foundation for cumulatively building on previous more or less viable ways of living” (28), and this is because engagement with a culture gives rise to more developed habits that equip human subjects to meet the demands of the interpersonal and cultural sphere in which they are situated. This occurs in part by way of what Gibson (1979) calls “education of attention” (254): skilled practitioners selectively introduce novices to affordances offered by particular aspects of the environment, and caregivers help children to learn what to notice and how to engage effectively with their surroundings (Rietveld & Kiverstein, 2014, p. 331). Dewey further emphasizes that we develop our habits in conditions set by prior customs, that is, institutionalized sets of social activities. Through “training” provided by various modes of socialization, including both formal and informal educational processes, individuals develop the habits, capacities, and skills that mark them out as members of a particular social and cultural group. Directly changing an individual’s habits is not possible, according to Dewey. Instead, we can change habits indirectly “by modifying conditions, by an intelligent selecting and weighting of the objects which engage attention and which influence the fulfillment of desires” (1922). However, customs tend to persist because each generation is brought up under the conditions established by the previous generation and therefore acquires its

53

Transformative Learning and the Affordance of Flexible Habits of Mind

1327

set of habits. This can become problematic in the event that customs encourage rigid and inflexible habits and do not allow space for the questioning of traditional ways of acting and thinking. Many of the habits developed within social institutions can be understood as bodily skills. What Buhrmann and Di Paolo (2005) call “sensorimotor coordinations” are particular sensorimotor patterns that an agent reliably uses to perform a task and which depend on that agent’s environment, body, and context. Examples include the bodily habits associated with driving a car, handling a tool, or cooking dinner. But in addition to these bodily skills, social institutions also encourage the development of specific habits of judging, reasoning, and weighing evidence (Dewey, 1916). Consider, for example, how many philosophers develop the habit of presenting their arguments as syllogisms or how scientists are trained to formulate hypotheses and value “objectivity.” It is clear that the educational system is one set of social institutions which shapes subjects’ habits of mind, assumptions, mindsets, and learning styles. Learning is a dynamic, fully embodied process that allows subjects to become more adept at completing various sorts of tasks, solving problems, and thinking about things in new ways; and educational institutions allow people to engage in cognitive activities that they are unable to partake in purely on their own or apart from environmentally structured activities. Clearly the nature of the institution in question, including the pedagogical practices and strategies employed, shapes subjects’ frames of reference and forms of knowing. To see this, compare the sort of learning enabled in a traditional classroom environment to the sort of learning enabled by a hands-on apprenticeship. It is unfortunate that learning environments sometimes “demand an unthinking and repetitive style of action from social individuals that ossify habitus and stunt the development of capacities” (Burkitt, 2002, p. 235). Consider, for example, classes that ask students to memorize a long list of facts and then answer a series of multiple choice exam questions or classes that rely almost entirely on lecture and do not invite students to reflect on or discuss the material. Such settings afford (and demand) particular habits of engagement and response, ones that centrally involve mechanical repetition, rote memorization, or routine modes of thinking and problem-solving. Conversely, these settings provide few opportunities for reflective thinking, challenging the status quo, or interrogating dominant assumptions. There is a danger that as learning experiences become “more encased in repetitive behavior and routine conduct” (Carden, 2006, p. 33), individuals will become less capable of self-growth and less open to new forms of knowledge and experience. Transformative learning environments, in contrast, afford the development of flexible learning and interpretive habits and provide students with opportunities to break away from old and ossified habits of action, thought, and emotional dispositions that hinder their development, self-understandings, or relations with others. A subject with flexible mental habits regulates her conduct; adjusts her responses in light of her particular situation, capabilities, and interests; and coordinates her engagement with the environment so as to move toward “optimal grip” on available action possibilities. Flexible habits then “can be understood

1328

M. L. Maiese

as an agent’s having a grip on a rich, dynamic, and varied field” (Ramírez-Vizcaya & Froese, 2019) of relevant action possibilities; this encompasses a capacity to manipulate or modify existing possibilities, envision alternative possibilities, and create new ways forward. In recent years, fostering such habits has become a central concern for many educators. The central aim of The Institute for Habits of Mind, for example, is to disseminate knowledge and resources that can help to transform schools into learning communities that cultivate habits for life-long learning. In their work on habits of mind that enable individuals to become intelligent, creative problem-solvers, Costa and Kallick (2000) emphasize the importance of metacognition, questioning, applying past knowledge to new situations, gathering data through all the senses, responding with wonderment and awe, and responsible risk-taking. Other key examples of flexible habits of mind include empathy, curiosity, imagination, and humility, all of which help someone to remain continually open to new insights. Empathy is one significant way to “open oneself up to different ways of knowing and new forms of intersubjectivity, with the potential to dislodge and rearticulate dominant assumptions, truths and boundaries” (Pedwell, 2012, p. 164). Through empathetic identification and coming face-to-face with what others feel and experience, subjects may encounter a shift in perspective that allows them to see reality in new ways. Carse (2005) emphasizes the need for “morally contoured empathy—empathy that is properly felt and expressed” and appropriately sensitive to the particular situation (171). This capacity for “contextually attuned emotional engagement” (Carse, 2005, p. 170) can be understood as a habit of mind that is crucial for ongoing self-growth and transformation. Empathy should be complemented by habits of curiosity and imagination. Being curious involves becoming appropriately engaged, connected, and interested in another person’s life experiences and felt condition. In her work on care, Noddings (1984) has emphasized the importance of adopting an open and curious stance and “stepping out of one’s own personal frame of reference into the other’s” (24). This centrally involves questioning assumptions, rethinking frames of reference, considering the perspectives of others who are differently situated, and imagining things otherwise. In Dewey’s sense, imagination is the ability to perceive what is in front of us in light of what could be; he emphasizes the importance of possibility, of openness, and of meanings that proliferate rather than stagnate (Cuffari, 2011, p. 539). Flexibility, fertility of imagination, and creativity of thought help subjects to respond dynamically to whatever internal and external conflicts they encounter and envision new sorts of solutions. Likewise, habits of questioning, taking up a stance of wonderment and awe, and engaging all of one’s senses can pave the way toward fresh perspectives. Individuals with these sorts of flexible habits of mind will become capable of acting as “maverick perceivers,” those who can attend to aspects of reality not countenanced by the dominant conceptual scheme (Concepcion & Eflin, 2009, p. 194). Carse (2005) further emphasizes that humility can help us to remain open in our interpretations and to resist foisting our own interpretations too rigidly onto others (191). The recognition that our current frames of reference may be inaccurate or limited also can help us to remain open to alternative ways of thinking, feeling, and behaving.

53

Transformative Learning and the Affordance of Flexible Habits of Mind

1329

Pedagogy that Affords Habits of Transformation I have suggested that transformative learning centrally depends upon the development of flexible habits of mind and that educational institutions can play a central role in such development. Along these lines, Dewey maintains that it’s the duty of educators to manipulate objective conditions so as to construct an environment that enables students to have certain kinds of worthwhile experience (1938). Likewise, Kegan (2000) rightly notes that a more explicit rendering of transformational learning should attend to the deliberate efforts and designs that support changes in the “dynamic architecture” of a learner’s form of knowing. This raises a question about just what sorts of pedagogical strategies and designs are needed to afford this sort of learning. Indeed, the theoretical considerations raised in the paper, which are grounded in an enactivist conception of the mind and psychology, motivate us to seek out new ways to cultivate flexible habits of mind. As Mezirow describes the transformative learning process, it centers on pedagogical practices that foster critically reflective thought, group deliberation, and group problem-solving (Mezirow, 1997, p. 10). Some sort of disorienting experience or personal crisis drives someone to engage in critical reflection and to reevaluate their assumptions about themselves or their world. After reflecting on their meaning perspectives, assumptions, and habits of mind, people engage in reflective discourse and talk to others about their new perspective. This process of revising one’s forms of meaning-making results in what Mezirow (1990) calls “perspective transformation,” which is associated with a “more inclusive, discriminating, permeable, and integrative perspective” (p. 14). It is difficult to deny that critical reflection has great potential to increase selfunderstanding and allow people to revise pre-existing frames of reference. But are there other ways, beyond critical reflection, of promoting “good” habits of mind? While some theorists (Baumgartner 2001; Clark & Wilson, 1991) have criticized Mezirow for ignoring the affective dimension of the learning process, Mezirow’s more recent work acknowledges the importance of affect, intuition, and imagination. Mezirow (2009), for example, emphasizes that a frame of reference encompasses cognitive, conative, and affective components and maintains that transforming problematic frames of reference is in part a matter of making them more “emotionally able to change” (p. 92). Still, he goes on to insist that “transformative learning is essentially a metacognitive process of reassessing reasons supporting our problematic meaning perspectives” (p. 96). In my view, because a person’s frames of reference sometimes are constituted by emotionally charged feelings that remain largely unconscious, increased self-insight requires not just conscious attention and critical reflection, but also “a methodology that allows these powerful energies gradual expression within conscious awareness” (Dirkx & Mezirow, 2006, p. 136). Another concern is that much of Mezirow’s insights about transformative learning appear to favor a particular kind of transformation, namely, a transition to a selfauthoring frame of reference (Kegan, 2000). This involves negotiating one’s own purposes, values, feelings, and meanings rather than simply acting on those of other people or the surrounding culture. Along these lines, Dewey depicts education as the

1330

M. L. Maiese

way “to foster conditions that widen the horizons of others and give them command of their own powers, so that they can find happiness in their own fashion” (1922). Likewise, Cranton and Roy (2003) suggest that when individuals transform a habit of mind, they question and reject assumptions and perspectives that previously were assimilated uncritically. Transformative learning requires that one be able to distinguish between is truly one’s self and what one has absorbed from one’s community or culture (Cranton & Roy, 2003, p. 95). While this sort of transformation no doubt is important, it is not the only way in which a “paradigm shift” within the individual (Concepción & Eflin, 2009, p. 180) can occur. Transformative learning is possible over the whole life span and might involve the development of a capacity for abstract thinking, imaginative abilities, or a profound shift in values. Indeed, there are many different ways in which our frames of reference and modes of knowing may change in significant ways. Moreover, it is important to keep in mind that even those frames of reference that have been “self-authored” may be incomplete, inadequate, or overly simplistic and therefore in need of revision, updating, or even overhaul. We need to think more carefully about what sorts of pedagogies productively disrupt pre-existing habits of mind and solicit openness, empathy, curiosity, and imagination on the part of students. To accomplish this, we must acknowledge that alongside their ambiguity between stability and plasticity, habits involve two primary affects: ease and anxiety (Proctor, 2016, p. 255). On the one hand, because habits give us ways of being in the world that are useful in multiple contexts, they provide us with a sense that we recognize the world and know what we are doing. Such ease and familiarity crucially give us the confidence to try something new but also have a tendency to block self-development by preventing us from seeing (or even considering) that things could be different. On the other hand, habits also can result in anxiety in cases where individuals encounter disruption, customary ways of behaving prove detrimental, or habits fail to function as expected. In cases where students experience unusually high amounts of incoherence between their pre-existing understandings and the new ideas that they encounter in class, there is a danger that they will retrench, “shut down,” and retreat into pre-existing ways of thinking (Concepcion & Eflin, 2009, p. 182) or “overly rigid patterns of interaction” (de Haan, 2017). It is difficult to get a student to shift her modes of thinking when such a shift would disrupt her social and material world and there are no supports in place to compensate for this disruption. Indeed, inflexible habits cause a subject to become “stuck” (Maiese, 2018), in a sense, in maladaptive habits of engagement and response, so that the capacity for fluid and flexible engagement associated with “online intelligence” diminishes. In addition, these characteristic patterns of thinking, feeling, and action may very well overrule or inhibit the expression of other situationally relevant habits (Ramírez-Vizcaya & Froese, 2019), so that subjects find it increasingly difficult to gain an “optimal grip” on available options and interpretive possibilities. In some cases, subjects may even develop cognitive walls: these are habitual ways of thinking and feeling that function as an effective screen against reliable evidence or rational argument. In cases of the so-called backfire effect (Nyhan & Reifler, 2010), for example, subjects double down on their mistaken beliefs when presented

53

Transformative Learning and the Affordance of Flexible Habits of Mind

1331

with new factual information that challenges those beliefs; that is, rather than bringing about a shift in perspective, the provision of new information “backfires” and actually reinforces a subject’s commitment to her mistaken views. This is especially likely to occur in cases where the topic under discussion is somehow central to the subject’s sense of identity. Challenging assumptions that are linked to someone’s identity, sense of self, or central ideological commitments can, indeed, bring about great unease. However, feelings of discomfort and anxiety are crucial insofar as they afford an opportunity for people to reflect on how they want to be in the world or experiment with new ways of thinking and behaving. Indeed, plasticity and possibility are valuable precisely because of the risk they pose to the self (Cuffari, 2011, p. 548); but we must not forget that “doing things differently entails becoming more comfortable with and learning to employ these anxieties” (Proctor, 2016, p. 257) and that transformative learning environments will need to strike a balance between ease and anxiety. That is, they must provide enough safety for students to question and explore without losing their footing in the world and yet be unsettling enough to promote self-transformation. This balance is difficult to achieve, and some students may not be psychologically ready for self-transformation, no matter how skillfully educators conduct their classes. Still, for those students who are ready, it is important to consider what sorts of pedagogical practices can facilitate transformative learning. Weis and Fine (2001) suggest that classrooms can serve as “counterpublics,” spaces in which students can engage in “extraordinary conversations” about controversial public issues. In these relatively safe, yet uncomfortable spaces, students can develop “the habits of mind and skills required to confront dominant social relations and to reconstruct more equitable possibilities” (Sheppard, Ashcraft, & Larson, 2011, p. 72). Establishing counterpublics requires that we find ways to balance certain tensions between ease and anxiety. On the one hand, we need to establish border-crossing opportunities that are contentious enough to expose students to diverse perspectives and move beyond polite tolerance of vastly different points of view. On the other hand, it is crucial that such engagements should not be so contentious that they preclude productive discussion (Sheppard et al., 2011, p. 77). Habits of openness, curiosity, and imagination are best developed in a “community of inquiry” where the emphasis is on questioning to provoke understanding rather than providing ready-made answers. Concepción and Eflin (2009) recommend approaches that inspire students to ask questions, rather than pedagogy that lists answers, as means of preparing students to engage deeply with ideas (p. 194). Teachers should employ pedagogies that make students’ revision of their pre-existing understandings as emotionally and socially bearable as possible while ensuring that students carefully consider the new material and understand it as accurately as possible (Concepción & Eflin, 2009, p. 187). However, they further note that discussion, even dialogue which is well-structured and facilitated by a skilled instructor, is not yet enough. Concepción and Eflin (2009) also recommend that teachers utilize learningcentered pedagogy, which centers around activities that facilitate certain kinds of valuable experience. Such pedagogy is embodied in the sense that students

1332

M. L. Maiese

participate in exercises wherein they live and do not merely get told about or discuss relevant ideas. Learning activities also encourage students to speak from their life experiences and reflect upon them. The authors discuss one exercise that they have used to “bring to life” the topic of feminist separatist communities. These professors asked the women in their feminist philosophy class to assemble in one classroom and the men to assemble in another. After having separate conversations for part of the class session, they all reassembled and were given an opportunity to share highlights from their respective conversations. Many of the women reported that it was a relief to have men out of the room, and sometimes they chose not to report key aspects of their discussion. The women’s choice to reduce men’s access and the men’s frustration about this reduction in access were among the topics that these professors hoped to discuss when evaluating separatism (Concepción & Eflin, 2009, p. 191). On another day, they asked students to throw a football to each other and see whether they noticed any gender-related differences in bodily comportment; and another day, they asked them to observe other students’ behavior in the student union. Exercises such as these do not simply convey information or tell students something; rather, by embodying the course material, these pedagogies show them important insights and allow them to “live the idea” (Concepción & Eflin, 2009, p. 194). This allows them to feel the depth of topics and questions that typically are discussed at more of a distance and thus has the potential to open them up to new considerations and ways of thinking. As a result of participating in such activities, students’ ability to empathically imagine perspectives other than their own grows, their sensitivity increases, and they feel emotionally connected to the course material. Mayes (2010) likewise emphasizes the importance of learning encounters that engage students’ bodies, senses, and emotions. He describes his own experience, in elementary school, of learning that light was faster than sound. His teacher stood at home plate of a baseball field with a track-and-field starting gun, and Mayes and the rest of the class stood out at center field. One of the girls in the class had a stopwatch and was instructed to start the watch when she saw the plume of smoke come out of the gun and stop it when she actually heard the shot. Such exercises cultivate a habit of engaging all of one’s senses and also allow for emotionally salient learning experiences. Mayes notes that “to see and hear the speed of light had allowed [him] to internalize that fact [that light was faster than sound] and thus to make it [his] own as something real, proximate, and subjectively potent” (p. 32). Similarly, to see and hear and feel another student giving voice to an alternative perspective is to have that point of view become proximate and subjectively potent and to have it become connected to one’s own lived experience. Listening to personal testimony from other students together with perspective-taking has the potential to affectively “move” students, disrupt their existing meaning schemes, and change the way that they frame the topics under discussion (Maiese, 2017). Coming face-to-face with another student’s sadness or anger, for example, may arouse feelings that disrupt pre-existing thought patterns, open someone up to an alternative perspective, and allow someone to see complex issues in new ways.

53

Transformative Learning and the Affordance of Flexible Habits of Mind

1333

Another method that holds great potential to foster flexible habits of mind and the ability to analyze problems from a wide range of perspectives is what Brent and colleagues (2019) call “problem-centric analysis.” They outline how courses in the Integrated Science and Technology program at James Madison University utilize holistic approaches to problem-solving that emphasize the interrelationships between science, technology, and society. In a two-semester undergraduate course sequence, students focused their attention on the real-world problem of global water scarcity. Teams of students were assigned to examine specific regions of the world where there are significant water crises. During the first semester, they worked together to define the problem, identify stakeholders, gain familiarity with relevant knowledge domains, and describe the system dynamics relevant to the particular region they were studying. During the second semester, they worked to develop integrative solutions and tested those solutions using dynamic systems. Throughout the academic year, student teams documented their progress on a WIKI space that was subject to ongoing review by their peers, professors, and an external panel of professionals. The hope was that through such educational experiences, students would learn more than content; they would develop habits of mind that pave the way for lifelong learning, innovation, and flexible problem-solving. Another approach that has potential to cultivate flexible habits of mind is “personalized learning.” As Kallick and Zmuda (2017) describe it, such learning gives students a voice by involving them in the processes of goal setting, lesson planning, and assessment. Students act as co-creators who identify the concepts and problems they wish to tackle and then work to outline the actions they will take. They then pursue these goals not in isolation, but with others: students work together to devise theories and create products. Personalized learning also involves an element of self-discovery. Students are encouraged to reflect on the development of their new skills and knowledge so that they gain a better understanding of how they learn. Such pedagogical practices put students “at the center” in the sense that they empower them to investigate problems they themselves have selected, collaborate with other students, and design solutions. Such learning is self-directed, dynamic, and tailored to students’ particular interests and aspirations. Similarly, Cuffari (2011) stresses that “habits of transformation” can be developed via practices that involve the accumulation of implicit knowledge (knowledge-how) as well as intelligence and reflection (p. 546). Drawing from McWhorter (1999), she suggests that pleasure can be used as a tool of habit cultivation and growth. McWhorter relates two of her own practices of pleasure that each develop habits of getting beyond oneself, opening oneself up to other people, and opening oneself up to new self-understandings: line dancing and gardening. While such activities did not take place in the context of formal education, it is not so difficult to envision how creative and pleasurable activities might be utilized in the classroom to encourage habits of openness, flexibility, and imagination. Teachers who hope to nurture their students’ imaginations should attempt to design “exercises to boot them out of complacency, indifference, or self-absorption, to encourage them to reach beyond their own perspectives, to feel and see along with others” (Carse, 2005, p. 190).

1334

M. L. Maiese

Because it provokes inquiry and innovation but also relies on fun and pleasurable activity, creative expression holds great potential to navigate a middle way between ease and anxiety. This suggests that pedagogical practices that incorporate expressive arts might be one powerful way to afford the development of flexible habits of mind and foster openness, sensitivity, and affective and imaginative engagement on the part of students. Along these lines, Yorks and Kasl (2002, 2006) maintain that by “affording glimpses into the other’s world of experience” (Yorks & Kasl, 2002, p. 187), “expressive ways of knowing” such as story-telling, drawing, and dance can help learners to shift their perspective. What they (2002) call “learning-within-relationship” is “a process in which persons strive to become engaged with both their own whole-person knowing and the whole-person knowing of their fellow learners” (p. 185). Expressive activities provide learners with a “brief portal of entry” into another person’s experience so that they can share in that experience and relate it to their own experiential knowledge (Yorks & Kasl, 2006, p. 52). Dance, for example, can help to establish empathic connections and contribute to shared understandings. Movement statements might be understood as “kinetic explorations” (Pallaro & Fischlein-Rupp, 2002) of images and themes that often are outside conscious awareness and yet characterize the participants’ outlooks on the world. As the members take turns executing new movements and mirroring each other, they become more aware of feelings and states of mind associated with interactional movements, more sensitive to the feelings of others, and more capable of expressing nonverbal empathy (Pallaro & Fischlein-Rupp, 2002, p. 38). Because dance provides an opportunity to experiment with new movement patterns, postures, gestures, and bodily styles, it also has potential to open up students to new ways of being in the world. Like dance, drama activities can be used to foster flexible habits, innovative engagement, and what Carden (2006) terms “creative intelligence” (p. 33). Staging and performing in a play can allow students to adopt the point of view of a character very different from themselves and to seriously consider alternative perspectives. In order to adopt the roles of different characters, students need to imagine themselves differently and behave differently, i.e., they must “experiment” with a wide range of bodily habits and points of view. Another potentially transformative element of drama practice is that it allows students to collaborate and work together to create and explore artistic acts (Cody, 2015). Such cooperation requires that participants negotiate diverse viewpoints, exercise empathy, and trust those with whom they are working. In addition, students who act as co-artists and are able take ownership of creative work (e.g., by writing a script or co-directing a short play) invest more and care more about the quality of their work. That is, they become highly emotionally invested in the learning process. On a more modest scale, drama games that involve humor and high levels of participation can be used to promote active listening, cooperation, and openness to different ways of being in the world. Greenwood (2012) further maintains that drama can help to stimulate critical engagement and foster the attitudes and skills associated with citizenship. In addition to expressing their agency and exploring various social roles, students can examine

53

Transformative Learning and the Affordance of Flexible Habits of Mind

1335

various sorts of human conflict and live through it vicariously via the character they are performing. Participating in drama activities also gives students an opportunity to explore ideas through mythic and metaphorical lenses, which can open up space for raising questions about power, identity, and society. Requiring that students evaluate their work from both performer and audience perspectives, utilizing structured reflection (Cody, 2015), can serve as a way to combine creative expression with the sort of critical reflection that Mezirow recommends. Thus, like other expressive art activities, drama can help to deepen students’ understanding and afford the development of flexible habits of mind.

Concluding Remarks The overarching aim of the approaches sketched in the previous section is to engage “the whole person” and catalyze some sort of experiential break, one which affectively moves students and disrupts their pre-existing frames of reference. This general aim fits well with Dirkx’s conception of transformative learning as soul work or inner work and his call for an “integrated understanding of subjectivity, one that reflects the intellectual, emotional, moral, and spiritual dimensions of our being in the world” (Dirkx & Mezirow, 2006, p. 125). This chapter has argued that such learning depends centrally on flexible habits of mind and that educational institutions can play a pivotal role in affording the development and exercise of such habits. By engaging with pedagogical practices that foster “creative intelligence” (Carden, 2006, p. 33), students can become more capable of innovative activity and open inquiry and avoid the dangers associated with sinking deeper into routine, retrenching, or becoming antagonistic toward change. However, the discussion presented here is just the beginning. As noted in the introduction, the aim of this chapter is to deepen our theoretical understanding of the sorts of flexible habits that are central to transformative learning. However, these recommendations remain rather speculative until they are implemented by more teachers and studied by education theorists. Further empirical research should be done to investigate what sorts of pedagogical practices and strategies afford the development of flexible habits and thereby facilitate transformative learning.

References Barandiaran, X., Di Paolo, E., & Rohde, M. (2009). Defining agency: Individuality, asymmetry, and spatio-temporality in action. Adaptive Behavior Journal, 1, 1–13. Baumgartner, L. (2001). An update on transformational learning. New Directions for Adult and Continuing Education, 89, 15–24. Brent, R., Deaton, M., Tang, J., & Handley, M. Incorporating habits of mind into science and technology curricula. Retrieved from https://www.academia.edu/1346592/Incorporating_ Habits_of_Mind_into_Science_and_Technology_Curricula on March 7, 2019.

1336

M. L. Maiese

Buhrmann, T., & Di Paolo, E. (2015). The sense of agency – A phenomenological consequence of enacting sensorimotor schemes. Phenomenology and the Cognitive Sciences. https://doi.org/ 10.1007/s11097-015-9446-7 Burkitt, I. (2002). Technologies of the self: Habitus and capacities. Journal for the Theory of Social Behavior, 32(2), 219–237. Carden, S. (2006). Virtue ethics: Dewey and McIntyre. New York, NY: Continuum International Publishing Group. Carse, A. (2005). The moral contours of empathy. Ethical Theory and Moral Practice, 8, 169–195. Chemero, A. (2009). Radical embodied cognitive science. Cambridge, MA: MIT Press. Clark, M., & Wilson, A. (1991). Context and rationality in Mezirow’s theory of transformational learning. Adult Education Quarterly, 41(2), 75–91. Cody, T. (2015). Transformative classroom drama practice: What is happening in New Zealand schools? p-e-r-f-o-r-m-a-n-c-e, 2 (1–2). http://www.p-e-r-f-o-r-m-a-n-c-e.org/?p=2513 Colombetti, G. (2014). The feeling body: Affective science meets the enactive mind. Cambridge, MA: MIT Press. Concepcion, D., & Eflin, J. (2009). Enabling change: Transformative and transgressive learning in feminist ethics and epistemology. Teaching Philosophy, 32(2), 177–198. Costa, A., & Kallick, B. (2000). Discovering and exploring habits of mind. Alexandria, VA: Association for Supervision and Curriculum Development. Cranton, P., & Roy, M. (2003). When the bottom falls out of the bucket: Toward a holistic perspective on transformative learning. Journal of Transformative Education, 1(2), 86–98. Cuffari, E. (2011). Habits of transformation. Hypatia, 26(3), 535–553. de Haan, S. (2017). The existential framework in psychiatry. Mental Health, Religion, and Culture, 26(6), 528–535. Dewey, J. (1916). Democracy and education: An introduction to the philosophy of education. New York, NY: The Macmillan Company. Dewey, J. (1922). Human nature and conduct: An introduction to social psychology. New York, NY: Henry Holt and Company. Dewey, J. (1938). Education and experience. New York, NY: Simon and Schuster. Di Paolo, E. (2005). Autopoiesis, adaptivity, teleology, agency. Phenomenology and the Cognitive Sciences, 4, 429–452. Dirkx, J., & Mezirow, J. (2006). Musings and reflections on the meaning, context, and process of transformative learning: A dialogue between John M. Dirkx and Jack Mezirow. Journal of Transformative Education, 4(2), 123–139. Froese, T., & Di Paolo, E. (2011). The enactive approach: Theoretical sketches from cell to society. Pragmatics and Cognition, 19(1), 1–36. Gibson, J. J. (1979). The ecological approach to visual perception. London, England: Houghton Mifflin. Greenwood, J. (2012). Strategic artistry: Using drama processes to develop critical literacy and democratic citizenship. Asia-Pacific Journal for Arts Education, 11(5), 104–125. Haslanger, S. (2012). Resisting reality: Social construction and social critique. Oxford: Oxford University Press. Käll, L., & Zeiler, K. (2014). Bodily relational autonomy. Journal of Consciousness Studies, 21(9–10), 100–120. Kallick, B., & Zmuda, A. (2017). Students at the center: Personalized earning with habits of mind. Alexandria, VA: ASCD. Kegan, R. (2000). What “form” transforms: A constructive-developmental perspective on transformational learning. In J. Mezirow et al. (Eds.), Learning as transformation: Critical perspectives on a theory in progress. San Francisco, CA: Jossey-Bass. Kelso, J. A. S. (1995). Dynamic patterns: The self-organization of brain and behavior. Cambridge, MA: MIT Press. Krueger, J. (2009). Knowing through the body. Journal of Chinese Philosophy, 36(1), 31–52. Krueger, J. (2014). Affordances and the musically extended mind. Frontiers in Psychology, 4, 1–12.

53

Transformative Learning and the Affordance of Flexible Habits of Mind

1337

Levine, S. (2012). Norms and habits: Brandom on the sociality of action. European Journal of Philosophy, 23(2), 248–272. Maiese, M. (2015). Embodied selves and divided minds. Oxford, UK: Oxford University Press. Maiese, M. (2017). Transformative learning, Enactivism, and affectivity. Studies in Philosophy and Education, 36(2), 197–216. Maiese, M. (2018). Getting stuck: Temporal desituatedness in depression. Phenomenology and the Cognitive Sciences, 17(4), 701–718. Mayes, C. (2010). The organic infrastructure of transformative education. Encounter: Education for Meaning and Social Justice, 23, 31. McWhorter, L. (1999). Bodies and pleasures: Foucault and the politics of sexual normalization. Bloomington: Indiana University Press. Mezirow, J. (1990). How critical reflection triggers transformative learning. In J. Mezirow et al. (Eds.), Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning (pp. 1–20). San Francisco, CA: Jossey-Bass. Mezirow, J. (1996). Contemporary paradigms of learning. Adult Education Quarterly, 46(3), 158–172. Mezirow, J. (1997). Transformative learning: Theory to practice. New Directions for Adult and Continuing Education, 74, 5–12. Mezirow, J. (2009). Overview of transformative learning theory. In K. Illeris (Ed.), Contemporary theories of learning: Learning theorists. . . in their own words (pp. 90–105). New York, NY: Routledge. Noddings, N. (1984). Caring: A feminine approach to ethics and moral education. Berkeley, CA: University of California Press. Nyhan, B., & Reifler, J. (2010). When corrections fail: The persistence of political misperceptions. Political Behavior, 32(2), 303–330. O’Sullivan, E., Morrell, A., & O’Connor, M. (Eds.). (2002). Expanding the boundaries of transformative learning: Essays on theory and praxis. New York, NY: Palgrave Press. Pallaro, P., & Fischlein-Rupp, A. (2002). Dance/movement in a psychiatric rehabilitative day treatment setting. The USA Body Psychotherapy Journal, 1(2), 29–51. Pedwell, C. (2012). Affective (self-) transformations: Empathy, neoliberalism, and international development. Feminist Theory, 13(2), 163–179. Proctor, S. (2016). The temporal structure of habits and the possibility of transformation. International Journal of Applied Philosophy, 30(2), 251–266. Ramírez-Vizcaya, S., & Froese, T. (2019). The enactive approach to habits: New concepts for the cognitive science of bad habits and addiction. Frontiers in Psychology, 10, 301. Ramstead, M. J., Veissiere, S. P., & Kirmayer, L. J. (2016). Cultural affordances: Scaffolding local worlds through shared intentionality and regimes of attention. Frontiers in Psychology, 7, 1090. Rietveld, E., & Kiverstein, J. (2014). A rich landscape of affordances. Ecological Psychology, 26(4), 325–352. Ruth, S. (1973). A serious look at consciousness-raising. Social Theory and Practice, 2(3), 289–300. Sheppard, S., Ashcraft, C., & Larson, B. (2011). Controversy, citizenship, and counterpublics: Developing democratic habits of mind. Ethics and Education, 6(1), 69–84. Standal, Ø. F., & Aggerholm, K. (2016). Habits, skills, and embodied experiences: A contribution to philosophy of physical education. Sport, Ethics, and Philosophy, 10(3), 269–282. Thompson, E. (2007). Mind in life: Biology, phenomenology, and the sciences of the mind. Cambridge, MA: Belknap Press. Weber, A., & Varela, F. (2002). Life after Kant: Natural purposes and the autopoietic foundations of biological individuality. Phenomenology and the Cognitive Sciences, 1, 97–125. Weis, L., & Fine, M. (2001). Extraordinary conversations in public schools. Qualitative Studies in Education, 14(4), 497–523. Wheeler, M. (2005). Reconstructing the cognitive world: The next step. Cambridge, MA: MIT Press.

1338

M. L. Maiese

Yorks, L., & Kasl, E. (2002). Toward a theory and practice for whole-person learning: Reconceptualizing experience and the role of affect. Adult Education Quarterly, 52(3), 176–192. Yorks, L., & Kasl, E. (2006). I know more than I can say: A taxonomy for using expressive ways of knowing to foster transformative learning. Journal of Transformative Education, 4(1), 43–64.

Michelle Maiese is Professor of Philosophy at Emmanuel College in Boston, MA. She earned a Ph.D. in Philosophy from the University of Colorado in Boulder. Her research focuses on issues in philosophy of mind, philosophy of psychiatry, and the emotions. She is the author of Embodied Minds in Action (co-authored with Robert Hanna), Embodiment, Emotion, and Cognition, and Embodied Selves and Divided Minds.

Achieving Education for Sustainable Development (ESD) in Early Childhood Education Through Critical Reflection in Transformative Learning

54

Şebnem Feriver, Refika Olgan, and Gaye Teksöz

Contents Introduction and Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Teachers as Supporters of Social Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transformative Sustainability Learning in Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transformative Sustainability Learning in Early Childhood Education . . . . . . . . . . . . . . . . . . . Critical Reflection for Transformative Sustainable Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Facilitators of Critical Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Collection Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Collection Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assessing the Levels of the Participants’ Reflection and Non-reflection . . . . . . . . . . . . . . . . . . Participants’ Levels of Reflection in Relation to the Data Collection Instruments . . . . . . . Participant’s Levels of Reflection and Their Transformative Journey . . . . . . . . . . . . . . . . . . . . . Features of Reflective Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion and Implications for Education, Practice, and Research . . . . . . . . . . . . . . . . . . . . . . . . . . Critical Reflection and Individual Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Content for the Critical Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Process for the Critical Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1340 1342 1344 1345 1346 1347 1348 1348 1352 1353 1353 1355 1355 1357 1360 1361 1364 1365 1367 1368 1369

Ş. Feriver (*) · R. Olgan Faculty of Education, Department of Elementary and Early Childhood Education, Middle East Technical University, Ankara, Turkey e-mail: [email protected]; [email protected] G. Teksöz Faculty of Education, Department of Mathematics and Science Education, Middle East Technical University, Ankara, Turkey e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_154

1339

1340

Ş. Feriver et al.

Appendix 1: Critical Reflection Opportunities in the Training in Relation to Mezirow’s Three Forms of Critical Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 2: The Categorization Scheme (Wallman et al., 2008) Used for the Analysis of Learning Diaries and Learning Activities Open-Ended Questions . . . . . . . . . . . . . . . Non-reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1370 1373 1373 1374 1375

Abstract

The central role of education in creating a more sustainable future has been already recognized by educators and policy-makers alike. This chapter argues that this can only be truly achieved through the efforts of teachers in implementing an “education of a different kind,” a general educational shift that seeks to encompass a converging transformation of the priorities and mindsets of education professionals. In this regard, the professional preparation of teachers, as the leading actors in shaping children’s learning processes, and their continuous professional development are vital considerations for Education for Sustainable Development (ESD) to be successfully achieved. Linking transformative learning and ESD has emerged as a distinct and useful pedagogy because they both support the process of critically examining habits of mind, then revising these habits and acting upon the revised point of view. This study aims to describe and evaluate the potential of transformative learning in innovating mainstream education toward sustainability by focusing on the role of critical reflection in a capacity building research project realized in Turkey. The data was gathered from 24 early childhood educators using a mixed-method research design involving learning diaries, a learning activities survey, and follow-up interviews. This chapter identified content, context, and application method of the in-service training as factors that have contributed to the reflective practices of the participants. In addition, presenting the implications regarding the individual differences in how learners engage in critical reflection practices, this research offers a framework for a content- and process-based approach derived from Mezirow’s conception of critical reflection. Keywords

Transformative learning · Education for sustainable development · Early childhood education · Critical reflection · In-service teacher training

Introduction and Background In 1992, jointly written by the Union of Concerned Scientists and more than 1700 independent scientists, among them the majority of living Nobel laureates, “World Scientists’ Warning to Humanity” cautioned that “a great change in our stewardship of the Earth and the life on it is required, if vast human misery is to be avoided”

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1341

(Union of Concerned Scientists, 1992). On the 25th anniversary of their appeal, a second warning has been given to humanity, clearly stating that humanity has failed to achieve adequate progress in responding to the environmental challenges, and worryingly, most of them are deteriorating in a severe manner (Ripple et al., 2017). Commonly referred to as wicked problems, which cannot be explained in a single definition, the current challenges in the planet earth, such as diminishing biodiversity, poverty, depletion of resources, food shortages, inequity, and chronic nutrition deficiency, are submerged under the conflicts of interest among multiple stakeholders. The shared features of these issues can be considered as “highly complex and systemic, ambiguous and contested, and urgent and existential” (Wals, 2015, p. 4). The Global Action Programme (GAP) on Education for Sustainable Development (ESD) was formed to give a tangible response to the urgent need for an overhaul of our living styles, which is sensitive to and respectful of the scarcity of our planet’s resources while improving our collective well-being (UNESCO, 2017a). In line with that, the current 2030 Agenda for Sustainable Development (UN, 2015) precisely echoes this vision which stresses the importance of appropriate educational action. Seventeen Sustainable Development Goals (SDGs), which are at the core of the 2030 Agenda, spread transformational and comprehensive SDGs aimed to achieve a maintainable, peaceful, prosperous, and equal life for everybody in the world both now and in the future. To make this adjustment, totally novel abilities, morals, and behaviors that result in more practicable social orders are needed. Therefore, in order to address this critical need, an alteration in the framework of education is strongly recommended (UNESCO, 2017b). As a matter of fact, education is not only characterized as an objective in itself but also a means to achieve the SDGs. Thus, education is not simply seen as a fundamental part of sustainable development, but rather as an indispensable enabler in the process. On the other hand, it has been discussed whether ESD is the solution to the challenges our planet faces or it is part of the problem (Balsiger et al., 2017). According to some critics, as utilitarian and neoliberal discourses on education and sustainability became dominant in the process, the prevailing growth paradigms reduce the natural world to a secondary role, which is only to be of use to human beings (e.g., Huckle & Wals, 2015). Approaching the issue from a different perspective, experts working in the field of ESD draw attention to the need for new pathways in teaching and learning to overcome current obstacles and continue to foster ESD (Tilbury, 2011). It has been argued that due to its capacity to engage learners to acquire a new set of skills and encourage them to undertake activities in a sustainable manner in complex circumstances, ESD advances the ideas of the integrity of nature, economic reasonability, and a fair society for present and forthcoming generations: What ESD requires is a shift from teaching to learning. It asks for an action-oriented, transformative pedagogy, which supports self-directed learning, participation and collaboration, problem-orientation, inter- and transdisciplinarity and the linking of formal and informal learning. Only such pedagogical approaches make possible the development of the key competencies needed for promoting sustainable development. (UNESCO, 2017b, p. 7)

1342

Ş. Feriver et al.

This concept is based on the premise that instead of only conformative and reformative learning, transformative learning is needed as well (Sterling & Thomas, 2006). This has been a challenge for those implementing educational policy at all levels because even the new, reformist and innovative education attempts have not escaped from repeating the failures of the old programs and result in seeking remedies to the problems of today in yesterday’s solutions (Sterling, 2001; Thornton, Peltier, & Perreault, 2004). Thus, as demonstrated by a number of research studies over the last three decades, education that is designed to raise awareness about environmental issues does not have a major effect on behaviors (Orr, 2004). Moreover, the notion that increasing education levels would inevitably translate into awareness to address the challenges of local to global concepts of an unsustainable lifestyle and economy has been rejected. This is supported by Sauter and Frohlich (2013), who pointed out that largest ecological footprints are left on Earth by people with highest education levels who live in the most developed economies (WWF, 2018). There are two different perspectives from which to shed light on this situation. First, it has been argued that among the factors which are effective in determining the outcomes of schooling, the economic, social, and political structures of the respective societies figure as predominantly as the educational curricula (Kubow & Fossum, 2007). After all, teaching is considered as a political act emerged as a result of cultural, racial, economic, and political tensions (Freire, 1998). Second, due to the intertwined nature of the social, economic, and ecological aspects, issues concerning education for sustainable development have gained a multifaceted character. This complexity necessitates a new approach in order to develop requisite learning experiences “of a different kind” (Schumacher, written 1974, published 1997). Consequently, any discussion concerning the priorities of educational provisions and reform should take place upon a research base and current thinking about sound educational practice (Rickinson, 2006).

Teachers as Supporters of Social Change Incorporating various aspects of sustainability into education necessitates educators to start thinking critically and creatively about the structuring (and possible restructuring) of didactic arrangements. This specific circumstance is underscored because the mediating effect of every teacher is most likely the most indispensable component of a student’s leaning toward sustainability in formal settings (Wals, 2006). Accordingly, the preparation and continuous professional development of teachers as the leading actors in shaping children’s learning processes across differences of perspectives, goals, and practice are vital concerns in the attainment of ESD goals (Hanushek, Rivkin, & Kaim, 2005; Pramling Samuelsson & Park, 2017). Woodrow and Caruana (2017) argue that given the increasingly diverse student population, increasing the level of critical consciousness of preservice teachers

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1343

(Freire, 1997) has become an indispensable step in preparing them for their roles as change agents. In their task to cultivate teachers as change agents, teacher educators must enhance preservice teachers’ awareness of overbearing circumstances and inequalities based on structural categories of difference and foster their ability to take a critical stance toward systems of power which is dismissive of individual’s and community’s rights (Ladson-Billings, 1998). Indeed, “informed by the critical theory tradition, reflection becomes critical when it’s focused on teachers understanding power and hegemony” (Brookfield, 2017, p. 9). This is supported by numerous well-documented and recurrent challenges to the development of preservice teachers’ critical consciousness (Woodrow & Caruana, 2017). One deduction from those challenges is that when their notions of self, society, and their interaction is challenged, preservice teachers often exhibit resistance toward critical education practices (e.g., Gay & Kirkland, 2003; Johnson, 2006). Being uninformed about the political nature of education, many teachers are uncertain about becoming involved in the wider context that affects their working environment, which eventually have an influence on their students’ learning conditions (Picower, 2013). Any process of reflection on social conditions is hindered by this resistance, which in return impedes the teacher educators’ ability to train teachers as change agents, while “educators who demonstrate critical consciousness have the ability and will to theorize and politicize their experiences” (Nieto & McDonough, 2011, p. 366). Thus, in order to accomplish “learning experience of a different kind,” it is necessary to rethink the content of education and the competencies of teachers. In the same vein, an essential requisite is the overhaul of the thinking of how to improve the abilities of the teachers. As Sterling (2010, p. 19) argues, “where there is a call for re-examination of assumptions and values, critical thinking and new creativity, the concept of transformative learning is coming more to the fore.” He considers that the main objective of adult educators is to guide learners to transformation, which embodies growing and maturing intellectually and as a result changing as a person through critical reflection on their assumptions, beliefs, and values. Indeed, “transformative learning refers to processes that result in significant and irreversible changes in the way a person experiences, conceptualizes, and interacts with the world” (Hoggan, 2016, p. 71). In this context, Freire (1985) argued that teachers should consistently approach their profession from a critical stand point and ask themselves for whom and on whose behalf they are working. Teachers who accept the role of a transformative intellectual regard students as critical agents, who engage in inquiry about how knowledge is generated and disseminated, make use of intellectual exchange, and make knowledge meaningful, critical, and emancipatory (McLaren, 2003). However, teachers involved in traditional educational programs are not expected to approach the system in which they work from an analytical point of view, and thus they are not encouraged to comprehend their potential role in a political and market-driven system. One of the reasons which are considered to cause this situation is the problematic design of higher education.

1344

Ş. Feriver et al.

Transformative Sustainability Learning in Higher Education Cranton and King (2003) affirmed that in its least complex shape, higher education (HE) today might, at best, create no more than dutiful citizens who are prepared to work within society’s institutions, accepted occupations, and organizations. There is little indication that HE is accomplishing more than essentially fortifying patterns that enable students to assimilate new experiences into what Belenky and Stanton (2000, p. 71) alluded to as inherited “mental maps,” which are conditioned frames of reference through which people channel their apparent learning experiences. Habermas (1984) referred to this kind of learning as instrumental learning in the sense that its main objective is to equip learners with information and skills. Mezirow (2000) advocated that instrumental learning supports a society’s “cultural canon, socioeconomic structures, ideologies, and beliefs about self [that] often conspire to foster conformity, and impede development of a sense of responsible agency” (p. 8). At the point when students are reduced to mere replicators, they adopt inherited mental maps, which might be questionable in terms of exploring the present elements of postmodern life (Glisczinski, 2007). New models of interdisciplinary education stimulate student cooperation in moving toward transformative, experiential, and collaborative learning (Cranton, 1996). Unfortunately, collaborative models are troublesome (but not unachievable) to create within current academic frameworks that underscore singular evaluating and other competitive models of accomplishment (Moore, 2005). One of the reasons for this lack of incorporation of these models is that HE sector has been energized by an internationally hegemonic neoliberal ideology (Sterling, 2017). According to Sterling, the paradigm created by this ideology renders obsolete the older (and more educationally defensible), liberal, holistic, and humanistic philosophies regarding the nature and purpose of education. Moreover, in spite of the prevailing academic freedoms in instruction and research, only a limited number of professors engage in alternative models for teaching and learning in their classrooms or accentuate social change as an outcome of their classes (▶ Chap. 49, “Are Students and Faculty Ready for Transformative Learning?”; Moore, 2005). Consequently, learners who pass through HE exit as individuals with only instrumental knowledge, joining the ranks of those with a higher level of education but who place a heavier ecological burden on our planet. Still, the potential of the HE should not be underestimated as it has the capacity to develop a new conscientiousness, understanding, perception and transformation as a result of stimulating proactive thinking, integrating variety of perspectives and promoting dialogue (Daloz, 1999). Belenky and Stanton (2000) underscored that “not only would participation and reflective dialogue support [students’] development as individuals, it could also support the development of a more inclusive, just, and democratic society” (p. 74). A tremendous potential exists for colleges to be pioneers in scrutinizing the present state of affairs, testing paradigms and straightforwardly honing better approaches for living, thinking, teaching, and learning (Moore, 2005). Yet, voices for change are becoming more

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1345

unyielding. For instance, Escrigas (2016) called upon to universities, to “learn to read reality,” and “understand the wider impacts of their actions and the costs of what they are not doing at a time when societal transition is urgently needed” (p. 3). In the same vein, it is argued that one of the vital issues facing the sector is the kind of role HE will play in creating the leaders of tomorrow and fostering graduates duly prepared to act in future scenarios (Blake, Sterling, & Goodson, 2013). The starting point of such approaches is the contention that students should be outfitted with the essential knowledge, aptitudes, qualities, and states of mind to manage intricate and ambiguous sustainability issues in society (Lambrechts & Van Petegem, 2016). One of the other gains of this process is the growing appeals for the greater alignment of HE to the issues and possibilities that sustainable development offers. In accordance with this idea, there is a development in colleges worldwide to advance strategies and processes for creating more sustainable campuses. This development was started with various worldwide international declarations and commitments made by colleges around the world (Wright, 2002). This chapter supports the idea that there is a need for a significant readjustment in the way pre-service teachers are taught and learn within HE (Dawe, Jucker, & Martin, 2005), which requires the academics to consider pedagogy through alternative perspectives (Sterling, 2004). Explicitly linking education for sustainability and transformative learning, not only because they encompass socioeconomic and political analysis, has come to the fore as a distinct and useful pedagogy, a cultural politic of schooling, learning, and teaching (McLaren, 2003). Indeed, many scholars described a meaningful connection between transformative and sustainability learning (i.e., Harmin, Barrett, & Hoessler, 2017; Sterling, 2010). The anticipated outcomes of transformative learning which often follows some variation of ten different stages are nurturing individuals who are more comprehensive in their perceptions of the world, prepared to distinguish progressively its different angles, open to different perspectives, ready to incorporate varying dimensions of their encounters into significant and all-encompassing relationships, and willing to trade thoughts with others and to gain assistance from others (Mezirow, 2000).

Transformative Sustainability Learning in Early Childhood Education This chapter agrees with the view that “wisdom was not at the top of the graduateschool mountain, but there in the sandpile at Sunday School” (Fulghum, 1986, p. 6). To address the wicked sustainability problems, arguably the most effective way is to engage with the young children at an early stage (Pramling Samuelsson, 2011; SirajBlatchford, Mogharreban, & Park, 2016) with the purpose of cultivating them as change-makers and models of sustainable behavior with the ability of thinking critically in an enhanced educational paradigm (Davis & Elliott, 2014). As demonstrated in recent research, the significance of “start early” has been highlighted in early childhood education for sustainability (ECEfS) (Boyd, Hirst, & Siraj-

1346

Ş. Feriver et al.

Blatchford, 2017), and other endeavors to further develop young children’s sustainability-related skills raised significant interest in making the case of sustainable living. As corroborated by recent studies on young learners, this new generation seems to possess the potential to make a difference in terms of more sustainable living (Bonnett, 2002). However, in spite of the growing interest in the field in recent years (Hedefalk, Almqvist, & Östman, 2015; Somerville & Williams, 2015), currently there is only scarce information on early childhood education for sustainable development (Siraj-Blatchford, Smith, & Pramling Samuelsson, 2010). Research in this area does not have sufficient involvement of preschool children as participants of a sustainable society (Boldermo & Ødegaard, 2019; Davis, 2009; EPSD, 2010; Pramling Samuelsson, 2016). As one of the main actors in giving direction to the learning processes of young children, the professional preparation of teachers and their continuous professional development are crucial contemplations for education for sustainability to be effectively achieved (Ärlemalm-Hagsér & Sandberg, 2011). A literature review undertaken by the authors of this chapter revealed the significant inadequacy in terms of papers which detail pre-service and in-service teacher education that adopted a sustainability-related transformative learning experience for ECE teachers. This was not a surprising result because this process requires a reconsideration of longheld frames of reference and altering them. Consequently, it is foreseen that new actions need to be undertaken (Barlas, 2001) as a way to critically reflect on social and political issues that may challenge a teacher’s deterministic form of existence (Freire, 1970). This chapter supports the opinion that the contrasts between priorities of the neoliberal education policies and transformative sustainability have been hindering the development of transformative sustainability learning in field of teacher education.

Critical Reflection for Transformative Sustainable Learning Critical reflection is considered as vital to foster transformative learning (Kreber, 2012). Being a reasoning process aiming to make meaning from an experience, it basically hangs a question mark on the validity of a long-taken-for-granted meaning perspective based on a presumption about oneself (Mezirow, 1990). The transformation of the meaning perspectives of an individual or group can be realized through critically reflective assessment, which Mezirow termed as epistemic, sociocultural, and psychic distortions of knowledge (Mezirow, 2000). “It describes the process by which people learn to recognize how uncritically accepted and unjust dominant ideologies are embedded in everyday situations and practices” (Mezirow, 2000, p. 128). Mezirow (1991, 2000) makes a distinction between three types of reflection: content reflection, where the data content is viewed as more profoundly for its accuracy; process reflection, where the systems that created the data are subjected to scrutiny; and premise reflection, which is reflection on basic premises, convictions, and assumptions. Despite its importance in bringing about transformative

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1347

learning, critical reflection is rarely deconstructed in depth when used in the research concerning fostering transformative learning (Taylor, 2017), and there is scarce information regarding the implementation of critical reflection and transformative learning in the teaching of sustainability (Brunnquell, Brunstein, & Jaime, 2015).To address this need, empirical research is presented with a view to examining the interaction of critical reflection with transformative sustainability learning experience created for in-service ECE teachers in Turkey. Coupled with investigating the contributions and effects of critical reflection on participants’ perspective transformation, the study addresses the following key questions: 1. How do Turkish ECE teachers transform their role as teachers through a transformative sustainability learning in-service program? a. What are the participants’ levels of reflection in terms of the provided in-service training? b. What are the participants’ levels of reflection revealed by data collection instruments? 2. What are the features and the themes of the participants’ reflection processes? a. What are the features and themes related to content reflection? b. What are the features and themes related to process reflection? c. What are the features and themes related to premise reflection?

The Study This study is a part of transformative sustainable learning experience research project (Feriver, Teksöz, Olgan, & Reid, 2016) realized with the participation of 24 Turkish early childhood educators in Turkey. In this project, it was aimed to assess the in-service transformative sustainability learning experience which was constructed in accordance with Mezirow’s ten-stage transformative learning approach to offer a viable framework that would encourage early childhood teachers to develop a “learning experience of a different kind” in the context of ESD. The data was gathered through a mixed-method research design using learning diaries, a learning activities survey, and follow-up interviews. The findings revealed the range of transformations that were seen as possible in the teachers’ perspectives during and after the training workshops. One of the influential factors in facilitating perspective transformation was the content, context, and sequencing of the training which allowed continuous critical reflection among the participants. For this reason, the present study examined the interaction of critical reflection with the transformative learning experiences which the participants underwent. While so doing, this study not only focused on individuals but tried to achieve a greater understanding concerning cultural difference and at the same time emphasizing “the individual within his or her socio-cultural context” (Taylor, 2007, p. 185) on the basis that in this respect there is room for theoretical development.

1348

Ş. Feriver et al.

Method The current study utilized a mixed-methods approach for the data collection and analysis within a sequential transformative design framework (Creswell, 2014). The data collection occurred in two phases: first, qualitative data was collected on a daily basis via Learning Diaries (LDs) throughout the training program. Then, at the end of the program quantitative and qualitative data was gathered via a Learning Activities Survey (LAS) and Interview Form (IF). The transformative sustainability learning in-service training program consisted of 21 sessions, lasting 28 h in total, spread over 7 consecutive days. The results from both methods of data collection were combined during the interpretation phase at the end of the study. The three forms of critical reflection presented by Mezirow were taken into account in the design of the content of the in-service training, the way the content was delivered and the type of data collection instruments, and in each of these three basic elements, the participants were provided with opportunities for critical reflection.

Facilitators of Critical Reflection Content of the Training as a Facilitator of Critical Reflection The teachers’ training program was constructed utilizing Mezirow’s ten-stage transformative learning approach (Table 1). The activities were in the following five main sections: (1) the state of the planet and our impact on it; (2) root cause analysis of the dominant paradigms, practices, and power relationships; (3) cradle-to-cradle thinking (McDonough & Braungart, 2002), understanding sustainability, its integration into early childhood education; (4) creation of early childhood ESD projects to be applied in the participants’ educational contexts; and (5) the integration of sustainability into one’s life. In this study, at every stage of the implemented training program, efforts were made to provide the participants with opportunities for critical reflection. At the various stages of the study, opportunities were provided for content reflection through examination of the content and description of the problem, both at the system level and that of the individual. For the process reflection, this was undertaken by focusing on new approaches capable of facilitating the solution of the issue being addressed, and for the premise reflection, by taking a critical view of one’s own distorted presuppositions, whether epistemic, sociocultural, or psychic. The relations between the content of the training and critical reflection opportunities and Mezirow’s three forms of critical reflection are given in tabulated form in Appendix 1. Application Method of the Training as a Facilitator of Critical Reflection To present an example of “learning of a different kind,” the training program aimed to provide the participants with a different experience from the teacher-centered, instrumental learning-oriented in-service training that is widely practiced in Turkey.

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1349

Table 1 Content of the in-service training corresponding to Mezirow’s stages of transformative learning Perspective transformation stages Stage 1 and 2. Disorienting dilemma and self-examination

Stage 3. A critical assessment of epistemic, sociocultural, or psychic assumptions

Stage 4. Recognition of one’s own and others’ discontent and sharing of transformation Stage 5. Exploration of new roles, relationships and actions

Stage 6. Planning a course of action Stage 7. Acquisition of knowledge and skills for implementing one’s own plans

Training content Nine dots: Encouraging thinking outside the box Data discussion: Undertaking an analysis of the factual data regarding the planet, reaching a conclusion regarding the state of the planet Ecological footprint: Providing a tool for participants to understand their own impact on the unsustainable situation of the planet Stations of cause: Undertaking a root-cause analysis of unsustainability Commercials: Deciding whether needs are born out of necessity or merely taken-for-granted assumptions Reading assignment-technology prisons in China: Reflecting on the basic assumptions on production processes Circles: Perceiving interaction among society, economy, and ecology Story of stuff/video film: Discovering the facts behind the current system Trading game: Understanding how economic activity in society has come to dominate the other components of the system Recognizing the discontent of others during the sharing of the process of transformation throughout the training Life of a chair and an apple tree: Discussing the production patterns of simple materials, we use in our daily lives with the production patterns in nature Cradle-to-cradle thinking/Reading assignment: Discovering the details of cradle-to-cradle thinking and its application to real life situations We are building sustainable schools: Discussing sustainable school models Ecological intelligence/Reading assignment: Learning about the new concept of ‘ecological intelligence’ developed by David Goleman Characteristics of a sustainable lesson plan: Deciding on components and characteristics of a sustainable lesson plan by creating rubrics Who told us that we cannot fly a plane?: Constructing our own descriptions of ‘sustainability’ Sustainability eyeglasses: Making relations between sustainability and the preschool learning outcomes prepared by the Ministry of National Education (continued)

1350

Ş. Feriver et al.

Table 1 (continued) Perspective transformation stages Stage 8 and 9. Provisional testing of new roles and building competence and selfconfidence in new roles and relationships

Stage 10. Reintegration into the new perspective

Training content Micro-teaching: Reflecting and applying what has been learnt by presenting a lesson concerning the context of sustainability Traffic lights: Creating preschool sustainability projects and discussing their applicability Expectation that participants will move to stage 10 as a result of the seven-day-training process

The majority of in-service training sessions organized in Turkey take place in a conference-room setting; the participants are passive listeners and interact with one another only during short intervals. Thus, the design of the content of the teacher training in this study included various opportunities for all the participants to actively interact with one another in different ways. This was achieved by seating the participants in small groups facing each other. During every session, the participants were called on to interact with one another, engage in deliberations about various aspects, produce various outputs as a result of these discussions, and share these outputs with all the other participants. Starting with the first activity, the participants played a variety of games to make it easier for them to get to know one another and share their feelings and thoughts. This activity was conducted to overcome personal barriers and facilitate a collaborative learning experience and communicative learning, because “feelings of trust, solidarity, security and empathy are essential preconditions for free full participation in discourse” (Mezirow, 2000, p. 12). It was also considered that playing games might ignite the child-like curiosity of the participants and create suitable grounds for them to question the frames of reference acquired from their early childhood years. The training took place in a state-run preschool located in Sakarya in the northwest of Turkey. The different classrooms and the playground of the preschool in which the training took place were used for various purposes throughout the duration of the training. For example, on the sixth day of the training, in which the participants’ capacities for sustainability were expected to develop, the activity took place in the indoor courtyard. The participants were given pieces of colored paper and were asked to create their own definitions of sustainability, considering what they had acquired from the training, and to write their definitions on the pieces of paper. The pieces of paper were then folded to make airplanes, and these planes were freely launched into the air. As they fell, each participant picked up one of the planes and assessed the sustainability definition that had been written on it, and added his or her comments to the piece of paper. Then the pieces of paper were converted back into planes and launched them again. Once again, each participant opened one of the paper planes that fell near to him or her and added his or her own views and appraisals to the exchange of information that had begun. After this game had been played for a while longer, the activity ended with the paper planes being displayed on the wall of the indoor courtyard.

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1351

Factual evidence was presented in order to reinforce the topics under discussion, to ensure instrumental learning among the participants, and develop their skills. Opportunities were provided for all the participants to confront their own frames of reference and assess their own roles in the development of these frames of reference. All these strategies are considered to achieve the goals of creating an atmosphere of social and intellectual freedom and fostering greater autonomy in terms of reflecting.

Context of the Training as a Facilitator of Critical Reflection The training took place at a time of year when the state preschools were on holiday and the teachers had to take part in in-service training. All the participants were preschool teachers, since it was assumed that (1) teachers of similar professional backgrounds would have similar frames of references, (2) this similarity would facilitate critical reflection on the common frames of reference, and (3) the quality of communication between the participants would be higher as a result of their common profession, and it would be easier for them to empathize with one another. Moreover, the trainer conducting the sessions was also a trained preschool teacher because it was thought that working with a trainer from their own discipline would ensure a quality professional relationship. The other primary elements of the context of the training were the trainer and the participants. The Trainer As shown in the previously published article (Feriver et al., 2016), both the IF and the LD revealed that the trainer was considered one of the main contributors to any perspective transformation. Therefore, it was regarded as meaningful to provide brief information about the trainer as a part of the context of the training. Most of the content of the training activities was holistically designed by the first author (“the trainer”). The trainer is a Turkish citizen and has previously held positions such as senior trainer and project manager in a variety of educational projects supported by international and national funding programs. During these assignments, she gained extensive experience in educational materials, lesson planning, and curriculum development for trainers, teachers, and children. She was the project manager of the Green Pack Project, which was awarded “Good Practice in Education for Sustainable Development in the UNECE Region” by UNESCO. Developed as part of the Green Pack Project with teachers of different subjects Turkey-wide, the trainer implemented the content from the project in the first half of the training content used in this study. For this study, together with the role of training content implementer, she also adopted the roles of co-learner and provocateur in line with the framework of the reformist perspective offered by Cranton (1994). The Participants Twenty-four early childhood educators volunteered for the study; they all worked in various public schools in the town of Sakarya in northwestern Turkey. Most of the teachers (95.8%) involved in the study were female. All the participants had a

1352

Ş. Feriver et al.

university degree and were familiar with the idea and ideals of reflective practice, but they had neither formal nor extensive ESD experience in their teacher education or subsequent professional development. Almost 50% of them were between 20 and 29 years old and almost all of them (95.8%) were less than 40 years old. Seven participants had teaching experience of between one and 4 years, 11 participants had between 5 and 9 years of teaching experience, and 5 participants had teaching experience of 10 and more years. Finally, almost all of the participants had participated in in-service training in the last 18 months.

Data Collection Instruments The following instruments were used for data collection: LAS, LD, and IF. The LAS and IF used in this study were composed of items structured and sequenced in accordance with King’s (2009) recommendations. LD was constructed according to the reflective model created by Rolfe, Freshwater, and Jasper (2001). Learning Diaries According to Taylor, “the strength of using journals is they have the potential to both capture and foster reflection” (2017, p. 83). The participants were encouraged to use their diaries as a space to document their thinking about the workshop issues, ask and explore critical questions, consider the integration of theory with practice and vice versa, and promote reflexive professional development. In addition to enhancing participants’ learning through the process of writing and thinking, the participants completed their LD at the end of each workshop day, responding to four open-ended questions constructed on Rolfe et al.’s (2001) simple three-step reflective model (what, so what, now what). The four questions generated on this model were: What did I do today?, What did I learn today? (what); What were the issues that kept my mind busy today? (so what); and How can I use this experience? (now what). To ensure a private space and confidentiality in the course of critical reflection, the participants chose a pseudonym on the first day of the training and continued to use this pseudonym in LD which they completed every day and LAS which they completed at the end of the program. This made it possible to monitor the transformative journey of each of the participants at the individual level. Learning Activities Survey This instrument was designed to produce quantitative data focused on three dimensions associated with the learning activities. The first dimension utilized Mezirow’s ten stages of perspective transformation and assisted in documenting the participants’ experiences through a checklist. The second dimension solicited views of what might have caused perspective transformation experiences, in relation to the impact of training activities, the influence of other people or the support received, and the changes that occurred in the person’s life. Lastly, the third dimension generated information on the demographic characteristics of the sample. The current study reports the results of the analysis of the sections of this instrument which are related to critical reflection.

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1353

Interview Form The interview protocol was developed to extend the scope and depth of themes of the survey. The excerpts from the IF were also used to corroborate and/or elaborate data collected through LD and LAS. IF comprised two dimensions. The first dimension of IF was related to Mezirow’s ten stages of perspective transformation, while the second explored attributions in perspective transformation.

Data Collection Procedure Data collection through LD took place throughout the workshop period, while for the other instruments (LAS and IF), the data were collected after the completion of the workshop and training activities, at the end of the seventh day. The LD was completed at the end of each day of training, at the training venue, within a period of about 15 min during which the trainer left the participants alone. The completed LDs were then submitted to the trainer, who reviewed the forms regarding the content and context of the training from the point of view of the participants. Where necessary, she made adjustments to the content of the training on the basis of what the participants had written. The LDs were photocopied and the originals were returned to the participants the next day. LAS was completed at the training venue in a 15-minute period after the training had ended. Finally, interviews in Turkish lasting approximately 15–20 min were conducted by the trainer at the training venue. The interviews were conducted with six participants who volunteered immediately after the initial analysis of the LAS data to explore and contextualize the findings from the survey. The purpose of each interview was explained to the participant before the interview was conducted. Each interview session was audiotaped with the permission of the participant and transcribed.

Data Analysis The data in this study was gathered and analyzed in Turkish. Subsequently, for reporting purposes, the interview texts and the codes were translated into English. Extracts from the participants (and the reporting of the study) were scrutinized by a native speaker with translation background to determine whether they were accurate in terms of reflecting the true meaning of a word/phrase. The study used qualitative and quantitative data analysis techniques formulated in three consecutive levels as described in Fig. 1.

Data Analysis Level 1 In the first step of the qualitative analysis, the framework of critical reflection in Mezirow’s theory of perspective transformation was used for systematic coding, coupled with an open coding phase, based on established grounded theory techniques (Strauss & Corbin, 1998). The examination of data was undertaken on the

Ş. Feriver et al.

1354

Data analysis level 1

Data analysis level 2

Data analysis level 3

•Coding responses provided to LD questions provided on a daily basis •Coding responses provided to the LAS open-ended questions given at the end of the training

•Supporting and validating the findings achieved at level 1 via the LAS checklist items

•Corroborating and/or elaborating data collected through the LD and LAS via IF

Fig. 1 The data analysis procedure of the study

basis of the responses to each question posed in LD completed at the end of each training day and the answers to the two open-ended questions posed in LAS. In the design and implementation of the training content since the Mezirowian adult education approach was exhibited in measuring critical reflection, it was decided to conduct the analysis in the framework of Mezirow’s levels of reflection approach. The LD completed by the participants at the end of each training day and their responses to the two open-ended questions in LAS were analyzed by adapting the coding schema developed by Wallman, Lindblad, Hall, Lundmark, and Ring (2008) based on Kember et al.’s (1999) categorization scheme used for assessing learners’ levels of reflection in reflective journals. As opposed to the seven categories used by Kember et al. (1999), Wallman et al. (2008) utilized a modified categorizing scheme based on Mezirow’s six original levels of reflection. In the present study, the researchers returned to the theory as suggested by Taylor and Snyder (2012) and revised the descriptions of the levels based on the theory, content of the intervention, and the data gathered from the field. The analysis units were the responses provided to the open-ended questions in LAS and each of the questions in LD. Since the participants were asked four questions in LD, there were generally four codes for each LD. However, there were also some exceptions; sometimes it was considered appropriate to assign two codes to some responses, and there were situations in which some participants did not answer all the questions in LD; thus, less than four codings were allocated. As shown in Table 3, a high proportion of the participants completed five and six LD, but there were also participants who completed fewer LD. The original version of the coding scheme demonstrated that it had good inter-rater reliability, feasibility, and responsiveness. In the present study, care was taken over inter-rater reliability. The first and second authors of the study coded the data separately according to the coding scheme provided in Appendix 2,

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1355

and the results were compared. The total frequency of the codes was 598, of which the authors agreed on 475 (79%). The discrepancies were discussed and the coding scheme was revised again. After the final version of the coding scheme was developed, the first and the third authors coded 25% of the data to check for interrater reliability. This time inter-rater reliability was calculated as 91%. The different interpretations were further discussed until there was 100% agreement.

Data Analysis Level 2 One of the most common criticisms in the field of critical reflection assessment is that assessments are undertaken utilizing a single data source. In this context, the researchers paid attention to the importance of further validation. In the present study, efforts were made to validate the measurements undertaken by the researchers concerning the participants’ reflective diaries based on the definitions they made using the checklists provided in the first part of LAS. For this purpose, the individual was identified as the unit of analysis and basic descriptive statistics were utilized. Inferences were drawn about the nature of the reflective practices of the participants by observing their individual reflective experiences in conjunction with their individual transformative experiences. Data Analysis Level 3 The responses of six participants to the IF were used to enhance understanding of the role of critical reflection on perspective transformation. As was the case for the other two data analysis instruments, the interviews were primarily intended to support reflective experience, as well as provide an in-depth insight into the data collected through LD and LAS. In addition, appropriate extracts from the participant interviews were presented to further illustrate the critical reflection experiences of the participants.

Findings Assessing the Levels of the Participants’ Reflection and Nonreflection Table 2 summarizes the participants’ levels of reflection on their participation in the in-service training. Table 3 demonstrates the distribution of the codes according to the individual participants. Six different codes were assigned to the responses of the participants under the two categories of non-reflection and reflection. Table 2 shows that all the participants engaged in reflective practices, albeit at different levels (levels 4, 5, and 6). Of the total 598 codings under the non-reflection and reflection categories, 333 were non-reflective and 265 were classified as reflective practices. The distribution of the codes in the non-reflective category showed that the participants frequently displayed habitual actions, giving straightforward descriptions of

Ş. Feriver et al.

1356 Table 2 Distribution of the codes

Category: Non-reflective

Category: Reflective

Codes Habitual action Thoughtful action Introspection Content reflection Process reflection Premise reflection

Frequency

Percentage

Number of participants engaged

208

34.8

24

8.66

82

13.7

24

3.42

43 90

7.2 15.1

16 24

2.69 3.75

120

20.1

23

5.22

55

9.2

20

2.75

Average per participant

Note: The total number of the non-reflective codes was 333, constituting 56% of all the codes. The total number of the reflective codes was 265, constituting 44% of all the codes.

the experiences provided to them. The average number of habitual actions per participant was 8.66. All the participants were also seen to engage in the practice of thoughtful action, noting the choices of action based on the acquired knowledge from the training without mentioning why a certain choice was made or why no interpretation of this choice was offered. This practice was recorded an average of 3.42 times per participant. The practice regarded as non-reflective in which the participants engaged the least was introspection. Seventeen participants engaged in introspection an average of 2.69 times; however, they referred to their feelings during the process without questioning or evaluating why they felt that way. An examination of the distribution of reflective practices shows that process reflection was the commonest category. Apart from one teacher, all the participants engaged in process reflection an average of 5.22 times. It was observed that these participants were intensively focused on problem-solving strategies. During the analysis, the participants were frequently found to use expressions that provided evidence concerning the ways in which they handled certain experiences. The training was also seen to have encouraged all the participants to engage in content reflection. On average, the participants exhibited 3.75 times that they thought about their experience by either interpreting or questioning it. They evaluated the situation in question by examining their own roles on the focused issues. In terms of premise reflection, each of the 20 participants was recorded to have engaged in this practice on an average of 2.75 occasions. The responses of these participants included reflections on how they had become aware of their thoughts, beliefs, feelings, and actions as well as demonstrating that they had become more critical about these. They demonstrated that they had somehow pushed themselves in order to understand and evaluate their underlying assumptions by questioning the root causes of the issues.

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1357

Table 3 Distribution of the codes according to the individual participants Participant 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

# of LDs completed 6 4 4 5 5 5 6 6 6 3 6 6 5 4 6 6 6 6 4 6 6 6 6 6

Habitual action 10 6 4 3 10 4 6 12 7 3 8 14 13 7 10 16 14 10 6 8 8 11 10 8

Thoughtful action 4 3 1 5 4 5 3 3 4 1 5 5 6 1 3 2 4 3 5 1 3 6 3 2

Introspection 1 2 3 4 2 5 6 3 5 0 2 0 0 2 3 1 0 1 0 0 0 2 0 1

Content reflection 5 7 4 3 2 4 2 3 2 1 1 2 3 6 3 2 5 5 3 6 5 6 4 6

Process reflection 6 1 6 5 3 7 9 5 6 6 7 3 0 6 5 4 1 4 3 8 9 4 6 6

Premise reflection 3 0 2 4 0 3 2 2 4 1 4 1 0 1 4 0 1 4 3 3 3 3 3 4

Note: This table includes the coding of the LAS open-ended items, which were completed by all the participants. The total number of the LDs completed was 129.

As demonstrated in Table 3, more than half of the participants completed six LDs, four participants five LDs, and the remaining five participants three or four LDs. Eight participants did not engage in introspection, which is categorized as a non-reflective practice. Only one participant did not provide evidence that could be labeled as process reflection. Four of the participants did not reach the premise reflection level.

Participants’ Levels of Reflection in Relation to the Data Collection Instruments The distribution of reflective practices is detailed in Table 4 and categorized according to the six questions which the participants were asked. In their responses to the question in the LD, “What did I do today?,” the participants were found to engage mainly in a simple description of the happenings (habitual action). While

Ş. Feriver et al.

1358 Table 4 Distribution of the codes according to the LD and LAS questions

LD-what did I do today? LD-what did I learn today? LD-what were the issues that kept my mind busy today? LD-how can I use this experience? LAS-briefly describe what you experienced throughout the in-service training LAS-thinking back to when you realized that your views or perspective had changed, what did attending this in-service training have to do with this experience of change?

Non-reflective Habitual Thoughtful action action 101 2

Introspection 14

Reflective Content reflection 9

Process reflection 2

Premise reflection 3

64

6

4

18

32

14

39

2

18

41

8

16

4

67

0

0

55

7

0

5

5

9

14

9

0

0

2

13

9

6

LD learning diaries, LAS learning activities survey

doing so, it was noticed that some participants also engaged in introspection, referring to the emotions they had felt during the process. Similarly, a high proportion of the answers given to the question “What did I learn today?” were assessed as habitual action. This question can also be said to have prodded the participants to engage in content, process, and premise reflection. Content reflection practices were most commonly found in the answers to the question, “What were the issues that kept my mind busy today?” This was also the question that led the participants to engage in the most premise reflection and introspection. The question “How can I use this experience?” appeared to orient most of the participants toward the practices of thoughtful action and process reflection. Some participants were also observed to engage in premise reflection in response to this question.

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1359

While evidence of both non-reflective and reflective practices was encountered in the responses given to the questions in LD, the responses given to the questions posed in LAS mostly produced evidence of reflective practices. The first LAS question assessed for this study was: “Since you have been participating in this training, do you believe you have experienced a time when you realized that your values, beliefs, opinions or expectations had changed?” The participants who indicated the “Yes” option were directed to the following question: “Briefly describe what you experienced throughout the in-service training.” Most frequently, the response to this question was assessed as process reflection. In the responses which the participants gave to this question, they were observed to mentally travel back to the past and make statements about the way they handled the experience in question. In response to this question, the participants were also observed to engage in premise reflection. In this connection, they were seen to state that they had gained new perspectives by discussing what they had acquired from the content of the training with other participants, broadened their horizons, came to adopt a different perspective, and were observed to comment on the importance of all this. Furthermore, the responses of the participants to the same question showed that they continued to question the situation and engage in examination and interpretation of the problem, i.e., content reflection. The second LAS question assessed was “Thinking back to when you realized that your views or perspective had changed, what did attending this training have to do with the experience of change?” In their responses to this question, for the most part, the participants were observed to engage in content reflection, stating that they had started to think about things which they had never considered before, questioning their own roles in an unsustainable system, and indicating that they were part of the problem. With respect to the same question, the participants also remarked on the particular activities that had started the transformation within them; i.e., they engaged in process reflection. In this context, they mentioned the activity about the apple tree and the chair, the ecological footprint activity and the Story of Stuff video. Like the other LAS question, this question too made it possible for participants to engage in premise reflection, in which they made statements to the effect that the activity had held up a mirror to them, taken them on an internal journey and helped them to grasp issues in a more holistic way. In the following stage of the study, an examination was conducted of the distribution of the codes by the day of training (Table 5) from which many conclusions were drawn. First, the findings revealed that the participants engaged more in reflective practices in their responses to the open-ended questions posed in LAS than when responding to the questions in LD. Second, the participants had engaged in introspection most intensively on the first day of the training but the amount of introspection declined steadily of the remaining days of training. The first day of training was also the day on which the participants engaged least in premise reflection practices. Third, while the degree of content reflection among the participants was similar on each of the first four days of training, it emerged that the evidence of this kind of reflection fell dramatically on the fifth day, and that it was

Ş. Feriver et al.

1360 Table 5 Distribution of the codes according to the days of the training

First day (23 completed journals) Second day (23 completed journals) Third day (23 completed journals) Fourth day (23 completed journals) Fifth day (21 completed journals) Sixth day (16 completed journals) Seventh day (LAS 1 + LAS 2, each filled in by 24 participants)

Non-reflective Habitual Thoughtful action action 27 15

Introspection 15

Reflective Content Process reflection reflection 19 19

Premise reflection 3

41

9

7

16

17

10

36

12

6

15

16

9

31

14

5

14

16

11

44

17

2

4

15

1

28

11

1

0

14

6

0

5

7

22

23

15

The participants were asked four questions in LD for six days and two questions on the seventh day at the end of the training

completely absent on the sixth day. Finally, the highest incidence of premise reflection practices was registered on the second, third, fourth, and seventh days of the training, while the lowest incidence of premise reflection occurred on the first and fifth days.

Participant’s Levels of Reflection and Their Transformative Journey The assessments made by the researchers of the reflective practices in which the participants engaged in their responses to the questions in LD and LAS were then compared with their responses to the items in the LAS checklist. All the participants put a cross by the item, “I had a training experience that caused me to question the way I normally act.” This supports the findings obtained from the LD, which revealed that all of the participants engaged in reflective practices. The research, of which this study is a part, which explores the transformative experiences of the participants and the components of transformative learning that facilitate a

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1361

perspective transformation, thus arrived at the conclusion that all the participants had gained a transformative experience, though not to the same extent. One of the factors that may have affected this outcome was the “self-assessment throughout the training” item in the LAS checklist. Of the 24 participants, 19 selected this item. When the findings from the LAS checklist items were compared with the openended items in LD and LAS, a notable outcome was that the participants who did not engage in introspection in LD or in response to the open-ended questions in LAS did not mark the item, “After the training, I felt uncomfortable with traditional role expectations (values, habits, behavior patterns)” in the LAS checklist either. Another noteworthy finding from this comparison indicated that participants who did not select the item, “I had an experience that caused me to question my ideas about social roles,” were also those who engaged less in premise reflection. Of the 16 participants who engaged in introspection, 13 marked the item, “I began to think about the reaction and feedback from my new behavior.” No clear pattern was identified regarding the other checklist items. Accordingly, the conclusion reached was that the items on the LAS checklist only helped to validate the reflective practices of the participants to a limited extent.

Features of Reflective Practices The next level of analysis explored the features and themes of the participants’ reflection processes. The meanings attached to the participants’ content, process, and premise reflection during and at the end of the training were examined, identified, and described. The themes that emerged most repeatedly from LD and the openended items in LAS are described below with reference to the categories of reflection. In addition, the themes in question are supported by the data from the interviews.

Content Reflection Practices The topic on which the participants most frequently engaged in content reflection involved realizing that the social, political, and economic systems which people have established are unsustainable and understanding how these systems interact with one another. At the same time, this awareness formed the basis for the participants to question their own day-to-day decisions and their places within this system and shown in the following extracts from the participants’ LDs and interviews: I was surprised that I had not thought of unjust behavior toward women and all disadvantaged sections of society as one of the reasons for unsustainability. I realize that in the bustle of everyday life, I had not undertaken this level of analysis. (LD, Participant 2) I see that like everybody else I am not willing to live with less either. And when I run out of the things, I usually buy, I am simply hurrying to buy more too. (LD, Participant 20)

1362

Ş. Feriver et al.

Calculating our own ecological footprints resulted in important changes in my perspective. I realized that my decisions support the unsustainable situation of the planet. I looked at myself in this activity. I evaluated my actions and saw that I was wrong. I started being aware of the impact of my actions on the planet. (IF, Participant 6) The Trading Game showed me that money dominates everything and affects all the other components of the system in an ill-balanced way. We should stop and look at ourselves because we have forgotten what really matters. (IF, Participant 8)

The comments below are examples revealing how the participants realized they only had very limited knowledge about issues of vital importance both for themselves and for all living things. What I saw in the Story of Stuff video really shocked me. I was very upset to learn that in today’s world, even mother’s milk is full of chemicals that are harmful for the baby. Perhaps it would be better not to bring children into this world. (LD, Participant 15) I read the reading assignment [Technology Prisons in China] and I was horrified. This is a humanitarian crisis – mass killing. The technological devices we all use are costing people their lives. I was not aware of these issues. (IF, Participant 4)

Process Reflection Practices When engaging in process reflection, the structure of education was found to be the theme on which the participants commented most frequently. This finding is supported by the fact that 18 participants selected the item “unconventional structure of the training” when completing the checklist in LAS of factors that might influence them having had a transformative experience. Within this theme, the sub-theme that emerged most frequently concerned play as shown in these extracts. The games we played were both very entertaining and very thought-provoking. Thanks to the games, we integrated with each other very quickly and very effectively and entered into constructive discussions. (LD, Participant 6) The games made me want to get away from the usual ways of thinking. (LD, Participant 21)

The games are also understood to have helped to keep the participants involved in the process after the intensive feelings they began to experience in the early days of the training. After what I saw, I became even more pessimistic. I think every era creates its own kind of person. I don’t have much faith that the people of our era can solve these deep-rooted problems that they have created. On the other hand, I’m learning things here that I had never heard about before, and I’m enjoying myself very much. Every day I come to the training sessions very eager and excited. (LD, Participant 9)

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1363

The second most prominent sub-theme with respect to the unconventional structure of the training relates to the personal relationships developed both with the trainer and between the participants. IF and LD revealed that the democratic and playful structure of the training removed the personal barriers among the participants and facilitated collaborative learning, which helped the participants to develop a multi-perspective approach. During the sustainability airplanes activity, I saw how the same subject could lead to different kinds of conclusions when considered by different people. At the same time, I noticed that the group discussions enriched the available options. (LD, Participant 24) I noticed that by treating life and events as multi-dimensional, and looking at them from many different angles, and sometimes thinking outside the box, more productive results can be generated. (LAS open-ended item, Participant 21)

The other theme of process reflection that occurred in LD in all the participants and supported by the interviews was that they discovered the potential of teachers to be agents of change, both as teachers and as individuals, and they were making plans to fulfill this role: I realized that there are a lot of things that I can do as a preschool teacher. I can make a big difference with small interventions. I have started to carry out research to broaden and deepen my knowledge of these issues. I will continue looking for resources throughout the summer holiday. When school starts, I am going to start integrating sustainability into the curriculum on the one hand and put these issues onto the agenda of parents through their family education on the other. (LD, Participant 14) If it hadn’t been for this training, I would have remained an individual inside my own shell. As an individual on my own, I wouldn’t have been able to do anything. But the training has encouraged me now. Every day, after the training, I have told my flat mate about what I have learned and what we discussed. Soon I will be meeting up with my family and I am going to discuss these things with them too. I can’t wait for the schools to open. I want to work with the children and carry out projects with the other teachers in my own preschool straight away, before my enthusiasm wanes. (IF, Participant 7)

Premise Reflection Practices The conclusion has been reached that the participants who engaged in premise reflection in their LD revealed in their responses to the open-ended questions in LAS that their assumptions, values, thoughts, and beliefs are actually determined to a large extent by the political, social, and economic contexts in which they live: Thanks to this training, I have confronted bitter truths. It turns out I am just one of the cogs in this rotten order centering on economics. I realize that I have been trundling along in the imaginary world that they serve up for us. How helpless I was. (LAS open-ended item, Participant 15)

1364

Ş. Feriver et al.

Thus, the participants also realized how little they knew about the forces that direct the global society they form part of. Moreover, they recognized that their actions were enacted solely on a surface-level basis, composed of commonly accepted beliefs, or sometimes even distorted facts, but they had become critical of this situation: To be honest, I hadn’t been able to go into these things in so much detail. As I went more deeply into the issues that we dealt with, I gained a better understanding of the extent of the mistakes I am making. I realized that I think and question very little. (LAS open-ended item, Participant 6)

As a result of this awareness, the participants indicated that they had made plans to change their priorities, and they felt more enabled and empowered in this respect because they had become more aware of the external influences acting on them: I have become aware that the things I describe as obstacles are actually my own obstacles. This training has made me redefine my role as a teacher in this society. As a teacher, I now believe that my duty to leave a sustainable future to our children and to raise them to be able to build a sustainable future is more important than all my other duties. (LD, Participant 21) We, teachers, are the engines of the education system. We are the role models for our students. If we want to affect our students’ lives to improve them, we have to change our mindset to reach our ‘inner selves’. (LD, Participant 12) The training as a whole made a lot of difference in terms of how I look at myself and my profession. All the activities complemented each other and guided us throughout the process. At the beginning, I was terrified, now I am hopeful and full of energy that I can change my conventional point of view. (IF, Participant 4)

Discussion and Implications for Education, Practice, and Research This study presents a framework for critical reflection that teacher educators can employ to support and analyze ECE teachers’ reflection practices by paying attention to the individual transformative learning experiences of teachers created by the content and the process for critical reflection opportunities within a capacity building course. The objectives in this research were to assess the levels of reflection and breadth, concentrating on the elements of the reflective process and depth of the understanding of the reflective discourse. Toward the goal of assessing the levels of reflection of participants, the coding pattern created by Wallman et al. (2008) based on the work of Kember et al. (1999) was marginally overhauled by integrating the emphasis of the social change in transformative learning that was initially based on a political, social, and ideological critique (Kreber, 2012). Furthermore, the meanings of the codes were expanded by including the extracts from the participants in the

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1365

study. As a result of this adaptation, the anticipated outcome was achieved based on Mezirow’s six steps model which was developed for the purpose of reflective discourse within the transformative learning process.

Critical Reflection and Individual Differences The findings of the study indicated that critical reflection is an individualistic process. The participants in this study were given the same training content and they were all provided with the same critical reflection opportunities. However, considerable differences were recorded in terms of the critical reflection processes in which each participant engaged. In parallel with this outcome, Mann, Gordon, and MacLeod (2009), in the review of 29 different studies conducted about reflective practice in the framework of health professional education, came to the conclusion that the tendency and ability to reflect appears to differ among individuals. It is considered that there may be many reasons for that situation and the leading reasons are elaborated here. The first cause of the individual differences in reflective practices is thought to be personality. By nature, some participants may be more inclined to engage in critical reflection; thus, they may have a pattern of a particular kind of critical reflection. Similarly, in our study, there were findings in some participants’ data referring to feelings during the process but did not question or evaluate the reasons for those feelings, this was seen in the data from eight participants. Another example revealed that despite same conditions being provided, four participants did not perform premise reflection. This pattern shows that as in the process of transformative learning, the process of critical reflection also varies among people. One of the reasons for detected individual differences could be the resistance exhibited by some of the teachers who participated in the training to critical educational practices (Gay & Kirkland, 2003; Johnson, 2006) and ideological implications derived from this approach (Neumann, 2013). As Neumann (2013) explained, some of the participants might “continue to follow, and thus tacitly endorse, the common script not just in reaction to institutional pressures, but also from an emotional desire to fit into mainstream notions about teachers” (p. 140). Another explanation that sheds light on the relationship between critical reflection and individual differences could be the reasoning presented by Cranton (1994), who extended the variety of the way individuals engaged in transformative learning by focusing on Jung’s (1971) model of psychological types. According to this approach, people with an extraverted attitude prefer to engage in interaction with others, events, situations, and information, while introverted types have a tendency toward indirect stimulation from the world which consists of an inner set of processes. Jung (1971) also referred to the preferences in making judgments, using logic (thinking) and using values (feeling). He described the reality of senses and intuition as two ways of perceiving claiming that every person has attitudes, preferences, and perceptions that differ and work together in various ways. In

1366

Ş. Feriver et al.

relation to individuals’ reaction to transformative learning, Cranton (1994) explained that feeling types had the tendency toward making changes in their values and their perspectives through value-based judgments whereas thinking types were not necessarily the most likely to revise their meaning perspectives through focusing on their feelings. To collaborate this explanation, the responses given by the participants to the questions posed in LDs can be cited as examples. Among the answers to the first question of LD, “What did I do today,” for which a rather simplistic description could suffice, some participants preferred to engage in reflective practices at different levels, while other participants voiced their concerns which can be categorized as introspection. As argued by Mälkki (2010) based on the work of Mezirow (1981, 1991, 2000, 2009), far from being an easy process, reflection is an arduous process which can be painful for some people. The reflective abilities of the participants in the current study may be affected by internals factors, such as emotional maturity (Mezirow, 2000), metacognitive abilities (Mälkki, 2010), and the capacity for critical thinking (Bourner, 2003). Another reason for the individual differences could be that while probably being reflective in their work and possessing the ability to engage in reflective thinking, some participants might not be able to formulate this process in a written form. This is supported by Kreber (2005) who referring to instructors of science wrote that “it is possible that they really engage in reflection but do not know how to show it” (p. 352). In the current study, it is also likely that some participants may hesitate in writing down what they thought no matter the level of their reflection. As demonstrated by Grant, Kinnersley, Metcalf, Pill, and Houston (2006), some learners were less motivated as a result of the non-alignment between the written approach to assessment and the learners’ favored learning style. Similarly, in relation to the motivational approach, it was shown that the perceived importance of reflection can successfully predict the time and effort a person is willing to invest, meaning that those who do not expect a positive return are unlikely to reflect profoundly and critically (Sandars, 2009). These determinations, as corroborated by the findings of this research, could provide some of the answers to why some participants completed less LDs than others. From this perspective, it can be argued that it is necessary to offer participants, in addition to writing, varied different mediums, through which they can refer to their own learning styles and also duly express themselves (Lundgren & Poell, 2016; Taylor, 2017). It is important to acknowledge the potential of social desirability bias that might derive from the confluent relationship between the trainer and the participants that emerged throughout the training. Although while collecting data, special care was taken to keep the participants’ identities anonymous in order to address that bias, it is thought that this aim might not be fully met in the interviews. Once more, this has demonstrated the importance of triangulation of data sources in terms of validating the findings. In creating this triangulation, rather than depending solely on the details derived from the reflecting person, utilizing other methods such as making observations to lessen assessors’ dependency on a person’s interpretative description (Koole et al., 2011) can bring about insightful outcomes.

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1367

Content for the Critical Reflection The current research offered a framework for a specific content for reflection to be considered critical. The participants in this study were exposed to a learning experience which is distinctly different from their previous practices. When looking at the distribution of the reflective and non-reflective practices according to the day of the training and comparing them with the content of the training as reflective opportunities provided within the session, it seems that together with the training content, the questions asked by the trainer which aimed to provoke reflection had very noticeable impact on the reflective processes of the participants. This study occupies a distinct position in relation to many other reflection studies in terms of its emphasis on the content of the transformative experience. For example, as demonstrated by a review which focused on 11 research articles in higher education across a range of disciplines, only two of the 11 studies indicated a higher percentage of students being reflective (Dyment & O’Connell, 2011). In these studies, the content concerning which students perform reflection was not questioned, whereas, as mentioned in the beginning of this study, the main objective of contemporary higher education is to equip learners with information and skills; in other words, to develop instrumental learning. Moreover, learners in HE are only provided with communicative learning opportunities in a limited fashion. In such conventional learning environments where instrumental learning is the most important goal, it is no surprise that a high level of reflection fails to occur. In this study, it was recorded that the participants performed considerably more intense premise reflection on the second, third, and fourth days, on which the situation of the planet earth was discussed from various angles in the education content which was prepared with the objective of instrumental, communicative, and emancipatory learning. It was during these days when, on an individual basis, participants were given intensive premise reflection opportunities, in which they could scrutinize the impact of the socioeconomic and political systems on individuals by uncovering hegemonic assumptions (Brookfield, 2017), undertake a rootcause analysis, and enter into a rigorous process of self-questioning. Thus, when abundant opportunities were provided, the participants performed more premise reflection. Also, during these days, a significantly higher number of provoking questions were posed to participants in comparison with the other days. Therefore, it can be argued that when designing interventional studies, it is advisable to consider the kind of reflection opportunities that should be provided to obtain better results. As mentioned previously, the sequence of the content of the education was designed in accordance with Mezirow’s transformative learning stages. Thus, given the parallels of the content of the education and Mezirow’s transformative learning components, it is thought that there may be a relation or correlation between the disorienting dilemma; self-examination; critical assessment of epistemic, sociocultural, or psychic assumptions; and exploration of new roles, relationships, actions, and premise reflection. This determination, however, requires further validation since, currently, there seems to be more research available in the form of disorienting dilemmas (Mälkki, 2012) or critical incidents (Cope & Watts, 2000).

1368

Ş. Feriver et al.

Process for the Critical Reflection In this study, it is argued that rather than focusing solely on the content that may improve teachers’ reflective practices, teacher educators should also concentrate on the process of this specific challenge. The findings of this study concerning the process of critical reflection indicate that certain types of questions posed to participants in written forms directed them toward certain kinds of critical reflection practices. Furthermore, it was found that the participants were more engaged in critical reflection in their responses to the questions asked on the last day of the training, when their views on the entire process were sought. Apparently, during the process of completing LD, the content was too controversial for some of the participants to consciously reflect-in-action (as the incident happens) (e.g., Eraut, 1994). In the current study, the utilization of the three-step reflective model of Rolfe et al. (2001) produced positive, albeit limited results. The questions which helped participants to take a critical approach to what they learnt as opposed to questions asking what they had done produced better results in terms of critical reflection. Thus, it is thought that by means of questions concerning what they had learnt, some form of mental time-travel opportunities was made available which helped them to remember the reflective moments. Furthermore, this process could also help participants to engage their metacognitive skills. In response to the question of “What were the issues that kept my mind busy today?” the participants engaged intensively in content reflection. The question of “How could I use this experience” led the participants to become involved in process reflection. Based on the reflective writing of the participants, it was seen that the questions in LD led to findings that showed the role of emotions on critical reflection, and transformative learning occurred in a limited manner. It can be argued that in addition to “What did I do today” and “What did I learn today,” the question of “How did I feel today” could also produce more findings related to the affective side of critical thinking. Also, it is considered that adding “why” to the questions in the three-step model of Rolfe et al. (2001) could help learners to undertake a higher level of critical reflection. Nevertheless, it is advisable to take into account the risk that some participants may be alienated from the process if their limits are tested by this level of critical reflection. Taking all the above-mentioned factors into consideration, directing questions to the participants which can help them see the bigger picture at the end of their transformative learning experiences; formulating questions in a way that assists participants in viewing the process as a whole; asking them what they learnt in the process, what kept their minds busy, and how they feel during the process; and finally complementing this process with “why” questions in order to show them how to put into practice what they learnt can facilitate learners to engage in high-quality critical reflection experiences. The context and application method of the training presents significant insights in the framework of process of the critical reflection. As argued by Taylor (2017), when investigating transformative learning, there is an emphasis on the individual which

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1369

often overshadows the important details about the context. As revealed in this study, elements embedded in the context of the training could have motivated some participants to move in the direction of engaging in critical reflection. This professional development study conducted with Turkish early childhood teachers is a very fitting example for less-controlled, less-formalized, non-judgmental, and stimulating adult education which offers different opportunities for reflection. It was concluded that as with transformative learning, critical reflection should also be realized with free will as suggested by Mezirow (2000). As explained, in this study, the participants were given considerable freedom in completing LD and LAS. Accordingly, reflection practices which are performed using free will and pseudonyms might offer participants a sense of freedom to write more deeply and critically without the fear of judgment (Dyment & O’Connell, 2011). Hence, as demonstrated in the findings, during those 6 days, a significant portion of the participants completed their LD responding to all the questions and answered all the open-ended items in LAS. While reflection was generally considered as an entirely singular process, ideas are moving toward conceptualizing it as a process facilitated by social collaboration interaction (Li, Paterniti, Co, & West, 2010). Also, within this study, it was observed that social interaction played a prominent role in participants’ process reflections (Brookfield, 2006) and playfulness was one of the factors which facilitated this. This playfulness created such a supportive atmosphere that participants were able to more easily overcome the obstacles when faced with experiences that were “unexpected, unfamiliar, surprising, and perhaps even disturbing” (Kreber, 2012, p. 330). As manifested in play in child development theories, play can also be utilized for adults as a tool to help deal with negative feelings, due to its ability to create a private world that can be shared with others, but remains separate from the real world and ordinary time and space (Tanis, 2012). In this context, it is considered that play has the potential to bring about abundant opportunities. Hence, taking advantage of such possibilities, how to include play/playfulness in adult learning should be further explored, including conducting independent studies about the potential of play/playfulness to add new capacity in terms of critical reflection and transformative learning.

Conclusion This study presented a way of encouraging early childhood teachers to engage in critical practice toward transformative learning to offer them insights into more sophisticated understanding regarding the teaching profession to compensate for the current deficiencies in higher education institutions (HEI). A professional development opportunity to transform teachers’ way of learning was created so that the participants could become transformative intellectuals acting as change agents, question how knowledge is produced and distributed, utilize dialogue, and make knowledge meaningful, critical, and emancipatory. Based on the research questions,

Ş. Feriver et al.

1370

the study aimed to provide a content and process understanding of critical reflection as an important tool for teacher educators with the objective of stimulating transformative learning. In doing so, new possibilities were explored as to how to support the teachers who had been educated within HEI as “technicians rigidly and obediently following a prescribed curriculum” (Liu, 2015, p. 136) to achieve an awareness to look beyond the micro classroom environment of their teaching to a broader context of education, schooling, and society. As demonstrated in this study, transformative learning intensively supported with critical reflection practices, in a framework of sustainability, may provide educators and teachers with opportunities to critique and change their conventional approaches to ECE teaching and learning. A central argument in this study is that as a result of their overall experiences throughout the training process, the participants were significantly more inclined to undertake critical reflection. The unconventional training content, unconventional application methodology, and the social and intellectual atmosphere created during the training encouraged the participants to engage in critical reflection. This process was supported with the utilization of the data collection instruments which furthered the opportunities for critical reflection. In countries, such as Turkey, which are at an early stage in ESD and cannot provide sufficient exposure opportunities during the existing teacher education program, supplementary in-service training such as demonstrated in this study can be offered. However, this might not be sufficient; therefore, concurring with the foremost limitation of the study, namely the time and space needed to develop ESD practice, it is recommended that there is an implementation of broader transformative learning experiences and/or system-wide strategies rather than one-off attempts or interventions. The findings of this study showed that such actions will be instrumental in enhancing the competence and bolstering the self-confidence of early childhood teachers in transformative learning, and perspective transformation geared toward sustainability.

Appendix 1: Critical Reflection Opportunities in the Training in Relation to Mezirow’s Three Forms of Critical Reflection

Timing of the sessions First day

Training content and critical reflection opportunities (the questions posed by the trainer to the participants throughout the sessions) Nine dots: You were instructed to join up nine dots using four straight lines without lifting your pen from the paper. None of you were able to complete the task, because you did not consider the possibility that you could draw lines which extended beyond the box formed by the nine dots. Yet you were not given any instruction to the effect that you could not go outside this box. Now let’s think about what other examples of this exist in our everyday life. What might such situations stem from?

Mezirow’s three forms of critical reflection Content reflection Premise reflection

(continued)

54

Achieving Education for Sustainable Development (ESD) in Early. . .

Timing of the sessions

Second day

Third day

Fourth day

Training content and critical reflection opportunities (the questions posed by the trainer to the participants throughout the sessions) Data discussion: Based on the factual data provided, what can you say about the state of life on our planet? What other examples can you give from your own experience that support or contradict this information? Ecological footprint: What do you think about the situation that has emerged? Is knowing this important for you? Why? What do you think about your own ecological footprint? What made you think like that? What did this process lead you to think about your own choices? Stations of cause: Among the causes of unsustainability, which ones did you find most and least important? How did you select these causes? Are the experiences you have had about the causes of unsustainability in line with your previous assumptions? If so, in what ways do they match? If not, in what ways do they clash? Do you feel any need to review your previous assumptions at this point? Commercials: How do you construct your values? How are your consumption habits formed? Commercials create the impression that consumption brings happiness. What do you think about this? Life of a chair and an apple tree: What are the similarities and differences between these two life cycles? When you look at them as a whole, how do these two systems operate? What kind of inferences can you make about your own daily lives on this basis? Cradle-to-cradle thinking: What do you think about this model? What might be the reasons why we people do not put this model into practice? What would have to be done for this model to become a part of our lives? Story of stuff/video film: What kinds of cause-andeffect relationships did you observe in what you watched? What kind of place do you think these relationships occupy in our daily lives? Was there any moment when you felt surprised, sad, happy, or disappointed? Did you become aware of any connection between what you were watching and your own lives? Was there any moment that made you question your own situation? Can you tell more about it? What do you think about the impact of social and economic norms? After watching this film, did you feel any need to reconsider the way you have come to look on the global system? Trading game: This activity is a version of the production patterns that exist today turned into a game. By playing the game, the participants were

1371

Mezirow’s three forms of critical reflection Content reflection

Content reflection Premise reflection

Content reflection Premise reflection

Content reflection Premise reflection

Content reflection Premise reflection

Process reflection

Content reflection Process reflection Premise reflection

Content reflection Process reflection Premise reflection (continued)

Ş. Feriver et al.

1372

Timing of the sessions

Fifth day

Sixth day

Training content and critical reflection opportunities (the questions posed by the trainer to the participants throughout the sessions) given the chance to experience the way in which competition and the desire to gain increases in the level of production, the dominant role of money in this process, and the uneven distribution of resources. Later, the game was used to demonstrate that there is an underlying economic basis to human-made systems, and lead the participants to think about the difference between needs and wants, about sustainable production and about the various elements of social justice. At the end of the game, the following questions were asked: How did you feel during the game? How did you get on with one another? What kind of relationship did you have with the banker? Did the groups act in accordance with the economy or did they direct the economy in line with their own needs? Is this situation true to life? In your view, how much production was sufficient? Did the groups treat each another fairly? Was money more important? Or was the important thing always to have more? Circles: What kinds of relationship do you think there are between these three components? What is your thinking based on? What kind of a cause-and-effect relationship can you establish between the three components? We are building sustainable schools: What do you think about the sustainable school models that have been generated? Within these models, what do you think could be put into practice and what do you think could not be put into practice? Why do you think like that? What could be done to make these models more workable? Characteristics of a sustainable lesson plan: After all this process, what do you think would be the salient characteristics of a lesson plan developed within the framework of sustainability? Based on these plans, do you think you could develop and implement sustainable lesson plans for your own students? What would help you to do this? Who told us that we cannot fly planes?: When you consider your definitions of sustainability and the discussions of these definitions, do you observe anything that you had not noticed before? What kind of inferences did you draw from this activity? At the end of the day, has there been any change in your original definition of sustainability? What factors might have influenced this? Sustainability eyeglasses: What kind of connection can be made between the ECE curriculum and sustainability? Did you notice this connection before? If not, what might have led to this awareness?

Mezirow’s three forms of critical reflection

Content reflection

Process reflection Premise reflection

Process reflection

Content reflection Process reflection Premise reflection

Content reflection Process reflection

(continued)

54

Achieving Education for Sustainable Development (ESD) in Early. . .

Timing of the sessions Seventh day

Training content and critical reflection opportunities (the questions posed by the trainer to the participants throughout the sessions) Micro-teaching and traffic lights: Do you believe that these lesson plans and sustainable education projects can be put into practice? What sort of obstacles might you face in this respect? What can you do to overcome these obstacles?

1373

Mezirow’s three forms of critical reflection Process reflection

Appendix 2: The Categorization Scheme (Wallman et al., 2008) Used for the Analysis of Learning Diaries and Learning Activities Open-Ended Questions 6. Premise reflection 5. Process reflection 4. Content reflection 3. Introspection 2. Thoughtful action 1. Habitual action

Reflective

Non-reflective

Non-reflection 1. Habitual action. Habitual action is an unconscious act that takes place without thought and can be performed at the same time as another act. A description of an act performed without thought or having to focus could be, for example, driving a car. A description of the course of events can be categorized as habitual action. For example: “I started to learn names of the other participants. We discussed the relationship among human-money-tree, we made drawings about that. We played the trading game which replicates the competition among countries. We used stations technique to evaluate the past three days of the training.” 2. Thoughtful action. Thoughtful action draws upon existing knowledge with no critical appraisal. The starting point lies in the previously existing knowledge, and choices between different alternatives regarding how to perform the task are made either unconsciously or not at all. Why a certain choice is made is not questioned and no interpretation is made. No thought is given to the consequences of this particular choice. An example of this is a description of the participant how she/he is going to use the experience she/he gained from the training “Due to what I’ve learned from this training, I have decided to cut back on my consumption, buy something only if I need it, be more conscious about my consumption habits.” 3. Introspection. Introspection refers to thoughts about oneself, one’s own thoughts or feelings about performing a task. There is no comparison between the actual

1374

Ş. Feriver et al.

task and/or one’s previous experiences, nor are there any thoughts as to why these feelings occur or what they might lead to. An example of this is a description of how it feels to learn something, or how the participant feels when she/he faced the critical facts about the planet “Frankly, I became very pessimistic. I always keep my hope alive. But I am so pessimistic that I am thinking of myself as someone who harbors a faint hope wishing that a single candle could illuminate the entire room” or how she participants felt about the involvement of other participants “During the activities today, I saw that there is no difficulty which could not be overcome by friendship and solidarity. I really appreciated the stations technique.”

Reflection The definition of reflection as it is used below is that a situation is identified in relation to an actual experience. This problem must somehow be analyzed in order for the task to be executable. Previous knowledge is used in the specific situation and is questioned and criticized when necessary. 4. Content reflection. Content reflection pertains to what one perceives, thinks, or feels, or how one acts when undertaking a task. There should be a questioning or an interpretation of a behavior in order to be categorized as reflection, otherwise it is most often categorized as “2. Thoughtful action.” While engaging in content reflection, the person consciously thinks about the problem, his/her role on the examined problem and what she/he needs to do to solve the actual problem. This is similar to asking, “What is happening here? What is the problem?” (Cranton, 2006, p. 34). She/he does not, however, reflect upon why the action taken works or how his/her own behavior developed. What effect the thought, feeling, or act may have should be discussed. For example, “During the course of this training I realized that I started to think about the issues that had never crossed my mind before. I never imagined what a large ecological footprint I have. It never occurred to me that I had such an ecological impact on the planet.” 5. Process reflection. Process reflection refers to how one performs the functions of perceiving, thinking, feeling, or acting, and to an assessment of the efficacy of the performance. There should be a proposal for, or an interpretation of, problem solving for a categorization as process reflection. For example, the participant explains his/her ideas on how to integrate sustainability issues into his/her curriculum and she/he further thinks how this change might work out. In comparison with content reflection, there is more focus on problem-solving strategies: “It is asking questions of the form, how did this come to be?” (Cranton, 2006, p. 34). Reflection of process can also contain reflection of the person’s feelings and actions, as well as what she/he has been doing to handle the experience. For example, “I got involved in deep discussions with my colleagues which in itself was a new experience for me. The setting of the training enabled us to acknowledge and respect diverse perspectives. Consequently, I realized that my own

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1375

perspective has also been broadened. I believe that thanks to my enhanced ability of approaching a subject from various angles, my workshops with the parents would produce results of better quality.” One’s thoughts and beliefs about how the thought, feeling, or act has an effect should be discussed in addition to how others apprehend the act. For example, “I was terrified when I read the reading assignment about the technological prisons in China. I read it to my husband, then also to my neighbor. This piece of news had such an impact on me that I believed that we should disseminate it to as many people as possible.” 6. Premise reflection (Theoretical reflection). Premise reflection relates to why one apprehends, thinks, feels, or acts the way one does and the consequences of that existing knowledge sets the framework for how a person acts in different situations. This should include an analysis of the whole situation/problem, including the root-causes by incorporating the answers to the “what,” “how,” and “why” questions. The political, cultural, and social contextual factors should be considered so that they can be included in a deeper understanding or reinterpretation of the problem. If the participant explains that she/he will consider alternative methods such as changing his/her behavior patterns, she/he should also justify and interpret this new choice of action. While doing that becoming aware of the answers to “Why is this important to me? Why do I care about this in the first place?” (Cranton, 2006, p. 34) questions by examining deeply held assumptions about how an individual makes meaning of his or her self and the world is also critical as in the following example “When I reviewed over the issues we discussed during the training, I noticed that we are in fact in a vicious circle, a global exploitation setup. Nothing is what it seems. What have we turned into? On top of that, I was also one of those who has been feeding into this vicious circle and exploitation setup. As a matter of fact, we have a very good example to look at. Nature offers us all the answers we are looking for. I am thinking of simplifying and deepening my perspective and life style.”

References Ärlemalm-Hagsér, E., & Sandberg, A. (2011). Sustainable development in early childhood education: In-service students’ comprehension of the concept. Environmental Education Research, 17 (2), 187–200. Balsiger, J., Förster, R., Mader, C., Nagel, U., Sironi, H., Wilhelm, S., & Zimmermann, A. B. (2017). Transformative learning and education for sustainable development. GAIA: Ecological Perspectives for Science and Society, 26(4), 357–359. Barlas, C. (2001). Learning-within-relationship as context and process in adult education. In Fortysecond annual adult education research conference and proceedings. East Lansing, MI: Michigan State University. Belenky, M. F., & Stanton, A. V. (2000). Inequality, development, and connected knowing. In J. Mezirow (Ed.), Learning as transformation: Critical perspectives on a theory in progress (pp. 71–102). San Francisco, CA: Jossey-Bass.

1376

Ş. Feriver et al.

Blake, J., Sterling, S., & Goodson, I. (2013). Transformative learning for a sustainable future: An exploration of pedagogies for change at an Alternative College. Sustainability, 5(12), 5347–5372. Boldermo, S., & Ødegaard, E. E. (2019). What about the migrant children? The state-of-the-art in research claiming social sustainability. Sustainability, 11(2), 459. Bonnett, M. (2002). Education for sustainability as a frame of mind. Environmental Education Research, 8(1), 9–20. Bourner, T. (2003). Assessing reflective learning. Education and Training, 45(5), 267–272. Boyd, D. J., Hirst, N. J., & Siraj-Blatchford, J. (2017). Understanding education for sustainability across the UK. London, England: Routledge. Brookfield, S. D. (2006). The skillful teacher: On trust, technique and responsiveness in the classroom. San Francisco, CA: Jossey-Bass. Brookfield, S. D. (2017). Becoming a critically reflective teacher (2nd ed.). San Francisco, CA: Jossey-Bass. Brunnquell, C., Brunstein, J., & Jaime, J. (2015). Education for sustainability, critical reflection and transformative learning: Professors’ experiences in Brazilian administration courses. International Journal of Innovation and Sustainable Development, 9(3), 321–342. Cope, J., & Watts, G. (2000). Learning by doing: An exploration of experience, critical incidents and reflection in entrepreneurial learning. International Journal of Entrepreneurial Behavior & Research, 6, 104–124. Cranton, P. (1994). Understanding and promoting transformative learning. San Francisco, CA: Jossey-Bass. Cranton, P. (1996). Types of group learning. New Directions for Adult and Continuing Education, 71, 25–32. Cranton, P. (2006). Understanding and promoting transformative learning: A guide for educators of adults (2nd ed.). San Francisco, CA: Jossey-Bass. Cranton, P., & King, K. P. (2003). Transformative learning as a professional development goal. New Directions for Adult and Continuing Education, 98, 31–37. Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage. Daloz, L. A. (1999). Mentor: Guiding the journey of adult learners (2nd ed.). San Francisco, CA: Jossey-Bass. Davis, J., & Elliott, S. (Eds.). (2014). Research in early childhood education for sustainability: International perspectives and provocations. London, England: Routledge. Davis, J. M. (2009). Revealing the research ‘hole’ of early childhood education for sustainability: A preliminary survey of the literature. Environmental Education Research, 15(2), 227–241. Dawe, G., Jucker, R., & Martin, S. (2005). Sustainable development in higher education: Current practice and future developments—A report for the Higher Education Academy. York, England: Higher Education Academy. Dyment, J. E., & O’Connell, T. S. (2011). Assessing the quality of reflection in student journals: A review of the research. Teaching in Higher Education, 16(1), 81–97. Eraut, M. (1994). Developing professional knowledge and competence. London, England: Falmer. Escrigas, C. (2016, June). A higher calling for higher education. Great Transition Initiative. Retrieved from http://www.greattransition.org/publication/a-higher-calling-for-higher-education European Panel on Sustainable Development (EPSD). (2010). Taking children seriously – How the EU can invest in early childhood education for a sustainable future. Report No. 4. Göteborg, Sweden: The Centre for Environment and Sustainability, GMV. Retrieved from http://www.ufn. gu.se/digitalAssets/1324/1324488_epsd_report4.pdf Feriver, S., Teksöz, G., Olgan, R., & Reid, A. (2016). Training early childhood teachers for sustainability: Towards a ‘learning experience of a different kind’. Environmental Education Research, 22(5), 717–746. Freire, P. (1970). Pedagogy of the oppressed. New York, NY: Herder and Herder.

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1377

Freire, P. (1985). The politics of education: Culture, power, and liberation. South Hadley, MA: Bergin & Garvey. Freire, P. (1997). Education for critical consciousness. New York, NY: The Continuum. Freire, P. (1998). Pedagogy of freedom: Ethics, democracy, and civic courage. Lanham, MD: Rowman & Littlefield. Fulghum, R. (1986). All I really need to know I learned in kindergarten: Uncommon thoughts on common things. Great Britain, UK: Grafton Books. Gay, G., & Kirkland, K. (2003). Developing cultural critical consciousness and self-reflection in preservice teacher education. Theory into Practice, 42, 181–187. Glisczinski, D. J. (2007). Transformative higher education: A meaningful degree of understanding. Journal of transformative education, 5, 317–328. Grant, A., Kinnersley, P., Metcalf, E., Pill, R., & Houston, H. (2006). Students’ views of reflective learning techniques: An efficacy study at a UK medical school. Medical Education, 40(4), 379–588. Habermas, J. (1984). The theory of communicative action. Boston, MA: Beacon. Hanushek, E., Rivkin, S. G., & Kaim, J. F. (2005). Teachers, students, and academic achievement. Econometrica, 73(2), 417–458. Harmin, M., Barrett, M. J., & Hoessler, C. (2017). Stretching the boundaries of transformative sustainability learning. On the importance of decolonizing ways of knowing and relations with the more-than-human. Environmental Education Research, 23(10), 1489–1500. Hedefalk, M., Almqvist, J., & Östman, L. (2015). Education for sustainable development in early childhood education. A review of the research literature. Environmental Education Research, 21 (7), 975–990. https://www.tandfonline.com/doi/abs/10.1080/13504622.2014.971716 Hoggan, C. D. (2016). Transformative learning as a metatheory: Definition, criteria, and typology. Adult Education Quarterly, 66(1), 57–75. Huckle, J., & Wals, A. E. (2015). The UN Decade of Education for Sustainable Development: Business as usual in the end. Environmental Education Research, 21(3), 491–505. Johnson, A. G. (2006). Privilege, power, and difference (2nd ed.). Boston, MA: McGraw-Hill. Jung, C. G. (1971). Collected works, volume 6: Psychological types. Princeton, NJ: Princeton University Press. Kember, D., Jones, A., Loke, A., McKay, J., Sinclair, K., Tse, H., . . . Yeung, E. (1999). Determining the level of reflective thinking from students’ written journals using a coding scheme based on the work of Mezirow. International Journal of Lifelong Education, 18, 18–30. King, K. P. (2009). Handbook of the evolving research of transformative learning: The learning activities survey (10th anniversary ed.). Charlotte, NC: Information Age. Koole, S., Dornan, T., Aper, L., Scherpbier, A., Valcke, M., Cohen-Schotanus, J., & Derese, A. (2011). Factors confounding the assessment of reflection: A critical review. BMC Medical Education, 11, 104–114. Kreber, C. (2005). Reflection on teaching and the scholarship of teaching: Focus on science instructors. Higher Education, 50(2), 323–359. Kreber, C. (2012). Critical reflection and transformative learning. In E. W. Taylor & P. Cranton (Eds.), Handbook of transformative learning: Theory, research, and practice (pp. 323–341). San Francisco, CA: Jossey-Bass. Kubow, P. K., & Fossum, P. R. (2007). Comparative education: Exploring issues in international context. Upper Saddle River, NJ: Pearson. Ladson-Billings, G. (1998). Just what is critical race theory and what’s it doing in a nice field like education? International Journal of Qualitative Studies in Education, 1, 7–24. Lambrechts, W., & Van Petegem, P. (2016). The interrelations between competences for sustainable development and research competences. International Journal of Sustainability in Higher Education, 17(6), 776–795. Li, S. T., Paterniti, D. A., Co, J. P., & West, D. C. (2010). Successful self-directed lifelong learning in medicine: A conceptual model derived from qualitative analysis of a national survey of pediatric residents. Academic Medicine, 85(7), 1229–1236.

1378

Ş. Feriver et al.

Liu, K. (2015). Critical reflection as a framework for transformative learning in teacher education. Educational Review, 67(2), 135–157. Lundgren, H., & Poell, R. F. (2016). On critical reflection: A review of Mezirow’s theory and its operationalization. Human Resource Development Review, 15(1), 3–28. Mälkki, K. (2010). Building on Mezirow theory of transformative learning: Theorizing the challenges to reflection. Journal of Transformative Education, 8(1), 42–62. Mälkki, K. (2012). Rethinking disorienting dilemmas within real-life crises: The role of reflection in negotiating emotionally chaotic experiences. Adult Education Quarterly, 62, 207–229. Mann, K., Gordon, J., & MacLeod, A. (2009). Reflection and reflective practice in health professions education: A systematic review. Advances in Health Sciences Education, 14, 595–621. McDonough, W., & Braungart, M. (2002). Cradle to cradle: Remaking the way we make things. New York, NY: North Point. McLaren, P. (2003). Life in schools: An introduction to critical pedagogy in the foundations of education (4th ed.). Boston, MA: Allyn & Bacon. Mezirow, J. (1981). A critical theory of adult learning and education. Adult Education Quarterly, 32, 3–24. Mezirow, J. (1990). How critical reflection triggers transformative learning. In J. Mezirow (Ed.), Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning (pp. 1–20). San Francisco, CA: Jossey-Bass. Mezirow, J. (1991). Transformative dimensions of adult learning. San Francisco, CA: Jossey-Bass. Mezirow, J. (2000). Learning to think like an adult. In J. Mezirow (Ed.), Learning as transformation: Critical perspectives on a theory in progress (pp. 3–33). San Francisco, CA: Jossey-Bass. Mezirow, J. (2009). An overview on transformative learning. In K. Illeris (Ed.), Contemporary theories of learning. Learning theorists . . . in their own words (pp. 90–105). London, England: Routledge. Moore, J. (2005). Is higher education ready for transformative learning? A question explored in the study of sustainability. Journal of Transformative Education, 3, 76–91. Neumann, J. (2013). Critical pedagogy’s problem with changing teachers’ dispositions towards critical teaching. Interchange, 44(1/2), 129–147. Nieto, S., & McDonough, K. (2011). “Placing equity front and center” revisited. In A. F. Ball & C. A. Tyson (Eds.), Studying diversity in teacher education (pp. 363–384). Lanham, MD: Rowman & Littlefield. Orr, D. W. (2004). Earth in mind: On education, environment and the human prospect. Washington, DC: Island Press. Picower, B. (2013). You can’t change what you don’t see: Developing new teachers’ political understanding of education. Journal of transformative education, 11, 170–189. Pramling Samuelsson, I. (2011). Why we should begin early with ESD: The role of early childhood education. International Journal of Early Childhood, 43, 103–118. Pramling Samuelsson, I. (2016). What is the future of sustainability in early childhood? In A. Farrell, S. L. Kagan, & E. K. M. Tisdall (Eds.), The Sage handbook of early childhood research (pp. 502–516). London, England: Sage. Pramling Samuelsson, I., & Park, E. (2017). How to educate children for sustainable learning and for a sustainable world. International Journal of Early Childhood, 49, 273–285. Rickinson, M. (2006). Researching and understanding environmental learning: Hopes for the next 10 years. Environmental Education Research, 12(3–4), 445–457. Ripple, W. J., Wolf, C., Newsome, T. M., Galetti, M., Alamgir, M., Crist, E., . . . 15,364 scientist signatories from 184 countries. (2017). World scientists’ warning to humanity: A second notice. BioScience, 67(12), 1026–1028. Rolfe, G., Freshwater, D., & Jasper, M. (2001). Critical reflection in nursing and the helping professions: A user’s guide. Basingstoke, England: Palgrave Macmillan. Sandars, J. (2009). The use of reflection in medical education: AMEE Guide No. 44. Medical Teacher, 31(8), 685–695.

54

Achieving Education for Sustainable Development (ESD) in Early. . .

1379

Sauter, M. B., & Frohlich T. C. (2013, October 15). The most educated countries in the world. 24/7 Wall Street. Retrieved from http://247wallst.com/special-report/2013/10/15/the-most-educatedcountries-in-the-world-2/ Schumacher, E. F. (1997). This I believe’ and other essays. Dartington, England: Green Books (essay first published in 1974). Siraj-Blatchford, J., Mogharreban, C., & Park, E. (2016). International research on education for sustainable development in early childhood. Dordrecht, the Netherlands: Springer. Siraj-Blatchford, J., Smith, K. C., & Pramling Samuelsson, I. (2010). Education for sustainable development in the early years. Organisation Mondiale Pour L‘Education Prescolaire (OMEP). Retrieved from http://www.327matters.org/Docs/ESD%20Book%20Master.pdf Somerville, M., & Williams, C. (2015). Sustainability education in early childhood: An updated review of research in the field. Contemporary Issues in Early Childhood, 16(2), 102–117. Sterling, S. (2001). Sustainable education. Devon, England: Green Books. Sterling, S. (2004). Higher education, sustainability, and the role of systematic learning. In P. B. Corcoran & A. E. J. Wals (Eds.), Higher education and the challenge of sustainability: Problems, promise and practice (pp. 49–70). Dordrecht, the Netherlands: Kluwer Academic Publishers. Sterling, S. (2010). Transformative learning and sustainability: Sketching the conceptual ground. Learning and Teaching in Higher Education, 5, 17–33. Sterling, S. (2017). Assuming the future: Repurposing education in a volatile age. In B. Jickling & S. Sterling (Eds.), Post-sustainability and environmental education: Remaking education for the future (pp. 31–45). London, England: Palgrave Macmillan. Sterling, S., & Thomas, I. (2006). Education for sustainability: The role of capabilities in guiding university curricula. International Journal Innovation and Sustainable Development, 1(4), 349–370. Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: Sage Publications, Inc. Tanis, D. J. (2012). Exploring play/playfulness and learning in the adult and higher education classroom (Unpublished doctoral dissertation). The Pennsylvania State University, Pennsylvania. Taylor, E. W. (2007). An update of transformative learning theory: A critical review of the empirical research (1999–2005). International Journal of Lifelong Education, 26(2), 173–191. Taylor, E. W. (2017). Critical reflection and transformative learning: A critical review. PAACE Journal of Lifelong Learning, 26, 77–95. Taylor, E. W., & Snyder, M. (2012). A critical review of research on transformative learning. In E. W. Taylor & P. Cranton (Eds.), The handbook of transformative learning: Theory, research and practice (pp. 37–55). San Francisco, CA: Jossey-Bass. Thornton, B., Peltier, G., & Perreault, G. (2004). Systems thinking: A skill to improve student achievement. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 77(5), 222–227. Tilbury, D. (2011). Education for sustainable development. An expert review of processes and learning. Paris, France: United Nations Education, Scientific and Cultural Organization (UNESCO). UN. (2015). Transforming our world: The 2030 agenda for sustainable development. Resolution adopted by the General Assembly on 25 September 2015. Retrieved from https://sustainablede velopment.un.org/post2015/transformingourworld UNESCO. (2017a). Education for sustainable development: Partners in action. Paris, France: Author. UNESCO. (2017b). Education for sustainable development goals: Learning objectives. Paris, France: Author. Union of Concerned Scientists. (1992). World scientists’ warning to humanity. Cambridge, MA: Union of Concerned Scientists. Retrieved from https://www.ucsusa.org/sites/default/files/attach/ 2017/11/World%20Scientists%27%20Warning%20to%20Humanity%201992.pdf

1380

Ş. Feriver et al.

Wallman, A., Lindblad, A. K., Hall, S., Lundmark, A., & Ring, L. (2008). A categorization scheme for assessing pharmacy students’ levels of reflection during internships. American Journal of Pharmaceutical Education, 72(1), 1–10. Wals, A. E. J. (2006). The end of ESD. . .the beginning of transformative learning. Emphasizing the ‘E’ in ESD. In M. Cantell (Ed.), Proceedings of the seminar on education for sustainable development held in Helsinki. Wals, A. E. J. (2015). Beyond unreasonable doubt – Education and learning for socio-ecological sustainability in the anthropocene. Inaugural address held upon accepting the personal Chair of Transformative Learning for Socio-Ecological Sustainability at Wageningen University. Woodrow, K., & Caruana, V. (2017). Preservice teachers’ perspective: Transformations as social change agents. Journal of Transformative Education, 15(1), 37–58. Wright, T. S. A. (2002). Definitions and frameworks for environmental sustainability in higher education. Higher Education Policy, 15, 105–120. WWF. (2018). Living planet report 2018: Aiming higher. Gland, Switzerland: WWF International.

Şebnem Feriver earned her Ph.D. in Early Childhood Education from Middle East Technical University. She has been working as project manager and senior trainer for various national and international projects. Her research interests are transformative learning, lifelong learning, teacher education, and education for sustainability. Refika Olgan is Associate Professor in Early Childhood Education at the Middle East Technical University in Turkey. Her research interests include science education in early years, environmental education, education for sustainable development, and teacher education. Gaye Teksöz is a Professor of Education for Sustainable Development at the Middle East Technical University in Turkey. Her research interests are focused on the theory and applications of environmental education, education for sustainable development, and climate change education for sustainability. Her research has been published in a wide range of international journals, including Environmental Education Research, International Research on Geographical and Environmental Education, and International Journal of Science Education.

Transformative Experience: A Critical Review and Investigation of Individual Factors

55

Kevin J. Pugh, Cassendra M. Bergstrom, Leah Wilson, Sarah Geiger, Jacqueline Goldman, Benjamin C. Heddy, Simon Cropp, and Dylan P. J. Kriescher

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Is a Transformative Experience? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dewey’s Aesthetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining and Illustrating Transformative Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Why Is Learning Transformative for Some Students but Not Others? . . . . . . . . . . . . . . . . . . . . . . . . Existing Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Speculations and Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exploratory Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Study 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Study 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1382 1383 1383 1385 1387 1387 1388 1397 1397 1397 1401 1407 1408 1409

K. J. Pugh (*) · C. M. Bergstrom · L. Wilson · S. Cropp · D. P. J. Kriescher School of Psychological Sciences, University of Northern Colorado, Greeley, CO, USA e-mail: [email protected]; [email protected]; [email protected]; simon. [email protected]; [email protected] S. Geiger Department of Graduate Psychology and Counseling, Andrews University, Berrien Spring, MI, USA e-mail: [email protected] J. Goldman Division of Counselor Education and Psychology, Delta State University, Cleveland, MS, USA e-mail: [email protected] B. C. Heddy Department of Educational Psychology, Oklahoma University, Norman, OK, USA e-mail: [email protected] © Springer Nature Switzerland AG 2023 J. M. Spector et al. (eds.), Learning, Design, and Technology, https://doi.org/10.1007/978-3-319-17461-7_155

1381

1382

K. J. Pugh et al.

Abstract

Transformative experiences are characterized by application of school content in everyday experience, expansion of students’ perception of the world, and increased value for aspects of the world re-seen through the lens of science content. An open question is why some students undergo transformative experiences and others do not. In this contribution, we review current research findings and propose promising areas for future research including investigating the role of interest, emotions, task values, dispositions and personality traits, a willingness to “surrender,” epistemological beliefs, and supportive out-of-school contexts. We then present the results of two initial studies targeting some of these future directions. In study 1, we investigated interest and personality traits as predictors of transformative experience. Maintained, but not triggered, situational interest was a significant predictor. The personality trait of openness to experience was also a significant predictor, and this relationship was fully mediated by maintained situational interest. In study 2, we investigated positive and negative emotions, anxiety, and task values as predictors of transformative experience. We used cluster analysis to develop profiles of affect and value. These profiles differed significantly in levels of transformative experience. The level of transformative experience was highest among students who reported high levels of positive emotions and task values with lower levels of negative emotions, anxiety, and perceived cost. Keywords

Transformative learning · Transformative experience · Science education · Motivation · Engagement · Interest · Dewey

Introduction “We live in a state of permanent third gradeness. We want this to be something that feels like it is yours again.” This comment comes from Jad Abumrad, one of the hosts of Radiolab, an American science-oriented radio program produced by New York Public Radio (WNYC, 2008). Jad and his co-host Robert Krulwich have a knack for making science come alive in their programs. In the above quote, Jad is referring to one of the strategies they use when producing programs: be third grade. His point is that most adults express a moderate interest in science. But when asked if they were ever interested in science, they often comment nostalgically about their early education years when they loved science. For many of us, our first experience with science was a magical time of wonder and imagination. Take McKinley, the oldest daughter of the first author, as an example. She lived in an imaginary world of science from the ages of about 3 to 5. She believed she was Saturn and would carry around her stuffed cat “Titan” all day telling stories about her moons and the other planets. When she wasn’t Saturn, she was lava. She would tell

55

Transformative Experience: A Critical Review and Investigation of. . .

1383

stories about wanting to get out of the earth and how she would burn down a house when she got out. Everything in her world became lava too. Oil poured in a bowl to make zucchini bread. Lava. Honey poured on a peanut butter sandwich. Lava. Syrup poured on pancakes. Lava. McKinley became fascinated with telling stories about lava, seeing things that might look or act like lava, and trying to makes sense of what lava was. At one point she commented, “I wish I could read about and learn about lava all the time. I wish I could go to your school and learn about the earth. . .I like the earth book [How the Earth Works] better than your other books because it has things I say ‘why’ about” (recorded as part of graduate student project many years ago). What is it that is magical about these early science encounters? There are probably many answers to that question and many ways of trying to capture the essence of such experiences (see Hadzigeorgiou, 2016). In our work, we use Dewey’s (1938, 1980/1934) theory of aesthetic and educative experience as a guide combined with ideas from contemporary research on motivation and transfer. In doing so, we define a particular type of experience with science, one characterized by passion, engagement, and transformation. We refer to such experience as transformative experience (Pugh, 2011). Of particular interest to us is how to support such experiences in adolescent and adult learners. That is, how can we help older individuals have magical experiences with science like they did in third grade? One approach to answering this question is to investigate why some individuals undergo transformative experiences and others do not. Consequently, the purpose of this chapter is to review existing research on individual factors related to undergoing transformative experience and identify additional factors likely to be predictive of transformative experience. A further purpose is to investigate the relation of some of these proposed factors to transformative experience among adult learners.

What Is a Transformative Experience? Because many individuals are unfamiliar with transformative experience theory (e.g., Pugh, 2011; Wong, Pugh, and the Dewey Ideas Group at Michigan State University, 2001), we provide a brief background on the theory. We review pertinent aspects of Dewey’s theory of aesthetic and educative experience and illustrate how this work gave rise to the transformative experience theory. We then provide a definition and illustration of transformative experience as a research construct.

Dewey’s Aesthetics Is disco an art? Is disco dancing an art at the same level as, say, classical ballet? There is no right answer to that question, but we suspect Dewey would be inclined to say, “Yes, yes it is.” John Dewey was a great American philosopher and educator. He (1980/1934) wrote a book on aesthetics, Art as Experience, that begins with his concerns about the separation of art from everyday experience. He was concerned that as art became more formalized, as it attained “classic status,” it became more cut

1384

K. J. Pugh et al.

off from everyday experience. He feared that art would become something isolated in the theater, concert hall, or museum. Art would be ballet at the theater, but not dance in everyday contexts. This was highly problematic for Dewey because he believed the whole point of art was to enrich and expand everyday experience. As he wrote in Art as Experience, “[Art] quickens us from the slackness of routine and enables us to forget ourselves by finding ourselves in the delight of experiencing the world about us in its varied qualities and forms” (1980/1934, p. 110). Art should be transformative. Dewey proposed that such transformation was at the heart of a particularly important type of experience exemplified by the arts, to which he gave the underwhelming name of “an” experience. Jackson (1998) explained: Our interactions with art objects epitomize what it means to undergo an experience, a term with a very special meaning for Dewey. The arts do more than provide us with fleeting moments of elation and delight. They expand our horizons. They contribute meaning and value to future experience. They modify our ways of perceiving the world, thus leaving us and the world itself irrevocably changed. (p. 33)

Take the art of Andy Warhol as an example. Warhol made art out of ordinary objects like a Campbell’s soup can. His point was to show people that ordinary, everyday objects are wonderful if looked at from an aesthetic perspective. For many people, such art was transformative. Among those affected was the art critic, Danto (1992), who provided this account of how pop art transformed his experiencing of the everyday world: I have the most vivid recollection of standing at an intersection in some American city, waiting to be picked up. There were used-car lots on two corners with swags of plastic pennants fluttering in the breeze and brash signs proclaiming unbeatable deals, crazy prices, insane bargains. There was a huge self-service gas station on a third corner, and a supermarket on the fourth, with signs in the window announcing sales of Del Monte, Cheerios, Land O’ Lakes butter, Long Island ducklings, Velveeta, Sealtest, Chicken of the Sea. . .I was educated to hate all this. I would have found it intolerably crass and tacky when I was growing up an aesthete. . .But I thought, Good heavens. This is just remarkable! (pp. 139–140)

Likewise, a play such as Phantom of the Opera may transform the way we see and think about such things as inspiration, beauty, and obsession. According to Dewey, art achieves its greatest purpose when it transforms the way we see and experience the world. So what does this all have to do with education, particularly science education? Well, Dewey’s ideas on education closely parallel his ideas on art. Dewey (1938) was concerned that education was becoming disconnected from everyday experience in the same way that art was becoming disconnected from everyday experience. Further, he believed that the purpose of education was much the same as the purpose of art: to enrich and expand everyday experience. We often cite Dewey in claiming that experience should be a means to learning while failing to realize that Dewey also believed that learning should be a means to experiencing, to living richer, more

55

Transformative Experience: A Critical Review and Investigation of. . .

1385

expansive lives. Finally, Dewey’s descriptions of meaningful learning are not so different from his characterization of “an” experience. In fact, Dewey may have pursued aesthetics late in his career, not so much because he was into art but because he was seeking to make sense of what particularly meaningful and transformative experiences are all about. Pugh and colleagues (Girod, Rau, & Schepige, 2003; Pugh, 2011; Pugh & Girod, 2007; Wong, 2007; Wong et al., 2001) have written more extensively about the connections between Dewey’s aesthetics and his ideas on education. Their central point is that science ideas can transform how we see and experience the world in a way similar to how art does in the context of “an” experience. Further, they proposed the concept of transformative experience as a more precise way of defining this learning outcome. In the following section, we summarize this definition and provide examples of transformative experiences. For