Biochemistry education: from theory to practice 9780841236332, 084123633X

Citation preview

1337

UPGRADE THE WAY YOU TEACH BIOCHEMISTRY This volume brings together resources from the networks and communities that contribute to biochemistry education. Projects, authors, and practitioners from the American Chemical Society (ACS), American Society of Biochemistry

E D U C A T I O N

VOLUME 1337

BIOCHEMISTRY EDUCATION FROM THEORY TO PRACTICE

ACS SYMPOSIUM SERIES

ACS SYMPOSIUM SERIES

BIOCHEMISTRY EDUCATION FROM THEORY TO PRACTICE

and Molecular Biology (ASBMB), and the Society for the Advancement of Biology Education Research (SABER) are included to facilitate cross-talk among these communities. Authors offer diverse perspectives on pedagogy, and chapters focus on topics such as the development of visual literacy, pedagogies and practices, and implementation.

PUBLISHED BY THE

American Chemical Society SPONSORED BY THE

ACS Division of Chemical Education

BUSSEY et al.

BUSSEY, LINENBERGER CORTES & AUSTIN

Biochemistry Education: From Theory to Practice

ACS SYMPOSIUM SERIES 1337

Biochemistry Education: From Theory to Practice Thomas J. Bussey, Editor Department of Chemistry and Biochemistry University of California San Diego La Jolla, California

Kimberly Linenberger Cortes, Editor Department of Chemistry and Biochemistry Kennesaw State University Kennesaw, Georgia

Rodney C. Austin, Editor Department of Chemistry, Mathematics, and Physics Geneva College Beaver Falls, Pennsylvania

Sponsored by the ACS Division of Chemical Education

American Chemical Society, Washington, DC

Library of Congress Cataloging-in-Publication Data Library of Congress Cataloging in Publication Control Number: 2019042629

The paper used in this publication meets the minimum requirements of American National Standard for Information Sciences—Permanence of Paper for Printed Library Materials, ANSI Z39.48n1984. Copyright © 2019 American Chemical Society All Rights Reserved. Reprographic copying beyond that permitted by Sections 107 or 108 of the U.S. Copyright Act is allowed for internal use only, provided that a per-chapter fee of $40.25 plus $0.75 per page is paid to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. Republication or reproduction for sale of pages in this book is permitted only under license from ACS. Direct these and other permission requests to ACS Copyright Office, Publications Division, 1155 16th Street, N.W., Washington, DC 20036. The citation of trade names and/or names of manufacturers in this publication is not to be construed as an endorsement or as approval by ACS of the commercial products or services referenced herein; nor should the mere reference herein to any drawing, specification, chemical process, or other data be regarded as a license or as a conveyance of any right or permission to the holder, reader, or any other person or corporation, to manufacture, reproduce, use, or sell any patented invention or copyrighted work that may in any way be related thereto. Registered names, trademarks, etc., used in this publication, even without specific indication thereof, are not to be considered unprotected by law. PRINTED IN THE UNITED STATES OF AMERICA

Foreword The purpose of the series is to publish timely, comprehensive books developed from the ACS sponsored symposia based on current scientific research. Occasionally, books are developed from symposia sponsored by other organizations when the topic is of keen interest to the chemistry audience. Before a book proposal is accepted, the proposed table of contents is reviewed for appropriate and comprehensive coverage and for interest to the audience. Some papers may be excluded to better focus the book; others may be added to provide comprehensiveness. When appropriate, overview or introductory chapters are added. Drafts of chapters are peer-reviewed prior to final acceptance or rejection. As a rule, only original research papers and original review papers are included in the volumes. Verbatim reproductions of previous published papers are not accepted. ACS Books Department

Contents Preface .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 

ix

Visual Literacy 1. Quantifying the Types of Representations Used in Common Biochemistry Textbooks .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  3 Kimberly Linenberger Cortes, Chloe House, Rhodelle Lewis, Shreya Krishnan, Kimberly Kammerdiener, Morgan Tamayo, and Thomas J. Bussey 2. Virtual Exploration of Biomolecular Structure and Function .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  21 Shuchismita Dutta and Daniel R. Dries 3. Physical Models Support Active Learning as Effective Thinking Tools .  . . . . . . . . . . . . . . . . . . . . . . . . . .  43 Cassidy R. Terrell, Margaret A. Franzen, Timothy Herman, Sunil Malapati, Dina L. Newman, and L. Kate Wright Pedagogies and Practices 4. Skills and Foundational Concepts for Biochemistry Students .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  65 Ellis Bell, Joseph Provost, and Jessica K. Bell 5. Implementing Guided Inquiry in Biochemistry: Challenges and Opportunities .  . . . . . . . . .  111 Jennifer Loertscher and Vicky Minderhout 6. The Development and Use of Case Studies .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  127 Sarah Baas Robinson, Erin Dolan, Kathleen Cornely, Amy E. Medlock, Jin Kyu Lee, and Paula P. Lemons 7. Development and Use of CUREs in Biochemistry .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  143 Joseph J. Provost, Jessica K. Bell, and John E. Bell 8. Lab eNotebooks .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  173 Keri Colabroy and Jessica K. Bell 9. Formative Assessment to Improve Student Learning in Biochemistry .  . . . . . . . . . . . . . . . . . . . . . . . . . . .  197 Erika G. Offerdahl and Jessie B. Arneson 10. Best Practices in Summative Assessment .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  219 Heather L. Tienson-Tseng

vii

Implementation 11. Don’t Go It Alone: The Importance of Community and Research in Implementing and Maintaining Innovative Pedagogy .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  247 Rodney C. Austin and Tracey Arnold Murray 12. Vignette #1: Introducing Active Learning to Improve Student Performance on Threshold Concepts in Biochemistry .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  261 Mary A. Kopecki-Fjetland 13. Vignette #2: Making a Switch to In-Class Activities in the Biochemistry Classroom .  . . .  275 Emily J. Ragan 14. Vignette #3: Developing Student Proficiency in Reading Biochemical Literature .  . . . . . . .  291 Rhonda J. Scott 15. Vignette #4: Implementation of POGIL-Inspired Experiments in the Biochemistry Laboratory .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  295 Kyle R. Willian 16. Vignette #5: Incorporating Topic-Focused, Student-Derived Projects in the Biochemistry Laboratory Curriculum .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  313 DeeAnne M. Goodenough-Lashua Editors’ Biographies .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  329 Indexes Author Index .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  333 Subject Index .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  335

viii

Preface Biochemistry, as a discipline, represents the unique intersection of the exploration of the physical and chemical principles that mediate and constrain biological phenomena. As an inherently interdisciplinary field, biochemistry does not have a single academic home (1). In some institutions, biochemistry is found in Biology Departments. In other institutions, biochemistry is found in Chemistry or Chemistry and Biochemistry Departments; and in other institutions still, biochemistry is found within Departments of Biochemistry or professional school. Additionally, biochemists and biochemistry educators are spread over three main professional organizations: the American Chemical Society, the American Society for Biochemistry and Molecular Biology, and the Society for the Advancement of Biology Education Research. The diversity of organizational structures may well lend itself to the diversity of biochemical research. However, as an instructional pursuit, biochemistry education is equally intersectional and subsequently fragmented. No single guiding framework for the undergraduate biochemistry curriculum currently exists. As such, it was the intention of the Editors to bring together a broad group of biochemistry educators from across these various professional societies to bridge the multiple efforts and activities of these organizations and researchers and to bring these authors into a joint dialog. This work originated with a symposium of Biochemistry Education at the 2014 Biennial Conference on Chemical Education (BCCE). This symposium was well received and large enough that it was split into two symposia: one on the lecture learning environment and one on the laboratory learning environment. These symposia series were repeated at the 2016 and 2018 BCCEs (and will be organized again for the 2020 conference). As part of the American Chemical Society’s Symposium Series, this book is intended to highlight the work presented through this symposium series. However, the Editors felt that this was also an opportunity to engage the broader biochemistry education community in a more comprehensive and inclusive conversation about the state of biochemistry education. This book is organized around the translation of theory into practice. The first section addresses a common theme in science education: visual literacy. The educational research around the use of visualizations in science is well documented. However, biochemistry has a particular disciplinary practice that requires fluency in the visual language(s) of biochemistry. Chapter 1 presents an overview of the various aspects of visual literacy in biochemistry education and focuses in on the variety of representations presented to biochemistry students in introductory textbooks, i.e., static two-dimensional representations characterized by the Taxonomy of Biochemistry External Representation. Chapter 2 explores translation from two-dimensional to three-dimensional structures. The ability for students to visualize and analyze this type of structural data is highly dependent on the types of and access to virtual interactive tools. Finally, Chapter 3 discusses the implications for the use of 3-D printing to make biomolecular structures visible and tangible for biochemistry students. The second section of this book is organized around the pedagogies and practices of biochemistry education. Chapter 4 begins this section by presenting a holistic view of the curriculum ix

for biochemistry and molecular biology majors in light of the evolution of the various professional societies that have influenced this curriculum. Chapter 5 presents the first of several pedagogical perspectives on biochemistry education. The use of guided inquiry as an evidence-based instructional strategy is discussed. Chapter 6 goes on to discuss a second evidence-based instructional strategy, i.e., the use of case-based learning. The development and iteration of case studies along with their implementation in large lecture settings is discussed. Chapter 7 explores the use of course-based undergraduate research experiences as a strategy aimed to integrate more authentic research experiences into the teaching laboratory environment. Chapter 8 continues the discussion of the laboratory learning environment through an exploration of the use of electronic lab notebooks, a practice ubiquitous in industry. Finally, Chapters 9 and 10 present a discussion of formative and summative assessment practices in biochemistry. The third section of this book presents a unique window into the biochemistry classroom. Chapter 11 presents a perspective on the community of practice necessary to promote and maintain innovative, evidence-based instructional strategies in biochemistry. Chapters 12–16 then present a series of vignettes intended to present the myriad of implementation strategies used by faculty to make strategic and meaningful instructional changes in their classrooms. In total, this book is intended to spark conversation amongst the broader community of biochemistry educators and spur continued collaboration and development of the biochemistry curriculum, pedagogy, and assessment.

References 1.

Huang, P. C. The integrative nature of biochemistry: challenges of biochemical education in the USA. Biochem. Educ. 2000, 64–70.

Thomas J. Bussey, Ph.D., Assistant Teaching Professor Department of Chemistry and Biochemistry University of California, San Diego 9500 Gilman Drive #0314 La Jolla, California 92093-0314, United States Kimberly Linenberger Cortes, Ph.D., Associate Professor of Chemistry Department of Chemistry and Biochemistry Kennesaw State University 370 Paulding Ave NW Kennesaw, Georgia 30144, United States Rodney C. Austin, Ph.D., Associate Professor of Chemistry Department of Chemistry, Mathematics, and Physics Geneva College 3200 College Avenue Beaver Falls, Pennsylvania 15010, United States

x

Visual Literacy

Chapter 1

Quantifying the Types of Representations Used in Common Biochemistry Textbooks Kimberly Linenberger Cortes,*,1 Chloe House,1 Rhodelle Lewis,1 Shreya Krishnan,1 Kimberly Kammerdiener,1 Morgan Tamayo,2 and Thomas J. Bussey3 1Department of Chemistry and Biochemistry, Kennesaw State University,

370 Paulding Avenue NW, Kennesaw, Georgia 30144, United States 2Analytics and Data Science Institute, Department of Statistics and Analytical Sciences, Kennesaw State University, 370 Paulding Avenue NW, Kennesaw, Georgia 30144, United States 3Department of Chemistry and Biochemistry, University of California San Diego, 9500 Gilman Drive, La Jolla, California 92093, United States *E-mail: [email protected].

Biochemistry education relies heavily on students’ ability to visualize abstract cellular and molecular processes, mechanisms, and components. As such, biochemistry educators often turn to external representations to provide tangible, working models from which students’ internal representations (mental models) can be constructed, evaluated, and revised. Prior studies have looked at the impact of representations on students understanding of biochemistry. Additional work in this area has looked at the types of representations used to teach and assess biochemistry concepts. However, no study to date has looked at the representations available to both instructors and students in biochemistry textbooks. The study presented here analyzed the representations found in the top four purchased one-semester biochemistry textbooks on Amazon in spring 2015. Using the Taxonomy of Biochemistry External Representations (TOBER), representations were coded as symbolic, macroscopic, particulate, microscopic, or schematic. Results show that, regardless of textbook, particulate representations are most commonly used. Additionally, schematic images are typically used in conjunction with at least one additional type of representation particularly when depicting concepts related to metabolism.

© 2019 American Chemical Society

Introduction Visual Literacy and External Representations in Biochemistry Education Consider the visual nature of the language used by biochemists. Anecdotally, biochemists may refer to the beauty of a protein structure or the symmetry of an inverted repeat or the contours of an active site when describing the complementarity between enzymes and substrates. Science is often expressed as visual representations: graphs, flow charts, models, and pictures (1, 2); and scientists often use representations in their work to, inter alia, present data or visualize a phenomenon. Illustrations allow for the diagrammatic representation of complex relationships and processes that may not be easily expressed in words. As such, the ability of students to comprehend, evaluate, and construct visual representations in science is a necessary skill (3). Schönborn and Anderson (4) define visual literacy as “the ability to read (understand or make sense of) and write (draw) […] including the ability to think, learn, and express oneself in terms of images”. Visual literacy can promote the development of knowledge that students may not be able to acquire from text alone (5). Biochemists demonstrate expert visual literacy by decoding, evaluating, interpreting, manipulating, and constructing external representations (6). They also display other cognitive skills related to visual literacy, including the ability to translate between multiple external representations and between the various levels of organization, as well as the ability to “visualize orders of magnitude, relative size, and scale” (6). Clearly, the ability to construct meaning from visual representations (visual literacy) is a necessary skill for biochemists. As shown in a previous study, faculty identified that constructing and interpreting representations to solve a problem were the most important visual literacy skills for biochemistry students to develop (7). It is also a necessary skill for biochemistry students, who will be presented with a large number of representations over the course of their educational careers. However, just as verbal literacy is not innate or intuitive to students nor is visual literacy. Thus, it has been argued that this skill must be “identified and taught” to biochemistry students (8). In order to consider the instructional implications of external representations, it is critical to consider what types of representations faculty have to draw on and, similarly, what types of representations to which students are commonly being exposed. Types of External Representations in Biochemistry External representations in chemistry have traditionally been categorized into one of three of the domains proposed by Johnstone (9) including symbolic, particulate, and macroscopic. At one of the points lies the macroscopic domain which includes anything that is found on a physical level such equipment used in a laboratory (e.g. glassware, a Western blot, red solution of myoglobin). It includes anything that is tangible and can be seen by the human eye without assistance. A second point of the triangle contains the particulate domain where the nature of atoms and relationships between them is found (e.g. C6H12O2 is the formula for glucose, and the atoms within this structure are covalently attached). At the particulate level, students must be able to construct what is happening in their mind because it is not visible to the human eye. The final point of the triangle contains the symbolic domain which are the symbols found on the periodic table or an equation for the gas laws. The symbolic domain provides a reference for the identity of the atoms and the mathematical relationship between them (e.g. the ratio of carbon atoms to hydrogen atoms in methane is 1:4). The sides of the triangle are the paths that students should be able to use and move between the domains while connecting information. However, this movement can be difficult for many students 4

(10). Whereas the triplet described by Johnstone is adequate for describing the domains in which most physical sciences are represented, in biochemistry, a fourth domain addressing the microscopic domain must be considered. The microscopic domain contains no direct contact with objects of interest so it attempts to explain things that can only be observed using a microscope (11). This fourth domain has been added to the original triangle model set forth by Johnstone to create the biochemistry tetrahedron. While cataloging the types of representations used within biochemistry courses, Towns and colleagues (12) built upon the macroscopic, particulate, symbolic, and microscopic domains of external representations to develop the taxonomy of biochemistry external representations (TOBER). The TOBER model is broken into categories of particulate, symbolic, microscopic, montage, schematic, and animations. External representations, apart from schematic, that fall under the TOBER categories can be correlated to the biochemistry tetrahedron with slight differences in what is included within each domain. The particulate category includes representations which contain spatial information such as ribbon, Fischer, or Lewis structures. The symbolic category is an external representation that involves symbolic, numerical, or graphical information such as equations or graphs. The microscopic category relates to any microscopic representation of a cell. The montage category contains combinations of multiple categories, for example, a single external representation that includes any combination of particulate, microscopic, macroscopic and symbolic components (12). The schematic category is an external representation that contains a flow diagram or some method of depicting steps in a process. The final category of animation refers to any multimedia illustrations that comprise the external representation. A study asking biochemistry faculty to selfreport their use of these various types of representations found that symbolic representations were most prevalently used in both instruction and assessment however, there was significant differences in the reported use of particulate representations of proteins, animations, and microscopic representations during instruction compared to assessments (13). Modes of External Representations As students interact with external representations, there are many variables that may affect students’ understanding of the content depicted by the representation. Schönborn, Anderson, and Grayson (14) note that “there are at least three factors affecting the ability of students to interpret [an external representation], the ability of the students to reason with the diagram, students’ understanding (or lack thereof) of the concepts of relevance to the diagram, and the mode in which the desired phenomenon is represented.” While students’ prior knowledge, subsequent understanding of the concepts, and reasoning abilities may significantly influence the potential for learning from the students’ end, the teacher may also influence learning by selecting the way in which information is presented in the external representation, such as the representational mode (15, 16). For example in the 2002 study by Schönborn, Anderson, and Grayson cited above, the authors explored students’ understanding of and ability to reason with textbook illustrations of Immunoglobin G (IgG). They concluded that “in many cases, the students are focusing on surfacelevel features of the diagram when extracting meaning from it”. This attention to surface features of the mode of the representations, such as color or shape, may cause students to gloss over the deeper meaning of the representation. Schönborn and Anderson (4) have identified three modes of representation: static, dynamic, and multimedia modes. Mayer and Moreno (17) note that “pictures can be static (e.g., illustrations, graphs, charts, photos, or maps) or dynamic (e.g., animation, video, or interactive illustration)”. 5

Thus, the static and dynamic modes are solely visual modes of representation. The multimedia modality is distinguished as an integration of visual and verbal information, such as a narrated video (17). Graphic designers, textbook publishers, researchers, and instructors have a significant element of control over the way in which information is presented to students (15, 16). A considerable amount of research has shown that the nature of the external representations influences student learning. For example, Patrick and colleagues (18) demonstrated that students attend to different features of an external representation depending on the nature of the representation. They used eye tracking to identify which representational features students were focusing on when they were shown either “simple” 2D graphics or “rich” 3D graphics of DNA replication (18). Patrick and colleagues found that the twisting shape depicted in the 2D representation provided participants with more information about the helical shape of the DNA double helix as opposed to the 3D graphic. However, the “more realistic shapes” used to depict the enzymes in the 3D graphic were preferred by participants over the geometric shapes of the 2D graphic (18). While students might prefer information-rich graphics, that does not mean that they learn more or better from them. In fact, Mayer and colleagues document the potential for cognitive overload based on the nature of the external representation (17, 19–25). Level of Abstraction of the Representation In addition to the mode of representation, Schönborn and Anderson (4) also identify the level of abstraction of a representation as a factor that may influence students understanding of the content depicted by the representation. The level of abstraction refers to the degree of realism depicted in the representation. For example, a drawing of a flower might be considered less realistic as compared to a photo of a flower. In this way, realism is a measure of how similar a representation is to the actual object it is meant to depict (26). Schönborn and Anderson (4) note that “biochemists make use of a wide range of [external representations…] which can be placed on a continuum from abstract, to more stylized, to more realistic representations of phenomena”. Schönborn and Anderson (4) go on to speculate that students’ ability to translate between representations of varying levels of abstraction may not be well developed, thereby making it difficult for students to decode the information presented in representations of biochemical phenomena at varying levels of abstraction. As part of a larger meta-analysis, Höffler and Leutner (27) examined the role of the level of abstraction in promoting learning from dynamic and static representations. They concluded that the level of abstraction of an animation was correlated with student learning outcomes. They determined that highly realistic animations supported student learning more than did less realistic animations or static representations. This contradicted a review of the literature by Tversky, Morrison, and Betrancourt (28) on the role of animations in teaching complex systems, including biological systems. Tversky et al. (28) concluded that “animations should lean toward the schematic and away from the realistic”. Thus the question remains, what role—if any—does the level of abstraction play in determining the effectiveness of an external representation to promote student learning?

Potential Outcomes of Using External Representations There are many potential results–both positive and negative–of using external representations to present information to students. Several studies advocate the use of external representations as a preferred instructional strategy to convey information and to enhance student understanding of the 6

content (4). However, it has also been shown that students may develop alternative conceptions from external representations (15, 16, 29–31). Additionally, external representations may present students with too much information, leading to cognitive overload (32). Thus, it is important to understand not only how external representations may be beneficial to students but also how they might be harmful, as well as potential ways to proactively address any potential harmful effects of using external representations in an educational context. Potential Benefits of Using External Representations External representations have gained wide use as a pedagogical tool as of late (33). Within biochemistry “this is reflected by the exponential growth over the years in the number and range of visualization tools now available to the biochemist for teaching, learning, and research” (4). However, what are the potential benefits of using external representations? Ausburn and Ausburn observed six potential benefits of student instruction in visual literacy using external representations (34). “[An] increase in all kinds of verbal skills, [An] improved self-expression and ordering of ideas, [An] increase in student motivation and interest in subjects of all types and at all levels, [The ability to ‘reach’] students not being reached in traditional ways. Students such as the educationally disadvantaged, the truant, the socially underprivileged, the emotionally disturbed, the intellectually handicapped, the ethnic and bilingual, the dyslexic, the deaf, those with speech pathology problems—all respond and have been helped in terms of both interest and achievement, 5. [An] improved image of self and relationship to the world, 6. [An] improved self-reliance, independence, and confidence.” 1. 2. 3. 4.

In addition to the conventional wisdom that external representations can enhance student learning (33), the science education research literature has identified several ways in which external representations may be beneficial. Winn summarizes a collection of research on static external representations, specifically maps and diagrams, noting that these external representations are “wellsuited to illustrate inter-component relationships and sequences” (35). Winn goes on to acknowledge that external representations can be used to integrate new knowledge with prior knowledge and make abstract content more coherent (35). Similarly, Harrison and Treagust note that external representations can be a valuable tool for students to construct new or more intricate knowledge (36). Their study examined a high school student’s understanding of an atom over the course of instruction, which included exposure to a variety of external representations. They found that following instruction, the student displayed an understanding of atomic and molecular models that was much more consistent with scientifically accepted conceptions as compared to the student’s prior non-scientific conceptions (36). Bauer and Johnson-Laird discuss how the use of diagrams can improve students’ reasoning abilities (37). They asked 48 undergraduate students to answer a series of questions about either people and places or electrical circuits. Some students were asked questions verbally while others were asked diagrammatically. Bauer and Johnson-Laird found that the participants shown the diagrams were better able to answer the questions correctly, regardless of the domain (people and places or circuitry) (37). They concluded that “certain diagrams can help individuals to reason more 7

rapidly and more accurately” (37). As such, external representations can be an extremely beneficial resource to allow students to develop their conceptual knowledge and reasoning abilities. External representations can also play a much more practical role in science education. Linn, Davis, and Eylon describe four basic tenants or metaprinciples to scaffold knowledge integration in science education: (1) make science accessible, (2) make thinking visible, (3) help students learn from others, and (4) promote autonomy and lifelong learning (38). Of these four tenants, external representations address at least two of them: making science accessible and making science visible. For example, Kozma and Russell note that “the expansion of the universe, tectonic plate drift, evolution of species, and molecular structure and reactivity are all scientific phenomena that are not available to direct experience” (39). External representations can provide a way to make science topics like these accessible and visible. Potential Negative Results of Using External Representations While external representations can be an important and beneficial part of science education, they can also have negative effects on students’ learning. For example, they might lead to the development of alternative, non-scientific conceptions. As discussed previously, prior knowledge is an important factor in determining what students can learn. Lowe describes how meteorologists and non-meteorologists come to understand a static weather map differently (31). Lowe concludes that differences in content knowledge lead to differences in understanding of a content-related representation. He goes on to suggest that the representational design of the diagram “implies [that] viewers possess appropriate background knowledge concerning the depicted situation” (31). If viewers fail to possess the “appropriate background knowledge,” it is not simply that they will not be able to understand the representation; instead, viewers can develop alternative understandings of what is being depicted in the representation. Harrison and Treagust interviewed 48 secondary students regarding their understandings of atoms and molecules (40). As part of the study, students were shown six different diagrams of atoms and asked to rank them according to how closely the representations aligned with their understanding of an atom. Harrison and Treagust suggest that students may hold alternative conceptions of atoms due to inaccurate decoding and interpretation of external representations (40). Similar to Lowe (31), Harrison and Treagust suggest that students may not have sufficient knowledge of the topic, which results in their inability to “appropriately” understand what information is being presented in the representation and the subsequent formation of alternative conceptions based on the representation. Lowe (31, 33) and Kozma and Russell (39) offer another possibility as to why students may develop alternative conceptions based on their experience of an external representation: they focus on surface features. In Lowe’s study of meteorologists’ (expert) and non-meteorologists’ (novice) interpretations of a static weather map, he describes the non-meteorologists’ explanation of what was represented by the weather map, noting that they focused almost exclusively on the surface features of the diagram as opposed to the most meteorologically relevant details. In 2003, Lowe found a similar result using an animated weather map, noting that “the animation did not appear to be effective in making subjects any more sensitive to these less obvious dynamics aspects, despite the fact that they were explicitly depicted in the animation and a high degree of user interaction was provided for”. Kozma and Russell also found that novices tend to focus on the surface features of a representation instead of on the most conceptually important features (39). They asked chemists (expert) and undergraduate chemistry students (novice) to complete a card-sort in which they were to arrange various chemistry representations into “meaningful” groups. Each card corresponded 8

to a computer display showing an equation, a graph, an animation, or a video segment related to chemistry. Kozma and Russell found that the “experts formed their groups around concepts and principles in the domain,” whereas, “[n]ovices made smaller groups and were more likely to give reasons for these groupings that merely described the common surfaces” of the representations (39). They concluded that “the chemical understanding of novices was bound to the common surface features of the representations” (39). Therefore, because students focus on the surface features they may develop the inaccurate understanding from a representation or one that differs from what is intended by the teacher. This may be due to students’ inability to distinguish between surface features of a representation and meaningful features of that representation. A second potential negative result of using external representations is that students may experience cognitive overload. The idea of cognitive overload is based on Cognitive Load Theory. Ayres and van Gog describe Cognitive Load Theory as “a model of human cognitive architecture that assumes that working memory […] is very limited in terms of being able to store and process information […], whereas long-term memory […] has a vast capacity, able to store an almost limitless amount of information” (41). Information is thought to enter the human memory system through working memory before potentially being stored in long-term memory. However, if incoming information overwhelms the limited capacity of working memory, that information may not be able to be processed and will not be stored in long-term memory. Several studies have linked the use of external representations to cognitive overload. Chandler and Sweller describe a series of experiments in which students were shown one of two types of diagrams (42). The first type of diagram contained a block of text describing the diagram followed by the diagram. The second type of diagram integrated the text into the diagram such that each part of the diagram was labeled and described within the diagram. Chandler and Sweller found that the students shown the integrated diagrams outperformed students shown the diagram with the separated information (42). To explain this observation, they describe what they call the splitattention effect. They propose that the integrated diagram reduced the amount of cognitive resources required for students to pair up the appropriate information from the text with the appropriate part of the diagram to make meaning out of the representation. According to this theory, If student attention is split between the text and the diagram, they are more apt to experience cognitive overload and less likely to understand the information presented in the representation. Thus, the design of the external representation may cause students to fail to understand the representation due to cognitive overload. Another issue related to cognitive overload is the amount of information presented to students in a representation. Some studies suggest that animations may be more likely to cause cognitive overload because significantly more information is presented to students as compared to a static representation or text alone. For example, Schnotz, Böckheler, and Grzondziel describe a study in which 12 pairs of university students were presented with a learning environment containing either animated or static representations of circumnavigation (32). The study showed that students who were shown the animated pictures did worse (as measured by a pre-test, post-test evaluation) than students who were shown the static pictures. Schnotz et.al. conclude that “although animated pictures may provide external support for mental simulations, they do not appear to be generally beneficial for learning, because they can prevent individuals from performing relevant cognitive processes” (32). Therefore, the mode and content of a representation may lead to cognitive overload. Although external representations have the potential to cause cognitive overload and/or lead to the development of alternative, non-scientific conception in students, the potential for improved student learning and reasoning and the ability to make science accessible and visible to students 9

means that external representations can be powerful instructional tools if we understand how to best use them. External Representations in Textbooks Textbooks are a cornerstone of most college classrooms and biochemistry is no different. A review of the literature of textbook use found that a primary reason to require a textbook in the eyes of both students and instructors is for consistency in presented material (43). This includes the representations used within the textbook. Textbooks have been shown to be one of the most graphically populated scientific communications (44) with some even criticizing the excessive use of representations (45). A prior study looked at representation use in general chemistry textbooks and found an average of 4 representations per page (46). When looking at how these representations were used in the textbooks, it was found that while many of the representations served a “representational” function several only served as decoration. Many studies have investigated what types of representations can be found in the science textbooks. When looking at the representation types used within high school science texts, the majority of representations are photographs or pictures and fall within the macroscopic domain (47–52). However, this trend changes when looking at the types of representations in physical chemistry textbooks where it was found that 85% of the representations used fell within the symbolic domain (53). With the significant difference in the types of representations used at different levels and content, and the importance of representations used in biochemistry, it is important to determine how specifically, these representations are used in the textbooks most prevalently used in biochemistry courses.

Quantifying the Use of Representation Types in Common Biochemistry Textbooks Methodology The study described herein builds on the previous work on representation use in biochemistry to answer the following research questions: 1. What types of representations are most prevalently used in biochemistry textbooks? 2. Is there a difference in the types of representations used depending on the content discussed? Sample Collection and Analysis The sample consisted of the top four one-semester biochemistry textbooks according to Amazon in spring 2015. These four textbooks represented the majority of sales in biochemistry on the site. Supplemental information supplied with the textbooks were not analyzed in this study. The four biochemistry text books are as follows: 1. 2. 3. 4.

Fundamentals of Biochemistry (4th edition) by D. Voet, J. Voet, and C. Pratt. Biochemistry (8th edition) by M Campbell and S. Farrell Essential Biochemistry (3rd edition) by C. Pratt and K. Cornely. Lehninger Principles of Biochemistry (6th edition) by Nelson and Cox

10

To obtain the data, every representation (whether it was in a figure or in text) was coded according to the TOBER subcategories found in Table 1. Only representations that were included within the text of a chapter were included in the data set. This included representations in the body of the text, figures, boxes, tables, and in the margins. Any representations that were found in end of chapter problems were excluded from this data set. Additionally, only representations in the textbook were coded, no supplemental information, online resources, or embedded links were included in the data set. The categories were not mutually exclusive and as such a single image could consist of more than one type of representation. Five raters each initially coded the same 3 chapters from a single textbook using the subcategory descriptors and then came together to discuss. Initial agreement was 94% but following discussions 100% agreement was obtained. The remaining chapters were split between 4 of the raters. When there were representations that were of question as to the coding, a discussion would occur among the raters to ensure agreement and the sub-category descriptions would be updated. The data was processed using Microsoft Excel to determine percentage of categories. Pearson’s chi-square test was used to determine associations between variables as necessary given the research question. All statistical tests were conducted using SAS. Table 1. Taxonomy of Biochemistry External Representations Codes Macroscopic

Microscopic

Particulate

Anatomy and Physiology

Cartoon

Space-filling

Equations

Laboratory

Microscope Image

Ball and Stick

Graphs

Ribbon Diagram

Tables (numeric)

Fischer Projection

Tables (classifications)

Random Photograph

Schematic

Symbolic

Haworth Projection Skeleton Wedge-Dash Condensed Structural Lewis Structure Chemical Equations Sequential Subunit (Oligomer) Stylized/Cartoon

Results What Types of Representations Are Most Prevalently Used in Biochemistry Textbooks? Across the 4 textbooks analyzed there were a total of 7,306 representations identified and included in the sample. Of these representations, 54% of them included more than one type of 11

representation listed in Table 1. For the representations that only included one type of representation, these primarily depicted particulate representations as seen in Figure 1. Noticeably low in count is the schematic single representations with only 1% of the total sample. These trends are decently consistent across the four textbooks as shown in Figure 2. Nelson and Cox seems to balance the number of multiple representations with the use of single particulate level representations with about 35% of representations in each category. This is in comparison to Pratt and Cornely and Campbell and Farrell which were found to have 70% and 67% of their representations be comprised of multiple types of representations. A Pearson correlation shows a significant association between the types of representation used and the textbook analyzed r(15) = 345.44, p = 0.0000, ω = 0.2645 indicating a small effect.

Figure 1. Percentage of all representations categorized in each type of representation.

Figure 2. Percentage of representations in each type of representation by textbook. Due to the large percentage of representations that were found to include more than one type of representation, it was important to look more specifically at this subset of the data. While 46% of the 12

representations depicted a single type of representation, Figure 3 shows 31% of the representations included representation that were composed of multiple representations of a single type. For instance, 96% of these representations are depicting multiple representations at the particulate level. An example of this could be a single representation of a protein that includes the spacefilling representation overlaid with the wireframe of the backbone. In regards to visual literacy skills, in order to interpret this type of representation students would have to be able to translate horizontally between the representations of a similar type.

Figure 3. Percentage of representations based on the number of types of representations included. As you start to require students to interpret multiple representations of different types, for instance in the remaining 23% of representations depicted in Figure 3, a more complex level of translation skill is required. This includes 16 representations that require students to move between 4 different types of representations. Looking at what types of representations are most commonly put together a pattern starts to emerge. For representations combining 2 types of representations, 68% combine particulate and schematic representations. An example of this type of representation would be a metabolic pathway that includes the names of metabolites, enzyme and pathway arrows but also includes the metabolite structures. A similar analysis of the representations combining 3 types of representations shows 71% of these representations incorporate particulate, schematic, and microscopic as to typically provide context as to where in the cell the metabolic process is located. When comparing the different textbooks by the percentage of representations requiring students to interpret multiple representations in Figure 4, the differences between the texts become obvious. As mentioned previously, Nelson and Cox overwhelmingly tends to use single representations. This trend in similar to Voet, Voet, and Pratt but to a less dramatic degree. Both of these books require less translational visual literacy of students than both Pratt and Cornely and Campbell and Farrell. Pratt and Cornely requires a greater degree of horizontal translation between multiple representations of a single type of representation than of all the other texts. Whereas, Campbell and Farrell requires the greatest degree of complex translation between multiple representations of different types of representation. This text actually accounted for 8 of the 16 representations that required translation between 4 different types of representations. When looking at what types of representations are making up these combinations, the trend that was seen on the whole is retained at the individual textbook level. A Pearson correlation again shows a significant association between the number of

13

representation using different types of representation and the textbook analyzed r(12) = 386.14, p = 0.0000, ω = 0.2784 again indicating a small effect.

Figure 4. Percentage of representations by textbook based on the number of types of representations included. Is There a Difference in the Types of Representations Used Depending on the Content Discussed? While there were some nuances to each book, for the most part there was very little difference in the content presented. The chapters for each textbook followed a similar pattern of 1-2 chapters of introductory material including the chemistry of water, followed by 4-5 chapters discussing the structure and function of proteins including typically two chapters on enzyme kinetics and inhibition. This was then followed by either 2-3 chapters on lipids, membrane structure and transport, and cellular signaling or a chapter on carbohydrates. Both sections were included but which came first depended on the book. These were then followed with a large coverage (8-11 chapters) of metabolism including carbohydrate, lipid, amino acid, and sometimes nucleic acid catabolic and anabolic processes. A chapter or two on photosynthesis and the Calvin cycle were also included in this section. Finally, the textbooks concluded with 4-8 chapters of on nucleic acid structure, central dogma, and s discussion of DNA technologies. The order of content presented should be of little surprise to many biochemists as this is similar to the order that most biochemistry courses are presented and is consistent with prior research suggesting that 90% of teachers use textbooks for organizing their instruction (54). The representations were coded in relation to which chapters they were presented in order to determine what types of representations were used to present the material. Because there were different amounts of coverage of each content area, the average number of representations per chapter were calculated to determine to provide an idea of the prevalence of use of representation by content area. The results are shown in Figure 5. The introductory chapters and those presenting information related to protein structure and function on average tend to use the most representations per chapter, approximately 80 per chapter. This is in comparison to the metabolism content which has the most amount of coverage in the textbooks but uses the fewest representations (53 per chapter) to discuss the material. The remaining content areas average approximately 68 representations per chapter.

14

Figure 5. Average number of representations per chapter content across all 4 textbooks sampled. The representations were also analyzed to determine what types of representations were used within each content area, shown in Figure 6. The solid regions of the bar graphs in Figure 6 represent the types of single representations used. It can be seen that for there are similar usage of macroscopic and microscopic representations. The introductory material and the protein structure and function chapters tend to use the largest proportion of symbolic representations. This is primarily in the context of presentation of acid-base chemistry within the introductory chapters presenting mathematical formulas and titration graphs and presentation of kinetics formulas and graphs within the enzyme kinetics and inhibition chapters. Particulate representations tend to be most prevalent in the later content areas. Finally, single schematic representations while used sparingly are most prevalent in the chapters on carbohydrates and metabolism. Examples of these would be figures depicting methods of carbohydrate analysis and introductory metabolic pathways showing overall cycles.

Figure 6. Percentage of types of representations by content covered. Solid colored categories indicate single representations and patterned categories indicate use of multiple representations.

15

The patterned portions of the bars in Figure 6 represent the various types of multiple representations used in each content area. Sixty percent of the representations in the carbohydrate chapter use multiple representations of a single type of representation and 100% of these are depicting carbohydrate concepts at the particulate level. An example of a representation that would fall within this category would be one depicting the Fischer projection of glucose and then the respective Haworth projection in order to show the transformation between the two. The remaining content areas range from 23% to 36% of the representations being composed primarily of multiple particulate representations. It is evident by looking at Figure 6 that as the content progresses throughout the book, there is an increase in the proportion of multiple representations using 2 or more types of representation. The metabolism chapters see the greatest increase in the proportion of representations using 2 types of representations. Unsurprisingly, 82% of these representations were combinations of particulate and schematic representations. When comparing the 3-4% of the representations that combine 3 types of representations in the final three content categories (lipids, membranes, and signaling; metabolism; and nucleic acids and central dogma), there is a difference in what is being combined. Regardless of content area, the combination of microscopic, particulate and schematic representations are overwhelmingly depicted, but for the chapters on nucleic acids and central dogma, this only represents 58% of the representations depicting 3 types of representations. The remaining representations in the nucleic acid and central dogma chapters primarily combine macroscopic, particulate and schematic representations. While rarely used, the representations combining 4 different types of representations were most commonly used in chapters discussing lipids, membranes, and cellular signaling and metabolism. These primarily were composed of representations combining macroscopic, microscopic, particulate, and schematic. Discussion The study described herein sought to characterize the types of representations used within common one-semester biochemistry textbooks using the Taxonomy of Biochemistry External Representations (12). It was found that by far, regardless of textbook, the most used classification of representation was particulate. This category of representation also included the most sub-categories of representation according to the TOBER classification. Particulate representations were also most used even when combined with other types of representations. This shows that biochemists often utilize particulate representations to convey biochemical information. Additionally, there is an expectation that biochemistry students can understand and use these types of representations in introductory biochemistry courses. Therefore, there is a specific need for students to be introduced to these types of particulate representations prior to entering into a biochemistry course as the curriculum relies so heavily on these representations. The reliance on particulate representations is in direct contrast to what was found when analyzing physical chemistry textbooks where symbolic representations tended to be overwhelmingly used (52). However, it was found that these symbolic mathematic expressions were most prevalent in the biochemistry texts in the chapters that included high coverage of physical chemistry topics such as equilibrium and kinetics. Schematic representations were rarely used as a single representation but were one of the most used representations combined with another type of representation. Unsurprisingly, this was predominantly found in the chapters related to metabolism. When looking at how the representations are used in comparison to the textbook content, it was found that the textbooks seem to inherently scaffold the visual literacy skills as the content is presented. The texts typically begin with a large proportion of representations depicting a single 16

type of representation. This allows for students to build their visual literacy of learning to interpret, manipulate and use the various types of representations independently. Then as the textbook progresses, there seems to begin to show a move towards a higher proportion of use of multiple representations of a single type. This would push students to start to horizontally translate representations at the particulate level. This is a more challenging skill that requires students to interpret and move freely across different particulate representations, a very important skill for instance when learning carbohydrates. Finally, the textbook content concludes with a larger proportion of representations depicting multiple different types of representations where students have to interpret and move freely between particulate and schematic representations as is the case for metabolism. This is the most challenging skill for students as it forces them to think about different types of representations and incorporate them to form a single understanding. Implications, Limitations and Future Areas of Research This project has shed light on how the structure of the textbook and thus the progression of most biochemistry courses (assuming the course follows along with the textbook) inherently scaffolds the development of visual literacy via the types of representations used to present the content. This implied scaffolding is most likely a by-product of how content is presented in biochemistry curriculum and not necessarily a direct intention of the authors (55). However, the instructor is the one presenting and using the representations in the course. The biochemistry instructor should not only be cognizant of the types of representations being used to present content in their own class and how s/he is scaffolding that development of visual literacy, but they should also be aware of how these representations have been introduced and used in previous classes in order to create a learning progression of visual literacy skills across the curriculum. Additionally, the instructor should be cautioned as to the use of multiple representations and the amount of cognitive load inherent in interpreting these representations when initially presenting this type of information. While the development of translation between and among different types of representations is important to biochemists’ development, these should be intentionally used with explicit instruction prior to the introduction of the multiple representations of how to interpret the representations independently and then once given together how to integrate them in order to help facilitate the development of visual literacy in biochemistry. For instance, before moving into the metabolism chapter an instructor should introduce how to read a schematic and then introduce a short metabolic pathway with just names of the metabolites and coupled reactions. Then once students have a grasp on how to read and interpret the schematic portion of the pathway add the structures to help interpret and give a chemical basis for what is happening in each step of the pathway. While a large amount of research in biochemistry and chemistry education research has focused on student understanding at the particulate and symbolic levels, this study exposed the need for additional research to look at biochemistry students’ understanding of schematic representations and how incorporating additional types of representations such as particulate structures such as in metabolic pathways impacts this understanding. Finally, this study only focused on the top 4 textbooks used in a one semester biochemistry course and as such limits the results to only part of the picture of the types of representations students are exposed to. Faculty could draw on other resources such as simulations, animations, and videos that are not included in this analysis but help develop student visual literacy. Additionally, based on previous work which found that there is a large discrepancy in the types of representations used during instruction and on assessments (13), it

17

would be interesting to determine what types of representations were used in the textbook assessment questions and how those compared to the types of representations used in the body of the text.

References 1. 2. 3. 4. 5.

6.

7. 8. 9. 10. 11.

12.

13.

14. 15.

16.

Trumbo, J. Visual literacy and science communication. Sci. Comm. 1999, 409–425. Wellington, J.; Osborne, J. Language and Literacy in Science Education; Open University Press: Philadelphia, PA, 2001. Stanley, E. D. Taking a second look: Investigating biology with visual datasets. Bioscene 1996, 13–17. Schönborn, K. J.; Anderson, T. R. The importance of visual literacy in the education of biochemists. Biochem. Mol. Biol. Educ. 2006, 94–102. Mayer, R. E.; Bove, W.; Bryman, A.; Mars, R.; Tapangco, L. Why less is more: Meaningful learning from visual and verbal summaries of science textbook lessons. J. Educ. Psychol. 1996, 64–73. Schönborn, K. J.; Anderson, T. R. Bridging the educational research-teaching practice gap: Foundations for assessing and developing biochemistry students’ visual literacy. Biochem. Mol. Biol. Educ. 2010, 347–354. Linenberger, K. J.; Holme, T. A. Biochemistry instructors’ views toward developing and assessing visual literacy in their courses. J. Chem. Educ. 2014, 23–31. Avgerinou, M.; Ericson, J. A review of the concept of visual literacy. Br. J. Educ. Technol. 1997, 280–291. Johnstone, A. H. Why is science difficult to learn? Things are seldom what they seem. J. Comput. Assist. Lear. 1991, 75–83. Johnstone, A. H. Why Is Science Difficult To Learn? Things Are Seldom What They Seem. J. Comp. Assist. Learn. 1991, 75–83. Chu, Y. C. Learning Difficulties in Genetics and the Development of Related Attitudes in Taiwanese Junior High Schools. Ph.D. Thesis, University of Glasgow, Glasgow, Scotland, 2008. Towns, M. H.; Raker, J. R.; Becker, N.; Harle, M.; Sutcliffe, J. The biochemistry tetrahedron and the development of the taxonomy of biochemistry external representations (TOBER). Chem. Educ. Res. Pract. 2012, 296–306. Linenberger, K. J.; Holme, T. A. Results of a national survey of biochemistry instructors to determine the prevalence and types of representations used during instruction and assessment. J. Chem. Educ. 2014, 800–806. Schönborn, K. J.; Anderson, T. R.; Grayson, D. J. Student difficulties with the interpretation of a textbook diagram of Immunoglobulin G (IgG). Biochem. Mol. Biol. Educ. 2002, 93–97. Bussey, T. J.; Orgill, M. What do biochemistry students pay attention to in external representations of protein translation? The case of the Shine-Dalgarno sequence. Chem. Educ. Res. Pract. 2015, 714–730. Bussey, T. J.; Orgill, M. Biochemistry instructors’ use of intentions for student learning to evaluate and select external representations of protein translation. Chem. Educ. Res. Pract. 2019, Advance Article.

18

17. Mayer, R. E. The promise of multimedia learning: Using the same Instructional design methods across different media. Learn. Instr. 2003, 125–139. 18. Patrick, M. D.; Carter, G.; Wiebe, E. N. Visual representations of DNA replication: Middle grades students’ perceptions and interpretations. J. Sci. Educ. Technol. 2005, 353–365. 19. Mayer, R. E. Multimedia learning: Are we asking the right questions? Educ. Psychol. 1997, 1–19. 20. Mayer, R. E.; Johnson, C. I. Revising the redundancy principle in multimedia learning. J. Educ. Psychol. 2008, 380–386. 21. Mayer, R. E.; Moreno, R. A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. J. Educ. Psychol. 1998, 312–320. 22. Mayer, R. E.; Moreno, R. Aids to computer-based multimedia learning. Learn. Instr. 2002, 107–119. 23. Mayer, R. E.; Heiser, J.; Lonn, S. Cognitive constraints on multimedia learning: When presenting more material results in less understanding. J. Educ. Psychol. 2001, 187–198. 24. Mayer, R. E.; Mautone, P.; Prothero, W. Pictorial aids for learning by doing in a multimedia geology simulation game. J. Educ. Psychol. 2002, 171–185. 25. Mayer, R. E.; Hegarty, M.; Mayer, S.; Campbell, J. When static media promote active learning: Annotated illustrations versus narrated animations in multimedia instruction. J. Exp. Psychol. Appl. 2005, 256–265. 26. Rieber, L. P. Computers, Graphics, and Learning; Brown and Benchmark: Madison, WI, 1994. 27. Höffler, T. N.; Leutner, D. Instructional animation versus static pictures: A meta-analysis. Learn. Instr. 2007, 722–738. 28. Tversky, B.; Morrison, J. B.; Betrancourt, M. Animation: Can it facilitate? Int. J. Hum-Comput. St. 2002, 247–262. 29. Lowe, R. Background knowledge and the construction of a situational representation from a diagram. Eur. J. Psychol. Educ. 1996, 377–397. 30. Linenberger, K. J.; Bretz, S. L. Biochemistry students’ ideas about how an enzyme interacts with a substrate. Biochem. Mol. Biol. Educ. 2015, 213–222. 31. Linenberger, K. J.; Bretz, S. L. Biochemistry students’ ideas about shape and charge in enzyme–substrate interactions. Biochem. Mol. Biol. Educ. 2014, 203–212. 32. Schnotz, W.; Böckheler, J.; Grzondziel, H. Individual and co-operative learning with interactive animated pictures. Eur. J. Psychol. Educ. 1999, 245–265. 33. Lowe, R. Animation and learning: Selective processing of information in dynamic graphics. Learn. Instr. 2003, 157–176. 34. Ausburn, L. J.; Ausburn, F. B. Visual literacy: Background, theory and practice. Innov. Educ. Teach. Int. 1978, 291–297. 35. Winn, W. Learning from maps and diagrams. Educ. Psychol. Rev. 1991, 211–247. 36. Harrison, A. G.; Treagust, D. F. Learning about atoms, molecules, and chemical bonds: A case study of multiple-model use in grade 11 chemistry. Sci. Educ. 2000, 352–381. 37. Bauer, M. I.; Johnson-Laird, P. N. How diagrams can improve reasoning. Psychol. Sci. 1993, 372–378.

19

38. Linn, M. C.; Davis, E. A.; Eylon, B.-S. The scaffolded knowledge integration framework for instruction. In Internet Environments for Science Education; Linn, M. C., Davis, E. A., Bell, P. , Eds.; Lawrence Erlbaum Associates, Publishers: Mahwah, NJ, 2004; pp 47–720. 39. Kozma, R. B.; Russell, J. Multimedia and understanding: Expert and novice responses to different representations of chemical phenomena. J. Res. Sci. Teach. 1997, 949–968. 40. Harrison, A. G.; Treagust, D. F. Secondary students’ mental models of atoms and molecules: Implications for teaching chemistry. Sci. Educ. 1996, 509–534. 41. Ayres, P.; van Gog, T. State of the art research into Cognitive Load Theory. Comput. Human Behav. 2009, 253–257. 42. Chandler, P.; Sweller, J. Cognitive load theory and the format of instruction. Cogn. Instr. 1991, 293–332. 43. Skinner, D.; Howes, B. The required textbook-friend or foe? Dealing with the dilemma. Journal of College Teaching and Learning (Online) 2013, 133. 44. Lee, V. R. Adaptations and continuities in the use and design of visual representations in US middle school science textbooks. Int. J. Sci. Educ 2010, 1099–1126. 45. Woodward, A. Do illustrations serve an instructional purpose in US textbooks? In Learning from Textbooks: Theory and Practices; Britton, B. K., Woodward, A., Binkley, M., Eds.; Lawrence Erlbaum: Hillsdale, NJ, 1992; pp 115–134. 46. Nyachwaya, J. M.; Gillaspie, M. Features of representations in general chemistry textbooks: a peek through the lens of the cognitive load theory. Chemistry Education Research and Practice 2016, 58–71. 47. Kapıcı, H. Ö.; Savaşcı-Açıkalın, F. Examination of visuals about the particulate nature of matter in Turkish middle school science textbooks. Chem. Educ. Res. Pract. 2015, 518–536. 48. Roth, W.; Bowen, G. M.; McGinn, M. K. Differences in graph related practices between high school biology textbooks and scientific ecology journals. J. Res. Sci. Teach. 1999, 977–1019. 49. Dimopoulos, K.; Koulaidis, V.; Sklaveniti, S. Towards an analysis of visual images in school science textbooks and press articles about science and technology. Res. Sci. Educ. 2003, 189–216. 50. Pozzer, L. L.; Roth, W. Prevalence, function and structure of photographs in high school biology textbooks. J. Res. Sci. Teach. 2003, 1089–1114. 51. Han, J.; Roth, W. Chemical inscriptions in Korean textbooks: semiotics of macro- and micro world. Sci. Educ. 2005, 173–201. 52. Gkitzia, V.; Salta, K.; Tzougraki, C. Development and application of suitable criteria for the evaluation of chemical representations in school textbooks. Chem. Educ. Res. Pract. 2011, 5–14. 53. Nyachwaya, J. M.; Wood, N. B. Evaluation of chemical representations in physical chemistry textbooks. Chemistry Education Research and Practice 2014, 720–728. 54. Chiappetta, E. L.; Ganesh, T. G.; Lee, Y. H.; Phillips, M. C. Examination of science textbook analysis research conducted on textbooks published over the past 100 years in the United States. In Annual meeting of the National Association for Research in Science Teaching; 2006, San Francisco, CA. 55. Voet, J.; Voet, D. Experimental Biology; San Diego, CA. Personal communication, April 2015.

20

Chapter 2

Virtual Exploration of Biomolecular Structure and Function Shuchismita Dutta*,1 and Daniel R. Dries2 1Institute of Quantitative Biomedicine, Rutgers, The State University of New Jersey,

174 Frelinghuysen Road, Piscataway, New Jersey 08854, United States 2Chemistry and Biochemistry, Juniata College, 1700 Moore Street, Huntingdon, Pennsylvania 16652, United States *E-mail: [email protected].

Visual representations are critical for converting abstract concepts of molecular architecture into tangible experiences that have deep meaning for learning and research. Analysis of these representations can guide the design of proteins and other biomolecules with new properties and interactions for academic, therapeutic, and commercial applications. For educational purposes, structures of biological macromolecules are currently explored through static images, virtual interactive views, physical models, and animations. This chapter focuses on the virtual exploration of biomolecular structure and function. Two key components necessary for such exploration are the three-dimensional (3D) structural data and visualization tools and visual comprehension practices. The Protein Data Bank offers free access to experimentally-determined 3D structural data, which are commonly used for virtual exploration both in research and for education. The source of structural data, common assumptions, and possible errors are discussed. An overview of currently available tools to visualize, analyze, and virtually explore biomolecular structures is presented. Also discussed are examples of how scientists, educators, and students integrate their understanding of biological and chemical principles with virtual exploration of molecular structure for the explanation of biomolecular function in research and education. In addition, the chapter provides historical perspectives on the development of virtual interactive tools and presents considerations for future tools to explore multiscale, nearatomic, and dynamic data with information spanning the atomic level through organismal function. Overall, the chapter presents resources and inspirations for introducing students to virtual exploration of biomolecular structure and function.

© 2019 American Chemical Society

Introduction Visualization is integral to the scientific endeavor. It facilitates synthesis, analysis, and communication of complex ideas and abstract concepts (1). An excellent illustration of the power of visualization was demonstrated by Watson and Crick in developing a model for deoxyribonucleic acid (DNA) (2). Beginning in the 1940s, Avery, McLeod, and McCarty showed that DNA was the transforming principle (3). This was later independently confirmed by Hershey and Chase (4). Chargaff examined various organisms to establish rules for the composition of nucleic acid bases in DNA (5). By combining this information with diffraction data collected by Rosalind Franklin and an understanding of hydrogen bonding between DNA bases (guided by Jerry Donahue) (2), Watson and Crick built the double helical model of DNA. Visualizing the model immediately suggested a mechanism for DNA’s functions: replication and storage of genetic information (6). The model also paved the way for a host of major discoveries, technical advances, and inspired several specializations in biology, including molecular biology, genetics, and biotechnology. To date, many structures of DNA and its various complexes with proteins, ribonucleic acids, ligands, and drugs have been determined using a variety of different experimental methods. Visualization of models derived from these experiments have helped analyze the stabilizing interactions of DNA and in exploring the molecular mechanisms of its functions. In fact, the double helical DNA model has become so popular that in addition to communication within the scientific community, it is also widely used in graphics, animations, and logos to connote molecular and/or genetic affiliation. Today, three-dimensional (3D) structures of biomolecules are routinely explored to comprehend structure-function relationships. Researchers in biology and medicine visualize structures of biological macromolecules (e.g., proteins, nucleic acids, lipids, and carbohydrates) as a foundation for developing solutions to real world problems – i.e., to interpret observed or experimentally-derived data, to formulate new research questions, to design experiments, and to engineer molecules with novel properties and functions. Educators and students visualize these models to comprehend complex biochemical processes and abstract concepts, such as allostery. Even non-experts can benefit from visualizing biomolecular structures (e.g., while discussing their specific health condition) to understand available personalized medicine options, and to make informed decisions about them. In short, macromolecular structure visualization is not only essential to learning about and practicing science, familiarity with it can help us become informed citizens. Training in biomolecular visualization is critical to prepare the next generation of scientists, educators, and consumers. Guidelines from various disciplinary professional societies, such as the American Chemical Society (7) and American Society for Biochemistry and Molecular Biology (8, 9) focus on understanding structure-function relationships and often do so with the explicit recommendation of using visualization tools. Undergraduate curricula and research opportunities in these fields have made room for interactive exploration of 3D structures of biological macromolecules. Textbooks in biology, chemistry, and biochemistry routinely include molecular structural images and discussion about them. Some even offer supplementary animations and videos to visualize and learn about the relationships of biomolecules and their complexes to their functions. Educators and students have multiple modes of exploring molecular structures: static images in textbooks illustrate key features of a molecular structure, virtual interactive views allow users to interact with the molecular model in three-dimensional computational space, animations of biological molecules tell the story of molecular structure and function, and physical models provide an appreciation of overall shape, features, and properties of the molecule. This chapter specifically focuses on the use of virtual interactive exploration of biomolecular structure and function. It begins 22

by describing the evolution of virtual exploration of molecular structures and technological innovations that brought biomolecular visualization to education. It then discusses some requirements for virtual exploration, science practices leading to visual comprehension, suggestions for assessing visual literacy, and a few examples of virtual exploration of biomolecules in classrooms and beyond.

Visual Communication of Molecular Structures: A Brief History The history of visualizing molecular structures can be traced back to the early nineteenth century (10) (Figure 1). Credit for the first representation of invisible and abstract concepts such as atoms and molecules in pictorial form goes to Dalton (11) - he gave unique symbols to each atom and represented molecules as geometric combinations of atoms. Nearly 60 years later, Kekule used lines and chemical symbols to draw a two dimensional (2D) representation of benzene based on data from its chemical reactivity (Figure 1A). It was only in the latter part of the century that van’t Hoff discussed the spatial arrangement of atoms in a molecule (12). This visualization of the tetrahedral arrangement of bonds around carbon atoms revolutionized chemistry [see review (10)]. The twentieth century ushered in an exponential increase in knowledge about molecular structures. Advances in quantum mechanics and molecular orbital theory helped define the specific geometry of atoms in a molecule (13). Experimental methods for structure determination such as Xray crystallography (X-ray) and nuclear magnetic resonance (NMR) spectroscopy enabled empirical molecular structure determination of increasing size and complexity. Visual communication of large molecular structures demanded 3D representations. Whereas computational power was brought to bear in analyzing empirical data for the determination of molecular structures, physical models were used to communicate their properties and functions, e.g., myoglobin (14) and DNA (2). Ideas about representing atoms as spheres and covalent linkages as lines or bars persisted in these representations. To simplify the information content in protein structures, Kendrew created a representation that only focused on the path of polymer backbone atoms. Regardless of whether these models included all atoms or only the backbone atoms, these 3D models required careful construction, making replicas both costly and labor-intensive. These models were most often shared as 2D photographic images and stereo views, but that meant that the viewer was unable to fully explore the 3D shape and interactions. Virtual representations made molecular structures more accessible for exploration, analysis, and modeling. Motivated by an interest in understanding protein folding (15), Cyrus Levinthal displayed a protein structure on a computer for the first time in 1964. The model consisted of a rotatable image of a wireframe model displayed on an oscilloscope and offered the first opportunity to manipulate a molecular structure virtually (16). Around the same time, Douglas Engelbart introduced the prototype of a mouse and rollerball, marking the beginning of the democratization of computing, broadening the use and accessibility of computational power (17). In the late 1960s, a grassroots effort began in the crystallography community to establish an open access central repository for 3D coordinate data files, which led to the creation of the Protein Data Bank (PDB) in 1971 (18). Access to experimental 3D structural data was enabled through this resource. In the late 1970s, there was a shift in purpose for virtual exploration of biomolecules using computers. Instead of using computers for just visualizing empirical models, software was developed to enable the manual fitting of amino acid sequences into electron density maps. The Richardson group first solved a structure using such an approach (19) (i.e., without using the Kendrew-type physical models constructed earlier from electron density maps). The ability to easily calculate and 23

store the model coordinates for automated recall and sharing made computer-based modeling more favorable than using physical models. Software developed during this time period [e.g., FRODO (20)] also included a host of other features, such as multiple renderings and real-time rotation, and became popular among structural biologists for model building (21). Computers were now used routinely for structure determination. Advances in computational technology and specific visualization metaphors also facilitated communication of molecular structural data. In the 1980s, personal computers allowed more users to participate in virtual exploration of molecular structures. Graphics on these computers were also improving. Monochrome renderings used in the early days were replaced by color. This enabled use of Corey-Pauling-Koltun (CPK) coloring schemes when displaying atoms. Polymer chains or segments of polymer chains could also be rendered in different colors. Inspirations from graphical illustrators such as Irving Geis (e.g., Figure 1B) and Roger Hayward helped users focus on specific aspects of the molecular structure (10). Molecular representation metaphors such as “backbones,” “ribbons,” and “surfaces” also helped bring these molecules to life. Drawing molecular structures using backbone or ribbon representation (e.g., Figure 1C) greatly facilitated the comparison of unrelated protein structures (22). Around this time the first tool for teaching molecular structure using virtual images, the Teaching Aids for Macromolecular Structure (TAMS), was also developed (23). The dawn of the Internet in the 1980s set the stage for generating information-rich images from public coordinate archives to communicate molecular structure-function relationships. In the late 1980s, the number of structures in the PDB dramatically increased due to the International Union of Crystallography (IUCr) policy on data deposition requiring 3D coordinate data to be deposited upon publication of all structures (24). Research investigators, instructors, and students alike could query the PDB for their favorite protein, access these data, and manipulate the structures for their own exploration and analysis. The 1990s were marked by development of user-friendly visualization software and broad virtual exploration of biomolecular structures by general users. David and Jane Richardson’s software package, first developed in 1992, could generate kinemages, or kinetic images, that could be rotated in real time, annotated, toggled, and used for the scientific illustration of structure and function (25). In 1995, RasMol (26) became popular for its use of shading for depth perception and aesthetically appealing images. Free access and compatibility with different operating systems contributed to RasMol’s rapid adoption by the biochemistry education community. Over time, educators and researchers, including structural biologists, demanded a plug-in to enable immediate visualization of structures. In response, MDL Information Systems released Chime (27) as an internet browser plug-in. Now journal subscribers could access structures as they read an article, and instructors could embed structural content on their webpages. Together, these programs provided the foundation for a host of other visualization programs that are routinely used today [e.g., PyMOL (28), Jmol (29), VMD (30), Cn3D (31), Swiss PDB Viewer (32), and UCSF Chimera (33)]. Since these early versions, updates and newer programs have adapted to be able to rapidly render molecular structure representations of the large structures. One such example is the structure of ribosome (Figure 1D), first determined in the early 2000s.

24

25

Figure 1. Timeline of developments in theory and technologies related to molecular structure determination and visualization from the nineteenth century to current times. A. Kekule’s 2D representation of the benzene molecule; B. Ball and stick representation of lysozyme (Illustration, Irving Geis. Rights owned by the Howard Hughes Medical Institute (www.hhmi.org). Not to be used without permission); C. Ribbon representation of the structure of myoglobin. Image from the RCSB PDB (rcsb.org) of PDB ID 1mbo (34); D. Space-filling representation of the structure of the ribosome as rendered by David Goodsell. Reproduced with permission from reference (35). Copyright RCSB PDB. CC-BY 4.0. Molecular representations in this figure are not drawn to scale. Today, millions of scientists, engineers, educators and students use the PDB as a core data science resource. To ensure that wwPDB remained the single, authoritative archive for macromolecular structure data worldwide, the Research Collaboratory for Structural Bioinformatics (RCSB PDB) co-founded the Worldwide Protein Data Bank (wwPDB) (36) organization in 2003 in collaboration with partners in Europe (PDBe) and Japan (PDBj). Within the wwPDB, the RCSB PDB acts as the archive keeper, and all partners develop, maintain, and provide web-based visualization software for virtual exploration: RCSB PDB uses NGL (37), PDBe uses LiteMol (38), and PDBj uses Molmil (39). Many other standalone and web-based versions of visualization software packages are also used for virtual exploration of biomolecules. The democratization of molecular visualization, however, comes with the risk of misuse and/or mis- or over-interpretation. The responsible use of macromolecular structural data, then, requires an understanding of the data, of the tools, and of the best practices in the instruction of visual literacy.

Virtual Exploration: Basic Requirements The goal of virtual exploration of biomolecular structure is twofold: (a) to visualize its shape and interactions in order to help understand the molecular basis of biological function and (b) to facilitate the prediction, design, and engineering of biomolecular properties and functions. Accurate interpretation and instruction of biomolecular structure requires the thoughtful consideration of two key components: understanding the 3D structural data that is visualized and learning how to optimally use the visualization tools. Appreciating the contributions of both these components can make biomolecular visualization meaningful and can maximize learning and communication for all users. Structural Data: Source, Assumptions, Errors Structural data are obtained either empirically or theoretically. Empirical data come from experiments including X-ray, NMR, and cryogenic electron microscopy (cryo-EM). More than 150,000 experimental structures of biological macromolecules and their various complexes are freely accessible from the PDB (40). Theoretical models, on the other hand, are often computed using a combination of homology modeling, knowledge of physico-chemical properties and behaviors, thermodynamics, and molecular dynamics (41). A collection of theoretical models is available from the Protein Modeling Portal (42). All models – both empirical and theoretical – are based on assumptions. Understanding these assumptions can alert students and users about model limitations and guide them in making inferences based on these structures. The model assumptions and limitations are not just confined to virtual exploration and should be considered in all types of molecular structure analysis.

26

Accuracy of theoretical models depends on the degree of similarity with a template, the chemical environment used for the model calculation, and the choice of algorithms. When a template model is not available, ab initio models can be computed, for example using amino acid side chain properties and energy minimization. More recently, theoretical and empirical models may be strategically combined with data from a variety of experiments to generate integrative hybrid models. Such hybrid models provide protein interaction data in near-atomic detail to interpret and communicate the significance of biological observations. When virtually exploring these integrative hybrid models, it is important to keep in mind that some regions of these models may be more accurate and/or reliable than others, depending on their provenance. Empirical models are also based on assumptions. Three of these assumptions are discussed here as they relate to sample composition, experiment-specific limitations, and experimental errors. First, one must carefully consider the contents of the experimental sample. Frequently, the sample is modified to facilitate the determination of its structure. Some proteins or regions of proteins are inherently disordered and remain refractory to experimental determination, except under particular conditions and/or in the presence of specific partner proteins. Some protein modifications are introduced to enhance protein expression, aid in solubility, and/or facilitate crystallization. Example of some of these modifications include: • Determination of only a subregion of the macromolecule. Loops, flexible regions, and intrinsically disordered subregions of a protein may be removed. Similarly, only the structure of one domain or a short peptide sequence of a protein may be determined. One such example is the tyrosine kinase domain of the human insulin receptor (PDB ID 1irk (43); Figure 2A), the transmembrane domain of which is refractory to crystallization methods. • Addition of a protein tag, domain, and/or linker. Fusion proteins are often used to facilitate the expression, solubilization, and/or crystallization of a molecule of interest. Such fusions may be as simple as a peptide or as large as a partner protein. One such example is the fusion of the yellow fluorescent protein with glutaredoxin via a linker peptide (PDB ID 2jad (44); Figure 2B). • Non-covalent association with accessory macromolecules. Juxtaposed with fusion proteins, non-covalent interacting macromolecules can also assist in stability and/or crystallization. One example is the use of Fab fragments in the structure determination of the human insulin receptor ectodomain (PDB ID 4zxb (45); Figure 2c). • Modification of sequence from its native form. For uniformity of sample, post-translational modifications of proteins may be stripped off, for example through deglycosylation or dephosphorylation. Proteins may also be mutated to mimic a particular state of the protein (e.g., aspartate or glutamate may mimic a phosphorylated protein) or to capture a specific intermediate form of the molecule/complex. One such example is the active site D498N mutation in the nuclease domain of HIV-1 reverse transcriptase to prevent catalysis in the nuclease domain (PDB ID 3v6d (46); Figure 2D). • Structure determination with a ligand, substrate, or cofactor. Similar to the modification by adding a peptide/protein domain, short peptides, cofactors, and other ligands may stabilize or be trapped in protein complex structures. One such example is the determination of the insulin receptor tyrosine kinase domain complexed with a cofactor (ATP) and substrate peptide (PDB ID 3bu5 (47); Figure 2E). Note the change in structure

27

between the apo form (Figure 2A) and cofactor/substrate form (Figure 2E), as an α-helix closes down on ATP and rearrangement of loops allows the substrate peptide to bind.

Figure 2. Engineered proteins used for molecular structure determination. A. Cytoplasmic domain of the insulin receptor tyrosine kinase in ribbon representation, PDB ID 1irk (43); B. Yellow fluorescent proteinglutaredoxin fusion protein shown in ribbon representation, PDB ID 2jad (44); C. Ribbon representation of human insulin receptor ectodomain in complex with four Fab fragments (in grey and circled), PDB ID 4zxb (45); D. HIV-1 reverse transcriptase, with an active site residue mutation (D498N), PDB ID 3v6d (46), the nuclease and polymerase domains in the structure are marked with circles; E. Cytoplasmic domain of the insulin receptor tyrosine kinase in ribbon representation, complexed with a substrate peptide (magenta and circled) and ATP (ball and stick representation and circled), PDB ID 3bu5 (47). All structural images in this figure were created with UCSF Chimera. A frequent assumption in the empirical determination of these modified structures is that these designed proteins simulate the native protein’s structure and function. For example, protein domains and short peptide sequences derived from proteins are assumed to retain their shape and function. For these structures of non-native conditions, non-structural empirical data (such as biochemical/ functional assays) must be used to confirm that these engineered macromolecules retain their nativelike structures. Therefore, before using any of the empirical structures, it is important to understand the sample composition and its relationship to the topic being studied. Second, one must be aware of the method of structure determination and of the nature of the deposited coordinate files. Careful attention should be given to incomplete or missing coordinates. When visualizing coordinates from the PDB, some structures may have missing sequence (e.g., Figure 2B) due to limited data quality, local movement in samples, or errors in measurement. In addition, deposited coordinates may represent different structural perspectives depending on the experimental method used: • Deposited coordinates of X-ray structures represent the asymmetric unit, or the smallest portion of the crystal to which symmetry operations can be applied to generate the complete unit cell. Symmetry and arrangement of biomolecules within the crystal are both 28

sample- and experiment-specific. The asymmetric unit of some X-ray structures include multiple copies of the protein, while others include an incomplete biologically functional unit (also called the biological assembly). For example, coordinates of an oxyhemoglobin structure includes only one copy each of an α- and β-hemoglobin chain (PDB ID 1hho (48); Figure 3A, top panel), despite existing as an α2β2 heterotetramer in its native structure. By applying twofold symmetry operations, coordinates of the other pair of hemoglobin chains is generated to represent the entire functional hemoglobin tetramer (Figure 3A, bottom panel). • Deposited coordinates of NMR structures represent ensembles of superimposed structures (e.g., Figure 3B, top panel). The representative model (usually model 1) is used for analysis, comparison and communication of NMR structures. The other models of the ensemble, however, have useful information about the range of molecular movements. For example, the representative model of human translation initiation factor eIF1A (PDB ID 1d7q (49); Figure 3B, bottom panel) suggests a single structure; the ensemble, however, reveals unstructured N- and C-terminal regions of the protein (Figure 3B, top panel). • Deposited coordinates for cryo-EM experiments for large assemblies with internal symmetry (e.g., cyclic, dihedral, helical, or icosahedral) include the smallest unit of symmetry. While this symmetry is an asset for structure determination, specific operations must be applied to the coordinate files of the repeating unit before examining structurefunction relationships of the whole macromolecule. For example the deposited file of the Dengue virus contains three polymer chains and a set of 60 matrices from which the structure of the complete virus may be generated (PDB ID 1k4r (50); Figure 3C).

Figure 3. Method- and experiment-specific manipulations of PDB coordinate data prior to virtual exploration. A. Structure of the oxyhemoglobin asymmetric unit (top) and biological assembly (bottom), PDB ID 1hho (48); B. Structure ensemble of human translation initiation factor eIF1A (top) and representative model (bottom), PDB ID 1d7q (49); C. Structure of the Dengue virus icosahedral unit (top) and complete virus (bottom), PDB ID 1k4r (50). All structural images in this figure were created with UCSF Chimera.

29

Third, one must be aware of any errors in the model. All experimental data and their use in structure determination can have limitations and errors when building empirical models. Here we focus on identifying and annotating errors in structure determination. Validation tools are used to compare each PDB structure against known geometric standards of bond length and angles, allowed conformational space for the molecule(s), known chemical interactions, including protein sequence data, and other data from relevant bioinformatics resources. Any deviations from standards or disagreement with information from other data resources are flagged (51). Comparison of the model with the original experimental data crosschecks the quality of the structure determination. While older structures archived in the PDB continue to provide valuable insights into structurefunction relationships, current tools have diagnosed some errors in these data. The PDB provides validation reports for each structure with relevant error diagnostics. Users of the structures are strongly encouraged to review these reports to be aware of any errors and the structure’s limitations. To test the validity of deposited structures, users can access experimental data available from the PDB to recompute molecular structures and compare them with those that are deposited. Validation reports are generated at the time of deposition of structures in the PDB; such reports are used to diagnose and assist in correcting errors before structures are publicly released. Structures that are not deposited in the PDB also can and should be validated to learn about any model errors and limitations. Visualization Tools: Affordances and Constraints Virtual exploration is one of the most easily accessible and cost-effective ways to interact with biomolecular structural data. Virtual exploration requires both software (to read atomic coordinates and render structures in multiple representations) and hardware (including a display space with controllers for interacting with the structure). Software may either be installed on a standalone computer or may be web-based requiring no special installation. 3D coordinate files of biomolecular structures may be loaded from local files, or they may be fetched from a database, such as the PDB. Commonly used stand-alone and web-based visualization software include PyMOL (28), Jmol (29), VMD (30), Cn3D (31) /iCn3D (52), Swiss PDB Viewer (32), and UCSF Chimera (33). For hardware, the most commonly used display space is a computer screen connected to a keyboard and mouse and/or touch screen. Today, with access to a variety of mobile devices, virtual exploration can be performed on the screens of cell phones and tablets. A wall or external screen can also be used for projection of molecular images. In these setups, interactions with the images may either be done in an immersive virtual environment, such as wearing stereo glasses and using a wand [e.g., Cave Automatic Virtual Environment (53)] or using a shadow-sensing infrared camera to detect the observer’s hand motions for interacting with the molecular images [e.g., The Molecular Playground (54)]. Using current visualization tools, users can virtually explore and manipulate 3D biomolecular structures to gain insights into the mechanisms of their functions. All visualization platforms allow users to examine the overall shape of the molecule/complex and its composition. Different renderings – e.g., surface, ribbons, and backbone – can be used to highlight key structural features and their functional relevance. Colors can help identify different polymer chains, trace the path of polymer chains, or denote specific properties of amino acid side chains in the protein/complex, such as charge, hydrophobicity, or evolutionary conservation. Ligands, ions, cofactors, or inhibitors and their specific interactions can be examined closely, as can interactions between partner proteins. Most visualization software have some basic analysis tools, e.g., for measuring distances and angles. Many tools allow structure comparisons, including superposition of related structures. Finally, some 30

programs allow simple modeling functions – such as mutating specific residues or simulating the docking of small molecules. To ensure that the resulting model is reasonable, modifications of the original deposited coordinates should be combined with several cycles of energy minimization. In short, visualization software programs enable users to interact with molecular structures to discover, analyze, and design specific interactions that in turn help users understand or design/engineer new features and functions. Since molecular structures available from the PDB represent static snapshots, users may be misled to believe that these molecules are rigid when, in fact, molecules are quite dynamic (e.g., Figure 3B). The majority of the current versions of visualization tools do not support exploration of movement or dynamics in biomolecules; nor do they provide easy ways to fill in the loops and regions not included in the empirical models. One way to explore the movement of molecules would be to visualize a series of individual structures related to the specific function. Advances in timeresolved structural analysis, such as X-ray free electron lasers (XFELs) (55), allow such resolution of movement. Newer virtual exploration tools are adding functionalities to visualize such data. Finally, the missing structures of domains, loops, and other amino acid residues from theoretical and empirical structural data can be modeled with a combination of knowledge from the scientific literature, animation tools, and educated imagination. With new technologies for structure determination rapidly emerging and pushing the boundaries of what we can “see,” requirements and assumptions for virtual exploration need to evolve. For example, our current visualization tools need to be updated to effectively and easily visualize multi-scale, near-atomic models, with information on dynamics emerging from integrative hybrid methods of structure determination.

Visual Comprehension: Practice and Assessment The aim of visual comprehension is to understand molecular mechanisms of known/ observed phenomenon and to provide insights for engineering new properties and functions. Thus, visual comprehension requires meaningful integration of information from the scientific literature and various bioinformatics resources with representations of 3D biomolecular structures to synthesize new knowledge. Molecular visualization converts structural data (diffraction patterns, chemical shifts, computational predictions) into representations of submicroscopic objects or machines - that is, they make the “unseeable” seen. Visualization tools also enable renderings of biomolecules to rapidly display connectivity and contours, revealing structural features and properties. Guidelines for 3D visual literacy (56) can help students make sense of 3D molecular representations. However, for complete visual comprehension, students need to understand the context of the structure – they must learn to integrate information from their virtual exploration of the molecule(s), with that from the scientific literature and bioinformatics resources, just as practicing experts do. Scientific Practices for Visual Comprehension Understanding structure-function relationship has been classified as a threshold concept in biochemistry (9), so visual comprehension (also called visual literacy) is important for biochemistry education (57). Virtual exploration of molecular structures, as discussed herein, is a metacognitive skill that requires simultaneous processing of visual stimuli, abstraction of theoretical knowledge into a model, and inference from visual representations (58). The challenge in training non-structural biologists and students is helping them connect their knowledge of chemical and biological principles, understanding of experimental data, familiarity with common representations and metaphors used in molecular visualizations, and understanding of biological contexts to make sense 31

of the structure and function of submicroscopic, abstract biomolecules. Effective instruction requires thoughtful consideration of the ways in which students interact with, process, and understand virtual representations of biomolecules and molecular phenomena (57, 59). Therefore, instruction in visual literacy should engage cognitive psychologists in addition to educators and scientists (60). One approach for teaching visual comprehension could be adopting a molecular storytelling approach (61). Students can use virtual exploration of biomolecules to do the following: ask questions related to the subject; explore and read related scientific literature; gather and integrate information from authentic bioinformatics resources; identify relevant empirical structures in the PDB; visualize, analyze, and compare molecular structures to understand their functions or to design new molecules with novel properties; integrate various types of data and information to generate new knowledge; and justify their conclusions using evidence from these resources. These steps utilize virtual molecular visualization and this approach to learning is likely to lead to a deeper comprehension of the topic being studied. Virtual molecular visualization can also be used to excite, engage, and spark curiosity in students who might otherwise struggle to comprehend the abstractions of a submicroscopic world. Assessment of Virtual Biomolecular Visual Literacy Visual literacy occurs at all levels of Bloom’s taxonomy, from knowledge acquisition through synthesis (62). While some levels of cognition (e.g., knowledge and comprehension) are addressed through classroom demonstration, others (e.g., analysis and synthesis) require more hands-on activities. A recent survey of over 100 biochemistry educators showed that 90% of respondents used virtual visualization for classroom demonstrations (63). This suggests that opportunities to increase student knowledge and comprehension are growing. However, respondents in the same survey frequently requested access to a central repository of tools and assessment instruments for visual literacy. In the absence of such tools, students are often expected to intuit meaning from the myriad representations they see in the classroom, and educators do not have a clear way to assess learning. This lack of clarity is often compounded by the oversimplification of models which, in the process of simplification, can lead to or reinforce misconceptions (64). The simultaneous processing of visual information and conceptual knowledge can result in heavy cognitive load, where multiple pieces of information must first be integrated before learning can occur (65). For example, the inability of a student to identify the active site of a protein may be due to a host of misconceptions, from conceptual knowledge of what an active site is to the inability to distinguish between a substrate and its enzyme. Biomolecular visual literacy, then, requires deconstruction into discrete tasks that can be individually assessable. For example, Schonborn and Anderson identified eight cognitive skills that are critical to biochemical literacy, such as the ability to “decode the symbolic language composing an external representation (59).” Building on this work, Offerdahl, Arneson, and Byrne identified a five-component taxonomy to describe the different types of abstractions students are expected to understand throughout a biochemistry curriculum (66). By defining cognitive tasks, naming them, and classifying them, instructors can create assessment instruments to probe the ability of students to manage a cognitive load before attempting to assess students’ ability to interpret virtual representations of biomolecular structures, thereby disentangling cognitive load from visual literacy. Once students have established a visual literacy vocabulary, they can then begin to manage the cognitive load required for meaningful interpretation of virtual molecular representations. Arneson and Offerdahl have created the Visualization Blooming Tool (VBT, based on Bloom’s taxonomy 32

of learning), a catalog of the various levels of cognition and skills required for biomolecular visual literacy (62). The VBT allows for the creation of assessment tools for discrete visualization tasks, such as comparing between two different representations of the same biomolecule. In 2017, the BioMolecular Visualization (BioMolViz.org) group published a framework that unpacked biomolecular visual literacy into overarching goals, learning goals, and learning objectives (67, 68). The BioMolViz group has since built a network of biochemistry and molecular biology educators to develop tools and instruments for the explicit assessment of biomolecular visual literacy by crowdsourcing their development at regional workshops (69). Irby, Pelaez, and Anderson have also identified anticipated learning outcomes for the design of assessment instruments in Course-based Undergraduate Research Experiences (CUREs), such as the BASIL project, which combines wetlab experiments with virtual exploration of protein structure (70). Assessment efforts such as these are not only wanted by the biochemistry education community (63) but are also necessary for the explicit assessment of virtual biomolecular visualization distinct from the explicit assessment of cognition itself.

Virtual Exploration of Biomolecules in the Classroom and Beyond Educational efforts related to virtual exploration of biomolecular structures range from demonstrations to fully-immersive exploration, from classroom instruction to research laboratories and beyond. Examples of education in virtual exploration of biomolecules are presented herein, grouped under educational resources (contents available online for all users); student research (developed and implemented locally, by one or a few faculty); and community engagement (including citizen science projects). While this list of examples is by no means comprehensive, they are included to inform readers and inspire them explore and/or develop their own projects for virtual exploration of biomolecules. Some key resources for virtual explorations of biomolecular structures are summarized in Table 1. Educational Resources One of the earliest efforts to assemble a collection of macromolecular structural images was made by Richard Feldman in 1976 (87). With the advent of the internet, many other projects for virtual exploration of biomolecular structure and function led to the development of web-based resources. For example, Eric Martz’s compilation of tutorials [MolviZ.org (77),] introduces a variety of molecules beyond proteins and DNA, including lipid membranes, water, and the periodic table of elements. The website links to a multitude of secondary visualization tools for educators, including other tutorials, physical models and sculpture, and a history of molecular visualization. Martz also developed the Atlas of Macromolecules (84) for virtual representations of a selection of biomolecules. In 1996, David Marcey created a collection of virtual interactive biomolecular renderings, called the Online Macromolecular Museum (85), which continues to serve students and educators. Since 2000, David Goodsell has written the RCSB PDB Molecule of the Month series (73, 88) to present a new biomolecular structure every month. These pages feature the biological context of a molecule alongside a JSmol interactive window for rotation and multiple renderings. Discussion questions offer educators further material for class use. Proteopedia (86, 89) independently presents webpages for every structure in the PDB archive. It also provides tools for annotating these structures and interacting on a virtual platform to tell molecular stories. This ease of use makes Proteopedia a 33

popular resource in the biochemistry classroom. Some of the most popular molecular structures for biology and biochemistry education, such as hemoglobin, trypsin, and the ribosome, have received input from many educators worldwide and have expanded pages with information about active site analysis, catalysis, and clinical relevance. The Center for Biomolecular Modeling at the Milwaukee School of Engineering (75) focuses on creating physical models based on PDB data for studying biology (90) but also develops educational materials for virtual exploration, organizes challenges (91), and runs research programs for high school and undergraduate students (92, 93). Table 1. A Selection of Resources for Instruction and Assessment of Biomolecular Visual Literacya Biomolecular visualization software

Ref.

Classroom Resources

Ref.

Jmol

(29)

PDB-101 (Teach; Learn)

(71)

iCn3D

(72)

Molecule of the Month

(73)

PyMol

(74)

Center for Biomolecular Modeling

(75)

UCSF Chimera

(76)

MolviZ.org

(77)

Swiss PDB Viewer

(78)

Molecular CaseNet

(79)

Visual Molecular Dynamics (VMD)

(80)

Biochemistry Authentic Scientific Inquiry Lab

(81)

FoldIt

(82)

Databases and Libraries

Ref.

Assessment of visual literacy

Ref.

Protein Data Bank (wwPDB)

(83)

Cognitive skills for biochemical literature

(59)

Atlas of Macromolecules

(84)

Visualization Blooming Tool

(62)

Online Macromolecular Museum

(85)

Taxonomy of abstractions

(66)

Proteopedia

(86)

Biomolecular Visualization Framework

(68)

a This list is taken from the resources cited in this chapter and is by no means exhaustive.

Inspired by the popularity of the Molecule of the Month features, and to find a home for RCSB PDB’s growing collection of virtual exploration and education-related materials, an educational resource called PDB-101 was established (94). This resource is focused on training novice users and introducing the power of structural biology to diverse audiences. PDB-101 offers a host of other educational tools, including curricular modules developed in collaboration with educators, interactive animations, and printable templates for origami-like paper models of biomolecules. Of particular note is the Guide to Understanding PDB Data, an excellent resource for the explanation of structural data for the novice. Features include information on methods of structure determination, the purposes of multiple renderings, structure factors and electron density, and how to understand ligands within PDB entries. In 2018, a group of biology, chemistry, biochemistry undergraduate educators formed the Molecular CaseNet to develop molecular case studies for undergraduate education. Cases developed by this group focus on exploring and understanding specific processes in biology by visualizing them in atomic detail. These cases will be shared through PDB-101 for use by undergraduate educators.

34

Student Research Molecular visualization provides a unique opportunity to engage students in exploration and authentic scientific inquiry. The following are two examples of authentic scientific inquiry. First, starting in 2006, a theme-based, interdisciplinary honors seminar called Molecular Anatomy Project was developed (95). Students were introduced to the foundation of molecular structures and interactions, the course theme (a global health topic, such as diabetes and antimicrobial resistance), virtual exploration tools, and various bioinformatics resources. Through the course they learned to critically read the scientific literature, identify and explore molecular structures relevant to the global health course theme, and write research papers presenting a molecular perspective for causes, diagnosis, monitoring, and treatment options. Several of these student-authored reports were reviewed by expert scientists and/or clinicians before being published on the PDB-101 Global Health pages (96). As a second example, a multi-institute collaborative project was initiated in 2015 as a Course-based Undergraduate Research Experience (CURE). In the Biochemistry Authentic Scientific Inquiry Lab (BASIL) (81), biochemistry laboratory courses were redesigned to include modules in which students integrated computational and wet lab techniques to characterize protein structures with unknown functions (97). Projects such as these two move molecular visualization from a largely aesthetic activity to one of exploration and discovery. Community Engagement Biomolecular structures have inspired artists to share their interpretations of protein function and cellular machinery, using form, color, and symmetry to draw the viewer into a submicroscopic world (98). Such artwork can both engage and become a means for students and general users to participate in virtual exploration of biomolecular structure and function. One example is the Molecular Machinery poster (cdn.rcsb.org/pdb101/molecular-machinery/), developed by David Goodsell and the RCSB PDB staff, which presents a clickable showcase of protein structures with a wide array of geometries, sizes, and locations of macromolecules (99). Biomolecular structures have also inspired computer scientists to entice gamers into the exploration of biomolecular structure. Those interested in virtual reality and gaming may be intrigued by the FoldIt game (82) as a proxy for the virtual exploration of biomolecular structures. Students and educators can use this game as a means for learning about protein folding and intermolecular interactions. Serious gamers can participate in solving protein folding puzzles posted by scientists to help solve their structures, thereby contributing to authentic research (100). This mode of engaging students and gamers in a citizen science project has helped uncover new protein families and solve challenging problems in structural biology (101).

Conclusions Virtual exploration of biomolecular structures is less than five decades old. Initially, only specialists like structural biologists and scientists had access to the computational resources necessary for such visualization. However, the motivations for using and accessing virtual exploration tools have since changed. Access to computers (including personal computers, tablets, and cell phones), 3D structural data, and a number of software programs and resources has enabled virtual exploration of biomolecules in undergraduate education and research. Students can now use visualization software to display atomic coordinates of biomolecular structures and to virtually interact with the model by using command line instructions and/or menu-driven controls. Researchers can also use 35

virtual explorations of biomolecular structures to understand biological processes, integrate results from biochemical experiments to ask new research questions, and/or rationally design molecules with new properties and functions. Visual comprehension routinely facilitates the practice of science and has many academic, medical, and commercial applications. In preparing the next generation of scientists, educators, and citizens, novel approaches are being developed to engage students in learning about biological macromolecules (e.g., authentic scientific research projects, case studies, immersive environments, and games that double as citizen science projects). Rapid adaptability, portability, and universal access of data from public archives has enabled virtual exploration of biomolecular structures. All users would do well to understand the source, nature, and limitations of these biomolecular structures so that they can make meaningful inferences from their virtual explorations. Parallel development of educational resources and student assessment guidelines will truly help engage more educators in using molecular visualization in their curricula. Lessons, activities, and experiences that bridge the gap between virtual exploration and visual comprehension to synthesize new knowledge will truly integrate virtual exploration of biomolecular structures in the practice of science. As we enter an era of virtual and augmented reality, structural exploration becomes more and more immersive, bringing the user directly inside the molecule. Through virtual exploration, scientists and educators can present the beauty of biomolecular architecture to inspire student learning and creativity.

References 1. 2. 3.

4. 5. 6. 7.

8.

9.

Olson, A. J. Perspectives on Structural Molecular Biology Visualization: From Past to Present. J. Mol. Biol. 2018, 430, 3997–4012. Watson, J. D.; Crick, F. H. Molecular structure of nucleic acids; a structure for deoxyribose nucleic acid. Nature 1953, 171, 737–738. Avery, O. T.; Macleod, C. M.; McCarty, M. Studies on the Chemical Nature of the Substance Inducing Transformation of Pneumococcal Types : Induction of Transformation by a Desoxyribonucleic Acid Fraction Isolated from Pneumococcus Type III. J. Exp. Med. 1944, 79, 137–158. Hershey, A. D.; Chase, M. Genetic recombination and heterozygosis in bacteriophage. Cold Spring Harb. Symp. Quant. Biol. 1951, 16, 471–479. Zamenhof, S.; Brawermann, G.; Chargaff, E. On the desoxypentose nucleic acids from several microorganisms. Biochim. Biophys. Acta 1952, 9, 402–405. Crick, F. H. C.; Watson, J. D. The complementary structure of deoxyribonucleic acid. Proc. R. Soc. London, Ser. A 1954, 223, 80–96. ACS Committee on Professional Training. Undergraduate Professional Education in Chemistry: ACS Guidelines and Evaluation Procedures for Bachelor’s Degree Programs; American Chemical Society: Washington, DC, 2015. Tansey, J. T.; Baird, T., Jr.; Cox, M. M.; Fox, K. M.; Knight, J.; Sears, D.; Bell, E. Foundational concepts and underlying theories for majors in “biochemistry and molecular biology”. Biochem. Mol. Biol. Educ. 2013, 41, 289–296. Loertscher, J.; Green, D.; Lewis, J. E.; Lin, S.; Minderhout, V. Identification of threshold concepts for biochemistry. CBE Life Sci Educ 2014, 13, 516–528.

36

10. Perkins, J. A. A History of Molecular Representation, Part One: 1800 to the 1960s. Journal of Biocommunication 2005, 1. 11. Dalton, J. New System of Chemical Philosophy; Manchester: London, UK, 1808. 12. Van’t Hoff, J. H. The Arrangement of Atoms in Space; Longmans, Green, and Co.: London, 1898. 13. Pauling, L. The nature of the chemical bond: Application of results obtained from the quantum mechanics and from a theory of paramagnetic susceptibility to the structure of molecules. J. Am. Chem. Soc. 1931, 53, 1367–1400. 14. Kendrew, J. C.; Bodo, G.; Dintzis, H. M.; Parrish, R. G.; Wyckoff, H.; Phillips, D. C. A threedimensional model of the myoglobin molecule obtained by x-ray analysis. Nature 1958, 181, 662–666. 15. Levinthal, C. Are there pathways for protein folding? Extrait du Journal de Chimie Physique 1968, 65, 44–45. 16. Levinthal, C.; Barry, C. D.; Ward, S. A.; Zwick, M. Computer Graphics in Macromolecular Chemistry. In Emerging Concepts in Computer Graphics; Secrest, J. N. a. D., Ed.; New York, 1968. 17. Engelbart, D. The Demo. 1968.https://web.stanford.edu/dept/SUL/library/extra4/sloan/ MouseSite/1968Demo.html (accessed July 3, 2019). 18. Protein Data Bank Crystallography: Protein Data Bank. Nature (London), New Biol. 1971, 233, 223–223. 19. Beem, K. M.; Richardson, D. C.; Rajagopalan, K. V. Metal sites of copper-zinc superoxide dismutase. Biochemistry 1977, 16, 1930–1936. 20. Jones, T. A. FRODO: A graphic model building and refinement system for macromolecules. J. Appl. Crystallogr. 1978, 11, 268–272. 21. Martz, E. F. E. History of Visualization of Biological Macromolecules; 2004. https://www.umass. edu/microbio/rasmol/history.htm (accessed April 15, 2019). 22. Richardson, J. S. The anatomy and taxonomy of protein structure. Adv. Protein Chem. 1981, 34, 167–339. 23. de Leeuw, H. P. M.; Haasnoot, A. G.; Altona, C. Empirical correlations between conformational parameters in beta-d-furanoside fragments derived from a statistical survey of crystal structures of nucleic acid constituents. Full description of nucleoside molecular geometries in terms of four parameters. Isr. J. Chem. 1980, 20, 108–126. 24. International Union of Crystallography Policy on publication and the deposition of data from crystallographic studies of biological macromolecules. Acta Crystallogr. 1989, A45, 658. 25. Richardson, D. C.; Richardson, J. S. The kinemage: a tool for scientific communication. Protein Sci. 1992, 1, 3–9. 26. Sayle, R.; Milner-White, E. J. RasMol: biomolecular graphics for all. Trends Biochem. Sci. 1995, 20, 374. 27. Martz, E. Protein Explorer: easy yet powerful macromolecular visualization. Trends Biochem. Sci. 2002, 27, 107–109. 28. DeLano, W. L. The PyMOL Molecular Graphics System; 2002. http://www.pymol.org (accessed Nov 6, 2015). 29. Group, J. D. Jmol: An Open-Source Java Viewer for Chemical Structures in 3D; 2008. http://www. jmol.org/ (accessed Jan 1, 2018). 37

30. Humphrey, W.; Dalke, A.; Schulten, K. VMD: visual molecular dynamics. J. Mol. Graphics 1996, 14, 33–38. 31. Hogue, C. W. Cn3D: a new generation of three-dimensional molecular structure viewer. Trends Biochem. Sci. 1997, 22, 314–316. 32. Guex, N.; Peitsch, M. C. SWISS-MODEL and the Swiss-PdbViewer: An environment for comparative protein modeling. Electrophoresis 1997, 18, 2714–2723. 33. Pettersen, E. F.; Goddard, T. D.; Huang, C. C.; Couch, G. S.; Greenblatt, D. M.; Meng, E. C.; Ferrin, T. E. UCSF Chimera--a visualization system for exploratory research and analysis. J. Comput. Chem. 2004, 25, 1605–1612. 34. Phillips, S. E. Structure and refinement of oxymyoglobin at 1.6 A resolution. J. Mol. Biol. 1980, 142, 531–554. 35. Goodsell, D. Ribosomal Subunits; Molecule of the Month, October 2000. DOI: 10.2210/ rcsb_pdb/mom_2000_10. http://pdb101.rcsb.org/motm/10 (accessed April 15, 2019). 36. Berman, H. M.; Henrick, K.; Nakamura, H. Announcing the worldwide Protein Data Bank. Nat. Struct. Biol. 2003, 10, 980. 37. Rose, A. S.; Bradley, A. R.; Valasatava, Y.; Duarte, J. M.; Prlić, A.; Rose, P. W. NGL viewer: web-based molecular graphics for large complexes. Bioinformatics 2018, bty419. 38. Sehnal, D.; Deshpande, M.; Varekova, R. S.; Mir, S.; Berka, K.; Midlik, A.; Pravda, L.; Velankar, S.; Koca, J. LiteMol suite: interactive web-based visualization of large-scale macromolecular structure data. Nat. Methods 2017, 14, 1121–1122. 39. Kinjo, A. R.; Bekker, G. J.; Wako, H.; Endo, S.; Tsuchiya, Y.; Sato, H.; Nishi, H.; Kinoshita, K.; Suzuki, H.; Kawabata, T.; Yokochi, M.; Iwata, T.; Kobayashi, N.; Fujiwara, T.; Kurisu, G.; Nakamura, H. New tools and functions in data-out activities at Protein Data Bank Japan (PDBj). Protein Sci. 2018, 27, 95–102. 40. Berman, H. M.; Westbrook, J.; Feng, Z.; Gilliland, G.; Bhat, T. N.; Weissig, H.; Shindyalov, I. N.; Bourne, P. E. The Protein Data Bank. Nucleic Acids Res. 2000, 28, 235–242. 41. Khor, B. Y.; Tye, G. J.; Lim, T. S.; Choong, Y. S. General overview on structure prediction of twilight-zone proteins. Theor. Biol. Med. Modell. 2015, 12, 15. 42. Haas, J.; Roth, S.; Arnold, K.; Kiefer, F.; Schmidt, T.; Bordoli, L.; Schwede, T. The Protein Model Portal--a comprehensive resource for protein structure and model information. Database (Oxford) 2013, 2013, bat031. 43. Hubbard, S. R.; Wei, L.; Ellis, L.; Hendrickson, W. A. Crystal structure of the tyrosine kinase domain of the human insulin receptor. Nature 1994, 372, 746–754. 44. Hakansson, K. O.; Winther, J. R. Structure of glutaredoxin Grx1p C30S mutant from yeast. Acta Crystallogr., Sect. D: Biol. Crystallogr. 2007, 63, 288–294. 45. Croll, T. I.; Smith, B. J.; Margetts, M. B.; Whittaker, J.; Weiss, M. A.; Ward, C. W.; Lawrence, M. C. Higher-Resolution Structure of the Human Insulin Receptor Ectodomain: Multi-Modal Inclusion of the Insert Domain. Structure 2016, 24, 469–476. 46. Das, K.; Martinez, S. E.; Bauman, J. D.; Arnold, E. HIV-1 reverse transcriptase complex with DNA and nevirapine reveals non-nucleoside inhibition mechanism. Nat. Struct. Mol. Biol. 2012, 19, 253–259.

38

47. Wu, J.; Tseng, Y. D.; Xu, C. F.; Neubert, T. A.; White, M. F.; Hubbard, S. R. Structural and biochemical characterization of the KRLB region in insulin receptor substrate-2. Nat. Struct. Mol. Biol. 2008, 15, 251–258. 48. Shaanan, B. Structure of human oxyhaemoglobin at 2.1 A resolution. J. Mol. Biol. 1983, 171, 31–59. 49. Battiste, J. L.; Pestova, T. V.; Hellen, C. U.; Wagner, G. The eIF1A solution structure reveals a large RNA-binding surface important for scanning function. Mol. Cell 2000, 5, 109–119. 50. Kuhn, R. J.; Zhang, W.; Rossmann, M. G.; Pletnev, S. V.; Corver, J.; Lenches, E.; Jones, C. T.; Mukhopadhyay, S.; Chipman, P. R.; Strauss, E. G.; Baker, T. S.; Strauss, J. H. Structure of dengue virus: implications for flavivirus organization, maturation, and fusion. Cell 2002, 108, 717–725. 51. Gore, S.; Sanz Garcia, E.; Hendrickx, P. M. S.; Gutmanas, A.; Westbrook, J. D.; Yang, H.; Feng, Z.; Baskaran, K.; Berrisford, J. M.; Hudson, B. P.; Ikegawa, Y.; Kobayashi, N.; Lawson, C. L.; Mading, S.; Mak, L.; Mukhopadhyay, A.; Oldfield, T. J.; Patwardhan, A.; Peisach, E.; Sahni, G.; Sekharan, M. R.; Sen, S.; Shao, C.; Smart, O. S.; Ulrich, E. L.; Yamashita, R.; Quesada, M.; Young, J. Y.; Nakamura, H.; Markley, J. L.; Berman, H. M.; Burley, S. K.; Velankar, S.; Kleywegt, G. J. Validation of the Structures in the Protein Data Bank. Structure 2017, 25, 1916–1927. 52. Coordinators, N. R. Database Resources of the National Center for Biotechnology Information. Nucleic Acids Res. 2017, 45, D12–D17. 53. DeFantia, T. A.; Dawea, G.; Sandinb, D. J.; Schulzea, J. P.; Ottoa, P.; Giradoc, J.; Kuestera, F.; Smarra, L.; Rao, R. The StarCAVE, a third-generation CAVE and virtual reality OptIPortal. Future Generation Computer Systems 2009, 25, 169–178. 54. Martin, C. T. Molecular Playground: Architechture Scale Interactive Molecules. http://www. molecularplayground.org/ (accessed April 15, 2019). 55. Johansson, L. C.; Stauch, B.; Ishchenko, A.; Cherezov, V. A Bright Future for Serial Femtosecond Crystallography with XFELs. Trends Biochem. Sci. 2017, 42, 749–762. 56. Dries, D. R.; Dean, D. M.; Listenberger, L. L.; Novak, W. R. P.; Franzen, M. A.; Craig, P. A. An expanded framework for biomolecular visualization in the classroom: Learning goals and competencies. Biochem. Mol. Biol. Educ. 2016. 57. Schonborn, K. J.; Anderson, T. R. The importance of visual literacy in the education of biochemists*. Biochem. Mol. Biol. Educ. 2006, 34, 94–102. 58. Gilbert, J. K. Visualization: A Metacognitive Skill in Science and Science Education. In Visualization in Science Education; Gilbert, J. K., Ed.; Springer: Dordrecht, 2005; pp 9−27. 59. Schonborn, K. J.; Anderson, T. R. Bridging the educational research-teaching practice gap: Foundations for assessing and developing biochemistry students’ visual literacy. Biochem. Mol. Biol. Educ. 2010, 38, 347–354. 60. NSF. Molecular Visualization in Science Education; NSF: NCSA Access Center: Arlington, VA, 2001. 61. Dutta, S.; Eswaran, S.; Sanelli, A.; Bhattacharya, M.; Tempsick, R. Learning Biology Through Molecular Storytelling. The Science Teacher 2018, 86, 28–33. 62. Arneson, J. B.; Offerdahl, E. G. Visual Literacy in Bloom: Using Bloom’s Taxonomy to Support Visual Learning Skills. CBE Life Sci Educ 2018, 17. 39

63. Craig, P. A.; Michel, L. V.; Bateman, R. C. A survey of educational uses of molecular visualization freeware. Biochem. Mol. Biol. Educ. 2013, 41, 193–205. 64. Tasker, R.; Dalton, R. Research into practice: visualisation of the molecular world using animations. Chem. Educ. Res. Pract. 2006, 72, 141–159. 65. Chandler, P.; Sweller, J. Cognitive Load Theory and the Format of Instruction. Cognition and Instruction 1991, 8, 292–332. 66. Offerdahl, E. G.; Arneson, J. B.; Byrne, N. Lighten the Load: Scaffolding Visual Literacy in Biochemistry and Molecular Biology. CBE Life Sci Educ 2017, 16. 67. Dries, D. R.; Dean, D. M.; Listenberger, L. L.; Novak, W. R.; Franzen, M. A.; Craig, P. A. An expanded framework for biomolecular visualization in the classroom: Learning goals and competencies. Biochem. Mol. Biol. Educ. 2017, 45, 69–75. 68. BioMolViz-Group. Molecular Visualization Proficiency Rubric; 2017. http://cbm.msoe.edu/ crest/molviz/ (accessed July 3, 2019). 69. Jakubowski, H.; Franzen, M.; Dries, D. Do you see what I see? Regional workshops to assess biomolecular visual literacy. ASBMB Today; 2017. https://www.asbmb.org/asbmbtoday/ 201712/Perspective/VISUAL/ (accessed July 1, 2019). 70. Irby, S. M.; Pelaez, N. J.; Anderson, T. R. How to Identify the Research Abilities That Instructors Anticipate Students Will Develop in a Biochemistry Course-Based Undergraduate Research Experience (CURE). CBE Life Sci. Educ. 2018, 17, es4. 71. PDB-101. http://pdb101.rcsb.org/ (accessed July 2, 2019). 72. iCn3D; 2017. https://www.ncbi.nlm.nih.gov/Structure/icn3d/full.html (accessed July 3, 2019). 73. PDB-101. Molecule of the Month; 2000; http://pdb101.rcsb.org/motm/motm-by-date (accessed July 1, 2019). 74. DeLano, W. L. PyMOL, Schrodinger. https://pymol.org/ (accessed July 3, 2019). 75. Center for BioMolecular Modeling; 1995. http://cbm.msoe.edu/ (accessed July 3, 2019). 76. UCSF Chimera. https://www.cgl.ucsf.edu/chimera/ (accessed July 3, 2019). 77. Martz, E. MolviZ.org; 2002. https://www.umass.edu/microbio/chime/ (accessed July 3, 2019). 78. Guex, N. Swiss-PdbViewer; 1994. https://spdbv.vital-it.ch/ (accessed July 3, 2019). 79. Molecular CaseNet; 2018. https://molecular-casenet.rcsb.org/ (accessed July 3, 2019). 80. Visual Molecular Dynamics (VMD); 2006. https://www.ks.uiuc.edu/Research/vmd/ (accessed 3 July 2019). 81. BASIL: Biochemistry Authentic Scientific Inquiry Lab. http://basiliuse.blogspot.com/ (accessed July 3, 2019). 82. foldit. https://fold.it/portal/ (accessed July 3, 2019). 83. worldwide PDB. http://www.wwpdb.org/ (accessed July 3, 2019). 84. Martz, E. Atlas of Macromolecules; 2002; https://www.bioinformatics.org/molvis/atlas/atlas. htm (accessed July 3, 2019). 85. Marcey, D. The Online Macromolecular Museum; 1996. https://earth.callutheran.edu/ Academic_Programs/Departments/BioDev/omm/about.html (accessed July 3, 2019). 86. Sussman, J.; Prilusky, J. Proteopedia; 2008. http://proteopedia.org/wiki/index.php/Main_ Page (accessed July 3, 2019). 40

87. Feldman, R. J. AMSOM: Atlas of Macromolecular Structure on Microfiche; Tracor-Jitco: Rockville, MD, 1976. 88. Goodsell, D. S.; Dutta, S.; Zardecki, C.; Voigt, M.; Berman, H. M.; Burley, S. K. The RCSB PDB “Molecule of the Month”: Inspiring a Molecular View of Biology. PLoS Biol. 2015, 13, e1002140. 89. Hodis, E.; Prilusky, J.; Martz, E.; Silman, I.; Moult, J.; Sussman, J. L. Proteopedia - a scientific ‘wiki’ bridging the rift between three-dimensional structure and function of biomacromolecules. Genome Biol. 2008, 9, R121. 90. Herman, T.; Morris, J.; Colton, S.; Batiza, A.; Patrick, M.; Franzen, M.; Goodsell, D. S. Tactile teaching: Exploring protein structure/function using physical models. Biochem. Mol. Biol. Educ. 2006, 34, 247–254. 91. Herman, T. Teaching Science Through Protein Modeling. ASBMB Today, 2010. https://www. asbmb.org/asbmbtoday/asbmbtoday_article.aspx?id=9158 (accessed April 15, 2019). 92. Herman, T.; Colton, S.; Franzen, M. Rethinking outreach: teaching the process of science through modeling. PLoS Biol. 2008, 6, e86. 93. Franzen, M.; Herman, T.; Harris, M. CREST: Connecting Researchers, Educators and Students. In Proceedings of Envisioning the Future of Undergraduate STEM Education: Research and Practice Symposium; American Association for the Advancement of Science: Washington, DC, 2016. 94. Rose, P. W.; Bi, C.; Bluhm, W. F.; Christie, C. H.; Dimitropoulos, D.; Dutta, S.; Green, R. K.; Goodsell, D. S.; Prlic, A.; Quesada, M.; Quinn, G. B.; Ramos, A. G.; Westbrook, J. D.; Young, J.; Zardecki, C.; Berman, H. M.; Bourne, P. E. The RCSB Protein Data Bank: new resources for research and education. Nucleic Acids Res. 2013, 41, D475–482. 95. Dutta, S.; Zardecki, C.; Goodsell, D. S.; Berman, H. M. Promoting a structural view of biology for varied audiences: an overview of RCSB PDB resources and experiences. J. Appl. Crystallogr. 2010, 43, 1224–1229. 96. Global Health: Diabetes Mellitus. http://pdb101.rcsb.org/global-health/diabetes-mellitus/ (accessed July 3, 2019). 97. Craig, P. A. Lessons from my undergraduate research students. J. Biol. Chem. 2018, 293, 10447–10452. 98. Goodsell, D. S. Molecular Machines. In The Machinery of Life, 2nd ed.; Copernicus Books: New York, 2009; pp 9−28. 99. Goodsell, D. S.; Jenkinson, J. Molecular Illustration in Research and Education: Past, Present, and Future. J. Mol. Biol. 2018, 430, 3969–3981. 100. Cooper, S.; Khatib, F.; Treuille, A.; Barbero, J.; Lee, J.; Beenen, M.; Leaver-Fay, A.; Baker, D.; Popovic, Z.; Players, F. Predicting protein structures with a multiplayer online game. Nature 2010, 466, 756–760. 101. Horowitz, S.; Koepnick, B.; Martin, R.; Tymieniecki, A.; Winburn, A. A.; Cooper, S.; Flatten, J.; Rogawski, D. S.; Koropatkin, N. M.; Hailu, T. T.; Jain, N.; Koldewey, P.; Ahlstrom, L. S.; Chapman, M. R.; Sikkema, A. P.; Skiba, M. A.; Maloney, F. P.; Beinlich, F. R.; Foldit, P.; Popovic, Z.; Baker, D.; Khatib, F.; Bardwell, J. C. Determining crystal structures through crowdsourcing and coursework. Nat. Commun. 2016, 7, 12549.

41

Chapter 3

Physical Models Support Active Learning as Effective Thinking Tools Cassidy R. Terrell,*,1 Margaret A. Franzen,2 Timothy Herman,2 Sunil Malapati,3 Dina L. Newman,4 and L. Kate Wright4 1Center for Learning Innovation, University of Minnesota, 111 S. Broadway,

Rochester, Minnesota 55904, United States 2Center for BioMolecular Modeling, Milwaukee School of Engineering, 1025 N. Broadway, Milwaukee, Wisconsin 53202, United States 3Chemistry, Clarke University, 1550 Clarke Drive, Dubuque, Iowa 52001, United States 4Thomas H. Gosnell School of Life Sciences, Rochester Institute of Technology, 85 Lomb Memorial Drive, Rochester, New York 14623, United States *E-mail: [email protected].

From the perspective of a novice student, the molecular biosciences are inherently invisible. A challenge facing bioscience educators is to help students create detailed mental models of the biomolecules that make up a living cell and how they all work together to support life. With the advancement of rapid-prototyping, also known as 3D (three dimensional)-printing, physical models of biomolecules are entering undergraduate classrooms as tools to aid in constructing mental models of biological phenomena at the molecular-level. This relatively new pedagogical tool requires evidence-based practices for optimal use in aiding student conceptual and visual development. This chapter presents current evidence for the use of physical models as learning tools, while also introducing case studies on how physical models of biomolecules are designed and assessed in undergraduate molecular bioscience settings.

Introduction In this chapter, we focus on development, use and assessment of biomolecular physical models in undergraduate molecular bioscience education. Here “molecular biosciences” encompasses any course utilizing concepts featuring biomolecules, ranging from monomers to macromolecules that support life on the molecular level (e.g. introductory biology; general, organic, and biochemistry (GOB); biochemistry, molecular biology, and cellular biology courses). At the undergraduate level, Vision and Change identifies modeling and simulation as core competency and disciplinary practices © 2019 American Chemical Society

(1). The Next Generation Science Standards (NGSS) Framework definition of models includes diagrams, physical replicas, mathematical representations, analogies, and computer simulations that are tools for the student to engage in “developing questions, making predictions and explanations, analyzing and identifying flaws in systems, and communicating ideas (2).” Moreover, models provide an opportunity for the student to engage in an iterative process of “comparing their predictions with the real world and then adjusting them to gain insights into the phenomenon being modeled (2).” This is the vein in which the potential power of models lies, as many authors suggest that learning barriers, particularly those related to abstract concepts, arise from unchallenged incorrect ideas, flawed mental models and the inability to relate new concepts to other knowledge (3, 4). Here, we predict using physical models will further the student’s development of a robust mental model that is able to overcome misconceptions and further the student’s learning progression in the molecular biosciences. Since no model is identical to the concept it represents (else it would cease being a model), students need to be trained to be skeptical in analyzing any model (5–7). Students who can examine a model and explain how the model is both like and unlike the real thing it represents demonstrate a conceptual understanding of what the model represents. Furthermore, like the Hindu fable of the blind men and the elephant, each model only represents a part of the whole, and it is through transitioning among multiple representations that we gain a true sense of what models represent (8). As such, physical models offer an avenue to develop students’ visual literacy skills, a recognized compounding variable in the abstract nature of the molecular biosciences wherein students are inundated with a variety of representations containing differing levels of abstraction (1, 9–11). Several studies suggest that these representations can lead to student learning difficulties and propagate misconceptions (4, 12–16). For example, spectacular animations of molecular processes help to convey difficult concepts, yet they often provide a “wow” factor to the expert, while moving through the information too quickly for a novice to process (17). Molecular visualization software allows educators and students alike to rotate and spin structures in virtual 3D space, but our assumption that students are able to “see” the objects in 3D may be a false one, especially if they have never experienced similar, tangible structures in the real world (18). One possible explanation for these difficulties is that current molecular bioscience curricula include little to no explicit instruction on interpreting, evaluating and moving through levels of representations (4, 9, 19, 20). Physical models of abstract concepts can aid in developing visual literacy skills that also support the overall aim here – to develop robust learner mental models that enable students to think like scientists. Until recently, students’ primary exposure to physical models was limited to the use of small molecule modeling kits in a chemistry course. Advances in structural biology began to reveal the 3D structure of macromolecules, but it was impossible for students to construct physical models of these complex structures. During this same time, the development of molecular visualization software made it possible for students to experience virtual representations of macromolecules in a computer environment. Today, advances in an additive manufacturing process known as rapid prototyping – commonly referred to as 3D-printing – have made it possible to construct physical models of complex molecular structures (Figure 1) (21). As 3D-printing technology continues to evolve, it is becoming possible to create models with complex color schemes in a variety of materials from hard plaster to flexible plastic and rubber. The recent explosion of low-cost filament-based 3D printers now makes it possible for molecular bioscience educators to acquire this modeling technology for as little as several thousand dollars.

44

Figure 1. Physical models of molecular structures. A. An amino acid – constructed by students using a small molecule kit. Models B-E are “assemblies of atoms,” constructed by 3D printing. B. ATP with magnetdocked phosphate groups. C. A zinc finger (PDB ID: 1ZAA) D. A four-subunit potassium channel (PDB ID: 1J95). E. The 70S E. coli ribosome (PDB ID: 4V5D) with magnet-docked large and small subunits, three tRNAs and a short stretch of mRNA. A wide variety of models have been used in all scientific disciplines to represent complex, often abstract concepts. Most studies on physical models have been conducted in organic chemistry courses or K-12 education, with relatively few investigating student learning and behavior with biomolecular physical models at the undergraduate level. Among these studies, however, exists evidence for student learning gains in different molecular bioscience settings. In two related studies, Oliver-Hoyo and authors report not only increased student engagement but also integration of knowledge across biochemistry concepts after using a series of macromolecules with small molecules (22, 23). Another investigation demonstrated better learning gains for students participating in a molecular dissection activity involving a 3D physical DNA model compared to the comparative building activity (24). In biology, higher learning gains for females are cited after using a physical protein model in one class session (18), while another retroactive study using several physical models related to the flow of genetic information demonstrates learning gains independent of gender (25). In fact, lower achieving students demonstrated the highest absolute gains (25). This last study is discussed in greater detail later in this chapter. A few studies have examined the impact of combining physical models with virtual activities, and, although there is some disagreement on the degree of impact, these studies do conclude with higher learning gains associated with the complementary use of these visual tools (26–28). Although these results are promising, more research is needed on best practices with physical models and on how and when students learn using these tools. Much of this, however, will hinge on increased educator and student access to, and training with, physical models. Herein, we introduce three case studies involving physical models of biomolecules in an active learning undergraduate classroom setting. Particular consideration for the use of physical models for students requiring accessibility services is presented in the final case study. Each case study stems from and works closely with the Milwaukee School of Engineering’s (MSOE) Center for BioMolecular Modeling (CBM), and as such we begin with an introductory case study on the CBM’s history of engaging communities of students, educators and researchers in physical modeling.

45

Case Study I: The CREST Project Modeling Projects The CBM explores the use of tactile, physical models of macromolecules and their building blocks – created by 3D printing technology – as a novel way in which to introduce students to the invisible world of molecular bioscience. Initiated in 1999, the CBM was originally focused on the creation of accurate custom models of proteins for use by researchers. But in 2001, the development of new design software (RP-RasMol) made it possible for the first time to involve teachers and their students in the design of these physical models. As we began to incorporate model design into our professional development programs for high school science teachers, we quickly realized that not only were the models effective teaching tools, but that the process of modeling was an even more powerful experience. This realization led to the development of the student modeling projects, in which a small team of students and their teacher work closely with a research lab to create a physical model of a protein that is central to the work of the lab. Although our initial insight into the power of modeling as a successful pedagogical approach involved high school teachers and their students in the SMART Team program (Students Modeling A Research Topic), we soon began exploring the use of this approach at the undergraduate level (29). We launched a series of CREST Projects (Connecting Researchers, Educators and STudents) resulting in additional insights into the power of modeling at the undergraduate level. A CREST modeling project combines (i) a student-centered modeling project involving an active research lab with (ii) a collaborative instructional materials development project in which collaborative teams create materials that engage classroom students in an inquiry-driven exploration of the research project (30). The most recent iteration of CREST explores the value of engaging undergraduates in the community of science through meaningful conversations with researchers at a professional meeting (31). The program identifies an awardee whose research will be presented at the annual American Society for Biochemistry and Molecular Biology (ASBMB) meeting. Teams of undergraduates explore some aspect of the research topic and design and build a physical 3D model to tell the molecular story of the structure and function of one of the proteins involved in this research. Teams tell their molecular story at a poster session at the ASBMB meeting, attend the award lecture, then meet with the researcher and colleagues in a “CREST Conversation.” These physical models become the shared mental model among researchers and students and allow students to engage in authentic scientific conversations with the researchers (32). Although a number of scientific professional societies encourage undergraduate participation, both in attending meetings and in presenting posters, unless there is a specific lecture that pertains directly to students’ research, undergraduates can feel overwhelmed and inadequate as scientists when attending scientific sessions. The CREST Program allows teams of students to design a physical model to tell a molecular story, attend a scientific session on that specific topic, then meet with their peers and researchers to discuss the topic in depth within the context of a professional meeting. We hypothesize that this meaningful engagement will help students identify as scientists, one of the key affective traits required for retention in STEM fields, especially for groups underrepresented in the sciences (33, 34). Indeed, undergraduates who participated in extracurricular CREST projects expressed greater confidence and identity as scientists and viewed their faculty advisor as a collaborator more than a mentor as a result of participation in CREST (30). The greatest gains in 46

these affective domains was seen in undergraduates from primarily undergraduate institutions (PUI) which lacked research opportunities for undergraduates (30). Instructional Materials Development Recent calls for reform in delivering science instruction emphasize the need to shift from lecturing to student-centered learning (1). Yet educators are slow to adopt better learning strategies (35, 36). There are numerous obstacles to overcome to adopt new methods. Change is difficult (37). Educators need to be dissatisfied with the current methods, and there needs to be a viable alternative that is clearly advantageous (38, 39). Even the best methods, when executed poorly, yield less than ideal results. Furthermore, few things are done perfectly the first time they are tried. Therefore, educators need appropriate training in using new innovations and a management of expectations on their first attempt at implementation (40). Perhaps of the greatest significance is the need for time – time to reflect, time to fail, time to analyze progress, time to revise, and time to share successes and challenges with others. Educators benefit from a community of peer support, as well as ongoing professional development support and administrative backing to encourage persistence to success (40–42). In answer to these challenges to implementation of best practices, a second aspect of the CREST Project engages collaborative groups (researchers, educators, students) in creating student-centered instructional materials that make current research accessible to early career trainees. The CBM offers a summer faculty workgroup meeting that allows educators to get away from their routines and focus on developing innovative materials for their classrooms. These “micro-sabbaticals” provide focused time for educators to collaborate in developing new ideas, and ongoing interactions during implementation provide the peer support needed to work through obstacles to adoption. Since most teaching is done in isolation, educators value cross-disciplinary collaboration and recognition from peers that their ideas are valuable (30). Participants also reported that having a set time dedicated to working on projects, as well as collaborators with whom to work, allowed them to dedicate the time needed to develop materials and implement new teaching strategies in their classes (30). The three case studies below were seeded by CREST collaborations and highlight a progression from the design or choice of physical model to the learning assessment for molecular bioscience courses. In all, these case studies provide molecular bioscience educators with information on the design/choice of physical models, design of correlating assessment of student learning, and types of evidence from assessment analysis.

Case Study II: Carbohydrate Models Of the four major macromolecules (proteins, lipids, nucleic acids), the structure-function relationship of carbohydrates is relatively unexplored in molecular bioscience education, with greater instruction time devoted to metabolism and laboratory exercises focused on chemical reactivity differences (43, 44). As such little to no evidence exists on student learning with these biomolecules even though they possess a range of structure-function properties and offer an excellent opportunity to engage students in several threshold concepts in biochemistry. Carbohydrate curriculum typically begins by introducing vocabulary related to the structures of monosaccharides; this foundational scaffolding is then used to introduce structure-function concepts related to disaccharides and finally ending with structure-function comparison among various polysaccharides. Even at the start, students have a hard time understanding and visualizing 47

the small differences in structures of monosaccharides which then impedes understanding di- and polysaccharides. The conceptual and visualization skills needed to understand chirality may be a significant learning barrier in understanding carbohydrates’ structure-function relationships. For example, a simple hexose like glucose has four chiral centers and an additional chiral center on the anomeric carbon upon ring formation. Upper level students who have completed the organic chemistry curriculum can successfully build correct ring structures using organic model kits, such as Prentice-Hall, Darling or Maruzen sets, due to the familiarity with cyclohexane. However, forming α- and β-linkages from those glucose models is often beyond their skill set, and, even when they manage to do it, they cannot make the connections between the linkages and resultant polysaccharide properties. The linkages as shown in most line representations (e.g. Haworth projections or chair conformations) are meant to focus attention on identifying the linkages. The line representations, however, do not convey how α- and β-linkages impose the steric restrictions in three dimensions that lead to the different structural properties of amylose and cellulose, respectively. The 3D structure of amylose (PDB ID: 1C58) shows the helical nature of amylose as well as hydrogen bonding possibilities. While the structure of cellulose straight chain polymer is not available directly on the RCSB Protein Data Bank site, structures of enzymes bound to different cellodextrins containing three to nine β-D-glucose units are available (PDB IDs: 4C4C, 3QXQ, 4TF4, etc.). Instructors may utilize virtual visualization tools, such as Jmol, to show students the end results of repeated α- and β-linkages for amylose and cellodextrins, or skilled students may even manipulate the virtual structures themselves. However, the path from linkage to final structure still remains muddy for most students. Physical models where students can make different linkages and manipulate structures to “see” steric restrictions for themselves would be useful. Based on these observations, a simplified glucose model was designed with –OH groups on C2 and C3 fixed to the ring. Oxygen (-O‑, red color) and hydrogen (-H, white color) atoms were designed as in most model kits with holes and linkers. The hydroxyl groups could be attached to C4 (below ring), C6 (above ring, away from C1) and either in the α- and β-positions on the anomeric C1. The anomeric C1 was colored grey while all other atoms including the fixed hydroxyl groups were tan colored. Clearly visible numbers were printed for each carbon, which is proposed to aid students in proper orientation of the model during instruction. The dihedral angles were chosen by comparison with available coordinates for amylose and cellodextrins and the model built using Spartan 14 modeling software. The choice not to use CPK coloring throughout was explicitly made to focus attention on those hydroxyl groups forming linkages; as students using glucose molecules built with organic kits often struggled with finding the correct –OH groups for linkages. The models were used in an upper level Biochemistry course (6-8 students) and a lower level introductory GOB (General, Organic and Biochemistry) course for pre-nursing students (20-25 students) over two class periods. Each student had at least one glucose model to work with and first understood the α- and β-positions on C1. They then worked with partners to build maltose and cellobiose models. One unexpected benefit of working with physical models was the explicit connection between condensation and removal of water, since they physically had to remove water to make the glycosidic linkage. Students then worked in larger groups to extend maltose to amylose and cellobiose to cellulose. Numbered carbons helped with building 1,4 linkages. Even with five glucose residues linked together, the differences between the polysaccharide structures becomes unmistakable. The α-linkages result in a flexible chain that naturally curved and showed the beginnings of a helix. The β-linkages result in a rigid and linear fibril. (Figure 2)

48

Figure 2. Amylose and cellulose models (on the left) formed from repeated α- and β- linkages with glucose monomers. A single glucose monomer (on the right) shows the number and color scheme selected for the 3D printed models. Students were then instructed to locate and compare the linkages within the polymer, and with the model in-hand the differences were obvious. The α-linkages were exposed and readily seen while the β-linkages were harder to find even with numbered carbons. Students were directed to make the connection between availability of linkages and function of polysaccharides. Glucose in storage polysaccharides like amylose should be readily cleavable, and a helix exposes the linkages easily. Cellulose “hides” its linkages and prevents easy access and thus is a good structural polysaccharide. Different groups could come together to form longer chains or in the case of amylose, make 1,6 linkages to make amylopectin or glycogen. Additionally, the presence of a single free anomeric carbon with multiple branches illustrates how reducing ends are “used up” in storage polysaccharides. The glucose models were simple enough to be used with lower and upper level students in different capacities. Pre-nursing students appreciated having their own glucose model to study before they started forming linkages. Some students were a little confused by tan coloring for most atoms, however this became a teachable moment about the limited nature of any given model. Biochemistry students were more used to working with multiple models representing a single structure and had no such difficulty. After working with both GOB and biochemistry students, the models were simplified. The –OH group on C6 was fixed to the ring and colored tan. Only a few glucose models needed the –OH group on C6 to create branching and this focused the use of models solely on 1,4 linkages. The absence of CPK coloring does prevent students from visualizing hydrogen bonds between residues. The potential for using flexible linkers to simulate H-bonds is currently being investigated. In all, this set of physical models is designed to engage students in building mental models that integrate the unique and varied structure-function concepts of carbohydrate chemistry.

Case Study III: Serine Protease Active Site Models Educators allot a significant portion of molecular bioscience curriculum to proteins, compared to lipids, nucleic acids or carbohydrates. Protein status in these curricula is not surprising considering that this biomolecule offers an avenue to cover and integrate four of the five biochemistry threshold concepts: “physical basis of interactions, thermodynamics of macromolecular structure formation, free energy, and biochemical pathway dynamics and regulation (45).” As such, a plethora of intervention activities, from POGIL to virtual based tools, with corresponding learning outcomes and gains, are reported in the literature. Even with this, little research has explicitly identified student misconceptions related to this biomolecule. Compounding on the extensive time spent covering topics related to proteins are the vast array of representations students are exposed to throughout 49

the course of protein education. At the outset of this study we wanted to develop a series of physical modeling activities with accurate features to target student understanding of protein structurefunction concepts while concurrently building students’ visual literacy skills. We also sought to use a validated instrument for assessment to test the impact of these models. Here we describe the design of serine protease physical models that intentionally address the three primary misconceptions identified through the Enzyme Substrate Interactions Concept Inventory (ESICI): electronics, stereochemistry and geometric complementary (46–48). We chose to create a set of serine protease models for chymotrypsin, trypsin and elastase as these enzymes, their substrates and inhibitors are not only common examples in biochemistry textbooks but also because these enzymes are exceptional examples of: substrate specificity, transition state stabilization, acid-base and covalent catalysis, geographical differences between catalytic and binding residues, alteration of pKa values of active site residues to facilitate catalysis, and integration with enzyme kinetics and inhibitor concepts. Current instruction with the proteases usually focuses on one, primarily chymotrypsin, and then mentions others (typically elastase and trypsin) for comparison of binding pockets that highlight enzyme specificity. The visual representations of these enzymes in textbooks usually highlight either electronics or geometric complementary. Additionally, with 3D virtual modeling students are likely only analyzing one protease at a time. Here a set of proteases enables students to make direct comparison among all three enzymes. In particular after years of teaching with proteases we note that students struggle to build a mental model for geometric complementary, therefore we propose a set of physical models with several different renderings will best target this misconception. While students demonstrate misconceptions related to each phenomenon separately, a complete understanding of how enzymes interact with substrates requires synthesis of all three concepts. The set of serine protease models was designed with respect to each targeted misconception. These models were developed with a team of undergraduates in conjunction with the CBM. As a team we decided on three major components for each protease as described in Table 1, an example of which is shown in Figure 3. These physical models are used in an undergraduate biochemistry course taught with an explicit focus on increasing visual literacy skills through the use of models and modeling. The course follows a flipped classroom design in which students watch a concept-based video prior to class and complete a pre-class assignment. During the 50 minute course time students engage in active learning activities in groups of two or three students. The protease physical models are used across two course days. During these days a physical model set is shared between two groups, with each group completing their own activity. During the first day students are given an exploratory activity designed to have students identify and compare the enzymes, then identify and compare the substrate based on electronic, geometric complementary and stereochemical interactions among the pieces. During the second day with the protease model set, each group works through a problem-based learning (PBL) activity while having the models available for reference. To measure whether these models impact student learning and the targeted misconceptions, several assessments were used in control and intervention semesters. In the control semesters students worked through the PBL activity without the use of models. In both the control and intervention semesters students completed the ESICI at the start and end of the semester; this instrument is used as a pre/post measure of student learning and misconceptions. Additionally responses to student answers on the in-class activities were rubric-scored with each item tagged with a corresponding misconception (electronics, stereochemistry, geometric complementary or some

50

combination). The same analysis was performed on correlating questions for students on individual exams on this material. Analysis of these data is forthcoming. More recently the serine protease kit is employed in the NSF-funded Modeling for the Enhancement of Learning Chemistry (ModEL-C) longitudinal study aiming to define how models impact the learning process and cognitive load for students. In this study, biometric data is collected using electroencephalographic (EEG) and eye tracking tools and voice recordings from simulated learning environments where biochemistry students complete the serine protease activity described above. In addition, observational and rubric analyses of student responses to activities from the simulated learning environment and real classroom sessions provide further data for assessing student learning. From these findings, an iterative process for model and assessment design will be proposed for educators interested in designing 3D physical models with corresponding active learning assessments that optimize the cognitive load and target student misconceptions. Table 1. Serine Protease Model Design Features Features of the physical model

Coloring and rendering

Proposed misconception targeted by the physical model

Active site backbone model

Each backbone model shows the catalytic triad side chains and one - two key binding pocket side chains.

The key side chains are Electronics rendered in spheres with CPK coloring. The rest of the protein is rendered as alpha carbon backbone.

Active site surface

Each surface plate model shows the surface topology of the active site.

The first iteration rendering used CPK coloring. In a second iteration, the surface plates were constructed of clear plastic so students could see the underlying atoms.

Substrates and inhibitors

Each designed substrate was All substrate and inhibitor docked using AutoDock in molecules are rendered in Chimera. One small spheres with CPK coloring. molecules is a chymotrypsin inhibitor, Tosyl phenylalyl chloromethyl ketone (TPCK).

Electronics and geometric complementary

Electronics, stereochemistry, geometric complementary

Figure 3. Serine protease physical model. A. Backbone, surface place, substrate epimers, and inhibitor for chymotrypsin. B. Surface plate snapped on top of backbone with the substrate epimers and inhibitor on the left side. C. Substrate bound in the active site. Photos courtesy of Cassidy Terrell.

51

In all, this set of physical models is designed to engage students in building mental models that decrease student misconceptions while also increasing student visual literacy and conceptual understanding. Additionally, we propose an avenue for using physical models to investigate cognitive load and engagement to better understand how students learn with this pedagogical tool.

Case Study IV: Flow of Genetic Information Models The “Central Dogma of Molecular Biology” describes information flow in a cell, from storage in molecules of DNA through expression as functional products (proteins) (49). A thorough conceptual understanding of the purposes and underlying processes of genetic information flow is a crucial foundation on which numerous molecular biology topics are built. Hence, genetic information flow is one of five Core Concepts of Vision and Change, which has been further articulated in the Biocore Guide for interpreting the Core Concepts (2a) (1, 50). Genetic information flow is also one of the four “Big Ideas” in the Advanced Placement Biology Curriculum Framework, and is described in the first objective in the Next Generation Science Standards for life sciences in high school (HS‐LS1‐1) (2, 51). Typical undergraduate students, however, struggle with many ideas associated with genetic information flow and focus on superficial terms and representations such as “transcription” and “Punnett squares” but cannot visualize or articulate the underlying molecular processes behind the terminology (52–59). Many undergraduate biology instructors have recognized the learning struggles and are interested in finding ways to improve student learning on topics related to genetic information flow. And while active-engagement pedagogies, when compared to lecture-only strategies, result in higher learning gains in STEM disciplines, our recent work has highlighted that physical models are better active-learning tools for helping students grasp certain topics related to genetic information flow (25, 60–63). We studied the use of physical models in a sophomore-level Cell and Molecular Biology course at a large, private university in the northeastern U.S. Many of the models used in this course were different from the 3D-printed, atomic-level models described above, but instead were based on more stylized, manipulable pieces made of craft foam, presenting a molecular level view of interaction of the macromolecules. For example, the Flow of Genetic Information Kit (FGIK) uses foam pieces to demonstrate how DNA is built by DNA polymerases in the process of replication, how RNA is built by RNA polymerases in the process of transcription, and how proteins are built by ribosomes in the process of translation (see Figure 4). Through a retrospective analysis of student data on the validated Central Dogma Concept Inventory (CDCI) tool (55), the authors demonstrated significantly higher learning gains on CDCI questions that were associated with a physical model‐based in-class activity compared with questions that were associated with clicker questions or peer discussion problems (25). The CDCI tool employs a multiple-select format, which helped the authors unearth interesting and useful patterns of student responses. While all students learned from engaging with the model-based activities, the researchers found that higher performing students improved their overall score by refining their almost-fullycorrect responses. For example, if a correct question response was ABD, the higher pattern of responses from the higher performing students went from AB (pre) to ABD (post). Lower performing students entered the course with less content knowledge, overall. The authors found that lower performing students improved by recognizing more correct vs incorrect responses after engaging with the model-based activities. For example, if a correct question response with ABD, lower performing students went from choosing (ACE) to (AB). Lower performing students may 52

not have left the course with complete expert-like mental models, but this group made some of the greatest learning gains, strongly suggesting they had created new (and correct) knowledge.

Figure 4. The Flow of Genetic Information Kit (3DMD). Students explore the process of protein translation using a model-based activity. Photo courtesy of Leslie Kate Wright.

Figure 5. Model based activities showed dramatic learning gains for all students, including low performers (bottom quartile) and deaf/hard of hearing (D/HH). In this analysis, each multiple-select question was scored as right or wrong (no partial credit). Students were ranked in quartiles by their scores on the entire CDCI at the beginning of the semester, although their performance is shown only for questions related to model-taught concepts. Dotted lines show that on average, D/HH students fell between the 2nd and 3rd quartile, both pre and post instruction. Error bars are SEM, n=426 with 34 D/HH. Out of the 426 students included in the study, 34 of the students identified as Deaf or Hardof-Hearing (D/HH), a population that faces significant challenges in post-secondary settings and lags behind hearing peers in B.S. college degree attainment (64). D/HH students, who are also underrepresented in STEM, may experience communication barriers in undergraduate classes and may not be able to fully participate in and learn from typical lectures and discussions (65). Research also shows that D/HH students may overestimate their own understanding of the material which may contribute to the overall gap in B.S. degree achievement compared to undergraduate hearing peers (64, 66). For example, in 2015 33% of all hearing individuals (ages 25-64) had completed a B.S. degree compared to only 18% of D/HH individuals of the same age range. Similar to English Language Learners (ELLs), D/HH students may face additional challenges when trying to understand a spoken lecture (or spoken discussion) while also trying to take notes during class (65). 53

Thus, strategies that activate information channels, other than spoken language, such as manipulation of models and model-based activities, may be especially useful for learners who are D/HH or have diverse communication styles (Figure 5) (65, 66). Dissecting out the D/HH students from the above analysis showed that D/HH made the same dramatic learning gains as hearing students on modelbased questions. While future work is needed to probe more deeply into why models are such effective learning tools, several cognitive theories such as constructivism, zone of proximal development, reduction of cognitive load and shared mental models, support the notion that models and model-based activities are effective tools for deep learning. Here we briefly describe each of these theories and suggest why they may be particularly important in the context of D/HH, English Language Learners (ELL), or other special populations. Constructivism This theory postulates that students learn best when they construct their own explanations through guided activities (67). The dynamic, physical model‐based activities allow learners to explore and build (through manipulation of the actual models) but also allow for refinement and reorganization of students’ mental models of molecular processes. A constructivist approach may be especially beneficial for D/HH learners because these students may be supported during classes by sign language interpreters or real-time captionists. In a traditional lecture-based course D/HH may not be able to write things down or take notes for themselves because they have to split their attention between the board (or PowerPoint slide) and watching the interpreter or captioning screen. Using a model-based activity is very hands-on with D/HH students working directly with the model, and not relying on interpreters or captionists as the conduit for information. Zone of Proximal Development Learners often require scaffolding to learn new things; in other words, it is difficult to incorporate ideas that are too far away from their prior knowledge (68). Physical models may offer a “bridge” to learners with a shaky/incomplete mental model of transcription, when they are trying to learn about gene expression. Offering this bridge is beneficial to all students but may be especially helpful for deaf students because, compared to hearing peers, they enter post-secondary institutions with greater differences in academic preparation and educational experiences (69). Reduction of Cognitive Load Learners can be expected to hold only 5-9 pieces of information in their working memory at a time (70). Thus, the cognitive structure of humans restricts the types of environments that are ideal for deep learning (71, 72). A lecture that incorporates 15 new vocabulary or technical terms probably is not ideal for deep learning! Physical models serve as an extension of cognitive space for learners; instead of having to remember details and terms of a process, learners can look at, point to and manipulate a physical 3D structure. As with hearing peers, D/HH students often must balance cognitive load with new learning opportunities. Unlike hearing students, D/HH students who communicate using American Sign Language (ASL) may have another challenge; when interpreters encounter an unfamiliar term or a word that is not connected with a standard ASL sign (or a sign they are aware of), they may invent a brand new sign on the fly to communicate a term or idea (73). This phenomenon may increase cognitive load in D/HH learners as they may have to 54

keep track of another ASL sign during a class. Physical models may alleviate some of the cognitive load faced by D/HH learners since they will have to rely less on ASL interpreters and can handle the model themselves. Shared Mental Model As learners manipulate the physical model they alter their existing mental models to align more closely with the physical model. Students working in a group have a structure to which to refer as they share ideas and engage in discussion. Thus, physical models become the embodiment of a shared group mental model, improving the learning experience for all. There is a correlation with students’ sense of belonging to a STEM community and college persistence, but deaf college students, among others, may struggle to achieve that sense of belonging. Including model-based activities during class may help D/HH students form connections with their peers (74, 75). Active learning classrooms that incorporate physical models of molecular biological processes improve learning for all students, but particularly for low performers and individuals with communication difficulties. We suggest that models are helpful beyond simply encouraging active engagement because they 1) make the abstract more concrete, 2) provide a shared mental model for discussion, 3) do not depend on jargon or vocabulary, and 4) promote dynamic rather than static conceptions.

Conclusions and Future Directions Although each of the case studies detailed above has unique features, they demonstrate that physical models can be employed in a variety of ways to create shared mental models among researchers, educators and students. There are several common threads shared by two or more of these cases: • The first two cases engage undergraduates in designing models. Making decisions about which structures to depict to tell a molecular story, or to best compare similar structures, or how to resolve two conflicting pieces of data in the literature, requires students to think like scientists (76). • Cases II and III employ backward design by first identifying learning objectives, then targeting student misconceptions in the design of instructional materials (77). The physical models serve as mental models and thinking tools, optimizing cognitive load so that students can develop a deeper conceptual understanding. • Cases I-III utilize accurate physical models to explore molecular interactions and connectivity. Case IV, on the other hand, employs schematic models that focus on a molecular process (DNA replication, transcription, translation). There is value in using both types of models, as well as in transitioning among multiple models of the same structure (78–84). • The first time students use a new tool (ie models), they need to learn HOW to use the tool. Students experience frustration when they are expected to simultaneously learn how to use a new tool and master the concepts. Students need time to explore models before they are expected to master the concepts the model conveys. Educators can bridge the gap by orienting students to the models, discussing the use of color and renderings of the models as well as limitations of models. Just as in observing fine art, novices must be guided to an

55

understanding of what they are viewing by an expert in order to appreciate the nuances of models they explore. • To be the most effective, students should be exposed to models throughout a course/ curriculum. Multiple representations of the same molecular story allow students to layer details for greater complexity. Models of similar structures allow students to compare and contrast, growing their conceptual understanding. Multiple types of models (physical vs. virtual, schematic vs. accurate) allow students to develop skills in transitioning among models. As discussed in Case IV, incorporating physical models in the molecular biosciences classroom may be especially beneficial for D/HH students. The goal of universal design for learning is to optimize learning experiences for all students, recognizing that all students learn differently (85). Indeed, each of the cognitive theories discussed in Case IV as applying to D/HH students is applicable to all students. Careful design of physical models as instructional tools will make molecular visualization accessible to a wide variety of learning differences. Physical models serve as a tactile embodiment of mental models, eliminating the need for cumbersome vocabulary in developing a conceptual understanding. This is valuable for both D/HH students and those for whom English is a second language. Careful color selection and the addition of tactile distinctions make the molecular world accessible to color blind and visually impaired students, respectively. Along these lines, other limitations and challenges exist for educators using physical models in the classroom and/or laboratory environment. Monetary costs of acquiring the models remain a barrier for use. Even if some models are purchased there may not be enough models for every student, and sharing may impede the models’ intended use. The models also often require the educator to provide information on orientation, color and proper use. However, there are several options for educators who are interested in incorporating physical models in the classroom. The Milwaukee School of Engineering has a Model Lending Library with a variety of models (86). Borrowers schedule online for a three week loan period (one week for shipping out, one week for classroom use, and a third week for return shipping) and pay only return shipping costs. Educators interested in building their own models can take advantage of a free designs available online (87, 88). Guidance is available for those wishing to build models using their own tabletop extrusion printers, or purchased inexpensively through 3D printing services (89–91). As physical models become increasingly utilized as a pedagogical tool, educators and researchers may consider avenues for investigating student learning and optimal design elements to both the model and assessment. Such studies could investigate the impact of color, size and scale, types of renderings to use together, relevance/impact of spatial reasoning and visual literacy skills, and best practices for classroom use. Additionally, comparative studies using 3D printed models and traditional organic kits from Prentice-Hall, Darling or Maruzen could offer insights into which approach best supports student learning. Along these lines no evidence on student learning exists to determine the impact of students creating 3D physical models compared with using pre-made physical models.

Acknowledgments The CREST Project is supported by the National Science Foundation under award numbers DUE-1022793, DUE-1323414 and DUE-1725940. 56

The serine proteases kit, data collection and analysis is supported by the National Science Foundation under the ModEL-C project with award numbers: IUSE 1711402 and 1711425. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation.

References 1.

2. 3.

4. 5. 6. 7.

8. 9. 10. 11. 12. 13. 14. 15.

Bauerle, C.; DePass, A.; Lynn, D.; O’Connor, C.; Singer, S.; Withers, M.; Anderson, C. W.; Donovan, S.; Drew, S.; Ebert-May, D.; Gross, L.; Hoskins, S. G.; Labov, J.; Lopatto, D.; McClatchey, W.; Varma-Nelson, P.; Pelaez, N.; Poston, M.; Tanner, K.; Wessner, D.; White, H.; Wood, W.; Wubah, D. Vision and Change in Undergraduate Biology Education: A Call to Action. Brewer, C. A., Smith, D., Eds.; American Association for the Advancement of Science: Washington, DC, 2011. NGSS Lead States. In Next Generation Science Standards: For States, by States; The National Academies Press: Washington, DC, 2013. Chi, M. Three Types of Conceptual Change: Belief Revision, Mental Model Transformation, and Categorical Shift. In Handbook of Research on Concpetual Change; Vosniadou, S., Ed.; Erlbaum: Hillsdale, NJ, 2008; pp 61−82. Treagust, D. F.; Chittleborough, G.; Mamiala, T. L. Students’ Understanding of the Role of Scientific Models in Learning Science. Int. J. Sci. Educ. 2002, 24, 357–368. Box, G. E. P. Robustness in the Strategy of Scientific Model Building. In Robustness in Statistics; Launer, R. L., Wilkinson, G. N., Eds.; Academic Press: New York, 1979; pp 201–236. Bateman, R. C.; Craig, P. A. Education Corner: A Proficiency Rubric for Biomacromolecular 3D Literacy. PDB Newsl. 2010, 45, 5–7. Dries, D. R.; Dean, D. M.; Listenberger, L. L.; Novak, W. R. P.; Franzen, M. A.; Craig, P. A. An Expanded Framework for Biomolecular Visualization in the Classroom: Learning Goals and Competencies. Biochem. Mol. Biol. Educ. 2017, 45, 69–75. Saxe, J. G. The Blind Men and the Elephant. In The Poems of John Godfrey Saxe; James R. Osgood and Co.: Boston, MA, 1873; pp 259−261. Offerdahl, E. G.; Arneson, J. B.; Byrne, N. Lighten the Load: Scaffolding Visual Literacy in Biochemistry and Molecular Biology. CBE—Life Sci. Educ. 2017, 16, es1. Schonborn, K. J.; Anderson, T. R. The Importance of Visual Literacy in the Education of Biochemists. Biochem. Mol. Biol. Educ. 2006, 34, 94–102. Schonborn, K. J.; Anderson, T. A Model of Factors Determining Students’ Ability to Interpret External Representations in Biochemistry. Int. J. Sci. Educ. 2009, 31, 193–232. Dwyer, F. M. The Relative Effectiveness of Varied Visual Illustrations in Complementing Programed Instruction. J. Exp. Educ. 1967, 36, 34–42. Mayer, R. E. Multimedia Learning: Are We Asking the Right Questions? Educ. Psychol. 1997, 32, 1–19. Mayer, R. E. The Promise of Multimedia Learning: Using the Same Instructional Design Methods across Different Media. Learn. Instr. 2003, 13, 125–139. Ametller, J.; Pintó, R. Students’ Reading of Innovative Images of Energy at Secondary School Level. Int. J. Sci. Educ. 2002, 24, 285–312.

57

16. Pena, B. M.; Gil Quilez, M. J. The Importance of Images in Astronomy Education. Int. J. Sci. Educ. 2001, 23, 1125–1135. 17. Goodsell, D. S.; Franzen, M. A.; Herman, T. From Atoms to Cells: Using Mesoscale Landscapes to Construct Visual Narratives. J. Mol. Biol. 2018, 430, 3954–3968. 18. Forbes-Lorman, R. M.; Harris, M. A.; Chang, W. S.; Dent, E. W.; Nordheim, E. V.; Franzen, M. A. Physical Models Have Gender-Specific Effects on Student Understanding of Protein Structure-Function Relationships: Protein Structure-Function Relationships. Biochem. Mol. Biol. Educ. 2016, 44, 326–335. 19. Linenberger, K. J.; Holme, T. A. Biochemistry Instructors’ Views toward Developing and Assessing Visual Literacy in Their Courses. J. Chem. Educ. 2015, 92, 23–31. 20. Scaife, M.; Rogers, Y. External Cognition: How Do Graphical Representations Work? Int. J. Hum.-Comput. Stud. 1996, 45, 185–213. 21. Herman, T.; Morris, J.; Colton, S.; Batiza, A.; Patrick, M.; Franzen, M.; Goodsell, D. S. Tactile Teaching: Exploring Protein Structure/Function Using Physical Models. Biochem. Mol. Biol. Educ. 2006, 34, 247–254. 22. Cooper, A. K.; Oliver-Hoyo, M. T. Creating 3D Physical Models to Probe Student Understanding of Macromolecular Structure: Creating 3D Physical Models. Biochem. Mol. Biol. Educ. 2017, 45, 491–500. 23. Babilonia-Rosa, M. A.; Kuo, H. K.; Oliver-Hoyo, M. T. Using 3D Printed Physical Models to Monitor Knowledge Integration in Biochemistry. Chem. Educ. Res. Pract. 2018, 19, 1199–1215. 24. Srivastava, A. Building Mental Models by Dissecting Physical Models: Building Mental Models by Dissecting Physical Models. Biochem. Mol. Biol. Educ. 2016, 44, 7–11. 25. Newman, D. L.; Stefkovich, M.; Clasen, C.; Franzen, M. A.; Wright, L. K. Physical Models Can Provide Superior Learning Opportunities beyond the Benefits of Active Engagements: Physical Models Improve Learning. Biochem. Mol. Biol. Educ. 2018, 46, 435–444. 26. Harris, M. A.; Peck, R. F.; Colton, S.; Morris, J.; Chaibub Neto, E.; Kallio, J. A Combination of Hand-Held Models and Computer Imaging Programs Helps Students Answer Oral Questions about Molecular Structure and Function: A Controlled Investigation of Student Learning. CBE—Life Sci. Educ. 2009, 8, 29–43. 27. Roberts, J. R.; Hagedorn, E.; Dillenburg, P.; Patrick, M.; Herman, T. Physical Models Enhance Molecular Three-Dimensional Literacy in an Introductory Biochemistry Course. Biochem. Mol. Biol. Educ. 2006, 33, 105–110. 28. Geldenhuys, W. J.; Hayes, M.; Van der Schyf, C. J.; Allen, D. D.; Malan, S. F. Receptor Surface Models in the Classroom: Introducing Molecular Modeling to Students in a 3-D World. J. Chem. Educ. 2007, 84, 979. 29. Herman, T.; Colton, S.; Franzen, M. Rethinking Outreach: Teaching the Process of Science through Modeling. PLoS Biol. 2008, 6, e86. 30. Franzen, M.; Herman, T.; Harris, M. CREST: Connecting Researchers, Educators and Students; Presented at Envisioning the Future of Undergraduate STEM Education: Research and Practice Symposium [Online], Washington, DC, 2016; Project 1323414; American Association for the Advancement of Science. http://www.enfusestem.org/projects/crest-connectingresearchers-educators-and-students-5/ (accessed April 26, 2019).

58

31. Wenger, E. Communities of Practice: Learning, Meaning and Identity; Cambridge University Press: Cambridge, England, 1999. 32. Rahm, J.; Miller, H. C.; Hartley, L.; Moore, J. C. The Value of an Emergent Notion of Authenticity: Examples from Two Student/Teacher-Scientist Partnership Programs. J. Res. Sci. Teach. 2003, 40, 737–756. 33. Estrada, M.; Woodcock, A.; Hernandez, P. R.; Schultz, P. W. Toward a Model of Social Influence That Explains Minority Student Integration into the Scientific Community. J. Educ. Psychol. 2011, 103, 206–222. 34. Hurst, M.; Gilmore, J.; Maher, M. Exploring the Professional Identity Development of Researchers in Science, Technology, Engineering, Math and Science Education; University of South Carolina, 2010. https://uscreese.files.wordpress.com/2010/06/exploring-the-professional-identity. pdf (accessed April 26, 2019). 35. ASBMB. Biochemistry/Molecular Biology and Liberal Education: A Report to the Teagle Foundation; [Online] Teagle Foundation, 2008. http://www.teaglefoundation.org/Teagle/ media/GlobalMediaLibrary/documents/resources/Biochemistry_Molecular_Biology. pdf?ext=.pdf (accessed April 26, 2019). 36. Silverthorn, D. U.; Thorn, P. M.; Svinicki, M. D. It’s Difficult to Change the Way We Teach: Lessons from the Integrative Themes in Physiology Curriculum Module Project. AJP Adv. Physiol. Educ. 2006, 30, 204–214. 37. Ebert-May, D.; Derting, T. L.; Hodder, J.; Momsen, J. L.; Long, T. M.; Jardeleza, S. E. What We Say Is Not What We Do: Effective Evaluation of Faculty Professional Development Programs. BioScience 2011, 61, 550–558. 38. Henderson, C.; Cole, R.; Froyd, J.; Friedrichsen, D. G.; Stanford, C. Designing Educational Innovations for Sustained Adoption: A How-to Guide for Education Developers Who Want to Increase the Impact of Their Work; Increase the Impact: Kalamazoo, MI, 2015. 39. Andrews, T. C.; Lemons, P. P. It’s Personal: Biology Instructors Prioritize Personal Evidence over Empirical Evidence in Teaching Decisions. CBE Life Sci. Ed.. 2015, 14, ar7. 40. Henderson, C.; Dancy, M.; Niewiadomska-Bugaj, M. Use of Research-Based Instructional Strategies in Introductory Physics: Where Do Faculty Leave the Innovation-Decision Process? Phys. Rev. Spec. Top. - Phys. Educ. Res. 2012, 8. 41. Guskey, T. R. Evaluating Professional Development; Corwin Press: Thousand Oaks, CA, 2000. 42. Rogan, J. M. How Much Curriculum Change Is Appropriate? Defining a Zone of Feasible Innovation. Sci. Educ. 2007, 91, 439–460. 43. Bowman, K.; Friedman, D. Glycoscience: Integrating a Key Macromolecule More Fully into the Curriculum. CBE Life Sci. Educ. 2013, 12, 5–8. 44. Figueira, A. C. M.; Rocha, J. B. T. A Proposal for Teaching Undergraduate Chemistry Students Carbohydrate Biochemistry by Problem-Based Learning Activities: Proposal for Teaching Undergraduate Chemistry Students. Biochem. Mol. Biol. Educ. 2014, 42, 81–87. 45. Loertscher, J.; Green, D.; Lewis, J. E.; Lin, S.; Minderhout, V. Identification of Threshold Concepts for Biochemistry. CBE Life Sci. Educ. 2014, 13, 516–528. 46. Linenberger, K. J.; Bretz, S. L. A Novel Technology to Investigate Students’ Understandings of Enzyme Representations. J. Coll. Sci. Teach. 2012, 42, 45–49. 47. Linenberger, K. J.; Bretz, S. L. Biochemistry Students’ Ideas about Shape and Charge in Enzyme-Substrate Interactions. Biochem. Mol. Biol. Educ. 2014, 42, 203–212. 59

48. Linenberger, K. J.; Bretz, S. L. Biochemistry Students’ Ideas about How an Enzyme Interacts with a Substrate. Biochem. Mol. Biol. Educ. 2015, 43, 213–222. 49. Crick, F. Central Dogma of Molecular Biology. Nature 1970, 227, 561–563. 50. Brownell, S. E.; Freeman, S.; Wenderoth, M. P.; Crowe, A. J.; Wood, W. B. BioCore Guide: A Tool for Interpreting the Core Concepts of Vision and Change for Biology Majors. CBE Life Sci. Educ. 2014, 13, 200–211. 51. The College Board. AP Biology Curriculum Framework 2012-2013; The College Board: New York, NY, 2011. 52. Allchin, D. Mending Mendelism. Am. Biol. Teach. 2000, 62, 632–639. 53. Khodor, J.; Halme, D. G.; Walker, G. C. A Hierarchical Biology Concept Framework: A Tool for Course Design. Cell Biol. Educ. 2004, 3, 111–121. 54. Lewis, J.; Wood-Robinson, C. Genes, Chromosomes, Cell Division and Inheritance--Do Students See Any Relationship? Int. J. Sci. Educ. 2000, 22, 177–195. 55. Newman, D. L.; Snyder, C. W.; Fisk, J. N.; Wright, L. K. Development of the Central Dogma Concept Inventory (CDCI) Assessment Tool. CBE Life Sci. Educ. 2016, 15, ar9. 56. Marbach-Ad, G. Attempting To Break the Code in Student Comprehension of Genetic Concepts. J. Biol. Educ. 2001, 35, 183–189. 57. Pashley, M. A-Level Students: Their Problems with Gene and Allele. J. Biol. Educ. 1994, 28, 120–126. 58. Pelletreau, K. N.; Andrews, T.; Armstrong, N.; Bedell, M. A.; Dastoor, F.; Dean, N.; Erster, S.; Fata-Hartley, C.; Guild, N.; Greig, H.; Hall, D.; Knight, J. K.; Koslowsky, D.; Lemons, P.; Martin, J.; McCourt, J.; Merrill, J.; Moscarella, R.; Nehm, R.; Northington, R.; Olsen, B.; Prevost, L.; Stolzfus, J.; Urban-Lurian, M.; Smith, M. K. A Clicker-Based Case Study That Untangles Student Thinking about the Processes in the Central Dogma. CourseSource 2016, 3. 59. Smith, M. K.; Knight, J. K. Using the Genetics Concept Assessment to Document Persistent Conceptual Difficulties in Undergraduate Genetics Courses. Genetics 2012, 191, 21–32. 60. Freeman, S.; Eddy, S. L.; McDonough, M.; Smith, M. K.; Okoroafor, N.; Jordt, H.; Wenderoth, M. P. Active Learning Increases Student Performance in Science, Engineering, and Mathematics. Proc. Natl. Acad. Sci. 2014, 111, 8410–8415. 61. Adegoke, B. A. Impact of Interactive Engagement on Reducing the Gender Gap in Quantum Physics Learning Outcomes among Senior Secondary School Students. Phys. Educ. 2012, 47, 462. 62. Haak, D. C.; HilleRisLambers, J.; Pitre, E.; Freeman, S. Increased Structure and Active Learning Reduce the Achievement Gap in Introductory Biology. Science 2011, 332, 1213–1216. 63. Candler, L. Actively Engage Students Using Hands-on & Minds-on Instruction. K-12 News, Lessons & Shared Resources By Teachers, For Teachers. 2009-2019 K-12 Teachers Alliance; http://www. teachhub.com/actively-engage-students-using-hands-minds-instruction (accessed Sept 10, 2019). 64. Garberoglio, C. L.; Cawthon, S.; Sales, A. Deaf People and Educational Attainment in the United States; National Deaf Center on Postsecondary Outcomes, 2017; p 15.

60

65. Stinson, M. S.; Elliot, L. B.; Easton, D. Deaf/Hard-of-Hearing and Other Postsecondary Learners’ Retention of STEM Content With Tablet Computer-Based Notes. J. Deaf Stud. Deaf Educ. 2013, 19, 251–269. 66. Marschark, M.; Wauters, L. Deaf Cognition: Foundations and Outcomes; Marshark, M., Ed.; Hauser, P., Ed.; Perspectives on Deafness; Oxford University Press: Oxford, New York, 2008. 67. Lord, T. R. Using Constructivism to Enhance Student Learning in College Biology. J. Coll. Sci. Teach. 1994, 23, 346–348. 68. Vygotsky, L. S. Mind in Society: The Development of Higher Psychological Processes; Cole, M., John-Steiner, V., Scribner, S., Souberman, E., Eds.; Harvard University Press: Cambridge, MA, 1978. 69. Albertini, J. A.; Kelly, R. R.; Matchett, M. K. Personal Factors That Influence Deaf College Students’ Academic Success. J. Deaf Stud. Deaf Educ. 2012, 17, 85–101. 70. Miller, G. A. The Magical Number Seven Plus or Minus Two: Some Limits on Our Capacity for Processing Information. Psychol. Rev. 1956, 63, 81–97. 71. Paas, F.; Renkl, A.; Sweller, J. Cognitive Load Theory: Instructional Implications of the Interaction between Information Structures and Cognitive Architecture. Instr. Sci. 2004, 32, 1–8. 72. Sweller, J.; Merrienboer, J.; Paas, F. Cognitive Architecture and Instructional Design. Educ. Psychol. Rev. 1998, 10, 251–296. 73. Buckley, G.; Smith, S.; DeCaro, J.; Barnett, S.; Dewhurst, S. Building Community for Deaf Scientists. Science 2017, 355, 255.1–255. 74. PCAST. Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics; Report to the President from the President’s Council of Advisors on Science and Technology; Office of Science and Technology: Washington, DC, 2012. 75. Brown, E. R.; Thoman, D. B.; Smith, J. L.; Diekman, A. B. Closing the Communal Gap: The Importance of Communal Affordances in Science Career Motivation. J. Appl. Soc. Psychol. 2015, 45, 662–673. 76. Span, E. A.; Goodsell, D. S.; Ramchandran, R.; Franzen, M. A.; Herman, T.; Sem, D. S. Protein Structure in Context: The Molecular Landscape of Angiogenesis. Biochem. Mol. Biol. Educ. 2013, 41, 213–223. 77. Wiggins, G. P.; McTighe, J. Understanding by Design, expanded 2nd ed.; Association for Supervision and Curriculum Development: Alexandria, VA, 2005. 78. Al-Balushi, S. M.; Al-Hajri, S. H. Associating Animations with Concrete Models to Enhance Students’ Comprehension of Different Visual Representations in Organic Chemistry. Chem Educ Res. Pr. 2014, 15, 47–58. 79. Gilbert, J. K.; Treagust, D. Models and Modeling in Science Education: Multiple Representations in Chemical Education; Gilbert, J. K., Ed.; Treagust, D., Ed.; Springer Netherlands: Dordrecht, Netherlands, 2009; Vol. 4. 80. Kozma, R. B. The Use of Multiple Representations and the Social Construction of Understanding in Chemistry. In Innovations in Science and Mathematics Education: Advanced Designs for Technologies of Learning; Jacobson, M. J., Ed.; Kozma, R. B., Ed.; Erlbaum: Mahwah, NJ, 2000; pp 11–46. 61

81. Kozma, R. B.; Russell, J.; Jones, T.; Marx, N.; Davis, J. The Use of Multiple, Linked Representations to Facilitate Science Understanding. In International Perspectives on the Design of Technology-Based Learning Environments; Vosniadou, S., De Corte, E., Glaser, R., Mandl, H., Eds.; Erlbaum: Hillsdale, NJ, 1996; pp 41−60. 82. Kumi, B. C.; Olimpo, J. T.; Bartlett, F.; Dixon, B. L. Evaluating the effectiveness of organic chemistry textbooks in promoting representational fluency and understanding of 2D–3D diagrammatic relationships. Chem. Educ. Res. Pract. 2013, 14, 177–187. 83. Wu, H.-K.; Krajcik, J. S.; Soloway, E. Promoting Understanding of Chemical Representations: Students’ Use of a Visualization Tool in the Classroom. J. Res. Sci. Teach. 2001, 38, 821–842. 84. Cox, J. R. Enhancing Student Interactions with the Instructor and Content Using Pen-Based Technology, Youtube Videos, and Virtual Conferencing. Biochem. Mol. Biol. Educ. 2011, 39, 4–9. 85. Meyer, A.; Rose, D. H.; Gordon, D. Universal Design for Learning: Theory and Practice; CAST Professional Publishing, an imprint of CAST, Inc: Wakefield, MA, 2014. 86. Center for BioMolecular Modeling, Milwaukee School of Engineering. MSOE Lending Library. http://cbm.msoe.edu/lendingLibrary (accessed April 29, 2019). 87. National Institutes of Health, NIH 3D Print Exchange. Discover 3D Models. https://3dprint. nih.gov/discover (accessed April 29, 2019). 88. Department of Biochemistry, Digital Commons@University of Nebraska-Lincoln. 3-D Printed Model Structural Files. https://digitalcommons.unl.edu/structuralmodels/ (accessed April 29, 2019). 89. Center for BioMolecular Modeling, Milwaukee School of Engineering. 3D Printing for the Bioscience Classroom. http://cbm.msoe.edu/teacherWorkshops/printResources/ (accessed April 29, 2019). 90. Howell, M. E.; van Dijk, K.; Booth, C. S.; Helikar, T.; Couch, B. A.; Roston, R. L. Visualizing the Invisible: A Guide to Designing, Printing, and Incorporating Dynamic 3D Molecular Models to Teach Structure–Function Relationships. J. Micro.& Biol. Ed. 2018, 19, 1–3. 91. Roston, R. MacroMolecules; Shapeways. www.shapeways.com/shops/macromolecules (accessed April 29, 2019).

62

Pedagogies and Practices

Chapter 4

Skills and Foundational Concepts for Biochemistry Students Ellis Bell,* Joseph Provost, and Jessica K. Bell Department of Chemistry & Biochemistry, University of San Diego, 5998 Alcala Park, San Diego, California 92119, United States *E-mail: [email protected].

This chapter is intended for faculty and prospective faculty teaching biochemistry and molecular biology (or related molecular life science) courses, and for biochemistry and molecular biology program directors or departmental chairs responsible for the planning of and/or implementing appropriate degree programs. The chapter reviews the evolution of curricular suggestions for biochemistry and molecular biology majors from the turn of the millenium to the present day. The major driving force for curricula change has been the Vision and Change report that continued an emphasis on concepts and skills rather than simply content. From the perspective of Vision and Change, we discuss and compare curricular recommendations from the American Chemical Society and the American Society for Biochemistry and Molecular Biology, and compare them to recommendations for pre-medical school students taking biochemistry courses. We discuss some of the concept areas and skills, and emphasize student-centered teaching approaches and high impact teaching practices in the context of education in biochemistry and molecular biology. We illustrate several of these areas in sufficient detail, with scoring rubrics, to indicate appropriate levels of student performance, and discuss the ASBMB certification exam. The chapter concludes with a discussion of what an idealized curriculum should look like to best serve its students, including a brief discussion of the importance of inclusive teaching.

Introduction: Concepts Conquer Content In order to discuss current curricular recommendations of the major professional societies it is important to analyze how these have evolved. As early as 1986, with the undergraduate science mathematics and engineering education report from the national science board (1), there were calls to improve biology education at the college level. In 2001, an article in Nature Reviews, “The Future of Education in the Molecular Life Sciences” (2) called for increased student-centered education with more focus on skills and concepts and the central role that undergraduate research can play. Discussion continued with the 2003 publication of “Bio 2010 transforming undergraduate education © 2019 American Chemical Society

for future research biologists” (3) and other publications (4, 5). This coincided with the Education and Professional Development committee of the American Society for Biochemistry and Molecular Biology releasing a revised recommended curriculum (6), Tables 1 and 2. How these recommendations might be put into effect in various types of Colleges and Universities was illustrated by a series of articles in Biochemistry and Molecular Biology Education (7–9). These revisions signaled the start of a move from content to concepts and skills, and, instead of listing required courses, described concept and content areas and skills that should be developed across a biochemistry and molecular biology degree curriculum.

Vision and Change: Curriculum Not Courses Recognizing not only the changing nature of the biological sciences (2) but also the changing demographics of the nation, exacerbated by continued low retention in the sciences of students from underrepresented groups and projections for increased needs of a science literate workforce, the National Science Foundation,in 2006, in conjunction with AAAS, the National Institutes of Health and the Howard Hughes Medical Institute initiated what was to become known as “Vision and Change”. To achieve maximum buy-in from institutions and faculty, the final report of the Vision and Change initiative, “Vision and Change in Undergraduate Biology Education: A Call to Action (10)” resulted from national conversations involving hundreds of educators, faculty, administrators and students as well as the major funding agencies. Vision and Change continued the emerging trend of focusing on conceptual areas and skills and categorized these as shown in Figure 1 and Table 3.

Figure 1. Key features of the vision and change recommendations for biology education.

66

Table 1. 2004 Recommended Curriculum for a Biochemistry and Molecular Biology Undergraduate Major Level Introductory

Intermediate

67 Advanced

Laboratory Skills

Biochemistry and Molecular Biology Introductory enzyme kinetics, allosteric regulation, Bioenergetics and equilibria, DNA/RNA structure and function. enzyme kinetics, mechanisms of reversible and irreversible enzyme inhibitors, ligand binding, detailed chemical mechanisms of enzymes. Metabolism and regulation, signal transduction, supramolecular assemblies

Chemistry Core Topics

Biology Core Topics

Atomic structure, molecular structure and Cell structure, biomolecule structure and function, protein spectroscopy, periodicity, thermodynamics, structure/function kinetics, bonding (covalent and noncovalent), reactions and stoichiometry, acids/bases, descriptive inorganic, transition metals, redox Structure/bonding/nomenclature, functional groups, instrumental structure determination, stereochemistry, synthesis, reaction mechanisms and intermediates, molecular recognition, organometallics, combinatorial chemistry, bioorganic (amino acids, peptides, lipids, carbohydrates, nucleotides).

Advanced topics in protein Physical biochemistry: thermodynamics, structure and function: Advanced kinetics, molecular spectroscopy, solutions and equilibria, ligand interactions, molecular topics in protein structure and modeling function: enzyme kinetics, mechanisms of reversible and irreversible enzyme inhibitors, ligand binding, detailed chemical mechanisms of enzymes, protein folding, molecular basis for protein function, regulation of protein activity, proteomics Isolation and characterization of spectroscopy (e.g. UV/VIS, fluorescence, proteins and other biomolecules, NMR, MS), chromatography (HPLC, etc.), enzyme kinetics and inhibition, electrophoretic techniques (e.g. PAGE, IEF, CE, etc.).

Concepts of compartmentation and tissue specialization, including plant, animal, bacterial, fungal cells, etc. The “Central Dogma.”

Advanced discussion of classical genetics and the “Central Dogma”: DNA replication, transcription and translation, topics in DNA/RNA structure/function, genomics, regulation of gene expression in prokaryotes and eukaryotes, protein synthesis and processing, genetic engineering techniques, bioinformatics.

DNA isolation and sequencing, cloning, PCR, genetic engineering techniques, microscopy, aseptic techniques, microarrays

Table 1. (Continued). 2004 Recommended Curriculum for a Biochemistry and Molecular Biology Undergraduate Major Level

Research

Biochemistry and Molecular Biology

Chemistry Core Topics

Biology Core Topics

genetic engineering techniques, quantitative techniques, data acquisition/statistics, use of computer databases, Experimentally-based research, including a formal proposal, report, and presentation(s)

Table 2. Skills That Biochemistry and Molecular Biology Students Should Obtain by the Time They Have Finished Their Undergraduate Program Science Related

68 Communication

Quantitative

WorkPlace

Understanding of the fundamentals of chemistry and biology and the key principles of biochemistry and molecular biology. Awareness of the major issues at the forefront of the discipline. Ability to dissect a problem into its key features. Ability to design experiments and understand the limitations of what the experimental approach can and cannot tell you. Ability to interpret experimental data and identify consistent and inconsistent components. Ability to design follow-up experiments Ability to use oral, written, and visual presentations to present their work to both a science-literate and a science-non-literate audience. Ability to assess primary papers critically. Ability to think in an integrated manner and look at problems from different perspectives. Awareness of the ethical issues in the molecular life sciences. Ability to use computers as information and research tools. Good “quantitative” skills such as the ability to accurately and reproducibly prepare reagents for experiments. Awareness of the available resources and how to use them. Ability to work safely and effectively in a laboratory. Ability to collaborate with other researchers.

Table 3. Foundations of “Vision and Change” Conceptual Areas Evolution: Structure and function: Information flow, exchange , and storage : Pathways and transformations of energy and matter: Systems: Skills

ABILITY TO APPLY THE PROCESS OF SCIENCE: Biology is evidence-based and grounded in the formal practices of observation, experimentation, and hypothesis testing. ABILITY TO USE QUANTITATIVE REASONING :Biology relies on applications of quantitative analysis and mathematical reasoning. ABILITY TO USE MODELING AND SIMULATION: Biology focuses on the study of complex systems. ABILITY TO TAP IN TO THE INTERDISCIPLINARY NATURE OF SCIENCE: Biology is an interdisciplinary science. ABILITY TO COMMUNICATE AND COLLABORATE WITH OTHER DISCIPLINES : Biology is a collaborative scientific discipline. ABILITY TO UNDERSTAND THE RELATIONSHIP BETWEEN SCIENCE AND SOCIETY: Biology is conducted in a societal context.

As a result of the Vision and Change initiative, the American Society of Biochemistry and Molecular Biology, using the NSF Research Coordination Network-Undergraduate Biology Education funding mechanism, convened a nationwide series of meetings over the next five years that resulted in a series of papers in Biochemistry and Molecular Biology Education, focusing on giving further definition to the concept areas that constitute biochemistry and molecular biology (Tansey et al.), the requisite skills (White et al.) and interdisciplinary concepts (Wright et al.) (7–9) which are summarized in Figure 2.

Figure 2. Summary of concepts and skills recommended by the American Society for Biochemistry and Molecular Biology. 69

These conceptual areas and skills became the foundation of the American Society for Biochemistry and Molecular Biology’s accreditation program and exam, as well as Course Source’s ”Biochemistry and Molecular Biology Learning Framework” (11, 12). Current curriculum recommendation from the American Society of Biochemistry and Molecular Biology’s are based on these conceptual areas and skills. For example, in the section on the Foundational Concepts of Macromolecular Structure and Function, the necessary foundational concepts are framed in the form of eight questions (learning goals): What factors contribute to the size and complexity of biological macromolecules? What factors determine structure? How are structure and function related? What is the role of noncovalent intermolecular interactions? How is macromolecular structure dynamic? How is the biological activity of macromolecules regulated? How is structure (and hence function) of macromolecules governed by foundational principles of chemistry and physics? How are a variety of experimental and computational approaches used to observe and quantitatively measure the structure, dynamics and function of biological macromolecules? Each question in the framework is associated with a brief overview of the key content features and a series of sample learning objectives. With regard to non-covalent interactions, the key conceptual areas and how they are linked to enzyme structure and function are represented by Figure 3.

Figure 3. Concept (mind) map of key features of non-covalent interactions and the roles they play in protein structure and function. This information builds from simple concepts of chemistry and physics and clearly identifies the roles that both attractive and repulsive non-covalent interactions play in enzyme structure function relationships. Interestingly, very few biochemistry textbooks or pedagogical publications (13) 70

acknowledge the pivotal role that repulsive non-covalent interactions play in structure function relationships leading to both student and faculty misconceptions about non-covalent interactions. One of the hardest parts about teaching, whether it is concepts or content or skills is setting the level of expectations of the students clearly. For example, a suitable learning goal for the material represented above could be: “Understand the various roles that non-covalent interactions may play in the structure and function of an enzyme.” If this is accompanied by a clear rubric that is given to the students and used in grading any related assignments, it has been shown that student outcomes improve (14). A rubric that has been developed by a group of faculty from different institutions tha are part of the Malate Dehydrogenase CUREs community (15) is shown in Table 4.

71

Table 4. Learning Objectives and Assessment Rubric for the Learning Goal: “Understand the Various Roles that Non-Covalent Interactions May Play in the Structure and Function of an Enzyme.” Learning Objective

Category

Criteria

Compare and contrast the physical basis for Coulombic interactions and Hydrophobic interactions

Good

Coulombic interactions: charge - charge interactions, Coulomb’s law, attractive, repulsive, depend upon magnitude of charge, full, partial charges, depends upon surrounding dielectric, distance. Hydrogen bonds may be bifurcated, depend on angle, distance, polarity of partners, strong hydrogen bonds may have some covalent character. Hydrophobic interactions- only favorable, need polar solvent, depend upon entropy of the system, solvent cages around individual hydrophobes decrease entropy, hydrophobic interactions minimize entropy loss. Repulsion effect of polarnonpolar interactions. Misses 1-2 points of excellent

Acceptable

Misses 3-4 points of excellent

Unacceptable

Minimal or no aspects of answer

Excellent

Good

Hydrogen bonds- in helix between C=O and N-H in i to i+4 relationship- in beta sheets: between strands, C=O to N-H May get charge or hydrophobic stabilization between side chains in helices (helix wheel, ridges and grooves) but not necessary for formation. Misses 1-2 points of excellent

Acceptable

Misses 3-4 points of excellent

Unacceptable

Minimal or no aspects of answer

Excellent

72 Briefly outline the types of non covalent interactions you would expect to stabilize secondary structure in a protein

Briefly outline the types of non covalent interactions you would expect to be involved in maintaining a functional tertiary structure in a protein. How does this change in intrinsically disordered regions of proteins?

Table 4. (Continued). Learning Objectives and Assessment Rubric for the Learning Goal: “Understand the Various Roles that Non-Covalent Interactions May Play in the Structure and Function of an Enzyme.” Learning Objective

Category

Criteria

Good

Hydrophobic core (solvent exclusion), hydrogen bonds and charge -charge interactions, van der Waals interactions. Attractive and repulsive: importance of dynamic structure, intrinsically disordered regions/proteins. For folded proteins attractive > repulsive, but not by much . Usually greater in extremophile proteins. In intrinsically disordered proteins or regions repulsive forces> attractive forces. Misses 1-2 points of excellent

Acceptable

Misses 3-4 points of excellent

Unacceptable

Minimal or no aspects of answer

Excellent

Wide variety of attractive and repulsive interactions- Coulombic, Hydrophobic, van der Waals: govern stability of interface: attractive > repulsive: Functions: govern symmetry, homo and hetero-oligomers, Allosteric regulation, Protein-Protein Interactions

Good

Misses 1-2 points of excellent

Acceptable

Misses 3-4 points of excellent

Unacceptable

Minimal or no aspects of answer

Excellent

Van der Waals (steric) – size, Charge-Charge attractions (full and partial), Hydrophobic. Full charges > partial charges and hydrogen bonds- affected by polarity of local environment. Important for specificity as well as affinity- 3 point

Excellent

73

Proteins may have quaternary structure. What types of non-covalent interactions might you expect to see across subunit interfaces? What functions might these interactions play?

What types of non covalent interactions would you anticipate are involved in substrate binding to an enzyme?- compare the relative strengths of the interactions you suggest

Table 4. (Continued). Learning Objectives and Assessment Rubric for the Learning Goal: “Understand the Various Roles that Non-Covalent Interactions May Play in the Structure and Function of an Enzyme.” Learning Objective

Category

Criteria

Good

attachment theory, etc. Any given ligand may have repulsive as well as attractive interactions. Misses 1-2 points of excellent

Acceptable

Misses 3-4 points of excellent

Unacceptable

Minimal or no aspects of answer

Excellent

Good

Catalysis promoted by orientation (correct orientation increases number of productive collisions) and by strain of bonds involved in reaction. Strain may be physical or electronic (strained bonds more reactive-higher energy, less stable, better nucleophile, electrophile. Non-covalent interactions involved with stabilization of the transition state and or destabilization of the ground state. Misses 1-2 points of excellent

Acceptable

Misses 3-4 points of excellent

Unacceptable

Minimal or no aspects of answer

Excellent

Good

Ones that change the local environment of the protonatable group: mutations that lower polarity promote protonation, raising the pKa, mutations that increase polarity have opposite effect. Mutations that remove or create formal like charge impact pKa to favor the lower energy (more stable) state: e.g. two adjacent carboxyl groups- one will be protonated, one unprotonated Misses 1-2 points of excellent

Acceptable

Misses 3-4 points of excellent

How might non-covalent interactions in a proteinsubstrate complex promote catalysis

74 What types of mutations at nearby residues would you predict alter the pKa of a protonatable group on a protein?

Table 4. (Continued). Learning Objectives and Assessment Rubric for the Learning Goal: “Understand the Various Roles that Non-Covalent Interactions May Play in the Structure and Function of an Enzyme.” Learning Objective

Category Unacceptable

Criteria Minimal or no aspects of answer

Table 5. Concepts, Content and Skills Associated with Pre-Med Education Area Foundational

Medicine Related

Concepts

Content

Skills

75

underlie biological complexity, genetic the ability to synthesize information and grounding in scientific principles and diversity, interactions of systems within the collaborate across disciplines. knowledge body, human development, and influence of the environment. application of scientific knowledge and Read the medical and scientific literature of understanding how current medical scientific reasoning based on evidence. The one’s discipline, but to examine it critically knowledge is scientifically justified, and ability to evaluate competing claims in the to achieve lifelong learning. These activities how that knowledge evolves. Curiosity, skepticism, objectivity, and the use of medical literature and by those in medical require knowledge and skills in critical scientific reasoning are fundamental to the analysis, statistical inference, and industries. Effective practice of medicine experimental design. Application of practice of medicine. Medical professionals recognizes that the biology of individual scientific knowledge in medicine requires should demonstrate strong ethical patients is complex and variable and is attention both to the patient as an principles and be able to recognize and influenced by genetic, social, and individual and in a social context. manage potential conflicts of interest. environmental factors. Decision making in medical practice involves uncertainties and risks.

The Impact of “Pre-Med” Education: A Focus on Transferable Skills As the Vision and Change final report was released, a joint AAMC-HHMI committee released a report “Scientific Foundations for Future Physicians (16),” whose main conclusions are summarized in Tables 5 and 6, based upon “Overarching Principles.” While the AAMC-HHMI report is relevant to both undergraduate and medical school education, and can be summarized in the phrase “Context, Collaboration and Communication,” many of the skills and conceptual areas (though not in the same level of coverage) overlap those that form the centerpiece of undergraduate education in the molecular life sciences (17). The recommendations of the AAMC-HHMI report are also reflected in recent changes in the MCAT exam. Although a Biochemistry and Molecular Biology major is a popular “pre-med” major, the major is designed to serve a wide variety of career goals and prepares students for many different career paths including graduate school in many of the life sciences, other professional schools such as dental or veterinary medicine, or careers in science writing or patent law, as well as direct access to jobs in biotechnology, all of which benefit from the conceptual understanding, content and skills associated with a biochemistry and molecular biology major. Since many pre-med students are not biochemistry and molecular biology majors, a significant question that needs to be addressed is whether a “majors” course in Biochemistry is suitable for these pre-med students. Many Biochemistry and Molecular Biology programs cover the central core of biochemistry in a two or more semester sequence with significant biology and chemistry pre-requisites, however pre-med students from other majors often have not taken these pre-requisites. It would appear that, in those cases, a separate, one semester, “survey” course that includes essential concepts from pre-requisites a biochemistry major would have taken, could be taught for such students. Such a course could also serve as a service course for other science majors, including chemistry majors, who might not have taken the biology pre-requisites a biochemistry major has taken. Such a survey course should be program dependent as the biochemical conceptual understanding and content and skills would need to be paired with the program specific essential concepts from “allied fields” that non-majors may be lacking. An alternative approach would be to communicate transparently to the non-major the concepts, content and skills of the field and how a single semester of a biochemistry series will cover a fraction of these areas. For instance, the first course of the biochemistry series may discuss macromolecular structure, kinetics, and glucose metabolism so that a non-major requiring cell signaling knowledge could compliment the biochemistry experience with another course. Both approaches, depending on resources and student populations served, could be a starting point for addressing biochemistry and molecular biology curricula for majors and non-majors. Meaningful conversations about curricular content should take place between all involved departments and programs to best serve the wide population of students needing to take a biochemistry course.

What Is Modern Biochemistry and Molecular Biology? Finding a definition of Biochemistry and Molecular Biology in either professional society (ACS and ASBMB) is elusive and perhaps this is as it should be. The closest ASBMB comes to a definition is “Biochemistry and Molecular Biology are distinguished by their focus on information flow, structure, function and mechanism within overarching biological contexts” at the start of their discussion of curriculum (18). Which can be compared with the ACS statement (19) “Members of the Division of Biological Chemistry use the principles of chemistry to assist in the development of a deeper understanding of biological processes.” Some have used the term Molecular Life Sciences (2, 20). For example, ASBMB runs a very successful biennial conference entitled “Transforming 76

Undergraduate Education in the Molecular Life Sciences.” Perhaps a suitable definition might be “biochemistry and molecular biology contribute to understanding the processes of living systems at the molecular level,” which would bring in the multidisciplinary perspectives particularly of physics and mathematics/computational science to those of biology and chemistry. Such a definition would be consistent with both Vision and Change and recent discussions at both ACS and ASBMB about quantitative aspects of the science. Such a definition would certainly be consistent with recent publishing trends in journals invoking biochemistry and molecular biology with an emphasis on “and” rather than separately defining molecular biology as distinct from biochemistry. This definition would also encompass specialized tracks within a biochemistry and molecular biology degree such as ”Chemical Biology”, “Biophysical Chemistry” or “Computational Biology/ Biochemistry” or the more traditional Immunology, Microbiology, Cell Biology, Cell Physiology, etc. It would also allow the development of foci on increasingly important topics such as environmental biochemistry, molecular ecology, biotechnology, and molecular neurobiology. Table 6. Communication Skills Required for the Practice of Medicine Foundational Skills

• write logically and with clarity and style about important questions across disciplines; • articulate persuasively, both orally and in writing, focused, sophisticated, and credible thesis arguments; • be able to use the methodologies that particular disciplines apply for understanding and communicating results effectively; • approach evidence with probity and intellectual independence; and • use source material appropriately with scrupulous and rigorous attribution.

Medicine Related Skills

Scientific matters can and should be communicated clearly to patients and the public, taking into account the level of scientific literacy of these audiences and understanding the intellectual and emotional responses to medical diagnoses and therapies. For example, physicians should be able to explain to patients: • the complexity and variability of the human body to help them appreciate that there is no single approach to the prevention, diagnosis, and management of disease; • the influence of genetic, lifestyle, and environmental factors in health and disease, as well as the heritability of genetic factors; • in appropriate terms, the technologies for diagnosis and treatment of disease, their relative risks and benefits, and the advantages and disadvantages of alternative choices; • in appropriate terms, the rationale for treatment strategies, including lifestyle changes as well as pharmacological interventions, how the drugs work, their possible interactions with other drugs, their risks and benefits, and alternatives, both pharmacological and nonpharmacological; and • how the brain and other organ systems interact to mediate behavior throughout the lifespan in health and in disease.

Current Curricula Recommendations In the belief that all professional chemists need to know some biochemistry, the ACS guidelines require that approved programs offer and certified majors graduate with the equivalent of three semester hours of biochemistry (Table 7). Molecular aspects of biological structures, equilibria, energetics, and reactions should be covered in the required biochemistry experience for chemistry 77

majors. Sufficient introduction should be presented so that students can obtain the flavor of modern biochemistry and an appreciation of the important applications in biotechnology. Table 7. Content Areas Recommended by ACS Chemistry Oriented Biological Structures and Interactions

Biology Oriented

Fundamental building blocks (amino acids, Biopolymers (nucleic acids, carbohydrates, lipids , nucleotides, and prosthetic peptides/proteins, glycoproteins, groups. Supramolecular Architecture and polysaccharides) Membranes

Biological Reactions Kinetics and mechanisms of biological catalysis; Organic and inorganic cofactors

Biosynthetic pathways and strategies/metabolic engineering; Metabolic cycles, their regulation, and metabolomics

Biological Equilibria Acid-base equilibria; Thermodynamics of binding and recognition; Oxidation and and reduction processes Thermodynamics

Electron transport and bioenergetics; Protein conformation/allostery, folding, oligomerization, and intrinsically disordered proteins (IDPs)

Practical Topics

molecular biology techniques (including PCR), bioinformatics and –omics, molecular modeling, protein engineering, and isolation and identification of macromolecules and metabolites

error and statistical analysis of experimental data, spectroscopic methods; kinetics, chromatographic separations, electrophoretic techniques, protein purification

Notable in its lists are the absence of Evolution and Homeostasis, two unifying pillars of the Vision and Change Initiative and current ASBMB Curricular recommendations. While the American Chemical Society does not “offer” a degree in Biochemistry and Molecular Biology, or Biochemistry, or Biological Chemistry, its general degree requirements are more course oriented with regard to content rather than concept oriented, although all of the topics in Table 6 appear in some manner in the ASBMB Biochemistry and Molecular Biology Learning Framework. With regard to skills, ACS lists: 1) Problem Solving skills, 2) Chemical Literature and Information Management Skills, 3) Laboratory Safety Skills, 4) Communication Skills, 5) Team Skills, and 6) Ethics. Although the ASBMB “skills” are laid out with more detail, the only significant difference between the two professional societies is in the context of teamwork and collaboration. Both highlight the need to be a team player (valued greatly by industry), but ASBMB specifically recognizes collaboration which has become a central feature in much modern research and is distinct from teamwork. The American Society for Biochemistry and Molecular Biology Curricula recommendations remain largely based upon the 2004 curriculum recommendations that first introduced a focus of concepts and skills rather than recommended or required courses, refined by the Vision and Change recommendations to incorporate overarching emphasis on evolution and homeostasis throughout the curriculum, as reflected in Figure 2. The Society approved the”Biochemistry and Molecular Biology Learning Framework” which is published on both the Society web page and Course Source (14, 15).

78

Accreditation and Student Certification The American Society for Biochemistry and Molecular Biology initiated a national accreditation (18, 21) for programs offering either a B.S. or a B.A. degree in biochemistry and molecular biology (or closely related majors) with two main goals, 1) to indicate that the program offered a curriculum and used pedagogical approaches consistent with the expectations of ASBMB, and 2) to encourage the use of evidence-based high impact teaching practices. Furthermore, to mark achievement by individual students in such programs they developed an independently scored exam aligned with the vision of biochemistry and molecular biology described earlier (Figure 2) that could be used to assess student performance. Students, based upon their performance in the exam, have their degree certified by ASBMB. Students with exceptional performance on the exam are recognized by ASBMB as having graduated with distinction. The Certification Exam is based primarily on a free response format with minimal or no multiple-choice questions and is designed to be taken in a one hour period. The exam emphaseizes overarching concepts and critical thinking skills, and students are expected to demonstrate the ability to synthesize information into coherent responses, not simply retrieve facts. The exam has twelve to fourteen questions drawn from the four core concept areas illustrated in Figure 2 focusing on evaluation of concepts and competencies outlined in the ASBMB learning goals and objectives. Results from an independent, nationally-recognized exam not only assist faculty and programs in meeting the demands from accrediting bodies and university administrators for evidence-based assessment, but also provides formative assessment data on areas of curricula strengths or weaknesses. The use of an independently graded exam covering foundational concepts and skills by the American Society for Biochemistry and Molecular Biology to certify individual student degrees is distinct from the American Chemical Society approach of “certifying” student bachelor’s degrees solely on the basis of having been conferred by an ACS approved department.

The Central Role That Undergraduate Research Plays in Undergraduate Education in the Molecular Life Sciences The Vision and Change Report emphasized the need to incorporate undergraduate research into the curriculum of life science majors. Research experiences have major effects on persistence in science (22–28) and positive outcomes in conceptual understanding and skills development, essential for effective workforce development (29–35). The Council on Undergraduate Research (CUR) defines undergraduate research as inquiry or investigation conducted by undergraduates that makes original intellectual or creative contributions to the discipline (36), and can often be provided by Course Based Undergraduate Research Experiences (37). Such work is a high impact practice providing robust service learning for students, increases retention, and enhances student learning though mentorship by faculty. Furthermore, it develops a deeper critical thinking ability, as well as intellectual independence. Realization that authentic research was important for student development was evident in the Boyer Report (3), where smaller schools that provide research mentoring disproportionately produce more graduate school students than research intensive institutions. The central role that undergraduate research plays in the education of students in the molecular life sciences is accepted and well documented. Both the American Chemical Society and the American Society for Biochemistry and Molecular Biology encourage the incorporation of 79

undergraduate research in the curriculum. The format that undergraduate research experiences take varies significantly depending upon the nature of the institution. In institutions with large numbers of students in the major, providing research experiences for all students presents problems. Flexibility in the ways that research experience is gained is essential. This experience might be obtained by mentored research in the research lab. Alternatively, it could be via a summer research experience or through course-based undergraduate research. Course-based undergraduate research experiences (CUREs), as will be discussed in a separate chapter in this volume, provides a cost-effective way of engaging all students in research activities. The benefits of undergraduate research have been extensively studied and documented, and as discussed earlier undergraduate research is regarded as a high impact teaching practice (38). To inclusively provide a research experience to students, faculty and institutions will need to take advantage of all models of undergraduate research to more widely deliver this high impact experience to all students. Research experiences should engage the student in all aspects of research. The work of Dolan and others have established criteria that are the hallmarks of research (39–42). These include, as illustrated in Figure 4, student generated hypotheses and original research that provides new knowledge.

Figure 4. Essential elements in a student generated research hypothesis and proposal. This requires a number of abilities including access to prior knowledge and the ability to assess the validity of information. Participation in authentic research activities also provides students with the opportunity to present their work in a variety of formats, using a variety of communication skills. Undergraduate research should include opportunities to write formal research proposals, and, of course, final reports that may be in the format of contributions to the scientific literature. As with the earlier example of no- covalent interactions, providing students with a detailed rubric, Table 8 (developed by the Malate Dehydrogenase CUREs Community (15)), lets students know the level of expectation and detail that they should achieve.

80

Table 8. Learning Objectives and Assessment Rubric for Learning Goal: Create/Develop and Present a Testable and Falsifiable Hypothesis and Appropriate Experiments to Interrogate the Hypothesis Learning Objective

Category

Criteria

Describe how the proposed work fits into the field/fills a gap in knowledge Excellent

Relationship of big picture of project to the field clearly indicated and explained

Good

Relationship of big picture of project to the field indicated but explanation unclear

Acceptable

Relationship of big picture of project to the field clearly mentioned but not explained

Unacceptable

Relationship of big picture of project to the field not addressed

Excellent

Hypothesis or goal clearly stated. Gives appropriate background justification for hypothesis

Good

Hypothesis or goal clearly stated but lacking justification

Acceptable

Hypothesis or goal is not clearly stated. Cited studies cited may or may not support hypothesis or goal as written. Hypothesis or goals lacking

Clearly states their Hypothesis and the requisite background information that lead to the hypothesis

81

Unacceptable Clearly indicates the testable and falsifiable predictions the hypothesis makes Excellent

Briefly outlines the types of experiments and data that will be used to interrogate the hypothesis

Good

Clearly states and justifies testable and falsifiable predictions and relates to hypothesis. Indicates controls Limited predictions and justifications, some indication of controls

Acceptable

Limited predictions, no justification or controls indicated.

Unacceptable

Predictions lacking

Table 8. (Continued). Learning Objectives and Assessment Rubric for Learning Goal: Create/Develop and Present a Testable and Falsifiable Hypothesis and Appropriate Experiments to Interrogate the Hypothesis Learning Objective

Category

Criteria

Excellent

Summary of experiments is consistent with testing hypothesis or reaching goal. Types of data that will support or falsify hypothesis indicated

Good

Outlines experiments but not how the data will contribute to the interrogation of the hypothesis

Acceptable

Gives detail but some proposed studies are not consistent with hypothesis or goal.

Unacceptable

Minimal attention to how experimental data will be obtained or used to interrogate the hypothesis

Excellent

References added appropriately

Good

References placed in text but some references missing.

Acceptable

Significant omission of references.

Unacceptable

R writteneferences lacking.

Excellent

Logical flow from global to particular study point of view. Engaging writing style. Clearly connects ideas. Good use of graphics Solid order and structure. Inviting writing style. Effectively moves the reader through the text. Graphics present but not well explained Organization is functional; some order lacks logical pattern and structure. Minimal use of graphics

As appropriate cites necessary references

82 General flow/organization

Good Acceptable Unacceptable

Lacks cohesive structure, difficult to follow.

Excellent

No spelling or grammatical errors; includes all required sections; clearly written in language for reader familiar with biochemistry; well organized and legible Minor spelling or grammatical errors; includes all required sections.

Grammar/spelling/general attention to detail

Good

Table 8. (Continued). Learning Objectives and Assessment Rubric for Learning Goal: Create/Develop and Present a Testable and Falsifiable Hypothesis and Appropriate Experiments to Interrogate the Hypothesis Learning Objective

Category Acceptable Unacceptable

Criteria Some spelling and grammatical errors; some sections not complete or less well organized. Significant spelling and grammatical errors; disorganized, difficult to follow.

83

Experiential Learning A critical role in the education of biochemistry majors is also given to experiential learning. This can take a variety of formats including laboratory courses, service learning, and internships and should include laboratory safety and the recognition of common laboratory hazards and responses to accidents involving hazardous materials. Reiterative discussion of the principles of ethical conduct of research and scholarship, (plagiarism, appropriate citation, qualifications for authorship, appropriate use of images and confidentiality) should be embedded in appropriate classes. In all cases of experiential learning, there should be a common intellectual thread through the activity, where the student has ownership, engages in appropriate science-related activities, and is required to document and communicate that work. Whether with undergraduate research or experiential learning the activities should constitute a significant component (ASBMB recommends a minimum of 400 contact hours, ACS 400 hours beyond introductory chemistry laboratory) and be distributed throughout the curriculum.

A Student-Centered Curriculum The Vision and Change report emphasizes the use of evidence-based teaching approaches where studies have shown that the classroom approach actually leads to student gains. Further-more, Vision and Change emphasized the need for an active, student centered classroom. A number of studies over the years have suggested that the traditional lecture approach to teaching is not particularly effective. Despite growing evidence supporting the effectiveness of a wide variety of high impact, student-centered approaches (43–49), lectures still dominate the teaching profession. A recent study suggested that 55% of teaching involves traditional lectures with a further 25% involving some sort of lecture format (50). Figure 5 illustrates a variety of student-centered approaches where evidence has shown the approaches to be effective in the classroom. These include Think-Pair-Share activities (43), the use of clicker questions (44), peer teaching (45, 46), case study use (47), and the use of student modeling activities (48). A particularly effective approach involves so-called brainstorming, which is sometimes referred to as mind mapping or concept mapping (49). Other approaches include reflective writing (51), peer review (52), collaborative work (53), the so-called one minute paper (54) (where students write a summary of a brief lecture or create a concept or mind map), portfolios (55), service learning (56), and learning communities (57)). In all cases, these approaches actively involved students or groups of students in developing and reviewing information and concepts. Here we will discuss briefly several of these approaches which we have found to be particularly effective in our biochemistry classes. These are Think-Pair-Share activities, peer teaching, peer-review, and the use of concept or mind maps.

84

Figure 5. Student centered high impact teaching practices. Think-Pair-Share Think-Pair Share activities involved the instructor asking a question and giving the students in the class several minutes to think on their own and make notes about the question. The pair component of the activity involves students forming pairs or small groups and discussing the individual responses to the question. The group then decides on a consensus setoff points for the question. In the share component, a member of the group shares this consensus with the class. The share component often involves the instructor revealing the correct answer. Peer Teaching Based on the old Chinese proverb “Tell me and I’ll forget; show me and I may remember; involve me and I’ll understand,” peer teaching is a variant of the SODOTO approach (See One, Do One, Teach One) often used in Medical Schools (58) and related to Think-Pair-Share approaches. In peer teaching, a topic, perhaps represented by a paper from the original literature, is used as the basis of the peer teaching activities. The classroom is split into groups and each group assigned a particular aspect of the paper perhaps represented by a figure or table in the paper. The groups are given a period of time to discuss what features of the topic are essential to understand the topic. Each group then designates a “teacher” who will carry that information to the next group and teach the group the agreed-upon topics. This process continues during the class until all groups have discussed each topic. In the process, all students have played the role of both learner and teacher and had to think about how best to present the material. The faculty member listens in on the various discussions throughout the class and adds appropriate information as needed. Usually the class ends with a brief summary of the pertinent information on each topic. Peer Review Peer review activities are particularly appropriate for either laboratory classes or classes that involve significant writing components. Students, often as a homework assignment have written a paper on the assigned topic. The papers are collected, anonymized, and given to students to peer review (usually 2-3 per student). The review usually involves a detailed rubric and the reviewing students assign scores based upon the rubric. To be truly effective, the peer-review culminates in class presentations of the strengths and weaknesses of each paper and depending upon the context can 85

involve students ranking the papers based upon the scores they gave using the rubric. The instructor plays the role of agent provocateur, raising appropriate points as needed to direct the class discussion. For each paper reviewed, one student in the class is designated the “recorder” and is tasked with summarizing the points, good and bad, raised during the class discussion of a given paper. This type of peer review approach allows students to see and discuss a wide range of writing styles and actively decide which approaches are the most effective. After the peer review of all the papers, each student is given the blind reviews of their own paper and the recorders summary of the class discussion and has the chance to revise their paper before submission for grading. Such peer review approaches work particularly well with CURE based laboratory classes where students formulated their background information, a hypothesis, and predictions that can be explored experimentally. Mind Maps The construction of concept maps or mind maps requires students to be actively involved in creating connections between concepts or topics, and in organizing information in a logical manner. For example, prior to class, students can be assigned reading on the topic, and, at the start of class, are grouped and asked to create a mind map of some aspect of the topic. Each student group then presents their mind map on the assigned topic using, for example, a white board. The groups then rotate through all of the mind maps and are asked to add any comments they think necessary to improve the original mind map. After all of the groups have visited and commented on all of the mind maps, the groups returned to their original mind map and, as appropriate, revise the mind map. At the end of class, all of the mind maps are photographed (students always have their smart phones with them), and the photographs sent to the faculty member who then posts them to a class website. As appropriate, the faculty member also creates and circulates idealized mind maps of each topic presented in the class. For example, Figure 6 is a consensus mind map generated from in-class discussion of the basics of reactions and interactions necessary to understand enzyme function which aligns concepts from chemistry with enzyme function.

Figure 6. Consensus mind map of central concepts of reactions and interactions generated from class discussion. 86

Using mind maps in this way effectively combines aspects of peer teaching, Think-Pair-Share activities, and peer review.

The Problem of Large Class Size Although large class sizes are sometimes used as a justification for keeping to a traditional lecture format, in reality many of the high impact teaching practices suggested in Figure 5 and discussed here are easily incorporated into a large class format, and have been at a number of large institutions. Think-Pair-Share, Clicker Questions, Concept Mapping, Case studies and One Minute Paper ideas all translate essentially seamlessly to a large class format particularly when combined with well thought out rubrics, well trained teaching assistants and recitation or Review sessions. Many large universities (for example, the University of Texas) that routinely face the problem of large class sizes have centers for teaching and learning that prepare faculty and teaching assistants, and publish resources that are freely accessible to all faculty (59).

Gateway Concepts In recent years, there has been significant discussion of “Gateway” concepts (60–63) and a number of so-called concept inventories have been developed (64–69). Gateway concepts are ideas and theories that are foundational to understanding the biochemistry that is built from them. For example to understand the structure-function relationship of macromolecules students must have a firm grasp of at least 5 gateway concepts of chemistry (see below), and with this foundational knowledge, students should readily be able to evaluate macromolecular structures in terms of through bond and through space interactions. Likewise, understanding enzyme kinetics and regulation builds from foundational chemistry concepts of reactions and interactions. Some discussion of the core concepts of the “allied” fields that set the stage for biochemistry and molecular biology seems warranted. The structure (both in terms of through bond connectivity (covalent bonds) and through space interactions (non-covalent interactions, both favorable and repulsive) of a molecule determine its dynamic properties and reactivity (chemical and physical): i.e. its function. Five gateway concepts of bonding and interactions underlay the foundation for understanding macromolecular structure function relationships and the structure, activity and regulation of enzymes. These are: 1) Core Concepts of Covalent Bonds and Polarity, 2) Bond rotations and vibrations , 3) Hydrogen bonds and other non-covalent interactions, 4) The Hydrophobic effect and 5) Dynamic aspects of molecular structure. Likewise, there are a series of gateway concepts for understanding reactions. Enzymes are biological catalysts that enable cells to control the wide variety of chemical reactions that continuously occur in a cell. Enzymes enable these processes to occur at ambient temperatures, with the requisite specificity, and unlike chemical catalysts, exhibit the phenomenon of saturability. The reactions catalyzed by enzymes often have mechanisms for regulation of the rates of the reactions. While many enzymes are proteins, some RNA molecules also exhibit enzymatic activity and are termed “catalytic RNA.” In all cases, the interactions of enzymes with their substrates, products and, if appropriate, regulatory molecules are governed by the same foundational concepts that govern chemical reactions in general. Presented here are 5 “gateway” concepts necessary to understand the action of enzymes from a chemical perspective. 1) Collision theory, 2) Transition state theory, 3) Rate laws, Steady States and Equilibria, 4) The effects of temperature, and 5: Structure and reactivity. These gateway concepts are expanded in more detail in Appendix 1. 87

Understanding Student Misconceptions of Gateway Concepts Can Lead to More Effective Instruction Over the past 15 to 20 years there has been a significant effort on developing so-called concept inventories in biochemistry and molecular biology. Unfortunately, often these do not reflect the underlaying concepts, but are useful tools in developing an understanding of student misconceptions. Significant work on identifying student misconceptions in certain key areas of biochemistry and molecular biology have contributed to improved teaching approaches. To have a maximum effect on student outcomes, it is essential to link foundational or gateway concepts to effective assessment of student outcomes and identification of student misconceptions (Figure 7). This requires identification of the Threshold (Gateway) concepts at the level of introductory classes in Chemistry, Biology, Physics, and Mathematics and how they feed into introductory concepts of biochemistry and molecular biology. The work of Loertscher and Minderhout and colleagues (60–69) is an important starting place, but more work, especially in the areas of chemistry and mathematics, is needed. Allied with identification of such gateway concepts is the need for suitable assessment instruments capable of identifying student misconceptions, as discussed by Lewis and coworkers (66).

Figure 7. Enhancing student outcomes Through a cycle of formative assessment of foundational concepts and skills. Finally, as demonstrated by and emphasized in the work of Taylor et al. (70), it is necessary to continually reinforce all areas of conceptual, gateway knowledge that has been introduced in prior course work to enhance student retention. This can be in the form of using a “pre-test” at the start of a course based upon pre-course reading and review of “assumed” knowledge, or a brief bootcamp to quickly review such assumed knowledge.

Quantitative Skills in the Curriculum Figure 8 summarizes the quantitative skills that are essential for biochemistry and molecular biology.

88

Figure 8. Key elements of quantitative skills for biochemistry and molecular biology. While a number of articles and reports, including BIO2010 and Vision and Change, have emphasized the need for undergraduate students to be better educated in the area of quantitative mathematical models of biological phenomena to enable them to engage in an era in science increasingly dominated by “Big Data” (the Omics era and topics such as Genomic Enzymology), this is still an area of deficit. Students in the molecular life sciences seem increasingly “equationophobic” and generally have poor quantitative skills and intuition. It seems that, as a community of biochemistry and molecular biology educators, we assume that students are “learning” these quantitative skills elsewhere in the curriculum. We need to do better. The argument seems too often to be that 1) there is too much material (content) to cover, 2) we don’t have time to focus on those quantitative, 3) students should have acquired these quantitative skills elsewhere. Many faculty have a “bootcamp” week at the start of a course where they review essential background. Perhaps this idea should be extended to include a “quantitative methods” bootcamp at the start of every course in the molecular life sciences. Quantitative skills can be developed, allowing these skills to be fully integrated across the curriculum. As with foundational concepts, repetition is the key to student comfort, familiarity, and retention of these skills.

Communication: Writing and Presentation Skills Communication skills are highly valued by almost all employers, irrespective of the field of employment. Developing various forms of communication in the course of a student’s education is known as a high impact practice (38). The training of undergraduates in Biochemistry and Molecular Biology offer a wide variety of formats for student engagement in communication, as summarized in Figure 9, and can be incorporated in many ways throughout the undergraduate curriculum.

89

Figure 9. Key elements of communication come in many forms with various audiences. Whether in written, oral, or visual formats students should be able to communicate to a variety of audiences, and in today’s world should also be aware of how to communicate using a variety of social media formats. Developing the attributes of great communicators lays the groundwork for success, not only in the classroom, but also in one’s life and career. These attributes can be put into the three components of communication, When, What and How: 1)When: To be an effective communicator, someone must listen and be able to both understand and know when to respond appropriately, they must be able to relate to others and tailor their message to the audience at hand, they must be available and able to lead inclusive discussions and they must be able to ask questions that engage the audience and give some level of ownership to the audience for the answers. 2) What: They must be able to simplify the complex and make the complicated understandable. They must be specific, clear and concise, and on point so that the audience doesn’t have the opportunity to be distracted by off message points. And 3) How: They must be confident to earn audience trust and demonstrate their knowledge, and they should be audience appropriate in terms of body language and attire. In terms of the life sciences, the various types of “things” a student should be able to communicate, as well as the different audiences they should be able to communicate with, are summarized in Figure 9. Opportunities for such communication should be threaded throughout the curriculum (71–74).

The Central Role That Accessing and Assessing Knowledge Plays in Undergraduate Life Science Education In addition to conceptual areas of the science and foundational skills, to prepare students for the ever expanding content of life science disciplines, and the challenges of the workplace or graduate school, it is essential that the undergraduate curriculum include both exposure to current issues and advances in the science and arm them with the skills to assess the information that they hear/see/ find. As summarized in Figure 10, undergraduates should be aware of and use original peer reviewed literature, they should be able to access and use publicly available data (structural, sequence, and -omics data), and understand the conceptual basis of computational approaches such as quantum mechanics, molecular mechanics, and molecular dynamics. There are many ways that these aspects 90

can be incorporated into the curriculum including regular outside speaker seminar series, student/ faculty journal clubs, and seminar courses that involve a series of “current topics.”

Figure 10. Students must access, assess, analyze and utilize data and information from many sources in biochemistry and molecular biology.

The Important Role of Capstone Courses and Experiences Many of the pedagogical approaches discussed here are often incorporated into a capstone course or experience. Capstone courses play a significant role in helping students not only hone essential skills, but also integrate material from other courses. Capstone courses are often writing intensive courses which also augments communication skills (75–77).

Content versus Concept Revisited With a focus on conceptual knowledge rather than extensive content, a question arises as to whether there are certain areas of biochemistry and molecular biology that must be included as content areas in the curriculum. It can be argued that if student understanding of the concepts can be assessed, detailed knowledge of specific content areas should not be required, although detailed knowledge of instructor chosen areas leads to deeper student understanding. For example, if the concepts required for an effective signaling pathway are understood and can be recognized by a student, is it necessary that any given signaling pathway be covered? If the ways that an enzyme can enhance the rate of a reaction and be regulated is understood at a conceptual, mechanistic, quantitative way, does it matter whether the enzyme is Phosphofructokinase or Aspartate Transcarbamoylase? Exceptions, because of their universality, might be how ATP is synthesized, and how DNA is replicated, transcribed and translated. If metabolism is understood at a big picture level in terms of the needs of an organism and how homeostasis is maintained at a conceptual level, do we need students to demonstrate detailed understanding and knowledge of specific pathways rather than illustrate their conceptual understanding with specific knowledge? It also raises the issue of how to best assess student understanding. Traditionally, exams have been based largely on content, rather than conceptual knowledge. The changed focus to concepts suggests that we need to rethink how we measure student outcomes. It also suggests that we rethink 91

the way we assign grades in courses. In many courses, the large majority of the grade is based on periodic exams. Depending on the frequency of such exams this simply encourages student memorization of content. The American Society for Biochemistry and Molecular Biology is attempting to change this paradigm with its national accreditation exam which focuses on the conceptual areas of the learning framework approved by the society. Although this is a national exam not intended for incorporation into a course, nor designed to contribute to a student’s grade, it is groundbreaking with its focus on conceptual areas. As faculty we need to think about how to use such an approach as a component of our grading of students. As part of the move towards concepts and skills, it is important that we appropriately reward students for demonstrating their understanding and abilities. This suggests that the overall grade in a course should be based on a wide variety of assessments, not just on content-based exams. While there will always be a place for the traditional exam in most courses, this is not always the best way to assess student understanding of concepts or reveal misconceptions. Written exams are also not ideal for assessing many required skills and, depending upon the learning goals and objectives of a course, a variety of assessment tools should be used to assign grades. Consider the principles of backward design where one starts with what you want the students to be able to do, then determine the best way to assess whether the student can in fact do it, and finally establish the preferred way of “teaching” the student (78). This suggests that the best way to assign grades should mimic the ways that give evidence of student success at mastering the learning objectives or goals of the course. Even with traditional exams, the focus on conceptual knowledge and understanding should be reflected in the style and wording of the questions used. Too often, exams focus on lower level Bloom’s taxonomy types of questions, focusing on memorization, rather than conceptual questions which require higher level Bloom’s taxonomy responses and involve applying, analyzing, evaluating and creating conceptual understanding. Such questions usually provide information to the student rather than requiring memorization of content, and ask the student to apply conceptual understanding. As part of any discussion on “grading,” the role of rubrics for assignments cannot be overemphasized. Students should be provided with clear expectations. Well-constructed rubrics for graded assignments let a student know what is expected of them. (14). The skills, conceptual understanding, and content areas we want our students to get from our classes and curriculum should be demonstrated and modeled in our classes and laboratories. While on the topic of exams, it is important to point out that educational research (79) has shown that spacing and repetition of delivery of material and assessment of student understanding can play an important role in long term student understanding rather than short term memorization. If we want students to understand and utilize foundational concepts, these concepts should be re-emphasized, with students applying the concepts in a variety of settings, and their understanding of them assessed, throughout the curriculum. Teaching concept or content one week and assessing it the next and moving on to the next topic does not lead to long term understanding and ability to apply the concept to new situations. Biochemistry and molecular biology degree programs usually build from a base of introductory courses from other departments such as chemistry biology physics and mathematics. This is unfortunate, since the foundational concepts that biochemistry and molecular biology build from are usually introduced in courses that have no intentional biochemistry and molecular biology content or focus. As a result, students effectively silo introductory concepts from the “allied field” disciplines away from biochemistry and molecular biology, which they often first see only as juniors. To fully realize a biochemistry and molecular biology degree, it is essential that these introductory and gateway concepts to molecular understanding are taught in the context of biochemistry and molecular biology, the molecular life sciences. 92

The truly interdisciplinary nature of biochemistry and molecular biology, emphasized in both the Vision and Change report, and by the American Society for Biochemistry and Molecular Biology, has led in some institutions to the creation of introductory interdisciplinary courses. Such courses have the capability of illustrating to the students the interdisciplinary basis that underlies the molecular life sciences. At one extreme, courses have been created where for example 5 faculty, representing biology chemistry, mathematics, computer science, and physics, coteach an introductory course that not only lays the groundwork for further coursework in each of the disciplines, but addresses topics in the molecular life sciences in a cohesive interdisciplinary manner (80–82). Such courses lead not only to students that appreciate the interdisciplinary nature of most of the sciences, but also students able to readily cross the boundaries between the disciplines in their own studies and research. Other courses have been developed that link two disciplines to the benefit of both (80–82). For example, a faculty member teaching an introductory chemistry course might team up with a faculty member teaching an introductory biology course, and both courses emphasize the cross disciplinary connections. Although such approaches are clearly beneficial to the students, it is often difficult to create such courses because of the siloed nature of departments, teaching loads and financial constraints. Finally, whether we discuss biochemistry and molecular biology in terms of concepts or content, the question we really must answer given the realities of today’s universities and colleges, is what courses do we put in the curriculum. In terms of biochemistry and molecular biology, one can argue that overall there are three types of courses that we must teach. 1) There are courses that are required for the major, 2) there are courses that are in effect service courses for other majors, and 3 ) there are courses that will fulfill the science requirement for non-science majors. With regard to courses for biochemistry and molecular biology majors, as a result of the discussions associated with Vision and Change and endorsed by both the American Chemical Society and the American Society for Biochemistry and Molecular Biology, there is general agreement as to the conceptual areas that should be covered. How courses are structured to cover both the conceptual areas and skill areas necessary for today’s students is, and should remain, logically a departmental or programmatic decision. While biochemistry and molecular biology degrees offered by either chemistry or biology departments often have a more chemical or biological focus, the discipline is best served by cooperation and collaboration between the departments. When the degree is offered by a standalone biochemistry and molecular biology department, there is still a need for discussion and collaboration at the level of the introductory courses. To cover the requisite concepts skills and appropriate content in the major clearly requires more than a single semester course. When considering how a program fulfills its service teaching requirements to other departments or programs, it becomes a question as to whether, for example, students taking a biochemistry class as a non-major should take the same class that a biochemistry and molecular biology major would take. Practically speaking, the biochemistry and molecular biology major is best served by a sequence of courses with the requisite labs, as appropriate, that cover the topics identified by the major professional societies. So, the question becomes should a non-biochemistry and molecular biology major simply take one of this sequence of courses and expect to be well served for their own particular major. The answer would seem obvious - no.. What becomes of courses designed for non-science majors. Virtually every science department teaches one or more courses that are designed for non-science majors. The question that all science departments, colleges, and universities should ask is. “What do we want the average citizen to understand about science?” It should not be some physics, or any other specific science, but an understanding of what science is, and what science is based upon. Just as most science is becoming more interdisciplinary a strong argument can be made that the average non-science student should 93

have knowledge not only of how science is structured and how science is conducted, but also the big ideas and issues facing each of the disciplines. Conceptually this suggests that science requirements for all non-science majors should not only promote understanding of the scientific approach, but also present these big ideas and issues at a conceptual and understandable level for all students. Biochemistry, as an inherently interdisciplinary science might be particularly suitable for a nonscience majors course and often forms the core of laboratory based non science majors courses that introduce students to the approaches and practices of science in general. Furthermore, such courses do not need to develop the student skills appropriate to all majors since those skills should be developed in the context of the students major, that is, in courses specific to a particular major, offered by the home department.

Foundational Concepts of Interdisciplinary Science for the Molecular Life Sciences To be truly effective, the interdisciplinary aspects of science need to focus on overarching interdisciplinary concepts rather than simply take disciplinary terms and discuss them from multiple perspectives. In the Biochemistry and Molecular Biology learning framework, interrelated interdisciplinary concepts are introduced as outlined in Figure 11 Each of the conceptual areas (Figures 1 and 2) that make up Vision and Change or biochemistry and molecular biology benefit by inclusion of these interdisciplinary concepts, which focus students on broader conceptual understanding and principles of design and function of living systems.

Figure 11. Overarching interdisciplinary concepts can provide a lens to examine all aspects of biochemistry and molecular biology.

How Could the Ideal Curriculum be Structured? As discussed throughout this chapter, there is much common ground between the two major professional societies dealing with undergraduate biochemistry and molecular biology education. While these areas of agreement focus on concept/content areas and skills, it is worthwhile to think about how in an ideal situation a curriculum could be structured.

94

Recognizing that first year introductory courses must set the concept/content stage for multiple majors (Chemistry, Biology, Biochemistry and Molecular Majors, Other Life Science Majors) as well as excite students about science and introduce the requisite skills to allow students to successfully start undergraduate research, we would suggest that the first year curriculum contain at least one course that focuses on big picture issues that face the life sciences, and one course that emphasizes the scientific process and skills necessary for students to start research. Finally, as either a standalone course, or as a significant component of another required laboratory course, students should be engaged in CURE activities where they engage in hypothesis development and other researchoriented skills. Other courses in the first year should emphasize the interdisciplinary and conceptual areas that underlay what are often referred to as the “allied fields” of chemistry and biology, with a strong focus on quantitative and mathematical skills. The second year should build upon these foundations and contain a “Gateway” course that sets the stage for Junior level courses in biochemistry and molecular biology. This gateway course could be paired with a laboratory sequence that introduces students to common techniques as well as bioinformatics and computational approaches that increasingly play important roles in the molecular life sciences. Integrated into such a gateway course should be outside speaker seminars highlighting a variety of career options, and discussion of research activities within the department/program, etc. During the junior year, students should be completing all of the required concept/content area courses that form the core of the major and continuing research activities (either as a CURE or as mentored research) or participating in appropriate experiential learning activities. Regular research seminars and further career orientation activities should be integrated into a required course to ensure that all students participate in these critical activities. Senior year should be devoted to electives within the major, and/or a capstone course and continuation of research activities or experiential learning activities, which should culminate in a presentation as part of a required “senior seminar”. To be successful and to allow students to benefit to the fullest extent, the curriculum for the major, or the overall curriculum for the students should not be so crowded that students cannot participate in research or experiential learning in a meaningful way, and do not have the time for reflection necessary for them to take charge of their own education and prepare for lifelong learning beyond their bachelor’s degree. Courses and degrees in the molecular life sciences should be designed to encourage/allow/prepare students to think and act like scientists and open doors to careers where students can utilize their passion for science in the ways that they choose to do so.

The Critical Role of Inclusive Teaching Irrespective of the curriculum , concepts or content, to serve our students requires faculty that are well versed in inclusive teaching if we are to meet the challenges facing not only our discipline, but also science in general. Few faculty seem aware of the problems that stereotype threat in our classes present nor are trained in ways to avoid putting students at a disadvantage in our classrooms despite a number of studies on this topic specifically in science education (83–87). As a start to informing and discussing this issue amongst faculty, perhaps all faculty should be required to read “Whistling Vivaldi” (88) as part of their summer reading!

95

Appendix: Based upon, with Expanded Detail, Wright et al. (9) “Essential Concepts and Underlying Theories from Physics, Chemistry, and Mathematics for ‘Biochemistry and Molecular Biology’ Majors” Gateway Concepts from Physics Coulomb’s Law Interaction between charges, the charges, q, may be full, formal charges, +, ‑, or partial charges, δ+, δ‑, and from the equation below, the force between them will be attractive, opposite charges, or repulsive, like charges. The value of F being either negative, for opposite charges, or positive, for like charges. The separation between the charges is represented by r.

Coulomb’s law is a vector equation and includes the fact that the force acts along the line joining the charges. Particularly in a biological setting, you have to consider what is between the charges: with more polar compounds in effect shielding the charges and decreasing the force between them. This is represented by the dielectric constant, D, of the medium. In a biological setting the dielectric constant ranges from 80 for water to an estimated 4-8 in the hydrophobic interior of a protein, and hence has a significant effect on the force between two charges. Where the dielectric constant must be taken into account, Coulomb’s Law is represented by:

Energy and Stability Steric energy of a molecule arises from specific interactions within the molecule. These interactions include the stretching or compressing of bonds beyond their equilibrium lengths and angles, torsional effects of twisting about single bonds, the Van der Waals attractions or repulsions of atoms that come close together, and the electrostatic interactions between partial charges in a molecule due to polar bonds

Bond stretching, bending, stretch-bend, out of plane, and torsion interactions are bonded interactions because the atoms involved must be directly bonded or bonded to a common atom. The Van der Waals and electrostatic (qq) interactions are between non-bonded atoms. As a result, in general for a molecule, High Energy = Low Stability. Newton’s Laws of Motion First Law: an object will remain at rest or in uniform motion in a straight line unless acted upon by an external force. Second Law: F=ma Third Law: for every external force that acts on an object, there is a force of equal magnitude but opposite direction which acts back on the object which exerted that external force.

96

Friction For a moving object, frictional resistance is usually proportional to the “normal force” and designated by N.( the force perpendicular or “normal” to the surfaces). Friction is independent of the area of contact and the coefficient of static friction is slightly greater than that of kinetic friction. In general, kinetic friction is independent of velocity. Friction is proportional to the roughness of the surfaces in contact. The frictional resistance force may then be written: ffriction = μN μk = coefficient of kinetic friction μs = coefficient of static friction Hooke’s Law An elastic object, such as a spring, at equilibrium has a defined length. If the spring is either stretched or compressed , the change in length is called its extension and has either a +ve value (stretched) or a -ve value, (compressed). The extension of an elastic object is directly proportional to the force (F) applied to it, related by Hooke’s Law

Where F is the force in Newtons, k is the Spring Constant in N/meter,( the greater the value of k, the stiffer the spring), and e is the extension (compression) in meters. The Potential Energy stored in a spring, Uel(x) = ½ k x2, (where x = e). For a Harmonic Oscillator, the frequency of the oscillation, f, is related to k by the relationship And the wavenumber, υ is πc/2 k/μ Where μ = m1m2/(m1+m2), and c is the speed of light. Stronger bonds have larger values of k, and give faster vibrations. Bonds to lighter atoms have faster vibrations than bonds to heavy atoms. Concept of Diffusion Diffusion is the thermal motion of all (liquid and gas) molecules at temperatures above absolute zero. Diffusion rate is a function of only temperature, and is not affected by concentration. Brownian motion is observed in molecules that are so large that they are not driven by their own thermal energy but by collisions with solvent particles. They move at random because they frequently collide. Molecular diffusion is relevant only on length scales between nanometer and millimeter. On larger length scales, transport in liquids and gases is normally due to another transport phenomenon, convection. Free Energy, Enthalpy, and Entropy Spontaneous changes are ones in which the free energy (G) of a system decreases (ΔG is negative). Heat energy is also called enthalpy (H). When heat is released, the change in the enthalpy for the system that is releasing the heat decreases, whereas when heat is absorbed, the change in the enthalpy increases. While a decrease in the enthalpy makes a process more spontaneous (favorable), the change in enthalpy alone cannot be used to predict whether an overall change is spontaneous. There is another factor that must be considered and that is the entropy (S). Entropy is a measure of disorder; when a system becomes more disordered, the change in entropy is positive. When a change 97

in entropy is positive, it makes the change more spontaneous (favorable). In general reactions are favorable if ΔH° < 0 and ΔS° > 0, or unfavorable if ΔH° > 0 and ΔS° < 0, It is critical to appreciate the difference between ΔG° and ΔG. ΔG° indicates standard conditions and M concentrations of reactants and products at equilibrium since ΔG° = RTLognKequilibrium. ΔG° tells you the Equilibrium position attained if the reaction occurs.ΔG° does not tell you whether the reaction will occur, or in which direction the reaction will proceed. ΔG, on the other hand, is the free energy under the conditions at hand, with

Where Q, the so-called Reaction Quotient = [Products]/[Reactants] ΔG° and ΔG are related by the equation

If Kequilibrium > Q, the reaction will proceed forward, converting reactants into products. If Kequilibrium < Q, the reaction will proceed in the reverse direction, converting products into reactants. If Q = Kequilibrium then the system is already at equilibrium and ΔG = 0. Neither ΔG nor ΔG° tell you anything about how fast the reaction will proceed. How fast the reaction proceeds requires knowledge of the Activation Energy of the reaction. Gateway Concepts of Structure from Chemistry The structure (both in terms of through bond connectivity (covalent bonds) and through space interactions (non-covalent interactions, both favorable and repulsive)) of a molecule determine its dynamic properties and reactivity (chemical and physical): i.e. its function. 5 gateway concepts of bonding and interactions lay the foundation for understanding protein structure function relationships and the structure, activity and regulation of enzymes. 1: Core Concepts of Covalent Bonds and Polarity For small molecules (1 or a limited number of “central” atoms), 3-Dimensional structures are determined by bond angles and lengths and geometric considerations involving both bonding and non-bonding (lone pair) electrons based upon Coulombic repulsion of electron density [Lewis Dot Diagrams and VSEPR]. Combinations of atomic orbitals (usually s, p or hybridized in biochemical systems) to give bonding can be described by the appropriate Schrodinger equations [Wave-functions]and the BornOppenheimer approximation [scheme] or valence bond theory[scheme] to give bonding and antibonding molecular orbitals [energy levels] and can be used to construct a molecular potential energy surface. 2: Bond Rotations and Vibrations The chemical bond between 2 atoms vibrates as a harmonic oscillator to allow bond stretching. [diagram]. Such stretching may be symmetric or asymmetric in a molecule with a central atom. In such molecules bending motions may also occur [Energy of a Molecule].

98

The time scale of such bond vibrations is on the order of 10-13-10-14 seconds. As a result of the high energy cost of deforming bond lengths and angles, such vibrations are usually of small amplitude. Rotations about single covalent bonds can occur with energy barriers to rotation on the order of 10-12kJ/mole in simple molecules without steric hindrance, and time scales on the order to 5 x 10-10 seconds. 3: Hydrogen Bonds and Other Noncovalent Interactions A variety of non-covalent interactions are based upon Coulombic interactions between opposite (attraction) or like (repulsion) charges involving either full (ionic) or partial (Van der Waals or Hydrogen Bond) charges rather than sharing of electrons as in covalent bonds. Such interactions can involve Charge-Dipole, Dipole-Dipole, or Induced Dipole interactions and the magnitude of the interaction energy, as a result of Coulomb’s Law, shows a dependence on both distance and the local dielectric environment. Coulombic interactions can be attractive or repulsive and both play critical roles in molecular structure and dynamics. Hydrogen bonds can be classified as Strong (2.2-2.5Å, ΔG = 40-14kcal/mol), Moderate (2.53.2Å, ΔG = 15-4kcal/mol) and weak (3.2-4.0Å, ΔG < 4kcal/mol) with the strength of the bond progressively decreasing as the angle between the involved atoms deviates from 180°. Hydrogen Bonds may be bifurcated. There appears to be a potential “covalent” contribution (sharing of electron density) in some strong Hydrogen Bonds.[Comparison to Covalent and Ionic Bond strengths]. The force between two atoms in non-covalent interactions can be described by the LennardJones Potential involving the equilibrium distance of the two atoms (the Van der Waals radius), the attractive interactions, and the repulsive forces resulting from overlapping atomic orbitals. 4: The Hydrophobic Effect Non-polar molecules, as a result of their local dielectric constant, structure a cage of polar solvent molecules around them, resulting in a decrease in the entropy of the system. The hydrophobic effect is the coming together of 2 or more non-polar molecules in a high dielectric solvent with a concomitant decrease in the overall structuring of solvent molecules and an increase in the entropy of the system. The magnitude of the hydrophobic effect is proportional to the number of C-H bonds in the molecule excluded from the polar solvent by the interaction. The ΔG for the interaction of two non-polar molecules comes predominantly from the entropy increase in the polar solvent molecules of the system: ΔG = ΔH – TΔS, with ΔS being large and positive overall with the resultant ΔG being negative. 5: Dynamic Aspects of Molecular Structure As a result of the properties of bonds and interactions a molecule is not a static structure. Bond rotations and vibrations combined with inter-molecular non-covalent interactions (attractive and repulsive) allow a potential energy surface to be calculated (approximated for large molecules). The potential energy of a molecule (which is the sum of all of the possible bonded and non-bonded interactions) determines the “structure” of a molecule, which can be described in terms of Potential Energy minima, the local equilibrium structures, and saddle points which represent transition states from one local equilibrium structure to another. Molecular motion can be described in terms of small amplitude motions (within an energy well on the surface) or large amplitude (between energy wells on the surface). 99

An ensemble of chemically identical molecules will be distributed between accessible (depends upon the temperature of the system) potential energy minima on the surface proportional to the depths of the minima, and can freely interchange between these local energy minima. Gateway Concepts of Reactions (and Interactions) from Chemistry Enzymes are biological catalysts that enable cells to control the wide variety of chemical reactions that continuously occur in a cell. Enzymes enable these processes to occur at ambient temperatures, with the requisite specificity, and unlike chemical catalysts, exhibit the phenomenon of saturability. The reactions catalyzed by enzymes often have mechanisms for regulation of the rates of the reactions. While many enzymes are proteins, some RNA molecules also exhibit enzymatic activity and are termed “catalytic RNA.” In all cases, the interactions of enzymes with their substrates, products, and, if appropriate, regulatory molecules are governed by the same foundational concepts that govern chemical reactions in general. Presented here are 5 “gateway” concepts necessary to understand the action of enzymes from a chemical perspective. 1: Collision Theory Reactions require collisions between molecules. Not all collisions can lead to product formation because of necessary energy conditions, orientation or steric factors. Only collisions with the correct orientation and with sufficient energy to overcome the activation energy of the reaction will be successful. Collision Theory predicts the concentration dependence of a reaction. Collision Theory predicts the temperature dependence of a reaction. 2: Transition State Theory Reactions proceed through formation of an “activated complex” that lies at a saddle point on the Potential Energy Surface between reactants and products. This activated complex is the Transition State of the reaction. The transition state for the reaction is in Quasi-Equilibrium with the reactants and products of the reaction. The transition state can convert to products and kinetic theory allows calculation of the rate of this process. Transition State Theory allows for the calculation of the standard enthalpy of activation (ΔH#), the standard entropy of activation (ΔS#), and the free energy of activation (ΔG#) from experimental data using the Eyring Equation (which resembles the Arrhenius Equation) relating the reaction rate to the temperature. 3: Rate Laws, Steady States, and Equilibria Rate laws for a reaction (which are usually experimentally determined) describe the dependence of the rate of the reaction on either concentration of the reactant(s) (Differential Rate Law) or the time of the reaction with a fixed starting concentration (Integrated Rate Law) In Differential Rate Laws, depending upon the order of the reaction, the rate is proportional to the concentration of the reacting species and on a rate constant for the process. The Integrated Rate Law relates the concentration of reactant (or product) to the time of the reaction and the initial concentration of reactant. The format of rate laws (differential or integral) depends on the Order of the Reaction. 100

In a Steady State, the concentrations([x]) of the components of a system do not change over a period of time (t) and can be represented by δ[x]/dt = 0. While a steady state is not necessarily at Equilibrium, an equilibrium, by definition, is a steady state and defined by an equilibrium constant, KEQ = [Products]/[Reactants]. As a result of rate laws, the Equilibrium Constant for a reversible reaction is related to the individual rate constants for the forward and reverse reactions (KEQ = kf/kr) and the energy difference between reactants and products( ΔG° = -RTlnKEQ ). 4: The Effects of Temperature Increased temperature, in general, leads to increased reaction rates as a result of increased collision frequency (small effect-frequency of collisions is proportional to square root of K) and an increase in the proportion of the molecules that have sufficient energy to react (large effect due to shift in Maxwell Boltzmann distribution, etc.) when they collide correctly. Increased reaction rates, reflected in the rate constants for the reaction, may affect forward and reverse reactions differently resulting in equilibrium constant effects of temperature. In the energy diagram for a reversible reaction, increased temperature in effect “raises” the energy of the reactants while decreased temperature lowers the “energy” of the reactants. Temperature changes have little effect on the energy of the transition state. The effects of temperature on reaction rates or equilibrium constants are described quantitatively by the Arrhenius and Van’t Hoff equations, respectively, and allow the appropriate thermodynamic data to be calculated from the temperature dependence of either rate or equilibrium constants. As a general rule of thumb, in biological systems, a 10K rise in temperature approximately doubles the rate of a reaction. 5: Structure and Reactivity Chemical Reactions involve bond breaking or bond making events and involve changes in the electron sharing within or between molecules and are usually discussed in terms of three fundamental concepts: i) nucleophiles and electrophiles, ii) acid-base chemistry, and iii) redox reactions. Nucleophiles and electrophiles: the electron density of a given molecule determines its reactivity, which can be influenced by the electron density of the molecule with which it is reaction. Nucleophiles are regions of a molecule that are electron rich, while electrophiles are regions that are electron deficient. Nucleophilicity increases as you increase basicity.- The conjugate base of a compound is always a stronger nucleophile. The most common nucleophiles contain lone pairs or pi bonds (especially with electron donating groups attached), although sigma bonds can be involved in some cases. Substitution reactions often include water as the nucleophile. For leaving groups, the weaker the conjugate base, the better the leaving group. In biochemistry, ATP is often involved to make a good leaving group (). The strength of an electrophile is governed not only by its electron deficiency, but also by steric considerations and the stability of a potential carbocationic intermediate in sN2 reactions. Modern concepts of acids and bases are derived from the Arrhenius theory (acids produce hydrogen ions, bases hydroxide ions), and include the Bronsted-Lowery theory (an acid is a proton donor while a base is a proton acceptor) and Lewis theory (an acid is an electron pair acceptor while a base is an electron pair donor). Acid-base chemistry in the molecular life sciences mostly involves weak acids and bases (only partially ionized in water) where pKa = -log Ka where Ka = ([A-][H+])/[HA]. The Henderson101

Hasselbach equation (pH = pKa + log[A-]/[HA]) can be used to assess the state of ionization of a group at any pH. There is a relationship between weak acidity to structure (the stability of the conjugate base, etc. - eg serine vs cysteine). Changing the local environment (charge environment, polarity of environment, etc.) has an effect on the pKa of a weak acid. The Henderson-Hasselbach equation is also the foundation for understanding buffers and titration curves. In Redox reactions, oxidation is loss of electrons, while reduction is gain of electrons. In such a reaction you can consider two “half reactions,” but you cannot have one half reaction without the other-half reaction. An oxidizing agent gains e- during reaction and is therefore reduced during reaction, while a reducing agent loses e- during reaction and is therefore oxidized during reaction. The oxidized form of a molecule is the form of molecule lacking e- and the reduced form of a molecule is the form of the molecule having e-. Oxidation numbers are often used to consider molecules undergoing redox reactions. For atoms in their elemental form, the oxidation number is 0. For ions, the oxidation number is equal to their charge. For single hydrogen atoms, the number is usually +1 (but in some cases, it is -1). For oxygen, the number is usually -2. The sum of the oxidation number (ONs) of all the atoms in the molecule or ion is equal to its total charge. For the general redox reaction (written as a reduction) aA + n e-←→ bB the Nernst equation takes the form:

Where E is the measured electrode potential, E° is the standard reduction potential, R is the gas constant (8.314 J/mol K), T is temperature in K, n is the stoichiometric number of electrons involved in the process, F is the Faraday constant (96,485 C/mol) and AR and AO are the activities of the reduced and oxidized members of the redox pair, respectively. In considering redox reactions, you need to account for all of the electrons as they transfer from one species to another, and deal with each of the two half-reactions individually. Overview of Types of Enzyme Chemical Mechanisms: Six basic types of enzyme activities exist based in large part on the type of chemical reactions they catalyze. As categorized by the Enzyme Commission, these are: 1) Oxidoreductases- involve oxidation/reduction reactions- movement of electrons between molecules, 2) Transferases- mostly substitution reactions (usually with alcohol, amine, or thiol as nucleophile), 3) Hydrolases- mostly substitution with water as nucleophile, 4) Synthases/Lyases- reactions where two molecules are brought together (or broken apart) that is not already class 1 or 3; can be aldol reactions, eliminations, additions, etc., 5) isomerases- where you switch between isomers (constitutional or stereo). Can involve cofactors. Can involve acid/base chemistry or substitution, etc. and 6) Ligases/Synthetasesusually substitution reactions, but need ATP (usually) to make a good leaving group. Gateway Concepts from Mathematics Models Models play important roles in science and come in three types: Physical, Mathematical, and Conceptual and usually involve three components, i) Information input, ii) an Information processor, and iii) Information output (often some type of prediction). Physical and Conceptual models (sometimes referred to as Mental Models) are often qualitative, while Mathematical (and 102

Statistical ) Models are quantitative. All Models have limitations, but are useful in generating avenues to “test” the model, and, as a result, are often changed based upon new evidence. The most useful models are the simplest model that accommodates the available evidence Randomness and Stochastic Processes Stochastic vs Deterministic Processes. A stochastic process, X(t) or Xt, is a collection of random variables indexed by time, t. Discrete or continuous time Markov Chains. A process is deterministic if its future is completely determined by its present and past. A Markov chain is a stochastic process, but unlike a general stochastic process, Markov chains must be “memory-less” and the probability of future actions are independent of the preceding steps that gave rise to the current state. Probability Probability Models- 3 components: 1] sample space (set whose elements are the “outcomes” or “sample points” 2] class of “events” (all subsets of the sample space) and 3] Probability Measure (assignment of a nonnegative number to each outcome, with the restriction that these numbers must sum to one over the sample space). The probability of an event is the sum of the probabilities of the outcomes comprising that event. Deriving Equations Equations can be developed based upon a theoretical model (e.g. chemical and enzyme kinetics) or on experimental observations (e.g. Coulomb’s Law). Using Equations Equations describe many processes in the molecular life sciences: linear- y=mx + c, hyperbolicy=mx/(x+K), exponential rise- y=a(1-e-kx), exponential decay-y=ae-kx Sigmoidal-y=yo+a/(1+e-((x-xo)/b)) and allow descriptor constants, m, K, k, a, b, etc. to be calculated from experimental data. Equations also allow experimental data to be extrapolated to predict values for the observable, y, for inaccessible values of the independent variable, x. Populations, Averages, Normal Distributions, and Standard Deviations Reproducibility and error analysis: average, x = Σxi/n. Sample standard deviation is the square root of Σ(xi-xav)2/(n-1). If you compare two normal distributions and take 95% confidence limits: i.e. the values laying between the 2.5% and 97.5% values, of each distribution and the values do not overlap, there is only 2.5% of the possible estimates of the higher number that could, in a normal distribution, fall into the upper echelons of the normal distribution of the lower number: the numbers would be said to differ at the level of the 95% confidence limits, or at a p value of 0.025. Linear Regression and Residuals Least squares analysis involves making an initial guess to the values of the parameters of the equation being fit and calculating using these initial guesses the values of y for the chosen values of x. The difference between theoretical and experimental data points is the “residual” and is negative or positive depending upon whether the calculated value is bigger or smaller than the actual experimental value. The residuals are squared and summed. The sum of the squares of the residuals is hence an estimate of the fit of the line to the data. The parameter estimates are then changed and a sum of squares of the new residuals calculated. The process is repeated until the sum of squares of the residuals is a minimum. The parameters giving the lowest sum of squares of the residuals are the best 103

fit parameters. The residual is the difference between the actual data point and the computed best fit data point and can be negative or positive. The residuals calculated from these best fit parameters can be plotted against the value of x, and a random distribution of residuals indicates the correct equation was used. A pattern in the residuals indicates that a different equation is more appropriate

References 1. 2. 3.

4.

5.

6.

7.

8.

9.

10. 11. 12. 13.

14.

Undergraduate Science, Mathematics and Engineering Education, National Science Board; 1986. https://www.nsf.gov/nsb/publications/1986/nsb0386.pdf (Accessed August 13, 2019) Bell, E. The Future of Education in the Molecular Life Sciences. Nat. Rev. Mol. Cell Biol. 2001, 2, 221–225. National Research Council Committee on Undergraduate Biology Education to Prepare Research Scientists for the 21st Century: Bio2010: Transforming Undergraduate Education for Future Research Biologists; National Academies Press: Washington, DC, 2003. Boyer Commission on Educating Undergraduates in the Research University. Reinventing Undergraduate Education: A Blueprint for America’s Research Universities. State University of New York at Stony Brook for the Carnegie Foundation for the Advancement of Teaching; Stony Brook, NY, 1998. Wood, W. B. Inquiry-based undergraduate teaching in the life sciences at large research universities: A perspective on the Boyer Commission Report. Cell Biol. Educ. 2003, 2, 112–116. Voet, J. G.; Bell, E.; Boyer, R.; Boyle, J.; O’Leary, M.; Zimmerman, J. Recommended curriculum for a program in biochemistry and molecular biology. Bioch. Mol. Biol. Educ. 2003, 31, 161–162. Tansey, J. T.; Baird, T., Jr.; Cox, M. M; Fox, K. M.; Knight, J.; Sears, D.; Bell, E. Foundational concepts and underlying theories for majors in biochemistry and molecular biology. Biochem. Mol. Biol. Educ. 2013, 41, 289–296. White, H. B.; Benore, M. A.; Sumter, T. F.; Caldwell, B. D.; Bell, E. What skills should students of undergraduate biochemistry and molecular biology programs have upon graduation? Biochem. Mol. Biol. Educ. 2013, 41, 297–301. Wright, A.; Provost, J.; Roecklein-Canfield, J. A.; Bell, E. Essential concepts and underlying theories from physics, chemistry, and mathematics for “biochemistry and molecular biology” majors. Biochem. Mol. Biol. Educ. 2013, 41, 302–308. Vision and Change in Undergraduate Biology Education: A Call to Action. http://visionandchange. org/finalreport (accessed August 14, 2019) CourseSource Biochemistry and Molecular Biology Learning Framework. https://www. coursesource.org/courses/biochemistry-and-molecular-biology (accessed August 14, 2019) ASBMB, Concept Driven Teaching, Assessment Tools and Course Materials. http://www.asbmb. org/education/teachingstrategies/ (accessed August 14, 2019) Loertscher, L.; Lewis, J. E.; Mercer, A. M.; Minderhout, V. Development and use of a construct map framework to support teaching and assessment of noncovalent interactions in a biochemical context. Chem. Educ. Res. Pract. 2018, 19, 1151. Allen, D.; Tanner, K. Rubrics: tools for making learning goals and evaluation criteria explicit for both teachers and learners. CBE-Life Sci. Educ. 2006, 5, 197–203. 104

15. The Malate Dehydroenase Cures Community. https://mdh-cures-community.squarespace.com (accessed August 15, 2019) 16. Association of American Medical Colleges-Howard Hughes Medical Institute. Scientific Foundations for Future Physicians (online). http://www.hhmi.org/grants/pdf/08-209_AAMCHHMI_report.pdf (accessed August 14, 2019) 17. Brenner, C. Changes in chemistry and biochemistry education: creative responses to medical college admissions test revisions in the age of the genome. Biochem. Mol. Biol. Educ. 2013, 41, 1–4. 18. ASBMB Accreditation Overview. http://www.asbmb.org/accreditation/overview/ (accessed August 14, 2019) 19. ACS Approval Program for Bachelor’s Degree Programs. https://www.acs.org/content/acs/en/ about/governance/committees/training/acsapproved.html (accessed August 14, 2019) 20. Indiana University Bachelor of Science in the Molecular Life Sciences. https://college.indiana.edu/ academics/degrees-majors/major-guides/molecular-life-sciences-bs.html (accessed August 14, 2019) 21. Del Gaizo Moore, V.; Loertscher, J.; Dean, D. M.; Bailey, C. P.; Kennelly, P. J.; Wolfson, A. J. Letter to the Editor, Structuring and Supporting Excellence in Undergraduate Biochemistry and Molecular Biology Education: The ASBMB Degree Accreditation Program. CBE—Life Sci. Educ. 2018, 17, 1–3. 22. Russell, S. H.; Hancock, M. P.; McCullough, J. The pipeline. Benefits of undergraduate research experiences. Science 2007, 316, 548–9. 23. Lopatto, D. Undergraduate research as a catalyst for liberal learning. Peer Rev. 2006, 8, 22–25. 24. Lopatto, D. Science in Solution: The Impact of Undergraduate Research on Student Learning; Research Corp, 2009. https://www.grinnell.edu/sites/default/files/documents/science_in_ solution_lopatto.pdf (accessed August 14, 2019) 25. Linn, M. C.; Palmer, E.; Baranger, A.; Gerard, E.; Stone, E. Undergraduate research experiences: Impacts and opportunities. Science 2015, 347, 627–35. 26. Luckie, D. B.; Maleszewski, J. J.; Loznak, S. D.; Krha, M. Infusion of collaborative inquiry throughout a biology curriculum increases student learning: a four-year study of “Teams and Streams”. Adv. Physiol. Educ. 2004, 28, 199–209. 27. Myers, M. J.; Burgess, A. B. Inquiry-based laboratory course improves students’ ability to design experiments and interpret data. Adv. Physiol. Educ. 2003, 27, 26–33. 28. Characteristics of Excellence in Undergraduate Research (COEUR); Council on Undergraduate Research: Washington, DC, 2012. https://www.cur.org/assets/1/23/COEUR_final.pdf (accessed August 14, 2019) 29. Seymour, E.; Hunter, A. B.; Laursen, S. L.; Deantoni, T. Establishing the benefits of research experiences for undergraduates in the sciences: first findings from a three-year study. Sci. Educ. 2004, 88, 493–534. 30. Eagan, M. K., Jr.; Hurtado, S.; Chang, M. J.; Garcia, G. A; Herrera, F. A.; Garibay, J. C. Making a Difference in Science Education: The Impact of Undergraduate Research Programs. Am. Educ. Res. J. 2013, 50, 683–713. 31. Bell, E. Using Research to Teach an “Introduction to Biological Thinking”. Biochem. Mol. Biol. Educ. 2011, 39, 10–16.

105

32. Robic, S. Mathematics, Thermodynamics, and Modeling to Address Ten Common Misconceptions about Protein Structure, Folding, and Stability. CBE—Life Sci. Educ. 2010, 9, 189–195. 33. Saldanha, J.; Haen, K. Investigating the Effectiveness of Case Studies in Improving Student Learning in a 200-Level Anatomy Course; CIRTL Reports; 2016, 3. https://lib.dr.iastate.edu/cgi/ viewcontent.cgi?referer=https://www.google.com/&httpsredir=1&article=1002& context=cirtl_reports (accessed August 14, 2019) 34. Solanki, S.; McPartlan, P.; Xu, D.; Sato, B. K. Success with EASE: Who benefits from a STEM learning community?”. PLoS ONE 2019, 14. https://doi.org/10.1371/journal.pone. 0213827 (accessed August 14, 2019) 35. Auchincloss, L. C.; Laursen, S. L.; Branchaw, J. L.; Eagan, K.; Graham, M.; Hanauer, D. I.; Lawrie, G.; McLinn, C. M.; Pelaez, N.; Rowland, S.; Towns, M.; Trautmann, N. M.; VarmaNelson, P.; Weston, T. J.; Dolan, E. L. Assessment of course-based undergraduate research experiences: a meeting report. CBE—Life Sci. Educ. 2014, 13, 29–40. 36. Dolan, E. L.; Lally, D. J.; Brooks, E.; Tax, F. T. Prepping Students for Authentic Science. Sci Teach. 2008, 75, 38–43. 37. Bell, J. K.; Eckdahl, T. T.; Hecht, D. A.; Killion, P. J.; Latzer, J.; Mans, T. L.; Provost, J. J.; Rakus, J. F.; Siebrasse, E. A.; Bell, J. E. CUREs in biochemistry-where we are and where we should go. Biochem. Mol. Biol. Educ. 2016, 45, 7–12. 38. Kuh, G. High-Impact Educational Practices: What They Are, Who Has Access to Them and Why They Matter; AAC&U: Washington, DC, 2008. 39. Fukami, T.; Brownell, S. E.; Kloser, M. J.; Shavelson, R. Undergraduate Biology Lab Courses: Comparing the Impact of Traditionally Based Cookbook; and Authentic Research-Based Courses on Student Lab Experiences. J. Coll. Sci. Teach. 2012, 1–10. 40. Spell, R. M.; Guinan, J. A.; Miller, K. R.; Beck, C. W. Redefining Authentic Research Experiences in Introductory Biology Laboratories and Barriers to Their Implementation. CBE—Life Sci. Educ. 2014, 13, 102–110. 41. Auchincloss, L. C.; Laursen, S. L.; Branchaw, J. L.; Eagan, K.; Graham, M.; Hanauer, D. I.; Lawrie, G.; McLinn, C. M.; Pelaez, N.; Rowland, S. Assessment of Course-Based Undergraduate Research Experiences: A Meeting Report. CBE—Life Sci. Educ. 2014, 13, 29–40. 42. Mans, T.; Callahan, K.; Zhang, J.; Bell, J. E.; Bell, J. K. Using Bioinformatics and Molecular Visualization to Develop Sudent Hypotheses in a Malate Dehydrogenase Oriented CURE. Course Source 2019, submitted. 43. Allen, D.; Tanner, K. Approaches in Cell Biology Teaching. Cell Biology Education 2002, 1, 3–5. 44. Lewis, S. E.; Lewis, J. E. Departing from lectures: an evaluation of a Peer-Led Guided Inquiry alternative. J. Chem. Educ. 2005, 82, 135–139. 45. Vickrey, T.; Rosploch, K.; Rahmanian, R.; Pilarz, M.; Stains, M. Research-Based Implementation of Peer Instruction: A Literature Review. CBE—Life Sci. Educ. 2015, 14, 1–11. 46. Weiman, C. Why Not Try a Scientific Approach to Science Education? Change: The Magazine of Higher Learning 2007, 39, 9–15. 106

47. Kinchin, I. M. Solving Cordelia’s Dilemma: threshold concepts within a punctuated model of learning. J. Biol. Educ. 2010, 44, 53–57. 48. Stephenson, N. S.; Miller, I. R.; Sadler-McKnight, N. P. Impact of Peer-Led Team Learning and the Science Writing and Workshop Template on the Critical Thinking Skills of First-Year Chemistry Students. J. Chem. Educ. 2019, 96, 841–849. 49. Stokhof, H.; Bregje de Vries, B.; Bastiaens, T.; Martens, R. Using Mind Maps to Make Student Questioning Effective: Learning Outcomes of a Principle-Based Scenario for Teacher Guidance. Res Sci Educ. https://doi.org/10.1007/s11165-017-9686-3 (accessed August 14, 2019). 50. Stains, M.; Harshman, J.; Barker, M. K.; Chasteen, S. V.; Cole, R.; DeChenne-Peters, S- E.; Eagan, M. K.; Esson, J. M.; Knight, J. K.; Laski, F. A.; Levis-Fitzgerald, M.; Lee, C. J.; Lo, S. M.; McDonnel, L. M.; McKay, T. A.; Michelotti, N.; Palmer, M. S.; Plank, K. M.; Tamara, M.; Rodela, T. M.; Sanders, E. R.; Natalie, G.; Schimpf, N. G.; Schulte, P. M.; Smith, M.; Stetzer, M.; Stewart, J.; Valkenburgh, B. V.; Vinson, E.; Weir, L. K.; Wendel, P. J.; Wheeler, L. B.; Young, A. M. Anatomy of STEM Teaching in American Universities: A Snapshot from a LargeScale Observation Study. Science 2018, 359, 1468–1470. 51. Halim, A. S.; Finkenstaedt-Quinn, S. A.; Olsen, L. J.; Gere, A. R.; Shultz, G. V. Identifying and Remediating Student Misconceptions in Introductory Biology via Writing-to-Learn Assignments and Peer Review. CBE—Life Sci. Educ. 2018, 17, 1–12. 52. Liu, J.; Pysarchik, D. T.; Taylor, W. W. Peer Review in the Classroom. BioScience 2002, 52, 824–829. 53. Scager, K.; Boonstra, J.; Peeters, T.; Vulperhorst, J.; Wiegant, F. Collaborative Learning in Higher Education: Evoking Positive Interdependence. CBE—Life Sci. Educ. 2016, 15, 1–9. 54. Harwood, W. S. The One-Minute Paper: A Communication Tool for Large Lecture Classes. J. Chem. Educ. 1996, 73 (3), 229. 55. Bryant, L. H.; Chittum, J. R. ePortfolio Effectiveness: A(n Ill-Fated) Search for Empirical Support. International Journal of ePortfolio 2013, 3, 189–198. http://www.theijep.com/pdf/ IJEP_7_2.pdf (accessed August 14, 2019) 56. Celio, C. I.; Durlak, J.; Dymnicki, A. A Meta-analysis of the Impact of Service-Learning on Students. Journal of Experiential Education. 2011, 34, 164–181. 57. Chamany, K.; Allen, D.; Tanner, K. Making biology learning relevant to students: integrating people, history, and context into college biology teaching. CBE-Life Sci Educ. 2008, 7, 267–78. 58. Sandra, V.; Kotsis, S. V.; Chung, K. C. Application of See One, Do One, Teach One Concept in Surgical Training. Plast Reconstr Surg. 2013, 131, 1194–1201. 59. Teaching Professor Special Report - Large Classes. https://facultyinnovate.utexas.edu/sites/ default/files/faculty-focus-teaching-large-classes-special-report.pdf (accessed August 14, 2019). 60. Meyer, J. H. F.; Land, R.; Baillie, C. Threshold Concepts and Transformational Learning. https://www.lamission.edu/learningcenter/docs/1177-threshold-concepts-andtransformational-learning.pdf (accessed August 14, 2019). 61. Cooper, M. M.; Klymkowsky, M. W. The Trouble with Chemical Energy: Why Understanding Bond Energies Requires an Interdisciplinary Systems Approach. CBE—Life Sci. Educ. 2013, 12, 306–312. 107

62. Eaton, C. D.; Highlander, H. C.; Dahlquist, K. D.; LaMar, M. D.; Ledder, G.; Schugart, R. C. A “Rule-of-Five” Framework for Models and Modeling to Unify Mathematicians and Biologists and Improve Student Learning. PRIMUS 2019, 29, 799–829. 63. Kelley, T. R.; Knowles, J. G. A conceptual framework for integrated STEM education. International Journal of STEM Education 2016, 3, 11. 64. Howitt, S.; Anderson, T.; Costa, M.; Hamilton, S.; Wright, T. A concept inventory for molecular life sciences: How will it help your teaching practice? Aust. Biochemist. 2008, 39, 14–17. 65. Bretz, S. L.; Linenberger, K. J. Development of the Enzyme–Substrate Interactions Concept Inventory. Biochem. Mol. Biol. Educ. 2012, 40, 229–233. 66. Villafane, S. M.; Bailey, C. P.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Development and analysis of an instrument to assess student understanding of foundational concepts before biochemistry coursework. Biochem. Mol. Bio. Educ. 2011, 39, 102–109. 67. Villafañe, S. M.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Uncovering students’ incorrect ideas about foundational concepts for biochemistry. Chem. Educ. Res. Pract. 2011, 12, 210–218. 68. Loertscher, J.; Green, D.; Lewis, J. E.; Lin, S.; Minderhout, V. Identification of Threshold Concepts for Biochemistry. CBE—Life Sci. Educ. 2014, 13, 516–528. 69. Klymkowsky, M. W.; Underwood, S.; Garvin-Doxas, K. TheBiological Concepts Instrument (BCI), a diagnostic tool to reveal student thinking. 2010. https://arXiv:1012.4501v1 (accessed August 14, 2019). 70. Taylor, A. T.; Olofson, E. L.; Novak, W. R. Enhancing student retention of prerequisite knowledge through pre-class activities and in-class reinforcement. Biochem. Mol. Biol. Educ. 2017, 45, 97–104. 71. Brownell, S. E.; Price, J. V.; Steinman, L. A writing-intensive course improves biology undergraduates’ perception and confidence of their abilities to read scientific literature and communicate science. Adv. Physiol. Educ. 2013, 37, 70–79. 72. Renaud, J.; Squier, C.; Larsen, S. C. Integration of a communicating science module into an advanced chemistry laboratory course. J. Chem. Educ. 2006, 83, 1029–1031. 73. Reynolds, J. A.; Thaiss, C.; Katkin, W.; Thompson, R. J. Writing-to-learn in undergraduate science education: a community-based, conceptually driven approach. CBE Life Sci. Educ. 2012, 11, 17–25. 74. Rivard, L. P. A review of writing to learn in science–implications for practice and research. J. Res. Sci. Teach. 1994, 31, 969–983. 75. Aguanno, A.; Mertz, P.; Martin, D.; Bell, E. A national comparison of biochemistry and molecular biology capstone experiences. Biochem. Mol. Biol. Educ. 2015, 43, 223–32. 76. Konrad, R.; Hall-Phillips, A.; Vila-Parrish, A. R. Are Our Students Prepared? The Impact of Capstone Design Pedagogical Approaches on Student Skill Development During IndustrySponsored Fieldwork. INFORMS Transactions on Education 2018, 18, 183–193. 77. Redman, P. Going beyond the requirement: The capstone experience. Peer Rev. 2013, 15, 12–15. 78. Wiggins, G.; McTighe, J. Backward Design. In Understanding by Design; ASCD: Alexandria, VA, 1998; pp 13–34. 108

79. Balota, D. A.; Duchek, J. M.; Logan, J. M. Is Expanded Retrieval Practice a Superior Form of Spaced Retrieval? A Critical Review of the Extant Literature. In The Foundations of Remembering: Essays in Honor of Henry L. Roediger, III; Nairne, J. S. , Ed.; Psychology Press: New York, 2007; pp 83–105. 80. Ramsey, L. L.; Radford, D. L.; Deese, W. C. Experimenting with Interdisciplinary Science. J. Chem. Educ. 1997, 74, 946. 81. Schlegel, W. M. Creating Interdisciplinary Science Programs: Purposes, Progress, Potholes. Peer Review 2011, 13. 82. Cockcroft, R.; Symons, S. L.; Goff, L., Knorr, K.; Robinson, S. J.; van Wersch, G.; Charney, D.; Farquharson, M. J. New Interdisciplinary Science Course for First-Year Faculty of Science Students: Overview and Preliminary Results from the Pilot. CELT 2016, IX, https://files.eric. ed.gov/fulltext/EJ1104480.pdf (accessed August 14, 2019). 83. Schmader, T.; Hall, W. M. Stereotype Threat in School and at Work: Putting Science Into Practice. Policy Insights from the Behavioral and Brain Sciences 2014, 1, 30–37. 84. Appel, M.; Kronberger, N.; Aronson, J. Stereotype threat impairs ability building: Effects on test preparation among women in science and technology. Eur. J. Soc. Psychol. 2011, 41, 904–913. 85. Dewsbury, B. M. On faculty development of STEM inclusive teaching practices. FEMS Microbiology Letters 2017, 364. https://doi.org/10.1093/femsle/fnx179 86. Beasley, M. A.; Fischer, M. J. Why they leave: the impact of stereotype threat on the attrition of women and minorities from science, math and engineering majors. Soc. Psychol. Educ. 2012, 15, 427–448. 87. Dewsbury, B.; Brame, C. J. Inclusive Teaching. CBE—Life Sci. Educ. 2019, 18, 1–5. 88. Steele, C. M. Whistling Vivaldi: How Stereotypes Affect Us and What We Can Do (Issues of Our Time); W. W. Norton & Company: New York, 2011.

109

Chapter 5

Implementing Guided Inquiry in Biochemistry: Challenges and Opportunities Jennifer Loertscher* and Vicky Minderhout Department of Chemistry, Seattle University, 901 12th Avenue, Seattle, Washington 98122, United States *E-mail: [email protected].

Guided inquiry is an active learning strategy that engages students in speciallydesigned activities with the goal of helping them develop disciplinary knowledge and transferrable skills. This evidence-based pedagogy has been used widely in STEM, including in the molecular life sciences. In this chapter we describe defining features of guided inquiry learning and characterize the challenges and opportunities associated with implementing it in the biochemistry classroom. Specifically, we describe aspects of implementation that are particular to an upperdivision, interdisciplinary course and we address how activities and classroom facilitation can be structured for an application-based discipline like biochemistry.

Introduction to Active Learning Active learning has become increasingly prevalent across the STEM disciplines. Vision and Change represented a sea change for teaching in the molecular life sciences and shifted the national conversation towards evidence-based active learning. Specific recommendations from Vision and Change include engaging students as active participants, using multiples modes of instruction, ensuring undergraduate courses are inquiry driven, and facilitating learning within a cooperative context (1). These expectations were codified for biochemistry and molecular biology programs in the guidelines for accreditation defined by the American Society for Biochemistry and Molecular Biology (ASBMB) accreditation program: “Students also should have some level of active learning in one or more of its many forms. This approach to teaching can engage students in either the classroom or the teaching laboratory. There are many approaches of active learning that provide opportunities for students to think critically and that promote student engagement, synthesis, analysis, team skills and problem solving” (2). Active learning is defined in different ways, but most educators agree that it engages students in the process of learning, with an emphasis on development of transferrable skills such as problem solving, critical thinking, and communication. Many forms of active learning include structures to promote cooperative work among students (3). Commonly implemented active learning strategies © 2019 American Chemical Society

include use of student response systems, inquiry-based learning, problem-based learning, peer-led team learning, case-based learning, and flipped classroom (4–7). In all cases, instructional design plays a significant role in promoting effective learning, and instructional choices should account for the classroom context and desired learning outcomes. Ample evidence demonstrates that active learning improves students’ examination scores and decreases failure rates across the STEM disciplines (8, 9). Although active learning takes on many different forms, some aspects have been shown to be particularly important. First, the intentional and transparent course structure that accompanies many active learning pedagogies appears to contribute significantly to student success. Research in large biology classrooms has shown that highly structured, active learning courses that emphasize practice, problem solving, and other higher-order cognitive skills improve students’ performance and lower failure rates compared to students in traditional lecture courses (10). Notably, all students experienced the described benefits of structure, but the effects were greater for students who are underrepresented in STEM (11). Second, considerable research suggests that cooperative learning, in which students work in structured groups, has positive effects on student learning. Two meta-analyses of 25 studies in chemistry classes showed greater learning gains in cooperative learning classrooms as compared to traditional classrooms, with evidence that class size and instructor choices may influence outcomes (12, 13).

Guided Inquiry: History and Characteristics Inquiry-based learning is a type of active learning that asks students to construct their understanding of concepts through addressing key questions or issues. It is grounded in learning theories that posit that knowledge and skills are constructed in, rather than transferred to, the mind of the learner (14). These theories are logical extensions of Piaget’s work on intellectual development, which suggest that cognitive development occurs as children build new knowledge through observations and experimentation, and add that information to their existing knowledge (15–17). Engaging students in inquiry-based learning implies that students are conversing as they consider responses to questions. This social interaction is essential to cognitive development and was suggested originally by Vygotsky (18) and supported by Piaget (17) and Bandura (19). Effective cooperative learning environments cultivate positive interdependence, while simultaneously creating individual accountability (20). Pedagogical approaches that use inquiry vary in the degree to which the educator structures the questions and the path to solutions (3). In open inquiry experiences, students define the question and path to solutions, which mimics the process of science, but can be frustrating for the learner. In guided or structured inquiry, the educator provides questions and scaffolding for solutions, which ideally give the learner just enough support to move forward, while still promoting independent learning. Citing cognitive load theory, some have argued that the complexity of tasks required in inquiry-based learning places too high of a demand on working memory and therefore may impede learning (21, 22). However, other researchers distinguish between open and guided inquiry and have argued that scaffolding provided by educators reduces cognitive load, making guided inquiry an effective pedagogical strategy (23, 24). Recognizing how individuals take in information from their surroundings and incorporate it into their knowledge base is critical to designing effective learning materials. The information processing model of cognition is one way to explain how individuals acquire and construct new understanding. Originating in cognitive psychology, the model was further illuminated for chemists by Johnstone (25). At its core, the model suggests that all incoming information is subjected to a perception filter 112

that evaluates the relevance of the information. Relevance is determined by prior knowledge as well as by likes, dislikes, and beliefs. Once relevance is established, the information is placed in working memory, while information determined to be irrelevant is ignored. Working memory has limited space. Therefore, it is essential that learners are not expected to do too much at once since cognitive overload can occur and result in little learning and a feeling of frustration (18). Finally, if connections are made between information in working memory and prior knowledge, the information may be assimilated into long term memory for storage. With learning theory in mind, guided inquiry instructional materials are designed to support students to construct understanding of concepts. One way to accomplish this is through use of the learning cycle. The basic model of the learning cycle was suggested by Karplus and Their (26), further studied by Abraham and Renner (27) and Lawson (28), and implemented in general chemistry (16, 29). It consists of three phases: the exploration phase, the concept invention phase, and the application phase. In the exploration phase students investigate a model, which can be a figure, text, data or other relevant information. The model is a critical feature of many guided inquiry activities, and students working in small groups are prompted to explore it through a series of carefully designed questions (30). In the concept invention phase, questions prompt students to recognize relationships depicted in the model and discover or invent the concept being taught. The final phase in the cycle is to apply the newly-learned concept to novel contexts. Effective learning cycle activities prompt connections between new information and prior knowledge, thereby facilitating assimilation of new information into long term memory.

Process Oriented Guided Inquiry Learning Process-oriented guided inquiry learning (POGIL) is a specialized type of guided inquiry learning in which students work in structured small groups on specially designed materials that guide them to discover and construct their own knowledge, while simultaneously developing transferrable process skills (29, 31, 32). Collections of POGIL materials have been developed for use in high school and postsecondary contexts across of range of disciplines including analytical chemistry, anatomy and physiology, biochemistry, calculus, computer science, general chemistry, materials science, organic chemistry, and physical chemistry (33). Studies across a range of STEM courses, including general chemistry, organic chemistry, and anatomy and physiology, have showed that use of POGIL substantially improves the odds of passing a course and improves exam performance compared to courses that are taught using a more traditional approach (29, 34–37). POGIL is distinct from some other active learning pedagogies in that development of content knowledge and process skills are both integral parts of all learning activities. The process skills defined by the POGIL Project are oral and written communication, critical thinking, information processing, problem solving, teamwork, management, and self-assessment (32). In addition, educators may choose to support their students to develop additional process skills that are especially important in a given discipline. Development of process skills depends on intentional instructional choices and integration of in-class activities, classroom structure, and facilitation (38, 39). Educators using POGIL have a long history of integrating process skills development into instruction (29, 31). More recently, there has been a concerted effort to focus definitions of process skills and develop tools for use by educators and researchers to promote and investigate process skills development. For example, Ruder and co-workers reported an effective model in which students’ development of process skills was intentionally scaffolded by instructors. Importantly, instructors in this study used an iterative approach in which they monitored students’ learning using rubrics and modified 113

instructional materials and classroom facilitation in response (40). The Enhancing Learning by Improving Process Skills in STEM (ELIPSS) project is developing a set of rubrics and other resources to enable educators to more intentionally support students to develop process skills and to assess skills development over time (41). Ultimately, rubrics have the power to support reflective teaching related to process skills and can be used by researchers to investigate students’ learning in guided inquiry classrooms (42).

Guided Inquiry in the Biochemistry Classroom Some aspects of the design and implementation of guided inquiry are common to all classroom environments, while others are influenced by issues that are specific to learning in a given discipline. Biochemistry is an application-based discipline that lies at the crossroads of the physical and life sciences. Therefore, developing guided inquiry learning experiences for the biochemistry classroom presents unique challenges and opportunities, which are described below. Distinctive Features of Teaching and Learning in a Biochemistry Context Biochemistry is typically offered in the later years of college and as such it depends on students using information and skills acquired in prior courses including introductory biology, general chemistry and organic chemistry. Learning in biochemistry is built upon prior knowledge, yet students’ retention of previously learned concepts and their ability to apply prior knowledge to new contexts can be highly variable. Differences in the ability to draw on prior knowledge is a possible reason why some students struggle more than others. Therefore, educators should assess students’ prior knowledge and tailor instruction to address specific needs of students in a given course. Prior knowledge can be assessed formally through administering an assessment at the beginning of a course (43–47), or informally through monitoring students’ responses to in-class activities. The complex systems of biochemistry add another dimension to the challenges of the subject. In addition to connecting to prior knowledge, students are expected to integrate chemistry and biology concepts, while at the same time, addressing problems that are themselves more complex. In biochemistry, students encounter macromolecules that are larger and more complex than those that they studied in previous chemistry and biology courses. Likewise, the myriad sequenced, branched, and regulated reaction pathways in metabolism present layers of complexity that far exceed that of previously-encountered chemical reactions. Finally, students are expected to consider the cellular environment through a molecular lens that is often different than that encountered in previous biology courses. As a result, students are confronted with learning new biochemical information, while drawing upon often shaky prerequisite knowledge of foundational concepts. Guided Inquiry Teaching Materials for Biochemistry Guided inquiry in biochemistry includes classroom activities that help students develop an understanding of new concepts, integrate and deepen understanding of previously-learned concepts in new contexts, and strengthen process skills. For activities that introduce new concepts unique to biochemistry, the learning cycle approach, described above, is recommended (for practical tips see (30)). However, many activities in biochemistry do not involve learning new concepts, but require integrating and applying prior knowledge in new contexts. The cognitive load for these applicationbased activities can be significant, especially if previously learned concepts are poorly understood and/or if the new context is complex. Therefore, careful attention must be given to prepare students 114

for application activities and to scaffold learning before, during and, after the experience. Considerations for crafting effective application activities including the before-class preparations are summarized in Table 1 and described briefly below. To prepare students for learning and to leverage class time for more challenging learning tasks, we recommend that educators ask students to review prior knowledge before class and check foundational understanding with their teammates during class. Each educator should monitor their own students to evaluate the utility of preparatory assignments. Students can be motivated to engage in preparatory work by understanding the context in which their prior knowledge will be applied. Such understanding can be prompted through a short lecture, video, assigned reading, guiding questions, or vocabulary review. Finally, students could be asked to consider how their prior knowledge applies in the new context. During class students work in teams to address guiding questions through which they discover new relationships between familiar concepts in the new context. This could include the examination of data or figures relevant to the new context. Guiding questions direct students to access relevant principles, consider them in the new context, and discuss implications in the system. In some cases, application activities can be effectively structured to focus on problem solving. In such cases, the activities themselves lack some of the typical structure of guided inquiry activities, as students focus on addressing a semi-structured problem. Instead, structure for the activity is provided through use a problem solving methodology (48). Towards the end of class or after class, students could be asked to extend their newly-acquired understanding to new contexts or questions. For example, they could be asked to consider the effects of mutation on the system studied or to design an experiment in another system. As an example of an application activity, consider the integration of prior knowledge and understanding of complex systems required of students learning about potassium ion channels. These well-characterized proteins allow potassium ions to pass across cell membranes while excluding smaller sodium ions. Understanding this apparent contradiction presents a challenge to students and requires that they draw on knowledge of the physical properties of molecules and ions, knowledge of membrane structure, knowledge of the enthalpy of ion hydration/dehydration, and threedimensional visualization skills. Understanding is further complicated by the fact that many students do not fully grasp the physical basis of intermolecular forces, but instead bring memorized definitions from prerequisite courses (49). Similarly, students may have previously considered proteins in a general way, as opposed to wrestling with molecular details. Furthermore, deep understanding of this system also requires students to draw on biology knowledge including prokaryotic and eukaryotic cell biology, structure and function of membranes, and gene expression. Therefore, learning about potassium ion channels offers an opportunity to deepen students’ existing understanding of previously-encountered concepts, while simultaneously learning new and relevant biochemistry content. The structure of guided inquiry activities as well as a focus on process skills, described below, can enable educators to support student learning in complex contexts like these. In addition to the information about application activities included in Table 1, educators interested in developing their own guided inquiry activities, should recognize that the development process is iterative and that students play a role in helping refine the activity. Confusion can arise due to students’ actions (for example, lack of preparation) or educators’ actions (for example, expecting students to integrate too many pieces at once). As students work through an activity, the educator should note the questions asked by the teams. Rather than answering the questions directly, it is an opportunity for the educator to determine what further scaffolding might be needed in the activity to support learning and minimize frustration. One way to gather this information from the students is to determine what pieces they understand and which they don’t. If all pieces are understood, 115

perhaps additional guiding questions might be helpful and the educator can experiment in realtime with additional questions. If there is confusion about prior knowledge, it should be addressed immediately either by the educator or one of the team members. Either way, the educator gains valuable information about support necessary for successful learning and can include that in future versions of the activity. Table 1. Aspects of Application Activities Used to Integrate and Apply Concepts What is the learning goal?

How is the goal accomplished?

Explore the new context Assigned reading with of the application guiding questions to activity provide focus Review relevant prior knowledge

Directed questions related to anticipated prior knowledge

Master relevant new vocabulary

Directed questions related to new vocabulary

When is the goal accomplished?

Potassium channel example

Completed before class Students were asked to individually and reviewed read sections of a research during class with team paper describing the structure of the K+ channel. They were provided a summary of the research questions and directed to relevant figures.

Situate prior knowledge Directed questions to Completed before class in the new context place prior knowledge in individually OR during new context class with team depending on difficulty

Sample questions: Label a transmembrane protein diagram. How many helices comprise each subunit?

Discover new relationships among previously known concepts

Sample question: Describes what happens to a hydrated K+ ion as it moves into the selectivity filter. Discuss energy changes that occur.

Directed and/or open- Completed during class ended questions to guide with team students to make connections and integrate ideas

Extend new insights into Open-ended and/or data-based questions to another context foster student independence in new context

Completed during class with team OR completed after class individually

Sample question: Given what you know about the K+ channel, propose two mutations that could result in Bartter syndrome.

A number of published guided inquiry materials exist for those seeking ready-to-use, classroomtested activities. Foundations of Biochemistry, a workbook containing 39 activities that were developed and field tested by a community of biochemistry educators, is the most comprehensive collection of guided inquiry activities available for biochemistry. Activities, each of which seeks to develop several well-defined content and process outcomes, cover a range of topics related to macromolecular structure/function and metabolism (50). Murray has developed a collection of guided inquiry activities intended to guide students through reading papers from the biochemical literature (51). The four published activities connect to biochemical research related to articles on membrane transport, purine biosynthesis, DNA replication, and HIV reverse transcriptase inhibitors. An emerging 116

resource for guided inquiry activities is CourseSource, a peer-reviewed repository of activities that align with learning goals defined by professional societies in the life sciences (52).Currently, two guided inquiry activities are available: one related to noncovalent interactions and ligand binding (53) and another related to three-dimensional protein structure (54). Finally, Biochemistry and Molecular Biology Education is an ever-growing resource for guided inquiry classroom and laboratory activities (55–58). Supporting Process Skills Development in the Biochemistry Classroom Many biochemistry educators view development of transferable process skills as an essential component of biochemistry education (59–61). Development of process skills is most robust if it is threaded throughout the course and is considered as part of activity design, classroom facilitation, and overall course structure. Any of the process skills defined by the POGIL Project (problem solving, critical thinking, communication, information processing, teamwork, management, selfassessment) can be developed in a biochemistry context. In the following paragraphs we have chosen to focus on problem solving, a fundamental skill across STEM disciplines; self-assessment, an oftenoverlooked process skill; and visual literacy, a specialized form of information processing that is essential in biochemistry. Problem solving is defined by the POGIL project as “identifying, planning, and executing a strategy that goes beyond routine action to find a solution to a situation or question” (32). At its core, it challenges students to decide how to proceed when the path to a solution is not obvious. A primary way educators help students develop problem solving skills is by choosing appropriate classroom materials. Guided inquiry activities are well-suited to support problem solving skills because students often document steps to solutions instead of simply writing answers. Educators can further support problem solving by structuring context-rich problems, which ask students to analyze data and make decisions on the path to a solution (31). The complex biological context of biochemistry lends itself well to crafting high-quality problem solving experiences. For example, Halmo and co-workers describe a problem that presents students with an unknown protein and asks them to make predictions about the effect of changing amino acids on protein structure and function (62). Classroom facilitation further supports students to build conceptual understanding and develop process skills. Supporting effective argumentation during class time is a concrete way that facilitation can support development of problem solving skills. Prince and co-workers describe an activity in which students analyzed the mechanism of an aspect of the citric acid cycle. The instructor helped students elevate the quality of their argumentation by verbally providing students with a framework for arguments and providing prompts for students to use accountable talk such as asking How did you arrive at that solution? (63). Finally, Cooper and co-workers report that working in collaborative groups improves students’ problem solving abilities, suggesting that the structure of the classroom itself plays a role in students’ development of process skills (64). Interestingly, the authors of this study suggest that improved problem solving in collaborative groups may results from the fact that the group setting enables students to meaningfully engage in another process skill, metacognition. Self-assessment and metacognition are often-overlooked process skills that educators sometimes struggle to incorporate into the biochemistry classroom. Self-assessment is the process of reflecting on one’s experiences with the goal of making meaning or improving in the future, while metacognition is reflective thinking about one’s thought processes (38, 65). Research on how people learn has shown the importance of these skills in meaningful learning (66). Research suggests that students likely fall on a continuum with regard to their capacity for self-assessment 117

and metacognition and that instructors can help scaffold development of these skills (67). Although students may initially resist the invitation to reflect in the biochemistry classroom, a number of reflective activities can be built into course structure and facilitation to help students improve selfassessment skills. For example, students can be asked to set learning goals at the beginning of the term, and then reflect on progress towards achieving those goal throughout the course. Short reflective questions can be built into the daily routine. Students could be asked questions such as, What was the most useful thing you learned today? What were your learning strengths today and why are they strengths? What is a personal area for improvement and how will you make this change? What insights have you gained from classmates today? (68). Visual literacy is a specialized form of information processing and is central to learning in biochemistry. The ability to skillfully interpret images is vital because biochemistry relies on diverse visual representations and students are frequently presented with multiple different kinds of images (69). However, students are often poorly prepared to process the myriad pictures they encounter. To compound the problem, biochemistry educators use their expert tacit knowledge to process visual information without explicitly helping students to “see” salient details or the big picture in complex images. As a result, students often exhibit limited understanding of biochemically relevant images (43, 70–74). Biochemistry educators can support development of this vital process skill in a number of ways. The first step is to recognize that visual literacy is a skill that students need help developing in a biochemical context. To this end, visual learning frameworks (75) and taxonomies (76–78) have been developed to scaffold visual learning in biochemistry. These frameworks and taxonomies are organizational systems developed by biochemistry educators, which categorize elements of visual learning. The hope is that by naming specific elements of visual learning, educators can more easily design instructional strategies to support students in acquiring visualization skills. Such tools can be used to create guided inquiry activities to help students recognize and interpret visual conventions in biochemistry (79), or solve problems with a visual component (80). The active environment in a guided inquiry classroom is conducive to supporting development of visual literacy. For example, students could be asked to handle physical models or generate their own representations, both of which are thought to develop visual literacy (81). Teachers could facilitate small group discussion of the advantages and limitations of different images (82).

The Role of the Teacher in the Guided Inquiry Classroom Educators at the tertiary level are experts in their disciplinary areas, but many have had little training in learning theory, course design, teaching strategies, or classroom management. Such pedagogical knowledge is especially important in active learning classrooms, where instructional choices can affect student outcomes (83, 84). Unlike lecture-based teaching environments, in which content delivery is the focus, teachers in active learning classrooms interact heavily with students and facilitate learning through the instructional choices they make. In the best case, teachers act like coaches, modeling and supporting behaviors that lead to deep learning. For example, Daubenmire and co-workers found that general chemistry students’ performance on the ACS conceptual exam differed between two sections of general chemistry taught using guided inquiry by two different teachers. Videos of classroom facilitation revealed that teachers of both sections interacted with student groups with the same frequency. However, the teacher of the higher-scoring section did not answer students’ questions directly, but rather guided students through the problem with additional questions (85). Therefore, in order to maximize the potential of active learning, it is important to 118

plan for successful facilitation. Finally, excellent teachers strive for continual improvement and use reflective practice “to find the best fit between their subject, teaching skills, relationships built with students, research, and personality” (86). Indeed, studies examining the role of teachers’ classroom practices agree that implementation of active learning is highly variable, but that truly excellent teachers observe the effects of their instruction on students and adjust practices in response (84, 86, 87). Creating and Using Facilitation Plans Teaching using guided inquiry requires educators to reimagine their role in the classroom. For many, the transition from lecture to guided inquiry involves exchanging lecture notes for a facilitation plan (59, 88, 89). A facilitation plan is a document generated by educators to organize their teaching and track progress towards defined instructional outcomes. The most useful facilitation plans are living documents, which evolve with teaching experience. A high-quality facilitation plan helps educators focus all in-class and out-of-class learning experiences on achievement of desired conceptual and process learning outcomes. Table 2 summarizes aspects of facilitation to be considered before, during, and after class in a guided inquiry classroom. These steps should be considered a general roadmap; each individual educator will need to develop facilitation plans that suit the learning needs of their students. Particular instructional choices vary depending on the course characteristics, including number of students or classroom layout (for examples see (90, 91)). Table 2. Aspects of Facilitation to Consider before, during, and after Class Before Class

• Determine content and process goals for the upcoming class • Identify or design appropriate activity • Determine whether group roles will be used and which roles would be most effective • Design teams that use diverse knowledge, skills, and perspectives • Consider space and time constraints • Determine what prior knowledge will be needed for successful completion of the activity • Assign preparatory work to help students retrieve/review prior knowledge • Determine whether students’ prior knowledge will be assessed, and if so, how • Prepare set-up for the activity that includes communicating with students about context, goals, roles, etc. • Plan how facilitation will support development of process skills; identify rubrics or other necessary resources • Plan for activity closure

During Class

• Establish and follow daily routines • Set the pace and stick to a schedule • Provide real-time feedback to students on conceptual and process skills development • Model and support effective learning behaviors • Provide resources as needed • Manage emotions and group interactions

After Class

• Assess whether content and process outcomes were met and whether activities need to be modified to better support learning. • Reflect on facilitation including, strengths, areas for improvement, and changes to be made in the future related to preparatory work, pacing, or group structure. • Plan for future implementation

119

Navigating Challenges Implementing guided inquiry in the biochemistry classroom is not without challenges. For many students, working in groups on activities that ask them to wrestle with challenging material can be unfamiliar and anxiety-provoking. Fortunately, a number of strategies are available to increase buy-in and foster positive group dynamics. Many educators who have made the change from traditional teaching to guided inquiry notice student resistance. Students may express discomfort, report that they learn best from lecture, or complain that the new format requires them to teach themselves. Interestingly, an effective way to increase student buy-in is simply to persist; the longer novel curricula are implemented at a given institution, the more likely students are to accept it (92). Furthermore, we have found that as students recognize that they are learning, buy-in issues tend to diminish. Therefore, assignments and activities that ask students to track their learning over time not only help students develop better self-assessment skills, but also can lead to higher levels of engagement in an unfamiliar classroom environment. Some educators use specially-designed activities that prompt students to identify how learning through guided inquiry could help them achieve their future goals. For example, students could be asked to identify characteristics that are essential for success in future careers and then critically analyze how those skills could be developed in a guided inquiry classroom. We have found that some students are simply looking for reasons to trust that new teaching strategies will work. We address this by empowering students to ask questions about course structure, which allows us to provide our rationale in terms of benefits for student learning. We have also collected “words of wisdom” from previous students who have completed the course. These previous students often express anxieties felt by current students and offer reassurance that it is possible to succeed. Students also offer concrete strategies for success in the course. Finally, educators should asses students’ engagement using published surveys (92, 93) or questions they design themselves. In a classroom in which social interactions with peers is required on a daily basis, students’ experiences in their small groups can have a significant impact on learning and attitude. In one study, students who reported being comfortable in their groups exhibited greater mastery of course content than those who worked in groups in which one person dominated (94). Since student perception of group dynamics can affect performance, a quick survey at the beginning of the term can inform educators about students’ prior experiences working in teams, which can influence instructional choices and facilitation. Theory-based practices for supporting cooperative learning have been extensively described (20, 95). For example, students can be asked to periodically assess their own performance in groups and that of their peers. This information can be used by educators to make adjustments to groups or to form groups that are more likely to function productively. Some educators ask students to create team agreements outlining responsibilities of the individual to the groups and of the group to the individual. Use of team roles can act as a mechanism to support positive interdependence within groups, but some educators find that assigning roles is a distraction that doesn’t function well in their classroom culture. Regardless of the choices educators make to effectively support group dynamics in their own classrooms, social dynamics must be addressed and supported through intentional instructional choices. Many educators new to using guided inquiry in their courses find grading and assessment challenging. There is not one “right way” to grade in a guided inquiry classroom and there are as many grading strategies as there are educators. Guided inquiry activities are designed to support students to develop conceptual understanding and skills. Due to the developmental nature of guided inquiry activities, most educators opt not to assign a summative score or grade to the activities themselves. Some educators ask students to submit completed activities and provide formative 120

feedback, without assigning a grade. It can be helpful to assign points to students for completing preparatory activities, participating in class, or completing self-assessment assignments. Often such work is assigned full points for completion as opposed to grading for correctness. In addition to tasks that are scored based on completion, some educators opt to assign individual or group homework that applies knowledge acquired during guided inquiry activities and is scored for correctness. Although some educators assign large segments of the overall course graded based on group work, many continue to use individual exams or projects to assign the majority of points in a course.

Summary Guided inquiry is an effective teaching tool in the biochemistry classroom. By engaging students in structured investigation of biochemical concepts, educators can help them solidify prior knowledge, develop new understandings, integrate complex ideas, and strengthen key skills that will be vital for success in their futures. The evidence-based guided inquiry model provided here is adaptable and invites educators into a reflective teaching process, with the ultimate goal of improving biochemistry learning in their classrooms.

References 1.

American Association for the Advancement of Science. Vision and Change in Undergraduate Biology Education: A Call to Action; Washington, DC, 2011. 2. American Association for Biochemistry and Molecular Biology Accreditation Program for Bachelor’s Degrees in Biochemistry and Molecular Biology: Accreditation Application Guide. http://www. asbmb.org/uploadedFiles/Accreditation/Application/App%20Guide_032817.pdf (accessed April 4, 2019). 3. Frey, G.; Shadle, S. E. GI: The Guided Inquiry. In POGIL: An Introduction to Process Oriented Guided Inquiry Learning for Those Who Wish to Empower Learners; Simonson, S. R., Ed.; Stylus: Sterling, VA, 2019; pp 69−84. 4. Eberlein, T.; Kampmeier, J.; Minderhout, V.; Moog, R. S.; Platt, T.; Varma-Nelson, P.; White, H. B. Pedagogies of Engagment in Science. Biochem. Mol. Biol. Educ. 2008, 36, 262–273. 5. Kulak, V.; Newton, G. A guide to using case-based learning in biochemistry education. Biochem. Mol. Biol. Educ. 2014, 42, 457–73. 6. MacArthur, J. R.; Jones, L. L. A review of literature reports of clickers applicable to college chemistry classrooms. Chem. Educ. Res. Prac. 2008, 9, 187–195. 7. Brewer, R. Successful Stories and Conflicts: A Literature Review on the Effectiveness of Flipped Learning in Higher Education. J. Comput. Assist. Lear. 2018, 34, 409–416. 8. Freeman, S.; Eddy, S. L.; McDonough, M.; Smith, M. K.; Okoroafor, N.; Jordt, H.; Wenderoth, M. P. Active learning increases student performance in science, engineering, and mathematics. Proc. Nat. Acad. Sci. 2014, 111, 8410–8415. 9. Springer, L.; Stanne, M. E.; Donovan, S. S. Effects of Small-Group Learning on Undergraduates in Science, Mathematics, Engineering, and Technology: A Meta-Analysis. Rev. Educ. Res. 1999, 69, 21–51. 10. Freeman, S.; Haak, D.; Wenderoth, M. P. Increased course structure improves performance in introductory biology. CBE Life Sci. Educ. 2011, 10, 175–86. 11. Haak, D. C.; Hille Ris Lambers, J.; Pitre, E.; Freeman, S. Increased structure and active learning reduce the achievement gap in introductory biology. Science 2011, 332, 1213–1216. 121

12. Warfa, A. M. Using cooperative learning to teach chemistry: a meta-analytic review. J. Chem. Educ. 2015, 93, 248–255. 13. Apugliese, A.; Lewis, S. E. Impact of instructional decisions on the effectiveness of cooperative learning in chemistry through meta-analysis. Chem. Educ. Res. Prac. 2017, 18, 271–278. 14. Resnick, L. B. Mathematics and science learning: a new conception. Science 1983, 220, 477–478. 15. Bodner, G. Constructivism: a theory of knowledge. J. Chem. Educ. 1986, 63, 873–878. 16. Bauer, C. F.; Daubenmire P. L.; Minderhout, V. Not Just a Good Idea, POGIL Has a Theoretical Foundation. In POGIL: An Introduction to Process Oriented Guided Inquiry Learning for Those Who Wish to Empower Learners; Simonson, S. R., Ed.; Stylus: Sterling, VA, 2019; pp 3–22. 17. Piaget, J. The Development of Thought: Equilibration of Cognitive Structures; The Viking Press: New York, 1977. 18. Vygotsky, L. S.; Kozulin, A. Thought and Language; MIT Press: Cambridge, MA, 1986. 19. Bandura, A. Perceived self-efficacy in cognitive development and functioning. Educ. Psychol. 1993, 28, 117–148. 20. Johnson, D. W.; Johnson R. T.; Smith, K. A. Cooperative Learning: Increasing College Faculty Intructional Productivity ASHE-ERIC Higher Education Report No. 4; The George Washington University, Graduate School of Education: Washington DC, 1991. 21. Kirschner, P. A.; Sweller, J.; Clark, R. E. Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educ. Psychol. 2006, 41, 75–86. 22. Paas, F.; Renkl, A.; Sweller, J. Cognitive Load Theory and Instructional Design: Recent Developments. Educ. Psychol. 2003, 38, 1–4. 23. Kapur, M. Examining Productive Failure, Productive Success, Unproductive Failure, and Unproductive Success in Learning. Educ. Psychol. 2016, 51, 289–299. 24. Hmelo-Silver, C. E.; Duncan, R. G.; Chinn, C. A. Scaffolding and Achievement in problemBased and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006). Educ. Psychol. 2007, 42, 99–107. 25. Johnstone, A. H. Chemistry teaching-science or alchemy? J. Chem. Educ. 1997, 74, 262–268. 26. Karplus, R.; Their, H. D. A new look at elementary school science. In New Trends in Curriculum and Instruction; Michaelis, J. U., Ed.; Rand McNally & Co.; Chicago, IL, 1967. 27. Abraham, M. R.; Renner, J. W. The sequence of learning cycle activities in high school chemistry. J. Res. Sci. Teach. 1986, 22, 121–148. 28. Lawson, A. E. A better way to teach biology. Am. Biol. Teach. 1988, 50, 266–278. 29. Farrell, J. J.; Moog, R. S.; Spencer, J. N. A Guided Inquiry General Chemistry Course. J. Chem. Educ. 1999, 76, 570–574. 30. Kussmaul, C.; Sullivan, M. Activity Selection and Writing. In POGIL: An Introduction to Process Oriented Guided Inquiry Learning for Those Who Wish to Empower Learners; Simonson, S. R., Ed.; Stylus: Sterling, VA, 2019, pp 141–170. 31. Hanson, D.; Wolfskill, T. Process workshops-A model for instruction. J. Chem. Educ. 2000, 77, 120–130. 32. POGIL. www.pogil.org (accessed April 11, 2019). 122

33. POGIL Curricular Materials. https://pogil.org/educators/become-a-pogil-practitioner/ curricular-materials (accessed April 11, 2019). 34. Lewis, S. E.; Lewis, J. E. Departing from Lectures: An Evaluation of a Peer-Led Guided Inquiry Alternative. J. Chem. Educ. 2005, 82, 135–139. 35. Walker, L.; Warfa, A. M. Process oriented guided inquiry learning (POGIL) marginally effects student achievement measures but substantially increases the odds of passing a course. PLoS One 2017, 12, e0186203. 36. Hein, S. M. Positive Impacts Using POGIL in Organic Chemistry. J. Chem. Educ. 2012, 89, 860–864. 37. Brown, P. J. Process-oriented guided-inquiry learning in an introductory anatomy and physiology course with a diverse student population. Adv. Physiol. Educ. 2010, 34, 150–5. 38. Cole, R.; Lantz, J.; Ruder, S. PO: The Process. In POGIL: An Introduction to Process Oriented Guided Inquiry Learning for Those Who Wish to Empower Learners; Simonson, S. R. , Ed.; Stylus: Sterling, VA, 2019; pp 42–68. 39. Minderhout, V.; Loertscher, J. Facilitation: The Role of the Instructor. In Process-Orientied Guided Inquiry Learning; Moog R. S., Spencer, J. N., Eds.; American Chemical Society: Washington, DC, 2007. 40. Ruder, S. M.; Stanford, C.; Gandhi, A. Scaffolding STEM classrooms to integrate key workplace skills: Development of resources for active learning environments. J. Coll. Sci. Teach. 2018, 47, 29–35. 41. ELIPSS. http://elipss.com (accessed on April 11, 2019). 42. Bunce, D. M.; Vanden Plas, J. R.; Neiles, K. Y.; Flens, E. A. Development of a Valid and Reliable Student-Achievement and Process-Skills Instrument. J. Coll. Sci. Teach. 2010, 39, 50–55. 43. Villafañe, S. M.; Bailey, C. P.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Development and analysis of an instrument to assess student understanding of foundational concepts before biochemistry coursework. Biochem. Mol. Biol. Educ. 2011, 39, 102–9. 44. Smith, M. K.; Wood, W. B.; Knight, J. K. The genetics concept assessment: a new concept inventory for gauging student understanding of genetics. CBE Life Sci. Educ. 2008, 7, 422–30. 45. Garvin-Doxas, K.; Klymkowsky, M. W. Understanding randomness and its impact on student learning: lessons learned from building the Biology Concept Inventory (BCI). CBE Life Sci. Educ. 2008, 7, 227–33. 46. Newman, D. L.; Snyder, C. W.; Fisk, J. N.; Wright, L. K. Development of the Central Dogma Concept Inventory (CDCI) Assessment Tool. CBE Life Sci. Educ. 2016, 15. 47. Shi, J.; Wood, W. B.; Martin, J. M.; Guild, N. A.; Vicens, Q.; Knight, J. K. A diagnostic assessment for introductory molecular and cell biology. CBE Life Sci. Educ. 2010, 9, 453–61. 48. Morgan, J. Overview of Problem Solving. In Faculty Guidebook: A Comprehensive Tool for Improving Faculty Performance; Beyerlein, S. W., Holmes, C., Apple, D. K., Eds.; Pacific Crest: Lisle, IL, 2007. 49. Loertscher, J.; Green, D.; Lewis, J. E.; Lin, S.; Minderhout, V. Identification of Threshold Concepts for Biochemistry. CBE Life Sci. Educ. 2014, 13, 516–528. 50. Loertscher, J.; Minderhout, V.; Frato, K. Foundations of Biochemistry, 4th ed.; Pacific Crest: Hamptom, NH, 2015. 123

51. Murray, T. A. Teaching students to read the primary literature using POGIL activities. Biochem. Mol. Biol. Educ. 2013, 42, 165–173. 52. CourseSource. https://www.coursesource.org (accessed April 11, 2019). 53. Werth, M. T. Serotonin in the Pocket: Non-covalent interactions and neurotransmitter binding. CourseSource; 2017. 54. Hark, A. T. Understanding Protein Domains: A Modular Approach. CourseSource; 2017. 55. de Espindola, M. B.; El-Bacha, T.; Giannella, T. R.; Struchiner, M.; da Silva, W. S.; Da Poian, A. T. Teaching energy metabolism using scientific articles: Implementation of a virtual learning environment for medical students. Biochem. Mol. Biol. Educ. 2010, 38, 97–103. 56. Roche Allred, Z. D.; Tai, H.; Bretz, S. L.; Page, R. C. Using PyMOL to explore the effects of pH on noncovalent interactions between immunoglobulin G and protein A: a guided-inquiry biochemistry activity. Biochem. Mol. Biol. Educ. 2017, 45, 528–536. 57. Smith, A. L.; Purcell, R. J.; Vaughan, J. M. Guided inquiry activities for learning about the macro- and micronutrients in introductory nutrition courses. Biochem. Mol. Biol. Educ. 2015, 43, 449–59. 58. Emery, P.; Yezierski, E. J.; Page, R. C. Guided inquiry activity linking thermodynamic parameters of protein unfolding to structure using differential scanning fluorimetry data in the biophysical chemistry classroom. Biochem. Mol. Biol. Educ. 2019, 47, 67–75. 59. Minderhout, V.; Loertscher, J. Lecture-free biochemistry: A Process Oriented Guided Inquiry Approach. Biochem. Mol. Biol. Educ. 2007, 35, 172–80. 60. ASBMB Skills. www.asbmb.org/education/educationstrategies/foundationalconcepts/skills (accessed April 11, 2019). 61. White, H. B.; Benore, M. A.; Sumter, T. F.; Caldwell, B. D.; Bell, E. What skills should students of undergraduate biochemistry and molecular biology programs have upon graduation? Biochem. Mol. Biol. Educ. 2013, 41, 297–301. 62. Halmo, S. M.; Sensibaugh, C. A.; Bhatia, K. S.; Howell, A.; Ferryanto., E. P.; Choe, B.; Kehoe, K.; Watson, M.; Lemons, P. P. Student difficulties during structure-function problem solving. Biochem. Mol. Biol. Educ. 2018, 46, 453–463. 63. Prince, A. N.; Pitts, W. B.; Parkin, D. W. Exploring Power Distribution and Its Influence on the Process of Argumentation in a POGIL Biochemistry Classroom. J. Coll. Sci. Teach. 2018, 47, 92–106. 64. Cooper, M. M.; Cox, C. T.; Nammouz, M.; Case, E.; Stevens, R. An Assessment of the Effect of Collaborative Groups on Students’ Problem-Solving Strategies and Abilities. J. Chem. Educ. 2008, 85, 866–872. 65. POGIL Process Skills. https://pogil.org/educators/resources/general-resources/process-skills (accessed April 11, 2019). 66. How People Learn: Brain, Mind, Experience, and School; Bransford, J. D., Brown, A. L., Cocking, R. R., Eds.; National Academy Press: Washington DC, 2000. 67. Stanton, J. D.; Neider, X. N.; Gallegos, I. J.; Clark, N. C. Differences in metacognitive regulation in introductory biology students: when prompts are not enough. CBE Life Sci. Educ. 2015, 14. 68. Hanson, D. Intructor’s Guide to Process-Oriented Guided-Inquiry Learning; Pacific Crest: Lisle, IL, 2006. 124

69. Schönborn, K. J.; Anderson, T. R. The importance of visual literacy in the education of biochemists. Biochem. Mol. Biol. Educ. 2006, 34, 94–102. 70. Harle, M.; Towns, M. H. Students’ understanding of external representations of the potassium ion channel protein part II: structure-function relationships and fragmented knowledge. Biochem. Mol. Biol. Educ. 2012, 40, 357–63. 71. Harle, M.; Towns, M. H. Students’ understanding of external representations of the potassium ion channel protein, part I: affordances and limitations of ribbon diagrams, vines, and hydrophobic/polar representations. Biochem. Mol. Biol. Educ. 2012, 40, 349–56. 72. Harle, M.; Towns, M. H. Students’ understanding of primary and secondary protein structure: drawing secondary protein structure reveals student understanding better than simple recognition of structures. Biochem. Mol. Biol. Educ. 2013, 41, 369–76. 73. Linenberger, K. J.; Bretz, S. L. Biochemistry students’ ideas about shape and charge in enzymesubstrate interactions. Biochem. Mol. Biol. Educ. 2014, 42, 203–12. 74. Linenberger, K. J.; Bretz, S. L. Biochemistry students’ ideas about how an enzyme interacts with a substrate. Biochem. Mol. Biol. Educ. 2015, 43, 213–22. 75. Dries, D. R.; Dean, D. M.; Listenberger, L. L.; Novak, W. R.; Franzen, M. A.; Craig, P. A. An expanded framework for biomolecular visualization in the classroom: Learning goals and competencies. Biochem. Mol. Biol. Educ. 2017, 45, 69–75. 76. Offerdahl, E. G.; Arneson, J. B.; Byrne, N. Lighten the Load: Scaffolding Visual Literacy in Biochemistry and Molecular Biology. CBE Life Sci. Educ. 2017, 16. 77. Arneson, J. B.; Offerdahl, E. G. Visual Literacy in Bloom: Using Bloom’s Taxonomy to Support Visual Learning Skills. CBE Life Sci. Educ. 2018, 17. 78. Towns, M. H.; Raker, J. R.; Becker, N.; Harle, M.; Sutcliffe, J. The biochemistry tetrahedron and the development of teh taxonomy of biochemistry external representations (TOBER). Chem. Educ. Res. Prac. 2012, 13, 296–306. 79. Loertscher, J.; Villafañe, S. M.; Lewis, J. E.; Minderhout, V. Probing and improving student’s understanding of protein alpha-helix structure using targeted assessment and classroom interventions in collaboration with a faculty community of practice. Biochem. Mol. Biol. Educ. 2014, 42, 213–23. 80. Schönborn, K. J.; Anderson, T. J. Bridging the Educational Research-Teaching Practice Gap: Foundations for Assessing and Developing Biochemistry Students’ Visual Literacy. Biochem. Mol. Biol. Educ. 2010, 38, 347–354. 81. Oliver-Hoyo, M.; Babilonia-Rosa, M. A. Promotion of Spatial Skills in Chrmistry and Biochemistry Education at the College Level. J. Chem. Educ. 2017, 94, 996–1006. 82. Linenberger, K. J.; Bretz, S. L. Generating cognitive dissonacec ins student interview through multiple representations. J. Chem. Educ. 2012, 13, 172–178. 83. Chase, A.; Pakhira, D.; Stains, M. Implementing Process-Oriented, Guided-Inquiry Learning fof the First Time: Adaptations and Short-Term Impacts on Students’ Attitude and Performance. J. Chem. Educ. 2013, 90, 409–416. 84. Auerbach, A. J. J.; Andrews, T. C. Pedagogical knowledge for active-learning instruction in large undergraduate biology courses: a large-scale qualitative investigation of instructor thinking. Int. J. STEM Educ. 2018, 5.

125

85. Daubenmire, P. L.; Bunce, D. M.; Draus, C.; Frazier, M.; Gessell, A.; van Opstal, M. T. During POGIL Implementation the Professor Still Makes a Difference. J. Coll. Sci. Teach. 2015, 44, 72–81. 86. Kane, R.; Sandretto, S.; Heath, C. An investigation into excellent tertiary teaching: Emphasising reflective practice. High. Educ. 2004, 47, 283–310. 87. Murphy, K. L.; Picione, J.; Holme, T. A. Data-driven Implementation and Adaptation of New Teaching Methodologies. J. Coll. Sci. Teach. 2010, 40, 80–86. 88. Minderhout, V. Creating a Facilitation Plan. In Faculty Guidebook; Beyerlein, S.; Holmes, C.; Apple, D., Eds. Pacific Crest: Lisle, IL, 2007; pp 359–362. 89. Sullivan, M.; Loertscher J. Facilitation. In Faculty Guidebook: A Comprehensive Tool for Improving Faculty Performance; Beyerlein, S. W., Holmes, C., Apple, D. K., Eds.; Pacific Crest: Lisle, IL, 2007; pp 171–194. 90. Bailey, C. P.; Minderhout, V.; Loertscher, J. Learning Transferrable Skills in Large Lecture Halls: Implementing a POGIL Approach in Biochemistry. Biochem. Mol. Biol. Educ. 2011, 40, 1–7. 91. Ruder, S. M.; Hunnicutt, S. S. POGIL in Chemistry Courses at a Large Urban University: A Case Study. In Process-Oriented Guided Inquiry Learning; Moog R. S.; Spencer, J. N., Eds.; American Chemical Society: Washington, DC, 2008; pp 131–145 92. Shaw, T. J.; Yang, S.; Nash, T. R.; Pigg, R. M.; Grim, J. M. Knowing is half the battle: Assessments of both student perception and performance are necessary to successfully evaluate curricular transformation. PLoS One 2019, 14, e0210030. 93. Wiggins, B. L.; Eddy, S. L.; Wener-Fligner, L.; Freisem, K.; Grunspan, D. Z.; Theobald, E. J.; Timbrook, J.; Crowe, A. J. ASPECT: A Survey to Assess Student Perspective of Engagement in an Active-Learning Classroom. CBE Life Sci. Educ. 2017, 16. 94. Theobald, E. J.; Eddy, S. L.; Grunspan, D. Z.; Wiggins, B. L.; Crowe, A. J. Student perception of group dynamics predicts individual performance: Comfort and equity matter. PLoS One 2017, 12, e0181336. 95. Johnson, D. W.; Johnson, F. P. Joining Together: Group Theory and Group Skills, 12th ed.; Pearson: New York, 2017.

126

Chapter 6

The Development and Use of Case Studies Sarah Baas Robinson,1 Erin Dolan,1 Kathleen Cornely,2 Amy E. Medlock,1 Jin Kyu Lee,1 and Paula P. Lemons*,1 1Department of Biochemistry and Molecular Biology, University of Georgia,

Athens, Georgia 20602, United States

2Department of Chemistry and Biochemistry, Providence College,

Providence, Rhode Island 02918, United States *E-mail: [email protected].

Case-based learning (CBL) engages students in learning through real-world stories. The unique advantage of cases compared to other evidence-based pedagogies is that they foster student interest and motivation. Cases also create opportunities for students to engage in generative processing - to meaningfully learn material by organizing, integrating, and constructing biochemistry principles and interacting with peers and instructors. We use CBL in large biochemistry classes for science majors who are not biochemistry majors. We devote the majority of class time for students to work on cases. We structure and facilitate generative processing through questions about the case. We facilitate interaction through group work, assisted by Peer Learning Assistants (PLAs). We use formative and summative assessment to measure students’ knowledge of case questions and their ability to transfer what they have learned. Each of us implements CBL slightly differently, yet we find CBL is conducive to this variety. We work collaboratively with local and national colleagues to continuously update, refine, and improve our implementation of CBL.

Introduction There is a rich body of literature on the use of the case study method in science courses, beginning in 1994 when Clyde Herreid (1) proposed that science teachers use a method of teaching that had proven to be successful in the fields of business and law. Since then, educators have been quick to adopt this method of teaching, recognizing, as Herreid did, that the use of “real world” scenarios is an effective way to engage students, convey course content, and develop problem-solving skills (2–8). There are case collections available in print (9, 10) and online (National Center for Case Study Teaching in Science, NCCSTS) while other published sources provide instruction in the development of one’s own case studies (11). A study by Borrego, et al., supports the fidelity of the © 2019 American Chemical Society

implementation of case study teaching in engineering courses (12) and presumably this is true of case-based strategies used in other Science, Technology, Engineering and Math (STEM) fields as well. As a diverse group of faculty members who teach using case studies and collaborate on developing and assessing our teaching, we believe we can offer a unique perspective on the implementation of case-based learning. Each of us teaches sections of an introductory biochemistry and molecular biology course for science majors – who are not necessarily majoring in biochemistry and thus are presumably less likely to be intrinsically interested in the subject. Our sections range widely in enrollment from 50 to 200 students. Our group regularly exchanges ideas and resources, including case studies themselves and case study-related assessments that enable us to implement this pedagogy despite large enrollments without the support of graduate teaching assistants. We have collected learning data from our students that demonstrates that they are making noteworthy learning gains. Our collaborative work provides us with ongoing opportunities to improve our teaching and student learning. Our group includes basic biochemistry researchers as well as discipline-based education researchers (DBER) and thus we have collective expertise in basic science and the science of learning. We use our combined perspectives and expertise to address the following questions: 1. Given the variety of evidence-based pedagogies, why should I choose case studies? 2. How can I implement case studies in large classes?

Given the Variety of Evidence-Based Pedagogies, Why Should I Choose Case Studies? At its core, case-based learning (CBL) is distinctive because it builds instruction around cases. When we say “cases,” we mean real or realistic stories that typically involve a combination of data from primary literature and a compelling, relevant situations such as medical diagnoses, societal problems, or everyday occurrences. Cases convey to the students why what they are learning is important. An introductory undergraduate biochemistry course typically includes learning objectives related to the structure and function of biological molecules, thermodynamics, enzyme mechanisms and kinetics, and metabolic pathway dynamics and regulation. Thus, case studies for biochemistry allow students to make connections between atomic, molecular, and cellular concepts, principles, and phenomena (which are not directly observable and thus seem abstract or irrelevant to students) to observable, concrete, and meaningful events. For example, a compelling case that we use in our course involves a woman who obtained the compound 2,4-dinitrophenol sold without a prescription via the internet as a weight-loss drug. She doubled the dose for faster results, and ultimately died (13). Through a set of guided questions students determine how 2,4-dinitrophenol leads to weight loss in low dose but has the potential to be lethal at high doses; at the same time, students develop their understanding of the coupling of chemiosmosis and oxidative phosphorylation. No two students coming into our classrooms are the same. Some are excited to learn the intricacies of metabolic regulation or protein structure and function, while others are simply there to complete a degree requirement. This means that we, as instructors, need to find strategies to engage a broad range of learners in ways that allow all to make progress in their science knowledge and skills, even though some may be interested in the material and others may not. In our view, this is the main value of cases – they foster student interest and motivation, which in turn promotes learning. Hidi and Reninger (14, 15) have described a model of interest development that involves a growth of interest over time with engagement with the subject of interest. This model has broad 128

applicability to the courses that we teach. A student may begin the course with little if any a priori interest in biochemistry. The coursework has the potential to create a situation in which students develop an interest in the material, but only if it captures the students’ interest. Cases help to create such situations. They trigger and maintain students’ situational interest, meaning interest in the moment, so it can develop into individual interest, or interest that is maintained outside of and beyond the course. The transition from situational to personal interest involves an interplay among knowledge, positive emotion, and personal value. As students learn more about a topic, they become more knowledgeable and skilled, which results in their feeling more competent. The more competent they feel, the more willing they are to invest time and effort with the activity (e.g., figuring out a problem, studying the material). Students are also willing to spend more time and effort on an activity that has personal relevance. For instance, a case involving a Type 2 diabetes patient may clearly relate to a students’ career interests of becoming a physician or to the fact that they may have a parent or grandparent with the condition. This relevance encourages students to seek out additional information on their own and to try their best on the particular coursework, which again improves their knowledge and skills and increases their self-efficacy and positive affect. Collectively, these factors operate as feedback loop: students invest more time and energy to understand the material and thus become more skilled, which fosters their positive emotions about the material and their confidence in their abilities. Ultimately, they become more motivated – more willing to invest effort toward making sense of the material (16, 17). Although student interest is certainly desirable, the ultimate goal is student mastery of the concepts and practices of biochemistry. Fortunately, case studies also offer opportunities for critical aspects of student learning and success that go beyond interest, most notably generative processing. Generative processing refers to the idea that meaningful learning requires students to build a new mental representation based on their existing knowledge (18–21). Research from the 1930s to the present day converges on the idea that learning involves mental construction on the part of the learner (18–27). All learners bring to our classes a pre-existing set of ideas, experiences, attitudes, and beliefs. In order for them to learn biochemistry, they must start with this existing knowledge, add to it, and revise it to form new knowledge structures that can be used in new situations, such as solving novel problems (17, 21). Generative processing refers to this type of cognitive work. Although other pedagogies including Process Oriented Guided Inquiry Learning (POGIL) and guided inquiry provide opportunities for generative processing, we use cases because of their unique ability to not only pique student interest but also lead students through repeated opportunities to construct knowledge. The SLO Model and the ICAP framework are two models that can be used to illustrate the concept of generative processing and how it applies to a classroom setting. The SLO model of Generative Learning explains that learning involves three processes, Selecting, Organizing, and Integrating (25, 26). Instruction presents students with words, pictures, objects, and other elements held in sensory memory. Students then select which features they will pay attention to. Next, students use working memory to organize the selected features based on the underlying structures that make sense to them. That is, students determine the connections that exist in the material being presented (16). Students who organize the material in working memory are able perform a task, such as responding to a question, based purely on the organization of this new information. In the final step of the SLO model, students take the organized information and integrate it to develop concepts, categories, principles, or problem-solving methods in long-term memory. In the SLO model, generative processing occurs in the organization and integration steps.

129

The Interactive, Constructive, Active, and Passive (ICAP) framework focuses on overt student behaviors during instruction (21). Passive behavior occurs when a student simply takes in information without actively processing it, such as listening to the professor or another student. Active behavior occurs when students take an action without extending their thinking beyond the material to be learned. This could include writing verbatim notes or copying down what a friend said. Constructive behavior occurs when students generate a product that goes beyond the learning materials provided. Here students are processing the information for themselves and making sense of it. This could include summarizing, explaining, mapping, generating a hypothesis, making a prediction, or designing a new experiment. Finally, interactive behavior occurs when students engage in co-constructing knowledge and explanations. That is, they explain their thinking to one or more partners, such as other students or an instructor, who respond in a way that requires that the student defend ideas, reason further, or demonstrate other new forms of thinking that are prompted by social interaction. Importantly, interaction as defined by ICAP only occurs if all participants in the dialogue are constructing and if there is frequent turn taking. In the ICAP framework, generative processing occurs when students construct and interact (17). So how is it that CBL creates opportunities for generative processing? The case studies we teach and most of the ones we have observed in case study collections (e.g., NCCSTS) (28) provide opportunities for students to learn through summarizing, mapping, drawing, explaining, imagining, designing experiments, analyzing data, and forming conclusions. Instructors can increase the chances that students will construct their own knowledge and interact with one another by presenting case study materials in a classroom environment in which tasks and questions posed in the case requires these behaviors.

Figure 1. Visual representations of the potassium ion channel. A: a ribbon diagram of the ion channel in the plasma membrane, made up of 3 subunits of alpha helices labeled in yellow, lavender and cyan. B: The helices are shown as cylinders, using the same color scheme as A. Potassium ions are shown as green spheres. C: A close-up of the potassium binding site. Carbonyl oxygens are shown in red. Created using PYMOL Molecular Graphics System (Schrodinger, LLC) and PDB ID 1BL8. Consider a case study we use on Bartter’s syndrome and the potassium ion channel. Bartter’s syndrome is a kidney condition in which salts like NaCl are not properly reabsorbed in the kidneys, which leads to improper concentrations of salts throughout the body (29). One of the proteins involved in this process is the potassium ion channel encoded by the gene KCNJ1. One of the learning objectives for this case is to understand the impact of noncovalent interactions on protein structure and function. In the case, we present students with multiple visual representations of the 130

potassium ion channel, two of which are shown in Figure 1. We ask students a series of questions about the structure and function of the potassium ion channel. For example, we ask them to use these visuals, textbook passages, and a paper from the primary literature to explain how the structure of the selectivity filter enables the channel to transport K+ ions, but not Na+ ions. Imagine how students might generatively process this case study. First, they must select and attend to the selectivity filter of the potassium ion channel as the most relevant piece of the information. Then they must organize the information about the selectivity filter, noting that carbonyl oxygens extend into the filter and that the K+ ions participate in some type of interaction with the carbonyl oxygens. Finally, they integrate this information with prior knowledge, recalling the nature of ion-dipole interactions and reasoning that this must be a biological example of what they have seen previously. In these three steps, students generatively process the information, creating a modified knowledge structure by organizing new information and integrating it with prior knowledge. Let’s take the Bartter’s syndrome example further to consider how CBL creates opportunities for interaction. In our classes and most case study classrooms, time is provided for students to discuss the material. Imagine an interactive conversation between two of our students who are coconstructing new knowledge about the potassium ion channel. The first student raises a question, Student 1: If an Na+ ion is smaller than a K+ ion, why can’t Na+ get across the membrane? Student 2: I think it has something to do with the diameter of the selectivity filter. Maybe it has to do with the distance between the carbonyl oxygens and the K+ ion. Student 1: Oh, I see what you’re saying. You are saying that K+ is just the right size to participate in an ion-dipole interaction with the carbonyl oxygens. The partial negative charge on the oxygen is close enough to the K+ ion to attract it. Student 2: Yes, that’s what I was thinking. So what does that mean for sodium? Student 1: Well, sodium has a smaller atomic radius? I think it must be too small. It must be too far away from the partial negative charge on the carbonyl oxygen to interact with it. In this example, the two students have taken the information provided in the text and visuals to co-construct an understanding of the noncovalent interactions that occur between K+ and the selectivity filter. They have made sense of something that seems counterintuitive at first glance, i.e., Na+ is too small to cross the potassium ion channel. Indeed, cases typically provide opportunities for students to interact by explaining, asking or answering questions, elaborating on a statement, clarifying, correcting, defending an argument, or requesting justification (21). Thus, cases provide opportunities for generative processing, including organizing, integrating, constructing, and interacting. Note that generative processing requires a motivated learner (16, 25). Motivation is viewed as the power source that fuels student ability to engage in cognitive processing as we have described (17). Students engage in this type of demanding thinking only if they are willing. Because cases provide opportunities for generative processing they increase the probability of meaningful learning. Meaningful learning has been defined as the ability to comprehend information and convert that knowledge into a usable form (17). Meaningful learning aligns well with national calls to focus science learning on core concepts and scientific practices, such as those described by the American Association for the Advancement of Science (AAAS) (30). For example, biochemistry instruction should focus on core concepts like structure and function and the flow of energy. It also should promote practices like application of the scientific process, quantitative reasoning, modeling, and problem solving. We see CBL as an efficient method to approach this challenge by using reallife stories to capture students’ interest to capitalize on that interest with opportunities for generative 131

processing. As a result, case-based learning increases the probability that meaningful learning will occur and that students will develop the ability to transfer that comprehension to novel and increasingly complex problems. We will now discuss how we implement CBL in a large class, including a typical week in our course; how we structure and facilitate interaction; and how we assess our students.

How Can I Implement Case Studies in Large Classes? As with any instruction, we made numerous decisions. We considered what type of case study would work best with our population of students, how to sequence the case study work week to week, how to structure and facilitate group work on the cases, and how to assess student learning. The factors we considered included the following: • Our course is a one-semester survey of biochemistry for science and engineering students who are not majoring in biochemistry. • Our class sizes range from 50-200 students. • Sometimes we teach in classrooms with movable chairs, but often we teach in theater-style classrooms. • We do not have graduate teaching assistants. Described below are the choices we made and how we put CBL into practice. The Types of Cases We Use Multiple types of CBL exist and they differ in the complexity of the questions and the amount of faculty interaction and group work (11, 31). We primarily use directed cases (32). The questions in a directed case tend to be well-defined with one or a few reasonable solutions. Students work on the cases in groups both during and outside of class, but submit individual assignments related to each case (discussed below). We as instructors discuss each case and solutions with the whole class. We select directed cases because they allow us to cover fundamental content in a linear way and to provide students with time for constructive interaction; but other CBL types, such as clicker cases or progressive disclosure cases (1), could be used in a large-class setting as well. Each of our cases addresses a set of learning objectives that articulate what students should be able to do once they have completed the case. These objectives align with our formative and summative assessments (discussed below). The cases themselves begin with a real-life story that gives students context for what they will be learning. The case might open with a conversation between two people, details of an emergency room visit and the results of initial tests, or the description of a metabolic disease. The idea is to create a connection between the students and the story. Ideally, students imagine themselves in the conversation or picture themselves as the clinician or researcher. Once the context has been set with the story, the case unfolds through a series of questions. Our questions are more structured than problem-based learning, but less structured than POGIL. The questions prompt students to use knowledge they have learned in previous courses or through textbook reading done in advance of class (see discussion below). They must decipher what each question is asking and decide what prior knowledge should be used. Students may be asked to analyze molecular structures, models of biochemical relationships, or original data (e.g., graphs or tables). Additionally, we often include questions that require students to draw structures and pathways, to help improve visual literacy and spatial ability (33–35). Often questions at the end of the case ask students to synthesize an 132

explanation based on information from several earlier questions. We tailor the length and difficulty of questions to fit within the time limits of a class session. Cases typically include eight to fifteen questions. A list of the publicly available cases we have used is found in Table 1. Other case studies are available from NCCSTS and several case study books have been published (36–38). Table 1. Publicly Available Biochemistry Case Studies Used by the Authors Case

Vision and Change core concepts, competencies, disciplinary practices (30)

Biochemistry topics covered

The Inexplicable Disease (39)

Apply process of science, tap Prions, analysis of into the interdisciplinary experimental data nature of science, understand the relationship between science and society

Great for the first day of class, primer for protein structure and folding. Based on the true story of a disease, Kuru, and the events that lead to the discovery of prions.

I scream for ice cream: Lactase persistence in humans (40)

Structure and function, Lactase enzyme, pathways and carbohydrate transformations of energy structure and storage, use quantitative reasoning

Students relate well as lactose intolerance is common. We have modified this case to go more in depth with carbohydrate structure.

Why is Patrick paralyzed (41)?

Pathways and transformations of energy and storage

We have modified this clicker case to a directed case to teach pyruvate dehydrogenase complex and the citric acid cycle.

Pyruvate dehydrogenase complex, metabolism, fermentation

Comments

Acute aspirin Information flow, exchange Blood buffering Useful for helping students apply and storage, use quantitative system, Hendersen- and understand acid. base overdose: Relationship to the reasoning Hasselbach equation concepts at the organismal level. blood buffering system (36) Carbonic anhydrase II deficiency (36)

Structure and function, information flow, exchange and storage

Blood buffering system, protein structure

We have combined this case with the case on aspirin overdose. We have used this case to teach enzymes and energetics of chemical reactions.

Wresting with weight loss (42)

Pathways and transformations of energy and storage

Oxidative phosphorylation (uncoupling)

This is one of several case studies available on DNP (43). Student engagement with this topic is usually high.

The mermaid and the globins: hemoglobin function and regulation (44)

Structure and function, Enzymes, allosteric information flow, exchange regulators, binding and storage, use quantitative curves reasoning

133

We have used this interrupted case in an introductory course to understand levels of protein structure and the effect of amino acid substitutions on protein structure.

Table 1. (Continued). Publicly Available Biochemistry Case Studies Used by the Authors Case

Vision and Change core concepts, competencies, disciplinary practices (30)

Biochemistry topics covered

A can of bull: Do energy drinks really provide a source of energy (45)?

Pathways and Metabolism, nutrition transformations of energy and storage, understand relationship between science and society, communicate and collaborate with other disciplines

This case, which profiles four different energy drinks, can be used in a “jigsaw” format in which the class is divided into groups. Each group provides the analysis for one drink, and then reports their findings to the class.

Murder or medical mishap? Death on the metabolic ward (46)

Ability to use quantitative reasoning, pathways and transformations of energy and storage

A great case to review glycolysis. Students enjoy the murder mystery.

Glycolysis, fructose metabolism, regulation of blood glucose levels

Comments

A Typical Week in Our Course We structure our entire course around case studies, teaching approximately thirteen cases in a semester or one case per week. In a typical three to four-class session week, students begin a case during the first class session. They complete the case outside of class before the next class session. They participate in class discussion and formative assessment about the case during remaining class sessions in the week. Prior to the first class session, students complete assigned textbook readings and take a reading quiz, consistent with the flipped classroom model (47). During class, the instructor introduces the case and may also provide information that goes beyond the case or points students in a particular direction. The students then work on the case, asking questions as needed while the instructor and Peer Learning Assistants (PLAs, described below) circulate throughout the room addressing questions and checking on progress. The instructor may present additional material as needed. We allow students to use computers, tablets, and smartphones while they work on the case because we want them to access resources from our course management system, the textbook, and reliable internet sites to assist their construction of answers. However, many students don’t recognize that meaningful learning will not occur if they copy answers to cases rather than constructing the answers for themselves. These students will “Google” case answers in lieu of consulting the course resources or textbook or looking for internet sites that help them make sense of the material. To overcome this tendency, we encourage students to primarily rely on the references we provide on the course management system and the textbook. If they cannot find assistance there, we encourage them to consult reputable internet sites such as PubMed and Google Scholar. We remind students that the purpose of the case is meaningful learning - comprehension and application - not just completion and memorization. After in-class work on the case, students finish the case outside of class and submit it electronically before case discussion in subsequent classes. Students’ questions primarily drive case discussion. For example, sometimes we ask students to confer with each other to determine the most troublesome questions from the case and to write those questions on the board. We then focuses discussion on the most problematic of challenging parts of each case.

134

During class discussion, we administer formative assessments through clicker-type systems, such as Top Hat, or random call or by asking for volunteers to respond. Our formative assessments focus on ensuring that students are comprehending the material, achieving the learning objectives, and transferring their learning to similar but novel problems. Two types of questions we have asked during the Bartter’s syndrome case are shown in Figure 2.

Figure 2. Examples of formative assessment. At the end of the week, students have completed a case study, including multiple formative assessments, and we move on to the next case the following week. Structuring and Facilitating Group Work As we described in the background, CBL creates opportunities for interaction among students in the form of group work, defined as students working in small groups on shared tasks to achieve common learning goals (48). We all design and implement group work a little differently to suit our classes and our individual teaching styles. Regardless of how we form and use groups, we all create opportunities for interaction and we require students to submit their case study analyses individually. We find this eliminates many of the group work-related issues such as uneven contributions. Given that we teach large classes and devote the majority of the class time to working on cases, it is helpful to have additional facilitators. We do not have graduate teaching assistants, so we employ PLAs, undergraduate students who have successfully completed the course. Though PLAs enrich the classroom experience, instructors can teach with cases without PLAs, and we have done so in the past. The idea of PLAs was developed at the University of Colorado Boulder in 2003; evidence shows that PLAs can positively influence students’ attitudes toward science and improve their learning gains (49, 50). PLAs also facilitate group discussions by helping to promote student reasoning and criticalthinking skills by asking questions of students rather than offering explanations (50–52). Our PLAs are volunteers. They volunteer for a variety of reasons, which include a desire to teach and mentor students, to receive a strong letter of recommendation from the supervising instructor, to complete the university’s experiential learning requirement, or to review course material in preparation for an exam (e.g., Medical College Admissions Test). Regardless of their reasons, the number of applications for PLA positions increases every semester, suggesting that PLAs find the experience valuable. PLAs are expected to attend each class and help answer students’ questions. Some students find PLAs more approachable than the instructor and appreciate their presence in the classroom. To assist PLAs in gaining a strong grasp of the material, we meet with PLAs once a week to discuss the plan for that week’s classes and discuss the cases themselves. We focus on the most difficult questions in the case and how PLAs can help students address those questions. We 135

encourage PLAs to reflect on how they learned a difficult concept and consider what they can do to help students struggling with that concept. PLAs can also alert the instructor to concepts where students need more help or guidance and make the instructor aware of any knowledge gaps. We constantly remind PLAs that their goal is not to explain things to students but rather to listen to students’ ideas, point them to resources (e.g., from the textbook), ask leading questions, and assist students in making connections. We tell PLAs to reserve explanation for the times when students are truly stuck. In terms of classroom space, a SCALE-UP (Student Centered Active Learning Environment with Upside-Down Pedagogies) (53) room is the ideal room for CBL as well as other forms of active learning. These rooms have circular tables that facilitate peer interaction and walls lined with whiteboards for students to draw structures, create diagrams or figures, or jot down ideas. We offer several sections per year in SCALE-UP classrooms, but we teach numerous sections in traditional classrooms. In these settings, we encourage students to move their desks to work together or to work with a partner sitting directly beside them if desks are not moveable. Assessing Case Studies Students will invest themselves in learning from cases if the assignments are assessed, and in a way that aligns with the case learning objectives. A good assessment approach includes both formative and summative assessment. Summative assessment takes place when instructors administer tests that inform decisions about completed instructional activities, such as exams used to help an instructor assign a grade to a student (54, 55). Formative assessment is a process of collecting evidence about student learning while instruction is still in progress; it is meant to help instructors adjust their instruction or students adjust their learning tactics (54, 55). Summative Assessment In our courses, 50-75% of the students’ grade is based on performance on exams. The exam questions are written based on case learning objectives. We typically ask some exam questions directly related to the cases and questions that require students to apply what they have learned from the cases but do not otherwise resemble the cases (e.g., not the same storyline or datasets). We do not adhere to any one question format, though most of us include both multiple choice and constructedresponse questions. NCCSTS also suggests a case-based test although we have not chosen to do this in order to keep our grading workload manageable (28). Students may initially be uncertain of how to prepare for an exam in a CBL course because they are used to more explicit delivery of information on which they will be tested. Student fears may be allayed by having discussions early in the course in which students are provided with information about the exam as well as sample exam questions. Many students prepare a study guide based on the case learning objectives, which stresses the importance of writing clear and well-aligned learning objectives. Some students take the wording of the learning objectives very literally and may be confused if they do not echo the exact language of the case questions. Some learning objectives may only be for the case itself and will not be used for summative assessment, such as a solving a mystery or making an Excel graph.

136

Formative Assessment We assess student learning of case material during class using a variety of approaches such as clicker questions, minute papers, muddiest point, or drawing tasks. These questions often come directly from the case, but we also include questions that test student’s transfer of the content. To incentivize students’ engagement and completion of cases, and to ensure that students are understanding before they get to the summative exams, we assess case answers, and each of us does this a bit differently. One way is by assessing cases using single questions that require students to synthesize the main concept addressed in a case. These quizzes are administered after the case wrapup. Most of us assess students’ responses on the cases. Grading all of the questions in each case for a group of 50+ students is not practical, so we use a variety of techniques designed to reduce grading workload. Some of us “spot grade” three to five questions per case; students do not know which questions will be graded ahead of time. Another method we use is randomly grading a certain number of students for case question correctness while the rest of the class is graded for case completeness. This is done so that at the end of the semester, all students have had the same number of cases graded for correctness. Alternatively, cases are graded based on the student’s effort on the entire case, or simply for completion. We use simple grading rubrics to speed the grading process. We have tried several approaches based on research that students view comments that accompany a grade as secondary to the grade (56, 57). A study by Butler and Nisan (58) showed students receiving descriptive feedback (without a grade) performed better on subsequent assessments than students receiving either grades without feedback or no feedback. Regardless of our method of case grading, we make sure that students stay motivated to complete cases and we assess their learning before giving summative assessments. With all coursework, students must strictly adhere to the College or University’s honor code. Students may be tempted to copy and paste answers from other students (current or former) or from the internet. There are several websites where students upload exams, quizzes, lecture notes, and even case studies. Students may be able to view this material for free or for a monthly paid subscription. Students may also gain access to these materials once they upload course-related documents. These websites state that the uploaded materials must not be violation of any honesty policy, but the websites are easily misused. In order to minimize student’s cheating on case studies, we are clear about what is allowed and what is not allowed. We include clear language on our course syllabi and discuss the importance of academic integrity throughout the semester. We try to give students ample time in class to work on the cases to lessen the temptation to turn to unauthorized sources.

Closing Thoughts We have developed a system for using case-based learning in large classes. We use directed cases, group work, formative and summative assessment, and PLAs to provide multiple opportunities for students to engage in generative processing. The system we present here is the result of collaboration among colleagues, both locally and nationally. Each of us is interested in teaching with case studies, and we appreciate that we can accomplish more as a team to foster student learning while keeping our instructional workload feasible. We close with some of the lessons we have learned about creating productive opportunities for collaboration among instructors. First and foremost, we share all of our instructional materials: cases, PowerPoint presentations, clicker questions, exams, syllabi, course calendars, etc. This has involved local sharing among colleagues in the same department and also sharing with colleagues at other institutions. Shared materials have been an invaluable resource, especially for instructors who are new to CBL. 137

Second, we work collaboratively to write new cases and improve existing cases. Writing a new case takes time, so taking turns writing cases and giving others feedback lightens the workload for all. We rely on existing resources such as the National Center for Case Study Teaching in Science (which has 82 cases categorized under Biochemistry) for case ideas whenever possible. When we write new cases and then share them with colleagues, we grant our colleagues the freedom to adapt the cases to suit their teaching styles, scientific strengths, and students. We are open to critique and work to refine cases or develop new cases based on feedback from colleagues and our students. We are mindful that what works for one class may not work for another, so we maintain multiple versions of each case in our shared materials. Third, we discuss our successes and failures. This includes informal conversations about the challenges we face and ideas for overcoming the challenges. More formally, we discuss data on student learning. For example, we administer pre- and post-assessments (59, 60) that enable us to compare the learning gains within our sections with national datasets. We have found in case study teaching a pedagogy that inspires student interest and motivation, provides opportunities for generative processing, works well in large classes, and fosters community among teaching colleagues. We hope we have inspired you to try CBL in your setting.

References Herreid, C. F. Case studies in science - A novel method of science education. J. Coll. Sci. Teach. 1994, 23, 221–229. 2. Gabel, C. Using Case Studies to Teach Science. In National Association for Research in Science Teaching National Conference; Boston, Massachusetts, 1999. 3. Levin, B. The influence of context in case-based teaching: Personal dilemmas, moral issues or real change in teachers’ thinking? In American Educational Research Association; Chicago, IL, 1997. 4. Lundeberg, M. A.; Levin, B. B.; Harrington, H. Who Learns What from Cases and How: The Research Base for Teaching and Learning with Cases; Lawrence Erlbaum Associates, Inc.: Mahwah, NJ, 1999. 5. Dinan, F. Chemistry by the case. J. Coll. Sci. Teach. 2002, 32, 36–41. 6. Fasko, D. Case studies and method in teaching and learning. In Society of Educators and Scholars; Louisville, Kentucky, 2003. 7. Yadav, A.; Lundeberg, M. A.; DeSchryver, M.; Dirkin, K.; Schiller, N. A.; Maier, K.; Herreid, C. F. Teaching science with case studies: A national survey of faculty perceptions of the benefits and challenges of using cases. J. Coll. Sci. Teach. 2007, 37, 34–38. 8. Yadav, A.; Vinh, M.; Shaver, G. M.; Meckl, P.; Firebaugh, S. Case-based instruction: Improving students’ conceptual understanding through cases in a mechanical engineering course. J. Res. Sci. Teach. 2014, 51, 659–677. 9. Cornely, K. The use of case studies in an undergraduate biochemistry course. J. Chem. Educ. 1998, 75, 475–478. 10. Higgins, S. J.; Turner, A.; Wood, E. J. Biochemistry for the Medical Sciences: An Integrated Case Approach; J. Wiley: New York, 1994. 11. Kulak, V.; Newton, G. A guide to using case-based learning in biochemistry education. Biochem. Mol. Biol. Educ. 2014, 42, 457–473. 1.

138

12. Borrego, M.; Cutler, S.; Prince, M.; Henderson, C.; Froyd, J. E. Fidelity of implementation of research‐based instructional strategies (RBIS) in engineering science courses. J. Eng. Educ. 2013, 102, 394–425. 13. Tewari, A.; Ali, A.; O’Donnell, A.; Butt, M. S. Weight loss and 2,4-dinitrophenol poisoning. Br. J. Anaesth. 2009, 102, 566–567. 14. Hidi, S.; Renninger, K. A. The four-phase model of interest development. Educ. Psychol. 2006, 41, 111–127. 15. Renninger, K. A.; Hidi, S. E. The Power of Interest for Motivation and Engagement; Routledge: New York, NY, 2016. 16. Wittrock, M. C. Generative processes of comprehension. Educ. Psychol. 1989, 24, 345–376. 17. Fiorella, L.; Mayer, R. E. Eight ways to promote generative learning. Educ. Psychol. Rev. 2016, 28, 717–741. 18. Wittrock, M. C. Learning as a generative process. Educ. Psychol. 1974, 11, 87–95. 19. Piaget, J. The Language and Thought of the Child; Kegan Paul, Trench, Trubner and Company: London, 1926. 20. Katona, G. Organizing and Memorizing; Columbia University Press: New York, NY, 1940. 21. Chi, M. T.; Wylie, R. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educ. Psychol. 2014, 49, 219–243. 22. Bartlett, F. C. Remembering: An Experimental and Social Study; Cambridge University Press: Cambridge, England, 1932. 23. Bransford, J. D.; Franks, J. J. The abstraction of linguistic ideas. Cognitive Psychol. 1971, 2, 331–350. 24. Wittrock, M. C. Generative processes of the brain. Educ. Psychol. 1992, 27, 531–541. 25. Mayer, R. E. Research-based principles for designing multimedia instruction. In Applying Science of Learning in Education: Infusing Psychological Science into the Curriculum; Benassi, V. A., Overson, C. E., Hakala, C. M., Eds.; Division 2, American Psychological Association, 2014. 26. Mayer, R. E. Multimedia Learning, 2nd ed.; Cambridge University Press: New York, 2009. 27. Wertheimer, M. Productive Thinking; Harper & Row: New York, NY, 1959. 28. National Center for Case Study Teaching in Science. http://sciencecases.lib.buffalo.edu/cs/ (accessed June 25, 2019). 29. Cunha, T. D. S.; Heilberg, I. P. Bartter syndrome: causes, diagnosis, and treatment. Int. J. Nephrol. Renovasc. Dis. 2018, 11, 291–301. 30. AAAS. Vision and Change in Undergraduate Biology Education: A Call to Action. Final Report; Washington, DC, 2011. 31. Herreid, C. F. Start with a Story: The Case Study Method of Teaching College Science; NSTA Press, 2007. 32. Cliff, W. H.; Wright, A. W. Directed case study method for teaching human anatomy and physiology. Am. J. Physiol. 1996, 270, S19–28. 33. Offerdahl, E. G.; Arneson, J. B.; Byrne, N. Lighten the load: Scaffolding visual literacy in biochemistry and molecular biology. CBE-Life Sci. Educ. 2017, 16, es1. 34. Quillin, K.; Thomas, S. Drawing-to-learn: A framework for using drawings to promote modelbased reasoning in biology. CBE-Life Sci. Educ. 2015, 14, es2.

139

35. Castro-Alonso, J. C.; Uttal, D. H. In Spatial Ability for University Biology Education; International Conference on Applied Human Factors and Ergonomics; Springer: 2018; pp 283−291. 36. Cornely, K. Biochemistry Cases; John Wiley & Sons: New York, NY, 1999. 37. Higgins, S. J.; Turner, A. J.; Wood, E. J. Biochemistry for the Medical Sciences: An Integrated Case Approach; Longman Scientific & Technical: Harlow, England, 1994. 38. Ludena, R. F. Learning Biochemistry: 100 Case-Oriented Problems; Wiley-Liss, 1995. 39. Serrano, A.; Liebner, J.; Hines, J. K. Cannibalism, kuru, and mad cows: Prion disease as a “Choose-Your-Own-Experiment” case study to simulate scientific inquiry in large lectures. PLos Biol. 2016, 14, e1002351. 40. Sellami, N.; Morris, J. A.; Vemu, S. I Scream for Ice Cream: Lactase Persistence in Humans; National Center for Case Study Teaching in Science. http://sciencecases.lib.buffalo.edu/cs/ collection/detail.asp?case_id=876&id=876 (accessed June 6, 2019). 41. Knabb, M. Why is Patrick Paralyzed?; National Center for Case Study Teaching in Science. http://sciencecases.lib.buffalo.edu/cs/collection/detail.asp?case_id=482&id=482 (accessed June 8, 2019). 42. DeSimone, S. M.; Prud’homme-Genereux, A. Wrestling with Weight Loss: The Dangers of a Weight Loss Drug. National Center for Case Study Teaching in Science. http://sciencecases.lib. buffalo.edu/cs/collection/detail.asp?case_id=563&id=563 (accessed June 8, 2019). 43. Freeman, P. L.; Maki, J. A.; Thoemke, K. R. 2017 Evaluating the quick fix: Weight loss drugs and cellular respiration. CourseSource. https://doi.org/10.24918/cs.2017.17 (accessed June 25, 2019). 44. Beres, N. R. The Mermaid and the Globins: Hemoglobin Function and Regulation; National Center for Case Study Teaching in Science. http://sciencecases.lib.buffalo.edu/cs/collection/detail. asp?case_id=1007&id=1007 (accessed June 20, 2019). 45. Heidemann, M. K. Urquhart, G. A Can of Bull: Do Energy Drinks Really Provide a Source of Energy?; National Center for Case Study Teaching in Science. http://sciencecases.lib.buffalo. edu/cs/collection/detail.asp?case_id=203&id=203 (accessed June 6, 2019). 46. Boury, N. M. Murder or Medical Mishap? Death on the Metabolic Ward; National Center for Case Study Teaching in Science. http://sciencecases.lib.buffalo.edu/cs/collection/detail.asp?case_ id=662&id=662 (accessed June 20, 2019). 47. Berrett, D. How ‘flipping’ the classroom can improve the traditional lecture. The Chronicle of Higher Education, February 19, 2012. https://www.chronicle.com/article/How-Flipping-theClassroom/130857 (accessed Sept 10, 2019). 48. Wilson, K. J.; Brickman, P.; Brame, C. J. Group work. CBE—Life Sci. Educ. 2018, 17, fe1. 49. Otero, V.; Pollock, S.; Finkelstein, N. A physics department’s role in preparing physics teachers: The Colorado learning assistant model. Am. J. Phys. 2010, 78, 1218–1224. 50. Knight, J. K.; Wise, S. B.; Rentsch, J.; Furtak, E. M. Cues matter: Learning assistants influence introductory biology student interactions during clicker-question discussions. CBE—Life Sci. Educ. 2015, 14, ar41. 51. Lewis, S. E.; Lewis, J. E. Seeking effectiveness and equity in a large college chemistry course: An HLM investigation of peer-led guided inquiry. J. Res. Sci. Teach. 2008, 45, 794–811. 52. Quitadamo, I. J.; Brahler, C. J.; Crouch, G. J. Peer-led team learning: A prospective method for increasing critical thinking in undergraduate science courses. Sci Educator 2009, 18, 29–39. 140

53. Lundeberg, M. A.; Kang, H.; Wolter, B.; delMas, R.; Armstrong, N.; Borsari, B.; Boury, N.; Brickman, P.; Hannam, K.; Heinz, C.; Horvath, T.; Knabb, M.; Platt, T.; Rice, N.; Rogers, B.; Sharp, J.; Ribbens, E.; Maier, K. S.; Deschryver, M.; Hagley, R.; Goulet, T.; Herreid, C. F. Context matters: Increasing understanding with interactive clicker case studies. Educ. Tech. Res. Dev. 2011, 59, 645–671. 54. Tanner, K.; Allen, D. Approaches to biology teaching and learning: From assays to assessments—on collecting evidence in science teaching. Cell Biol. Educ. 2004, 3, 69–74. 55. Popham, W. J. Classroom Assessment: What Teachers Need to Know, Seventh Edition; Pearson: Boston, MA, 2014. 56. Brookhart, S. M. How to Give Effective Feedback to Your Students; Association for Supervision and Curriculum Development: Alexandria, VA, 2008. 57. Schinske, J.; Tanner, K. Teaching more by grading less (or differently). CBE—Life Sci. Edud. 2014, 13, 159–166. 58. Butler, R.; Nisan, M. Effects of no feedback, task-related comments, and grades on intrinsic motivation and performance. J. of Educ. Psychol. 1986, 78, 210–216. 59. Villafane, S. M.; Bailey, C. P.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Development and analysis of an instrument to assess student understanding of foundational concepts before biochemistry coursework. Biochem. Mol. Biol. Educ. 2011, 39, 102–109. 60. Villafane, S. M.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Uncovering students’ incorrect ideas about foundational concepts for biochemistry. Chem. Educ. Res. Pract. 2011, 12, 210–218.

141

Chapter 7

Development and Use of CUREs in Biochemistry Joseph J. Provost,* Jessica K. Bell, and John E. Bell Department Chemistry and Biochemistry, University of San Diego, San Diego, California 91977, United States *E-mail: [email protected].

Course-based undergraduate research experience (CURE) is an approach to integrate research into a teaching laboratory. CUREs are a developing pedagogy that broadens access to the high impact practice of research and a novel way to engage students using research as a teaching modality. The creation and use of CUREs are becoming increasingly popular to both engage students and broaden access to research for more students. As the pedagogy of CUREs is relatively young, the definitions and practices defining a CURE and how CUREs are implemented are highly diverse. This chapter will expand on the history and evolution of CUREs, and highlight what we have learned on the student learning gains, discuss assessment. The faculty practitioner is the primary audience for this work; the instructor in the trenches, whether at a two-year community college, a well-funded PUI, an underfunded school, or a sizeable land-grant research institution. We will examine the details necessary to understand the results of educational research and provide a framework to create, adopt, or develop a CURE. This chapter will be a resource for adopters as well as inform educational researchers.

Why Undergraduate Research and CUREs Can you remember the first time you had an interesting research question or idea ending in an exciting experiment? Is the reason you are passionate about science because the process of science involves bouncing ideas around with a group of colleagues, deeply reading literature to create a hypothesis or the design of an experiment? Are you excited by science because of the joy of getting results from a challenging assay? Was it was holding up that Western blot that finally worked, the PCR reaction that amplified or synthesis of an elusive compound. Do you recall that feeling when, for the first time, you held data in your hands that only you knew and was something of real interest for your scientific community? For most scientists, this is why we do what we do. Reading about science is not as motivating or enriching as is doing science; much like reading about being a concert pianist does not make one able to play Chopin. Providing a practical experience of being a scientist, rather than reading about science for our students makes perfect sense. CUREs are part of a continuum of © 2019 American Chemical Society

how we integrate research, discovery and other elements of the practice of science into our teaching laboratories and is one way we can help provide this experience for a large group of students. The impact of research on student learning naturally brings about the memory of working in a mentored research laboratory as an apprentice, now defined as an undergraduate research experience (URE). The influence on undergraduates involved in UREs has been studied with the most significant impact on the motivation and persistence of STEM students (1). Weaver et al. described the benefits of integrating research- into courses for undergraduates as similar to those seen with UREs (2). More specifically, the involvement of undergraduates in research promotes how students think and act as scientists, bolsters their feelings of belonging, and improves their confidence in STEM (3–6) Students rated the effectiveness and benefits of an undergraduate research experience using the Survey of Undergraduate Research Experiences (SURE) and reported a higher likelihood of persistence in STEM and independence (7, 8). A longitudinal study of over 2000 students as freshmen and again as seniors show that like the Lopatto work, student involvement in a URE is more likely to plan to pursue graduate or professional degrees in STEM by 14 to 17% (9). Another study by Peteroy-Kelly et al. used three validated instruments including the Genetics Concept Assessment in addition to the Classroom Undergraduate Research Experience (CURE) survey to demonstrate increases in the understanding of concepts and attitudinal gains for students involved in a year-long CURE in genetics and cellular and molecular biology laboratory (10). Others (11, 12). describe in stronger terms how CUREs help a student realize an increase in interest and motivation for and to continue in science, and increase in cognitive gains, especially for learning the scientific process. Detailed planning and careful analysis of the project found that the CURE enabled both high and poor-performing students to make more significant gains in conceptual understanding. The CourseBased Undergraduate Research Experience (CURE) also supported student increases in understanding, and participation in, the scientific process, including reading literature, analyzing data, communicating results and other activities required for success in STEM. The authors conclude that the work “suggests that CURE experiences do indeed lead to learning gains…” (10). At this time, there are only a limited number of robust studies that identify the causal mechanism for URE outcomes. However, there exists a large body of work, some referenced here, that provides ample support for UREs as a means to improve increases in graduation rates, retention and a stronger understanding of scientific processes including data analysis and the scientific process (6). Voices of the students: “[The CURE course] helped me realize that I wanted to become a research scientist, and I am now pursuing a Ph.D. in developmental biology at Stanford University. Probably the most valuable lesson I learned from this course stems from the fact that the outcomes of our experiments were unknown. This taught me the data are the data, and you cannot make data fit a hypothesis you like if they do not”. Graduate Student and former CURE undergraduate at Oxford College (13). Wei and Woodin (14, 15). describe and encourage the incorporation of an undergraduate research experience outside of the apprentice URE model. Recognizing the need to broaden access for research, Woodin describes a variety of approaches, including CUREs, to meet these needs. Supporting the call from both Vision and Change and President’s Council of Advisors on Science and Technology to increase access to research experiences, there is a growing number of faculty who have developed individual CUREs to provide these important research experiences in their courses. Others which will be discussed here, have clustered to create larger-supported systematic CUREs 144

(GEP, SEA Phages, and others) to meet these needs. However, like UREs, the impact on students has to be carefully examined. There are many compelling reasons to broaden the URE experience and by extension the CURE accessibility. As suggested in several studies, UREs and CUREs help to increase the STEM workforce. One benefit of a CURE is to provide the motivation, persistence, and other reported positive gains of UREs to more or even all students, including non-science majors. Overall there is low retention in STEM students, and, for at-risk populations this is an even more significant problem. Increasing the retention rate by providing such experiences would have a beneficial impact on the potential STEM workforce. The NSF reports a 33% retention rate for students in STEM undergraduate degrees, yet predict a 20.4% increase in employment in the biological, environmental life sciences, and 12.75% increase in the physical sciences (16, 17). Another study by the U.S, Education Department indicates that of those who started in a STEM degree program only 52% remained and the attrition was worse for those in community college programs where only 30% of those starting a STEM major (18, 19). Such results translate to a national concern about the ability to meet the demand for trained workers in STEM (20). Overall there has been an increase in STEM-related jobs. In the private sector, there is a predicted 34% increase in employment growth compared to past decade (20). Over 360,200 new physical and life jobs was forecast to be created between 2014 and 2024. Because of low retention rates and a changing demographic of college-bound students, the current rate of STEM degree growth is likely to create a concerning shortfall to meet the national needs. Causing concern is where the U.S. is situated in the global field of science and technology. Simply put, we are outpaced internationally, which impacts the nation in many ways (21). Further complicating the need to meet STEM workforce demands is whether the demographics of that future workforce will mirror those of society. By 2060 the White Non-Hispanic population is predicted represent 44.3% of the US total population followed by Hispanics at 27.5%, and Black or African Americans will represent 15% of the US population. Asian and American Indian / Alaska Native is predicted to represent 9.1 and 1.4% respectively in 2060. The increase in each population describes a decrease by 9.6% for White Non-Hispanic Americans and a relative increase of 40.6% for African American, 93.2% for Hispanic American, and a 37.3% for American Indian (22) Troubling is the proportion of black and other minorities in the physical and life sciences. White students make up 67-68% of these science majors, while Asian students represent 16 and 19% leaving 13-15% of the rest of the majors from the remaining communities.16 The differences of course are amplified when examining the workforce. In 2015, 28% physical sciences and 48% biological science employment were women while Hispanics and Blacks accounted for only 6% and 5% of science and engineering occupations respectively (19). Gender is also a concern for STEM careers. While in psychological sciences, biological sciences, and math women represent 77%, 58.7% and 42.9% of degrees awarded in 2015. Women still are behind in computer science (18%) and engineering (20%; 16, 20). Even with the near even ratio of female to male in biological sciences females who are interested in science more often move into non-diagnosing health practitioner fields instead of STEM workforce or other opportunities (19). Clearly, there must be an intervention to meet the needs of a changing workforce and to meet the needs of our students. We must recognize and address that many students, who cannot afford a summer internship or are not exposed to research depending on the institution and mentorship, will not be exposed to the gains of UREs. The difficulty of access for many students makes creating and providing CUREs for all students a critical initiative in STEM. 145

What Is a CURE? The most straightforward description of a CURE is the integrating of research into the classroom environment. This is the common theme among many of the early CURE or CURE-like courses. To better understand “research,” it is helpful to understand what the STEM community defines as research and how the description of CUREs has evolved. The Council on Undergraduate Research (CUR) defines undergraduate research as “an inquiry or investigation conducted by an undergraduate student that makes an original intellectual or creative contribution to the discipline” (23) while the ACS describes undergraduate research in chemistry as “self-directed experimentation work under the guidance and supervision of a mentor or advisor. Students participate in an ongoing research project and investigate phenomena of interest to them and their advisor”. See Box 1 for an expanded description (24). Historically the driving force for creating CUREs was to broaden access to the positive impact of research (UREs) on students. CUREs are distinct from traditional “cookbook” and inquiry laboratories. Inquiry based experiences have strong support across STEM disciplines and show unique and important student gains. Inquiry-based experiences do share some learning characteristics, including generating their questions, obtaining supportive evidence, and depending on the experience analyzing and connecting the results to the initial scientific question (25, 26). Inquiry shares some features with CUREs. The inquiry cycle includes a process where students are engaged in the scientific process and using literature or prior observations generate a question and hypothesis and then investigate, discuss, and reflect. However, for inquiry-based laboratories, the answer is either known or of limited use to stakeholders outside of the classroom. CUREs programs are distinct from Inquiry laboratories as CURE create opportunities for students to engage in the scientific method on problems where the answer is not known, supporting enhanced student persistence and identity in STEM. Research is the crucial difference between Inquiry and CURE; does anyone know the results of the experiment? Is this project broadly of interest to the scientific community (see Box 2)? If the answer to the first is yes and no to the second, then the activity is an inquiry and not a CURE. While antidotal, the comments by 2018 ACS President Peter Dorhout supports these findings when he writes “The “secret sauce” for the success of CUREs and related practices is that they provide the same benefits to students that a mentored research experience does” (27). Several groups describe the key or minimal components of a CURE. One of the early descriptions came from a poll of faculty engaged in undergraduate research who were asked to define the essential features of a successful undergraduate research project (28). This work was summarized and using organizational psychology leadership theory defined as structural items or those that create the structure for the research project and are more measurable. The responses include: • • • • • • •

Reading literature, Opportunities for students to design and conduct research while exploring creativity. The ability to work both independently and collaboratively (on a team) with peers The establishment of student-faculty mentorship or partnership Ownership of the project by students Careful reproducibility of results combined with mastery of techniques. Oral and written communication.

Focusing on larger size courses (50-500 students) the Australian funded Authentic Large-scale URE (ALURE) group created a consortium of 39 academics serving over 7000 students in creating 146

resources to support development and assessment of these class-based UREs (29, 30). In their final report (31) the four key elements (and the structure) of each element of an ALURE are: • Design and Logistics – What is the research question, what makes this research authentic, what are the student learning goals and how will they be assessed, and what equipment, training and resources might be needed. • Motivation – Why is the instructor implementing the ALURE, what are the overall outcomes and expected challenges, why would another faculty member be interested in the project and what challenges might another colleague encounter. • Support for Students – Recruiting student to the project, student learning objectives, support for students as they experience cognitive and other challenges, and training for teaching assistants. • Evaluation – Who are the stakeholders and how will they use the data, and how will the data be analyzed? The Genomic Education Partnership is a large-scale bioinformatics CURE. Using the SURE survey (student self-reported gains), they found that a key feature of their CURE is that gains on several learning benefit items increased with time spent on the research aspect of the project (32). Gentile, Brenner and Stephens led a study for the National Academies of Sciences Engineering and Medicine on UREs (6). As part of the panel’s conclusions, they encourage undergraduate participation in UREs and recognized that CUREs are part of the spectrum of this engagement. They have a more extensive list of characteristics that defined UREs as a whole. These activities must: • Engage students in research practices including the ability to argue from evidence. • Aim to generate novel information with an emphasis on discovery and innovation or to determine whether recent preliminary results can be replicated. • Focus on significant relevant problems of interest to STEM researchers and in some cases a broader (civic) community. • Emphasize and expect collaboration and teamwork, involve iterative refinement of experimental design, experimental questions or data obtained. • Allow students to master specific research techniques. • Help students engage in reflection about the problems being investigated and the work being undertaken to address those problems. • Require communication of results either through publication or presentations in various STEM venues. • Are structured and guided by a mentor with students assuming increasing ownership of some aspects of the project over time. Of course, these definitions include the recognition that not all URE experiences will have the same intensity and depth. Voices of the students: I realize the need for textbook knowledge coming out of college, but the ability to produce knowledge instead of just taking it in was the best educational experience of my college career. The CURE course I completed dramatically altered my career path, making me passionate about computational biology. I also think the ability as an undergraduate to discuss a rich research experience made me much more marketable as 147

researcher and certainly helped me transition into a highly competitive graduate program. Undergraduate student at Washington University in St. Louis (13).

In 2012, Fukami et al. described the divergence from traditional or cookbook laboratories to “authentic research-based courses” where the research was integrated into the curricula (33). Here, the inquiry format is extended to include “authentic” research as a teaching motif. Incorporating hallmarks of authentic research as defined by the AAAS and the National Academies, Brownell and colleagues define the factors in a yet to be labeled “CURE” as: • • • • •

Development of student-generated research questions whose answers are currently unknown Longitudinal focus on one set of research questions over the length of a course Implementation of experimental designs that are not predetermined Collaboration among peers Presentations by students of results and ideas for future research.

In this study the defining factor for a CURE is authentic research (34). Based on the frequency of the responses, authentic research was determined by either novel questions (if only one theme was mentioned) or the collections of themes that add to the process of science. In this study, biology faculty were asked what the essential components of authentic research experiences in introductory 148

biology classes were. In addition to research, most described elements important for a CURE as experimental design, data collection, and data analysis. Presentation or publication, hypothesis formation, student-generated questions, and new questions were reported at a lower frequency and thus defined as second or third-tier design elements of a CURE.

Another group of experts in UREs and CUREs led by Erin Dolan and others were gathered as part of the Course-based Undergraduate Research Experiences Network or CUREnet (35) and one of their outcomes was to develop a consensus definition for a CURE. Together they created a description of the dimensions that make a CURE unique from other laboratory experiences. These five features (also called dimensions) form both design features and build a framework for a logic model to measure the mechanism of effective learning and outcomes of a CURE. The design features or dimensions include: • Use of science practices: This includes the many activities involved in the scientific process such as researching the literature, designing scientific questions, building a hypothesis, designing approaches to test the hypothesis, creating methods, analyzing data and as done in a research environment “navigating the messiness of real-world data”. Basically, the design and methods of a research project. The level and depth of incorporation of the practices will depend on the length of the CURE and the audience and instructor although the CUREnet group proposes that several of the practices should be included in a CURE to make it unique from other laboratory experiences. • Discovery: The purpose of a research investigation that is not known is the discovery. The ownership can be either student or faculty directed or collaborative in nature, but the outcome must not be known. This is a key distinguishing feature of CUREs from inquiry and mirrors one of critical elements of research in the apprentice model. • Relevant or Important Work to a Broader Community: As described in Box 2 on authentic research, this is the opportunity for students to contribute new knowledge. Separate from the vague definition of authentic or genuine research, in this design feature of a CURE, students conduct work that others will find important. Such an audience could of course be the traditional scientific community via an eventual publication contributing to the larger 149

body of knowledge in their discipline or even something “a report of interest to the local community”. • Collaboration: In addition to the networking skill that develops as students collaborate via teamwork, it is important to recognize that, just as science is not conducted in a vacuum, modern research involves collaboration both within and outside of a research group’s laboratory. Increases in student metacognition takes place when teamwork/collaboration takes place. As described by Auchincloss et. al., collaboration may encourage the students to think and recognize the issues in both their understanding of the project and their reasoning. It would be curious to learn the nature of collaborations that mimic those in today’s research between groups of students or a second scientist (assuming a PI role rather than an instructor) might impact the CURE experience outcomes. • Iteration: Similar to the findings by Elgin’s Genomic Education Partnership where they believe this dimension increases duration (staying in STEM major), and has an important impact on other student outcomes. Iteration means that a CURE must have time for students to fail or repeat experiments, a key part of the scientific process. The Dolan group emphasized that each of these features can provide the context to examine the impact of elements of CUREs using theoretical and empirical evidence. While later we will address assessment, defining these key features can be used to assess how each feature is used in a CURE to either determine its difference from other pedagogies to test their impact on student outcomes. Together these features, when taken as a whole, define CUREs separate from inquiry or traditional cookbook laboratories and are what make a CURE like an internship or apprentice research experience. The simplest definition of a CURE is straight forward as the integration of research into a teaching laboratory. Yet unpacking of the dimensions of a CURE remains frustratingly elusive. Complicating matters is that biology and biochemistry have seemed to embrace and developed many CUREs while other STEM disciplines lag or have a very different approach and definition to integrating research. Terms such as inquiry, discovery, authentic, and others are used which are not delineated from traditional laboratory experiences. As such, we can expect more varied lists of key elements and definitions of CURE as our disciplines continue to evolve. As scientists, there are many unique developmental experiences from undergraduates, through graduate school and eventually becoming practicing scientists. These diverse pathways explain how various groups of faculty/instructors can come up with distinct lists of what a CURE should contain yet emphasize many of the same characteristics of research. The diversity of lists of dimensions, activities, and outcomes, elicits the question “is a universally accepted definition of a CURE possible?” If a CURE is created for internal use and not publication, the answer is “not exactly.” Using evidence-based and peer-reviewed educational research resources to understand the affective dimensions of a CURE is vital in creating a universal definition of a CURE. For assessment and to build a common framework to develop the instruments critical to measure the causal mechanisms for these CUREs, there must be at least some set of features agreed upon by the community. Building from these, we can ask questions that are both appropriate and transferable. The caveat is that these agreed-upon features form the core of what a CURE must minimally contain. From this common base, there is a place to add the individual needs. While there are merits to each of the lists of definitions described here the Auchincloss et al. dimensions of a CURE to distinguish a from inquiry and traditional laboratory experiences are influencing many groups and individuals. The evidence is found both in recent literature and 150

referenced in many poster and oral presentations in biochemistry and biology conferences (35). In our earlier review on CUREs in biochemistry, we adopted the same five dimensions in our description of a CURE (13). These five key dimensions of a CURE, if commonly accepted, can become the structure or framework by which the broader community can measure how important or useful the dimension in a CURE experience in a way that is broadly usable. As Auchincloss et al. explain, “The five dimensions compromise a framework that can be tested empirically by characterizing how a particular dimension is manifested in a program, developing scales to measure the degree or intensity of each dimension, and determining whether the dimensions in part or as a whole are useful for distinguishing CUREs from laboratory learning experiences” (35). We have yet to learn the relative importance for each dimension in achieving student gains. Most CUREs vary in the intensity and approach of how each dimension is employed. Not every CURE will employ every activity of each dimension. Thus, “Using this framework to identify critical elements of CUREs and how they relate (or not) to important student outcomes can inform both the design of CUREs and their placement in a curriculum” (35). There is a gap in our understanding of the relative importance of the CURE dimensions. The CUREnet group recommended gathering empirical evidence to characterize the impact of each dimension and the role various dimensional activities have on CURE outcomes (35). A shining example of a CURE activity lacking empirical evidence to support the impact on student outcomes is hypothesis development. Our opinion is that hypothesis development (one of the activities of the Scientific Practices dimension) is foundational and truly distinguishes a CURE from traditional laboratory experiences, and further has a strong impact on important student outcomes such as identity, ownership, and persistence. How much do the students engage in, and the level of accountability of hypothesis development should be examined for the contribution to the related student outcomes. To address this question, data needs to be collected using suitable validated instruments.

Developing a Hypothesis as Part of “Scientific Practices” Activity Examining the literature, we see that the accountability or involvement of hypothesis generation greatly varies in importance in many CUREs. In some ways, this level of incorporation of hypothesis generation reflects how we as scientists started our projects in graduate school, where many were given a project to develop with an existing hypothesis. While later in our career as a post-doctorate or newly minted faculty, we were able to create our own project and synthesized a truly independent project from an interesting scientific question. The process leading to hypothesis generation impacts how students understand the scientific question, develop methods, and design experiments to test the hypothesis. Such a process is what we do as scientists and should be reflected in a CURE. Box 3 indicates a model of minimal actions and approach to designing hypothesis in a CURE. One approach to incorporate hypothesis development as a key dimensional activity is described by Bell et al (36). In this work, several outlines of mini-curricula designed to guide students through the steps of hypothesis development CURE as student investigates malate dehydrogenase (MDH) in a range of classes from gateway first year classes, to community college courses to capstone senior level courses (36). The focus of the project is one or more his-tagged wild-type MDH to be expressed in bacteria. Students are asked to critically evaluate a feature of MDH using information in the literature and using bioinformatics, develop the hypothesis of the domain’s function, using sitedirected mutagenesis predict the outcome of an amino acid mutation and design experiments to test that hypothesis: 151

Project introduction, review and primary literature review: Presentation of big picture of the research project, provide simple review of structure and non-covalent interactions driving enzyme structure and function. • • • •

Group work dissecting key elements of critical/central MDH paper Given handout with major points about MDH including enzyme function Use think-pair-share to focus on MDH function and reaction Students find 5-10 papers on MDH that might be the foundation of an idea/question/ hypothesis and use mindmap to detail on publication • Students generate literature background with feedback Bioinformatics Tools to Develop a Hypothesis • Students given presentation on developing questions: big picture to detail – build from background to MDH. -

Include specifics on what is a good hypothesis Start mindmap on hypothesis development Small group discussion on potential areas of interest and generate ideas for project.

• Introduce to appropriate bioinformatics. i.e. clustal omega, pubmed, or other database/ software to compare known structure, features, sequences from literature to the clone they will be working on.

152

-

Students will conduct think-pair-share to decide on big picture question and how they are going to construct their bioinformatics approach. Students conduct analysis and further develop scientific question and hypothesis along with mindmap of evolving project Students share sample bioinformatics evaluation (e.g. clustal alignment) and discussion of conclusions drawn from it

Molecular Visualization to Refine Hypothesis • In-class presentation review on what is a good hypothesis emphasizing the fact that it must make testable predictions. • Hypothesis review leads to a discussion of non-covalent interactions and their roles in protein structure-function and the role they take in enzyme activity • Students think-pair-share and develop mini-mindmap to identify the amino acid(s) important to their hypothesis • Students conduct molecular visualization of the structure (Pymol or other software) workshop to make images of residues they are interested in that show what types of interactions the amino acids may engage with (other parts of the protein, substrate, cofactor, etc.…). • Students are challenged to start developing hypothesis detail – how do they think the residue(s) of interest interact with other parts of the protein, the substrate, etc. From Hypothesis to Predictions and Experiments • Student presentation of hypothesis. Must include background, amino acid sequence alignment, 3D structure of active site and cofactor binding site, and reasoning for the proposed mutation. • Mini-presentation (by faculty) on experimental approaches. Briefly review key aspects of available experimental approaches. The goal is to connect the student’s research question/ hypothesis to experimental approaches. • Small group discussion on proposed mutations, think-pair-share activity to allow students to further develop ideas of experiments to test hypothesis • Small student group meeting with faculty instructor to discuss planned mutations, the design of primers and provide guidance moving forward with their project Study Section: Peer Review & Critique of Proposals • Students participate in a “panel review” session of their draft proposals. • Student Scribe summarize strong and weak points of all proposal raised during discussions and “panel” rank proposals. Finalize Proposal • Students revise their own proposals having reviewed feedback from panel, presentation questions and faculty feedback. • Open time for students to ask follow-up questions • Final report submitted. 153

This is a thorough approach to guide students through the process of creating and designing a hypothesis and develop an experimental design approach based on the input to hypothesis generation. For those conducting non-protein/enzyme CUREs the concepts of active learning, background investigation, and the development of a hypothesis to drive experimental design could be fashioned from this approach. The example provided here requires five-six laboratory sessions to conduct and is an example of a comprehensive approach performed in a semester-long CURE. Depending on time constraints, an abbreviated version could easily be adapted. A complete description of a hypothesis module is integrated into the MCC CUREs community including lesson plans, goals, and key teaching discussion points specific for a variety of institutions including community colleges and large-research intensive universities and shorter examples of hypothesis development can be found on www.coursesource.org.

How Far Along Are We? Roger’s Bell Curve describes the various stages for adopting (or diffusing) new ideas, innovation, or technology (37). At the outside leading edge of the bell curve lie those who are the enthusiastic creators and most willing to use new ideas. These are the innovators and represent a small (2.5%) population willing to take risks in their careers. As CUREs are relatively new, those faculty who integrated research or adapted from an inquiry mode only a few years ago are our innovators. Individuals represented by the next 13.5% of the curve are considered early adopters who are also risk-takers and as described by Rogers, are more leadership than innovative oriented. Early adopters tend to be influential and along with the innovators are the people that push out ideas into the broader culture. Early majority individuals make up the middle left portion 34% of the bell curve, are willing to create new and innovative ideas, use more caution and work to convince the rest as they are willing to accept new ideas. The late majority and laggards finish the last 34 and 16% of adopters of innovation. They tend to be more traditional in approach and are not as interested in change. Laggards will find a reason to fight against change and only adjust when forced to do so. Voices of the students: [The CURE course] taught me the value of inquiry and how important questing what you know is to furthering your knowledge. I had not participated in research before because I had seen it as something way past my abilities. The lab setup helped me grow as a scientist. Undergraduate student at Oxford College (13). CUREs are in the mid to late Early Adopter mode of innovation diffusion. Historically the bulk of chemistry and biology laboratories have been traditional laboratory experiences where the problem and answers are not in question. Over the last 5-8 years, a real integration of research into teaching undergraduate biology and biochemistry laboratories has taken hold. This wave of focusing on engaging students in the teaching laboratory follows decades-long research into how we engage students in the classroom with active learning or pedagogies of engagement including process-oriented guided learning, case-based learning, flipped classrooms and others. Groups from the innovators are now creating mature CUREs and encouraging the next focus for CUREs (assessment, adoptive measures, sustainability and large scale implementation). However, as we will discuss later, the growth and maturation of CUREs is leading to a bifurcation of literature on CUREs, separating those with advanced educational research expertise and the practitioners (the faculty without specific and in-depth educational research training). 154

The Evolution of CURES An informal analysis of the education literature shows many of the early publications of inquiry started in the mid to late 1990s. In 2001 J.E. Bell called for a close examination of how we integrate interdisciplinary approaches to learning and called for a realistic experience of a research experience in teaching laboratories (38). Early descriptions of incorporating shorter elements of research in a curriculum followed several years later. The journal Biochemistry and Molecular Biology Education’s (BAMBED) early attention to CUREs include an initial call to incorporate research for undergraduates as the barriers from adopting research in the teaching lab, their benefits addressed, and these innovators suggested strategies to overcome the gap of “research-practice” (39). The first BAMBED publication describing the outcome of research and not inquiry in a teaching laboratory was described for a microbiology lab (40). In 2010, Parra et al. described bringing faculty research interest into the teaching laboratory as a modality to strengthen the research-teaching nexus (41). In 2011 J.E. Bell (42) furthered the idea as he described the impact of research on biological thinking and how to develop critical components to adapt the integration of research into the teaching environment. In the Journal of Chemical Education (JCE), inquiry as a method for simulated research has been described as far back as the 1950s with hundreds of publications using inquiry over the past ten years. While the importance of a research experience is emphasized by the American Chemical Society’s Committee on Professional Training (ACS CPT) and described by the ACS CPT’s leadership (43). Yet, even with this history of engaging students using inquiry and research projects, only a handful of publications specifically describe CUREs in the journal JCE. One interesting example published in JCE describes how instructors adapt the concept of a one-semester CURE/URE fusion taught over a one-month dedicated period (January term) using a formal research introductory course combined with a separate URE experience in a formal research lab course (44). It should be noted that the lack of CURE specific, publications in JCE is not entirely reflective how the chemistry community has embraced CURES. One might posit that the lack of ACS JCE publications on CUREs is in part, why the ACS focused literature has not fully embraced the CURE pedagogy in name while some chemists are certainly doing it in practice. Research Corporation for Science Advancement recognized the lack of formally developed CUREs in the physical sciences and created an impressive report of a meeting of experts and Cottrell Scholars. This report highlighted both the barriers to adoption for those in the physical sciences, the definition and assessment for CUREs and described several examples of CUREs being used by the Cottrell Scholars (27). One of the earliest publications on research in the teaching laboratory was published in Cell Biology Education in 2006 (CBE Life Sciences). Here, the impact of summer research in an apprentice model vs. a collaborative learning model where students conducted guided-curricula in a pseudo CURE format was presented (45). Another innovative work was published in 2007 that started students using guided inquiry approach and transitioned students to work on a novel research project (46). Since that time, CBE Life Sciences has published over 50 research articles, review articles and meeting reports describing both systems incorporating CUREs and individual faculty creating and examining the impact of their CUREs. One of the current approaches to encourage the adoption of CUREs is the CUREnet project and website. In its second evolution, the NSF funded CUREnet project created a network of faculty to develop, teach and assess CUREs. The website, CUREnet.org, provides a collection of deposited CUREs and supporting documents to design, operate, and assess CUREs. The CUREnet collection 155

database is searchable with terms of discipline, core competencies, nature of research, state, target audience, duration, and state. Early in its inception, the CUREnet leaders organized a meeting of working group experts to identify and address critical issues in CUREs. The meeting resulted in an influential report (35) helped summarize the current state of CURE assessment and identified gaps in our understanding of CUREs. Much of this work describes the approach and evidence needed to define learning framework and theories needed to understand which elements of CUREs are effective in promoting student gains and provide and advanced pathways by which instructors and design and test their CUREs. Voices of the students: This [CURE] course was different from other lab courses in that I was applying critical thinking skills and laboratory techniques I learned in my upper biology courses. This course has helped me prepare for a future career in pharmaceutical science. Undergraduate student at Georgia State University (13).

Examples of CUREs There are several examples of established national level CURE programs. These programs support the adoption of CUREs for a diverse range of institutions. Three of the most established programs are the HHMI funded Science Education Alliance – Phage Hunters (SEA-PHAGES) targeting first-year life science students, The Genome Consortium for Active Teaching (GCAT) and Genomic Education Partnership (GEP). These are highly successful because of the inclusive and rich set of resources supporting faculty engaging in CUREs. Each have a proscribed approach and a shared scientific theme. Faculty engaged in these national-level CUREs engage in training workshops, provided teaching materials, and a roadmap to easily adopt and reduce the activation barrier to starting a CURE. While the benefits to these large consortiums are many, the constant challenge is sustainability. Once funding ends, the costs to continue providing training and support are significant. Staff and faculty are needed to maintain stocks, publish new works, maintain websites, provide workshops as well as generate and share physical resources. The GCAT project is no longer active due to funding, limiting further adoption of the CURE and use of their tools. Thus how these effective and important programs are sustained will be a challenge. We will briefly discuss each as an introduction to these projects as they are well published and widely recognized. SEA-Phages (www.seaphages.org) This is a centralized, highly structured and well-supported program for first-year students to isolate and analyze the genome of soil bacteriophage to understand the genetic diversity and evolution as they generate data for future publications. Member faculty are provided mentorship and teaching help, access to databases and are encouraged to participate in ongoing symposia and workshops. For their involvement, participating faculty/institutions are expected to follow the guidelines for SEA-Phages CURE (described as authentic discovery-based phage research) curricula that involve a proscribed approach ultimately enriching the GenBank database and potentially cumulating in collaborative research publications. The key to this is the robust support system for adopting faculty. Self-described as a CURE-like, the inclusive Research Education Community (iREC) the participants across over 100 instructions were serving thousands of students. Using the student self-perception of learning gains, first-year students participating in the SEA-Phages program rated as well or better than those who participated in traditional UREs or CUREs (46). They also found that students involved in the project had an increased effect on intent to stay in sciences 156

(persistence) and suggested that this is due in part to project ownership and the agency of identity and community within the project (47, 48). Genomic Education Partnership (GEP; www.gep.wustl.edu). GEP is another, inclusive well-supported community of CURE faculty providing opportunities to upper-division students. GEP students analyze raw sequences and annotate the genome of Drosophila with the ultimate goal to publish on the evolution of the model organism. There is a complete and extensive set of curricula designed to train faculty to incorporate the sequence finishing and annotation with a base or minimal package to a much more extensive set of projects. The current focus is continuing on Drosophila but is expanding to studying the evolution of parasitoid wasps. There is a CourseSource publication describing the genomic training in modular form used to prepare students for independent research (49). The GEP large-scale bioinformatics approach to CUREs is easily implemented with reduced costs as the research does not require wet-lab space or costs to access the database. Several research publications have resulted from these CUREs which include many undergraduate student co-authors. In addition to the benefits of an inclusive centralized program, the GEP project benefits to students are independent of the type of institution which encourages a wide variety of universities to adopt the program and that there is a benefit to student gains when the duration of the CURE is increased (12). Genomic Consortium for Active Teaching (GCAT; www.bio.davidson.edu/gcat). The GCAT is a nationally supported CUREs with a more decentralized or directed approach. The three sub GCAT foci provide online resources as well as materials for faculty wishing to incorporate molecular biology into a CURE. GCAT Chip provided chips/microarrays from a range of species for analysis and assessment. While the chips are no longer provided, there remains a support system for those wishing to adopt the project for their own needs. The GCAT pClone and Synthetic Biology project are approaches in which faculty can use the materials to generate a CURE. While the workshops are no longer funded, the resources and training materials along with instructions to obtain plasmids and other materials are still in place. The pClone system gives the students the ability to use the scientific process to design and own projects asking about the effects of mutations or regulatory proteins on transcriptions using the pClone system (50). A newer and developing semi-centralized community of CURES is developing to fill the need for a protein or biochemistry centered approach. Each of the national projects described so far is genetics and molecular biology oriented. While there are plenty of persuasive examples of individual or small clusters of faculty creating a range of CURE research topics, there has been until recently a lack of more extensive programs. Two NSF funded programs Biochemistry Authentic Scientific Inquiry Lab (BASIL) and the Malate Dehydrogenase CUREs Community (MCC) are beginning to fill this need. Malate Dehydrogenase CUREs Community (MCC; www.mdh-curescommunity.squarespace.com/contact) Another protein-centric CURE project focuses on a single enzyme, malate dehydrogenase (MDH). Building on the experience creating CUREs in their courses and programs (13, 51, 52) and the research experience the MCC provides structure and support for faculty adopting CUREs in entry and advance courses for a range of institutions including community colleges. The CURE centers on MDH because the protein is stable, the enzyme assay straightforward, and inexpensive. Importantly there are many unanswered exciting research questions. Participants in the MCC have 157

access to a large range of His-tagged bacterial expression constructs of MDH from organisms ranging from plant and mammalian, to salt or cold-adapted organisms, parasites, and photosynthetic bacterium. Currently being developed are resources to support adopters including training resources, protocols, learning outcomes and rubrics. MCC CUREs conducted in modular form running either half a semester or a full semester and are all based on a hypothesis development and proposal module. Faculty wishing to focus on kinetics, allosteric properties inhibition or evolution and adaption of MDH will find validated protocols, learning goals, guides, and other resources in the mechanism cluster. Two other clusters supporting CUREs include the protein conformation cluster (structure, function, folding and dynamics) and cellular biochemistry cluster (post-translational modifications, protein-protein interactions, and genetic regulation). The MCC involves a consortium of 16 institutions that are asking pedagogical questions on the duration of a CURE and the impact of collaboration between students. Biochemistry Authentic Scientific Inquiry Lab (BASIL; www.basilbiochem.github.io/basil/ index.html) The BASIL project is intended for biochemistry laboratory adoption, and is amenable to lowerlevel courses and even for use in outreach activities. A collection of affinity tagged plasmids containing the genes for Protein Data Bank entries described as having an unknown function from the starting point for this project. From this database of over 4000 proteins (53) the BASIL group has focused on proteins predicted to be similar in structure or predicted function to a hydrolase. As described, this allows an open-ended research project using an enzyme assay that is reasonable and cost effective. While described as an inquiry lab, BASIL addresses the key dimensions, qualifying the project as a CURE. Using computational tools, students are asked to predict the function and suggest physiological substrates for the selected protein. The scientific process begins with welldefined modules guiding the students through bioinformatics and docking software. Students then use literature and propose a hypothesis and test their CURE. Paul Craig, one of the founders of the BASIL program states “In the CURE setting, we want our students to be fully engaged as scientists, including hypothesis creation, experimental design, data collection and analysis, scientific writing and presentation. To help students move from a “cookbook lab” setting to hypothesis creation, we get them started on the parts of an experiment that are well established, e.g., expression and purification of a protein, then we provide them with tools to explore that protein in the wet lab (molecular weight determination, protein concentration), as well as computational tools and exposure to the literature. We challenge them to identify the things they know and the things they don’t know about the protein and then create a hypothesis about the protein, such as its function or potential binding partners. Then they have to design experiments to confirm or deny that hypothesis”. Support for BASIL faculty members includes assessment, blog posts, online tutorials for the bioinformatics modules, a community of faculty to support the development and entry is inexpensive with a ten clone starter pack for under $50. As the assessment of this project is transferable to other CURE projects, we will discuss the BASIL assessment approach in the assessment section of this chapter. A description of the development of the project has been published and, along with the website, contains a support system for CUREs that should be selfsustainable (54). Other CURE Examples The Freshmen Research Initiative (FRI; www.cns.utexas.edu/fri) is a hybrid research and education program hosted by the College of Natural Sciences at the University of Texas at Austin 158

(55). Since its creation in 2015, Fri has served more than 6,000 students. It annually reaches more than one-third of each natural science freshmen class, including 40% of each cohort coming from underrepresented groups. Faculty create “research streams,” which are ongoing projects for students to engage in a research project. A research stream is a three-course research sequence serving 3540 students. The first course is a research methods class, and the second two courses in the Stream are semester-long CUREs. Faculty Stream mentors are supported by an educational post-doctorate or a PhD-level research educator and funding for the Stream. The program serves between 900 students in 2015 with 27 active research streams. UT Austin Fri advertises over 200 publications have been created with student co-authors in the Fri program. Six different institutions, including Iowa State, University of Maryland and Binghamton University SUNY have replicated the first-year CURE experience. The beauty in this program is the organized approach a university commits to bringing research faculty together with hundreds of freshmen students and supports the organization and structure. Like the outcomes from SEA-Phages, this first-year experience finds that students involved in the CURE program are more likely to graduate with a STEM degree (17% higher 6-year graduation rate in STEM) than those students who do not get the experience in the first year. Also particularly exciting is that the experience has the same impact regardless of race, gender, or firstgeneration status, indicating that first-year CUREs have the potential to truly broaden the outcomes for all students (56). Another structured, systematic approach to bring the research experience and positive outcomes to a broad audience at research-intensive universities include the Center for Authentic Science Practices at Purdue, the Vertically-Integrated Projects Program at Georgia Tech, and CU Boulder. Other examples of multi-university collaborations integrating research in introductory chemistry courses include the Center for Authentic Science Practice in Education (CASPiE (57); and Research Experiences to Enhanced Learning (REEL (58); program. Finally, UCLA has created a scalable framework to give students a supported CURE or URE option in their upper-division in the life sciences. The Competency-based research laboratory curriculum (CRLC) has served over 1000 between 2010 and 2016. Students who elect the CURE path (vs. a sponsored URE path) will take two 10-week laboratories (59). In addition to the system supported CUREs in large universities, community colleges also have several key examples. As part of a $1.1 million grant from the Howard Hughes Medical Institute, Hamline University partnered with two community colleges, Century College and North Hennepin Community College to develop the Engaging Science Students through Investigative Research Program. Through this partnership, biology, biochemistry chemistry, physics, premed, and prehealth science majors have opportunities to participate in research projects under the direction of faculty mentors. These projects are integrated as short two- to four- week modules into existing chemistry and anatomy classes with plans to expand the CUREs into additional courses. There is also a network of community colleges (Community College Undergraduate Research Initiative; www.CCURI.org) that supports the development of CUREs within a growing consortium of community college faculty.

Examples of Various CUREs Implemented across Scientific Disciplines Chemistry At Emory University, the second semester of physical chemistry was converted to a semesterlong CURE experience (60). This CURE focuses on the interaction of uremic toxins with the protein human serum albumin. Students were introduced to the research topic and techniques needed for the 159

CURE over the first four weeks. Students were given the overarching research problem and worked in rotating groups to test and create data that was iterative (repeating data through rotating groups) and worked as a class to analyze and interpret the results. Using the Lopatto CURE survey found similar positive gains across the survey as other CURE experiences. While the authors admit they did not expect these results from an upper-level class when many of the students had already been involved in research, the students reported increased tolerance for obstacles, collaboration, and increased ownership. Gourley and Jones recently published a series of concrete examples of CURE and CURE like activities to “fill the gap between generalized or holistic assessments and individual classroom/ laboratory innovations which can serve as models for adoption” an ACS book (61). An example is self-defined as “collaborative undergraduate research” in the classroom where over 15 weeks, students conduct structure-function protein research conducted in the department of chemistry, University of Wisconsin-Eau Claire (61). This, like much of the other works described by Gourley and Jones, have the essential elements of a CURE. A chemistry CURE reported in JCE utilizes Inorganic Chemistry (62). The University of Vermont created an inorganic chemistry laboratory to fill a set of needs in their inorganic discipline and used the opportunity to provide students with a CURE. The research project centered around creating a catalyst for the dehydrogenation of ammonia borane using transition metals. The course was designed to align with the key components of CURE as described by Auchenloss et al. (35). Using the CURE survey, showed gains in their understanding of the scientific process, ownership, and research activities. Biochemistry and Molecular Biology Familiar to many, lactate dehydrogenase (LDH) is an enzyme that has been used in inquiry labs for many years (63). Ayella and Beck converted the experience into a CURE where students generated a hypothesis, learned skills in expression, purification and enzyme assays, performed sitedirected mutagenesis based on their hypothesis and performed independent experiments examining their scientific questions (64). Collaborating between two institutions, private liberal arts McPherson College and the public Wichita State University, the students self-reported the types of gains seen for other CUREs and some students, who were not exposed to research before, continued this as an independent research project. The Peterson group provides another example of individual biochemistry and molecular biology CUREs that use gene expression as a research question (65). This research focuses on a sigma factor, RpoS, which regulates RNA polymerase in Escherichia coli and is itself controlled at many different levels. Three universities, Suffolk University, Wellesley College, and DeSales University, coordinate to create a 9-week or a 5-week CURE. Here, students identify genes regulating RpoS, then generate a hypothesis on how the genes might impact RpoS at the transcription or translation level. Using an overexpression screen with an RpoS′-′LacZ reporter, students discover their candidate genes and develop a hypothesis on how the candidate genes affect RpoS by analyzing the domains of the proteins. Finally, they tested their hypothesis with a series of different LacZ reporters. Using a range of internal assessments of learning outcomes and attitudinal surveys, students each showed an increase in learning of specific concepts and ability to analyze specific data, indicating that for this CURE students were able to learn concepts as well as more generalized outcomes. Biology A novel approach is described by Kowalski et.al where three interdisciplinary CUREs from biology, chemical biology, and neurobiology were created around a central research theme (66). 160

Students take one or more of the CURE courses set up in biochemistry, chemical biology or a neurobiology semester-long courses. In the design of this unique CURE, there was an intentional and collaborative element integrated to between each CURE in the various courses. Student learning outcomes of concepts and experimental skills were conducted using an in-house instrument as a pre-post course instrument and used the CURE survey. Five goals were assessed: Generate novel data relating to the common project, develop students’ experimental design and data analysis skills, promote positive attitudes about science and perceptions of learning gains, promote student retention in STEM disciplines, and promote faculty research productivity. Overall there was a significant and measurable increase in both new scientific knowledge and increased faculty involvement. Getting new data and keeping faculty involved is critical at a PUI. Several manuscripts have been generated using the data. In addition to the larger inclusive and system supported CUREs we have provided a list of institutional approaches to broaden access to research and a small sampling of individual-smaller scale CUREs to demonstrate the range of approaches to giving students the critical research experience. A longer list of CURE examples across diverse disciplines can be found in JCE, BAMBED, the Chemical Educator (TCE), Journal of Microbiology and Biology Education, and CBE Life Sci) and a growing list of CURE programs and individual CURE examples can be through the CUREnet website (67).

Pedagogical Research: CURE Learning Outcomes and Assessment “We operate on the principle that undergraduate research is not only the essential component of good teaching and effective learning, but also that research with undergraduate students is in itself the purest form of teaching.” Jim Gentile Past President Research Corporation, Dean Emeritus College Natural and Applied Sciences -Hope College, AAAS Fellow, National Associate-National Academies of Sciences (6). While there is a collective agreement that UREs and specifically CUREs are important in the development of undergraduates, missing is a robust understanding of the mechanism or connection between specific dimensions and activities to outcomes. Much of the published CURES do not examine which part of a CURE leads to an improvement in a given outcome(s)? Thus proper assessment must be conducted. Which type and level of assessment best used to evaluate a CURE depends on the final goal of the faculty involved. There are three reasons and depths of assessment. For those who want to locally (for the investigator’s own measures, a department or informally to report CURE outcomes within an institution) will be much different than the type of assessment needed for publication or (and possibly not different) from the assessment needed to assess student outcomes adding to the comparable data to effectively evaluate the causal effect of CUREs on student outcomes. To better understand how to plan and interpret assessment, one should start with a simple primer. First, are the six types of assessment of learning. 1) Diagnostic assessment or pre-assessment is to measure a students’ strengths, weaknesses, and knowledge before instruction. Such assessment is particularly important when considering what will be measured for an introductory vs. advanced or capstone CUREs. This type of assessment will inform how you will form a CURE and influence the outcomes. 2) Formative assessment is used to measure outcome progress during a project. The goal is to monitor progress to provide feedback and identify progress and gaps. 3) Summative assessment are aimed at measuring student gains at the end of a CURE. An assessment of the extent that a set of outcomes have been reached or the effectiveness of learning. 4) Confirmative assessment is used 161

after a CURE has been running and an examination if the instruction is still a success – sort of an extended version of summative assessment. 5) Norm-referenced assessment to compare a student’s performance against an average and often a national norm. As the SURE and CURE surveys are now closed, but with permission, can be used as a norm-referenced tool, faculty can assess their student’s gains against the national norms. And finally, 6) Criterion-referenced assessment measures a student’s performance against a goal or specific standard. Something they are expected to know. For some, this might be the type of assessment used to measure a specific domain of knowledge and skills. Outcomes – Most simply defined, the benefits of a CURE are the learning outcomes most desired for our students after they complete a CURE. Sometimes the term “learning goal” is used interchangeably with “learning outcomes.” Think of an outcome as a statement that describes the knowledge, skills, or behaviors a student should gain. Outcomes are specific and use active verbs making the outcome clear. These are different from learning goals, which are broad descriptions of what the CURE will accomplish, such as expose students to research methodology. Whereas a learning outcome, those benefits of the CURE such as content knowledge, motivation, understanding the process of science, and persistence in science. The source of CURE outcomes originates from the community of scientists. Some are published (35, 68–70), and others come from the experienced and consensus of practitioners of our collective disciplines. An outcome is measurable using verbs that specify the student gain (behavior, skill, concept, ability…), often using active verbs from Bloom’s Taxonomy. To create a more valid outcome, one must use multiple lines of analysis to be sure of the outcome. For some, this might be a conversation of faculty and educational experts. Irby et al. propose an exciting approach to moving what they propose as anticipated learning outcomes that are written before and without the feedback of what happens after a CURE or project has been completed (64). Using a defined a data-driven approach which involves a close analysis of the process of a proposed outcome, multiple faculty input and review and an alignment check with a review process to mature an outcome to a “verified outcome.” While the authors describe the process for a specific type of outcome, their work demonstrates how careful use of multiple inputs can create appropriate, useful, and assessable outcomes. Following a rigorous evaluation of publications describing CUREs, the Dolan group described a logic model for CURE instruction to organize variables in CUREs. In their model, the Dolan group evaluated outcomes based on the time of participation of CUREs. Early or short-term outcomes (analytical, technical, content skills) that are both more readily assessed with prepared assessment instruments and achieved early in the process of a CURE, medium-term (motivation, collaboration science appreciation sense of belonging to a larger community…) and long-term outcomes (selfauthorship, resilience and grit, persistence, science identity…) that may even need assessment after the CURE is completed (35, 70) Another more applied set of outcomes for CUREs is described by Irby et al. where they use a five-step evaluation method to validate CURE leaning outcomes (71). The next step in the assessment of a CURE program is mapping outcomes to CURE activities and dimensions. Aligning CURE activities and dimensions is critical to extending the understanding of the impact of a CURE, how the implementation of a CURE design is useful and leads to student outcomes while supporting the national dialogue on CUREs. To properly accomplish alignment, we must move beyond the approach of assessing only the outcomes. To measure the mechanism or causal elements of a CURE, educational research experts and discipline based-education research (DBER) encourage practitioners to use social learning theories to build a framework in which appropriate assessment and gains can be understood (6, 27, 35, 72) Practitioners, defined here, are those “faculty and instructors in the trenches without specific graduate training in science education 162

research”. Appropriate theories of student learning and development must be selected to make the connections and then apply existing or create validated assessment instruments to measure the outcome. Fortunately, a number of models describing pathways of dimensions/activates to specific outcomes aligned with potential assessment instruments have been suggested (72). These pathways and assessments apply a range of student and social learning theories, social cognitive theories, and applying epistemological development, several models linking a set of activities from CURE dimensions and outcomes are described for adoption. Learning theories include behaviorism, social learning theory, cognitive learning theory, constructivism, and social constructivism. Starting with various dimensional activities, early, mid- and late outcomes are aligned, and connect the activities to the outcomes. Such analysis helps define what might be done in a CURE, the outcome from that activity, and suggest off the shelf assessment to investigate the mechanism of learning gains. Choosing the right assessment instruments is critical to measure the appropriateness of the dimension or activity in question. The often-cited CURE and SURE surveys are self-reported assessments of students to measure student experiences in research like courses. Unfortunately, while often used, the surveys have been reported as limiting. A meta-analysis of CUREs and UREs published between 2010 and 2015, identified only a small number of studies used validated assessments beyond self-reported gains to determine the gains in conceptual learning or research highlighting a need to identify ways to best design both UREs and CUREs to promote learning (72). Similar results were reported by the National Academies which conclude that research on the efficacy of UREs and CUREs are 1) in the early stages of development, 2) that only a small number of studies have employed research designs that support inferences about causations and 3) call for CUREs to incorporate the types of assessment that move beyond described cases or correlational designs for a single CURE (6). After analysis of published CUREs, educational researchers find the theoretical framework (the principles used to explain, predict, and understand the outcomes of learning) important to ascribe learning outcomes missing (73, 74). Thus it is critically important, for the assessment of CUREs to have meaning to use more sophisticated and appropriate validated tools. Unfortunately only a few validated assessment instruments or “off the shelf” assessments are available. The Laboratory Course Assessment Survey (LCAS (75); uses student perception of the dimensions based on inputs from UREs to characterize if the student’s CURE experience is distinguished from a traditional laboratory. The four-point scaled (Likert like) instrument can be used to link particular elements of CURE design to activities and outcomes. The Experimental Design Ability Tool (EDAT) and the expanded EDAT are open-ended tools to measure student’s ability to design experiments using a simple experimental design prompt in a pre/post-test format (76, 77). The exam is a description and prompts given to students to think about and design a basic experiment. The format is a short essay. For instructor use, can be used as is. For pedagogical use requires training and validation of multiple evaluators to be useful. The EDAT does not measure quantitative skills, and as the content and terminology is general, the instrument works for a range of students. The Extended EDAT (EEDAT) has a more defined set of prompts and grading rubric to measure the same outcomes (77). Another survey using a multi-point response survey is the Project Ownership Survey (78). This survey asks students to rate the level of agreement of intellectual responsibility on a project. The survey investigates three factors, emotion of the experience of the laboratory course, project ownership, and the type of course in a series of questions. The ownership survey is described to help identify design features of experiences that enhance student ownership and is appropriate for full student/faculty use. An interesting instrument intended to measure the development of intent to persist in STEM throughout a summer URE is the Student Integration 163

into STEM Careers and Culture (79). Highlighting several factors that impact student persistence the instrument, with support from an educational expert should be amenable to a CURE. As an exemplar, the study on discovery, iteration, and collaboration of students in CURES used three of these instruments to measure the gains (80). Although many of these instruments are validated, many more are described in the various publications that must be adapted and tested before the instrument can be used for pedagogical research (Box 4).

164

Practical Approaches: Organizing Your CURE There are a number of resources that take different approaches on how to design a CURE including the backward design (88, 89) and a case-by-case approach with examples provided by the Research Corporation (26). Using the practice of backward planning, starting with outcomes is the preferred approach to designing a CURE and its assessment. Education has embraced this approach (88) to design curricula, courses, activities, and now CUREs (89). Another approach to create a CURE is to participate in one of the CUREnet workshops (67). Advice on how to start a CURE that includes a helpful checklist and a description of how to overcome barriers has been described by Bell et al. (13). Here are a few important points to consider when organizing and creating your CURE: • What are the broad goals of your CURE? Will it be an entire course or for a portion of a semester? Is this CURE for entry-level (gateway) courses or an upper-division experience? Are there departmental or institutional goals that need to be considered? • Identify a pilot CURE. Start with a simple pilot that you want to adopt (with the same experimental plan or your own research idea). Allow yourself to establish a proof of principle and gain some practical experience and allow your CURE to evolve over time, rather than make the one perfect large-scale CURE. Be flexible as the project evolves to adjust for scheduling, troubleshooting and the types of chaos that research will bring. • Consider the outcomes already defined in the literature described here and engage your stakeholders to create, review and define your outcomes. • Using one of the models analyzed map the outcome(s) to an activity and dimension. If creating new outcomes use the guidance of Corwin et.al. or work with a DBER or education research specialist (75). • While the effectiveness of hypothesis has yet to be fully examined, inclusion of a hypothesis module. • Define the research goals (experimental) for the students. What are the resources and time needed to ramp to the CURE or time to complete the CURE. Remember that iteration is one of the key dimensions along with duration that have shown to have positive impact on student gains. • Ask what are the skills and backgrounds students bring to the CURE. Will some be more capable with a URE experience? How will you balance the diversity of skills and attitudes brought to the CURE? • Articulate how you will mentor your students through the CURE. Will you have checkpoints, research milestones, expectations, mini-meetings, scientific and development goals to provide students guidance and grades. • Research the final output of student work – oral presentation, poster, final paper. • Define the balance of student responsibilities in the CURE. Hypothesis, method development, decision points, background and literature development. • Identify the training support will students need? Will students need help with skills such as pipetting, making solutions and buffers, statistical analysis, protocol and instrument support. Knowing what the need and providing the resources before the semester starts is a stress reducer. • Develop cohorts of faculty. Work with other faculty as you create your CURE. At the department level, discuss with your peers, review your plans with your chair/head, find a 165

group of like-minded faculty to present your “rational and data” for your CURE. Feedback of this nature is important in the iterative process of backward planning. • Map your research to the class size and outcomes. Will all students do a unique aspect of the project? Will there be overlap or duplication within the course? • Design the assessment. If this is a pilot or a CURE that you don’t plan to share or publish, use the appropriate assessment. If you plan to grow the CURE beyond your institution for grant, publication or sharing with the scientific community, engage a DBER or research educational specialist to guide through the assessment.

ACS and ASBMB Accreditation and CUREs One of the motivations to integrate research into the laboratory is to provide a research experience for a broad audience. The ACS and ASBMB each promote research as part of their accreditation; however, neither has a requirement for research in their accreditation evaluation. The ACS requires 400 hours of laboratory experience beyond general chemistry laboratories. If a program does not meet the 400-hour requirement with the traditional laboratory hours (i.e., organic, biochem, physical chemistry … labs) a program seeking accreditation can count up to 180 hours of the 400 hours of laboratory time to meet the minimum requirements. If research is used to count hours, a set of graded research reports must be submitted for evaluation. The ACS Committee of Professional Training (ACS CPT) will not accept laboratory reports for a research write-up. However, in addition to evaluating various aspects of a program, the ACS accreditation process values pedagogical approaches, development of student skills including problem solving, team and communications skills as well as a research experience for the students. A CURE provides evidence of these items for accreditation and renewal. Programs submitting materials with inquiry or CUREs as part of the student experience are viewed favorably. The ASBMB also has a minimum expectation for laboratory hours (400 hours, including general chemistry) and is recommended that at least one of these experiences be research/inquiry-based. The ASBMB accrediting body recognizes that it is difficult for large or small schools to provide all students with a research experience; they do appreciate the experience that CUREs bring to a program and outcomes for students.

Future of CUREs Two concerns need to be addressed: 1) the issues of persistence, and 2) enhancing the diversity of STEM graduates to sustain future STEM workforce needs. CUREs are important in solving these needs. First, we must continue to mature the appropriate assessment of dimensions and activities of CUREs to make their implementation both more effective and efficient. Second, we must continue to create better ways of lowering the energy barriers needed for faculty and programs to increase the broad and early implementation of CUREs in the curricula.

References 1. 2. 3.

Lopatto, D. Undergraduate Research Experiences Support Science Career Decisions and Active Learning. CBE Life Sci. Educ. 2017, 6, 297–306. Weaver, G. C.; Russell, C. B.; Wink, D. J. Inquiry-Based and Research-Based Laboratory Pedagogies in Undergraduate Science. Nat. Chem. Biol. 2008, 4, 577–580. Lopatto, D. Undergraduate Research as a High-Impact Student Experience, Association of American Colleges and Universities. Peer Rev. 2010, 12.

166

4.

5.

6.

7. 8. 9. 10.

11.

12.

13.

14. 15.

Lopatto, D.; Tobias, S.; Council on Undergraduate Research (U.S.); Research Corporation for Science Advancement. Science in Solution: The Impact of Undergraduate Research on Student Learning; Council on Undergraduate Research, 2010. Seymour, E.; Hunter, A.-B.; Laursen, S. L.; DeAntoni, T. Establishing the Benefits of Research Experiences for Undergraduates in the Sciences: First Findings from a Three-Year Study. Sci. Educ. 2004, 88, 493–534. National Academies of Sciences, Engineering, and Medicine. Undergraduate Research Experiences for STEM Students: Successes, Challenges, and Opportunities; The National Academies Press: Washington, DC, 2017. https://doi.org/10.17226/24622. Lopatto, D. Survey of Undergraduate Research Experiences (SURE): First Findings. Cell Biol. Educ. 2004, 3, 270–277. Lopatto, D. Undergraduate Research Experiences Support Science Career Decisions and Active Learning. CBE—Life Sci. Educ. 2007, 6, 297–306. Eagan, M. K.; Hurtado, S.; Chang, M. J.; Garcia, G. A.; Herrera, F. A.; Garibay, J. C. Making a Difference in Science Education. Am. Educ. Res. J. 2013, 50, 683–713. Peteroy-Kelly, M. A.; Marcello, M. R.; Crispo, E.; Buraei, Z.; Strahs, D.; Isaacson, M.; Jaworski, L.; Lopatto, D.; Zuzga, D. Participation in a Year-Long CURE Embedded into Major Core Genetics and Cellular and Molecular Biology Laboratory Courses Results in Gains in Foundational Biological Concepts and Experimental Design Skills by Novice Undergraduate Researchers. J. Microbiol. Biol. Educ. 2017, 18, 1–26. Jordan, T. C.; Burnett, S. H.; Carson, S.; Caruso, S. M.; Clase, K.; DeJong, R. J.; Dennehy, J. J.; Denver, D. R.; Dunbar, D.; Elgin, S. C.; Findley, A. M.; Gissendanner, C. R.; Golebiewska, U. P.; Guild, N.; Hartzog, G. A.; Grillo, W. H.; Hollowell, G. P.; Hughes, L. E.; Johnson, A.; King, R. A.; Lewis, L. O.; Li, W.; Rosenzqeig, F.; Rubin, M. R.; Saha, M. S.; Sandoz, J.; Shaffer, C. D.; Taylor, B.; Temple, L.; Vazquez, E.; Ware, V. C.; Barker, L. P.; Bradley, K. W.; Jacobs-Sera, D.; Pope, W. H.; Russel, D. A.; Cresawn, S. G.; Lopatto, D.; Bailey, C. P.; Hatfull, G. F. A Broadly Implementable Research Course in Phage Discovery and Genomics for First-Year Undergraduate Students. MBio 2014, 5, e01051–13. Shaffer, C. D.; Alvarez, C. J.; Bednarski, A. E.; Dunbar, D.; Goodman, A. L.; Reinke, C.; Rosenwald, A. G.; Wolyniak, M. J.; Bailey, C.; Barnard, D.; Bazinet, C.; Beach, D. L.; Bedard, J. E.; Bhalla, S.; Braverman, J.; Burg, M.; Chandrasekaran, V.; Chung, H. M.; Clase, K.; Dejong, R. J.; Diengelo, J. R.; Du, C.; Eckdahl, T. T.; Eisler, H.; Emerson, J. A.; Frary, A.; Frohlick, D.; Gosser, Y.; Govind, S.; Haberman, A.; Hark, A. T.; Hauser, C.; Hoogewerf, A.; Hoopes, L. L.; Howell, C. E.; Johnson, D.; Jones, C. J.; Kadlec, L.; Kaehler, M.; Silver Key, S. C.; Kleinschmit, A.; Kokan, N. P.; Kopp, O.; Luluck, G.; Leatherman, J.; Lopilato, J.; Mackinnon, C.; Martinez-Cruzado, J. C.; McNeil, G.; Mel, S.; Mistery, H.; Nagengast, A.; Overvoorde, P.; Paetkau, D. W.; Parrish, S.; Peterson, C. N.; Preuss, M.; Reed, L. K.; Revie, D.; Robic, S.; Rocklein-Canfield, J.; Rubin, M. R.; Saville, K.; Schroeder, S.; Sharif, K.; Shaw, M.; Skuse, G.; Smith, C. D.; Smith, M. A.; Smith, S. T.; Spana, E.; Spratt, M.; Sreenivasan, A.; Stamm, J.; Szauter, P.; Thompson, J. S.; Wawersik, M.; Youngblom, J.; Zhou, L.; Mardis, E. R.; Buhler, J.; Leung, W.; Lopatto, D.; Elgin, S. C. A Course-Based Research Experience: How Benefits Change with Increased Investment in Instructional Time. CBE—Life Sci. Educ. 2014, 13, 111–130. Bell, J. K.; Eckdahl, T. T.; Hecht, D. A.; Killion, P. J.; Latzer, J.; Mans, T. L.; Provost, J. J.; Rakus, J. F.; Siebrasse, E. A.; Bell, J. E. CUREs in Biochemistry-Where We Are and Where We Should Go. Biochem. Mol. Biol. Educ. 2017, 45, 7–12. Woodin, T.; Smith, D.; Allen, D. Transforming Undergraduate Biology Education for All Students: An Action Plan for the Twenty-First Century. CBE—Life Sci. Educ. 2009, 8, 271–273. Wei, C. A.; Woodin, T. Undergraduate Research Experiences in Biology: Alternatives to the Apprenticeship Model. CBE—Life Sci. Educ. 2011, 10, 123–131.

167

16. Pew Research Center. Diversity in the STEM Workforce Varies Widely across Jobs. https://www. pewsocialtrends.org/2018/01/09/diversity-in-the-stem-workforce-varies-widely-across-jobs/ (accessed June 8, 2019). 17. Report of a Workshop on Science, Technology, Engineering, and Mathematics (STEM) Workforce Needs for the U.S. Department of Defense and the U.S. Defense Industrial Base; National Academies Press: Washington, DC, 2012. 18. Chen, X. STEM Attrition: College Students’ Paths Into and Out of STEM Fields Statistical Analysis Report. National Center for Education Statistics, [online] Nov 2013, https://nces.ed.gov/pubs2014/ 2014001rev.pdf (accessed Aug 14, 2019). 19. nsf.gov - Women, Minorities, and Persons with Disabilities in Science and Engineering - NCSES - US National Science Foundation (NSF), 2017, https://www.nsf.gov/statistics/2017/nsf17310/ (accessed June 8, 2019). 20. NSF-National Science Foundation. S&E Indicators 2018; https://www.nsf.gov/statistics/2018/ nsb20181/ (accessed June 8, 2019). 21. Human Capital Report 2016-Reports-World Economic Forum; http://reports.weforum.org/humancapital-report-2016/measuring-human-capital/?doing_wp_cron=1486038808. 8636078834533691406250 (accessed June 8, 2019). 22. Vespa, J.; Armstrong, D. M.; Medina, L. Demographic Turning Points for the United States: Population Projections for 2020 to 2060 Population Estimates and Projections Current Population Reports; U.S. Census Bureau, [online] 2018. https://www.census.gov/content/dam/Census/newsroom/press-kits/2018/ jsm/jsm-presentation-pop-projections.pdf (accessed Aug 14, 2019). 23. Characteristics of Excellence in Undergraduate Research (COEUR); Council on Underegraduate Research, [online] 2012. https://www.cur.org/assets/1/23/COEUR_final.pdf (accessed Aug 14, 2019). 24. Undergraduate Research in Chemistry; https://www.acs.org/content/acs/en/education/students/ college/research.html (accessed Aug 14, 2019). 25. Barron, B.-H. L. Teaching for Meaningful Learning: A Review of Research on Inquiry-Based and Cooperative Learning. Book Excerpt. Georg. Lucas Educ. Found. 2008. 26. Lazonder, A. W.; Harmsen, R. Meta-Analysis of Inquiry-Based Learning. Rev. Educ. Res. 2016, 86, 681–718. 27. Waterman, R.; Heemstra, J. Expanding the CURE Model: Course-Based Undergraduate Research Experience. Research Corporation for Science Advancement, [online] June 2018. http://rescorp.org/news/2018/ 06/expanding-the-cure-model-course-based-undergraduate-research-experience (accessed Aug 14, 2019). 28. Lopatto, D. The Essential Features of Undergraduate Research. CUR Quarterly 2003 (March), 139–142. 29. Rowland, S. ALURE Project Undergraduate Research Experience. http://www.alure-project.net/ (accessed June 8, 2019). 30. Rowland, S. L.; Lawrie, G. A.; Behrendorff, J. B. Y. H.; Gillam, E. M. J. Special Section: Innovative Laboratory Exercises-Focus on Australia Is the Undergraduate Research Experience (URE) Always Best? The Power of Choice in a Bifurciated Practical Stream for a Large Introductory Biochemistry Class. BAMED. 2012, 40, 46–42. 31. Rowland, S.; Lawrie, G.; Pedwell, R. Engaging Undergraduate Students in Authentic Science Research. A Large-Scale Approach; HERDSA, 2019; ISBM 9780648550716. 32. Lopatto, D.; Hauser, C.; Jones, C. J.; Paetkau, D.; Chandrasekaran, V.; Dunbar, D.; MacKinnon, C.; Stamm, J.; Alvarez, C.; Barnard, D. A Central Support System Can Facilitate Implementation and Sustainability of a Classroom-Based Undergraduate Research Experience (CURE) in Genomics. CBE—Life Sci. Educ. 2014, 13, 711–723. 33. Fukami, T.; Brownell, S. E.; Kloser, M. J.; Shavelson, R. NSTA Science Store: Undergraduate Biology Lab Courses: Comparing the Impact of Traditionally Based “Cookbook” and Authentic Research-Based Courses on Student Lab Experiences. J. Coll. Sci. Teach. 2012, 1–10.

168

34. Spell, R. M.; Guinan, J. A.; Miller, K. R.; Beck, C. W. Redefining Authentic Research Experiences in Introductory Biology Laboratories and Barriers to Their Implementation. CBE—Life Sci. Educ. 2014, 13, 102–110. 35. Auchincloss, L. C.; Laursen, S. L.; Branchaw, J. L.; Eagan, K.; Graham, M.; Hanauer, D. I.; Lawrie, G.; McLinn, C. M.; Pelaez, N.; Rowland, S.; Towns, M.; Trautmann, N. M.; Varma-Nelson, P.; Weston, T. J.; Dolan, E. L. Assessment of Course-Based Undergraduate Research Experiences: A Meeting Report. CBE—Life Sci. Educ. 2014, 13, 29–40. 36. Mans, T.; Callahan K.; Zhang J.; Bell, J. E.; Bell, J. K. Using Bioinformatics and Molecular Visualization to Develop Sutdent Hypotheses in a Malase Dehydrogenase Oriented CURE. In Press 2019, Course Source. https://www.coursesource.org (accessed Aug 14, 2019). 37. Rogers, E. M. Diffusion of Innovations; Free Press: New York, 2003. 38. Bell, J. E. The Future of Education in the Molecular Life Sciences. Nature Reviews 2001, 2, 221–225. 39. Anderson, T. R. Bridging the Gap Bridging the Educational Research-Teaching Practice Gap. The Power of Assessment. T. BAMBED 2007, 35, 471–477. 40. Rasche, M. E. Laboratory Exercises Outcomes of a Research-Driven Laboratory and Literature Course Designed to Enhance Undergraduate Contributions to Original Research. Biochem. Mol. Biol. Educ. 2004, 32, 101–107. 41. Parra, K. J.; Osgood, M. P.; Pappas, D. L. Laboratory Exercises A Research-Based Laboratory Course Designed to Strengthen the Research-Teaching Nexus. Biochem. Mol. Biol. Educ. 2010, 38, 172–179. 42. Bell, J. E. Educational Issues at National Meetings. Biochem. Mol. Biol. Educ. 2003, 31, 349–349. 43. Wenzel, T. J.; Larive, C. K.; Frederick, K. A. Role of Undergraduate Research in an Excellent and Rigorous Undergraduate Chemistry Curriculum. J. Chem. Educ 2012, 89, 7. 44. Danowitz, A. M.; Brown, R. C.; Jones, C. D.; Diegelman-Parente, A.; Taylor, C. E. A Combination Course and Lab-Based Approach To Teaching Research Skills to Undergraduates. J. Chem. Educ. 2016, 93, 434–438. 45. Frantz, K. J.; DeHaan, R. L.; Demetrikopoulos, M. K.; Carruth, L. L. Routes to Research for Novice Undergraduate Neuroscientists. CBE—Life Sci. Educ. 2006, 5, 175–187. 46. Carson, S. A New Paradigm for Mentored Undergraduate Research in Molecular Microbiology. CBE—Life Sci. Educ. 2007, 6, 343–349. 47. Hanauer, D. I.; Graham, M. J.; Hatfull, G. F. A Measure of College Student Persistence in the Sciences (PITS). CBE—Life Sci. Educ. 2016, 15, ar54. 48. Hanauer, D. I.; Graham, M. J.; Betancur, L.; Bobrownicki, A.; Cresawn, S. G.; Garlena, R. A.; JacobsSera, D.; Kaufmann, N.; Pope, W. H.; Russell, D. A.; Jacobs, W. R., Jr.; Sivanathan, V.; Asai, D. J.; Hatfull, G. F. An Inclusive Research Education Community (IREC): Impact of the SEA-PHAGES Program on Research Outcomes and Student Learning. Proc. Natl. Acad. Sci. 2017, 114, 13531–13536. 49. Laakso, M. M.; Paliulis, L. V.; Croonquist, P.; Derr, B.; Gracheva, E.; Hauser, C.; Howell, C.; Jones, C.; Kagey, J. D.; Kennell, J.; Silver Key, S. C.; Mistry, H.; Robic, S.; Sanford, J.; Santisteban, M.; Small, C.; Spokony, R.; Stamm, J.; Van Stry, M.; Leung, W.; Elgin, S. C. R. An Undergraduate Bioinformatics Curriculum That Teaches Eukaryotic Gene Structure. CourseSource; [online] 2017, 4, https://www. coursesource.org/courses/an-undergraduate-bioinformatics-curriculum-that-teaches-eukaryoticgene-structure (accessed Aug 14, 2019). 50. Eckdahl, T. T.; Campbell, A. M. Using Synthetic Biology and PClone Red for Authentic Research on Promoter Function: Genetics (Analyzing Mutant Promoters). CourseSource; [online] 2015, 2, https://www.coursesource.org/courses/using-synthetic-biology-and-pclone-red-for-authenticresearch-on-promoter-function-genetics (accessed Aug 14, 2019). 51. Knutson, K.; Smith, J.; Wallert, M. A.; Provost, J. J. Laboratory Exercises Bringing the Excitement and Motivation of Research to Students; Using Inquiry and Research-Based Learning in a Year-Long

169

52.

53.

54. 55.

56.

57. 58. 59. 60. 61. 62. 63. 64. 65.

66.

67. 68.

69. 70.

Biochemistry Laboratory. Part I - Guided Inqury. Purification and Characterization of a Fusion Protein: Histadine tag, Malate Dehydrogenase and Green Fluorescent Protein. BAMBED 2010, 38, 317–323. Knutson, K.; Smith, J.; Nichols, P.; Wallert, M. A.; Provost, J. J. Bringing the Excitement and Motivation of Research to Students; Using Inquiry and Research-Based Learning in a Year-Long Biochemistry Laboratory. BAMBED 2010, 38, 324–329. Cormier, C. Y.; Mohr, S. E.; Zuo, D.; Hu, Y.; Rolfs, A.; Kramer, J.; Taycher, E.; Kelley, F.; Fiacco, M.; Turnbull, G. Protein Structure Initiative Material Repository: An Open Shared Public Resource of Structural Genomics Plasmids for the Biological Community. Nucleic Acids Res. 2010, 38 (suppl_1), D743–D749. Craig, P. A. A Survey on Faculty Perspectives on the Transition to a Biochemistry Course-Based Undergraduate Research Experience Laboratory. Biochem. Mol. Biol. Educ. 2017, 45, 426–436. Beckham, J. T.; Simmons, S. L.; Stovall, G. M.; Farre, J. The Freshman Research Initiative as a Model for Addressing Shortages and Disparities in STEM Engagement. In Directions for Mathematics Research Experience for Undergraduates. World Scientific 2015, 181–212. Rodenbusch, S. E.; Hernandez, P. R.; Simmons, S. L.; Dolan, E. L. Early Engagement in CourseBased Research Increases Graduation Rates and Completion of Science, Engineering, and Mathematics Degrees. CBE—Life Sci. Educ. 2016, 15, ar20. Henry, C. M. Getting a Head Start. Programs Introduce Undergraduates to Laboratory Research in their Freshmen Year. C&E News 2005, 83, 39–40. Clark, T. W.; Ricciardo, R.; Weaver, T. Transitioning from Expository Laboratory Experiments to Course-Based Undergraduate Research in General Chemistry. J. Chem. Educ. 2016, 93, 56–63. Sanders, E. R.; Toma, S.; Hirsch, A. M.; Shapiro, C.; Moberg-Parker, J.; Levis-Fitzgerald, M.; Lee, P. Y. Transforming Laboratory Education in the Life Sciences. Microbe Mag. 2016, 11, 69–74. Williams, L. C.; Reddish, M. J. Integrating Primary Research into the Teaching Lab: Benefits and Impacts of a One-Semester CURE for Physical Chemistry. J. Chem. Educ. 2018, 95, 928–938. Best Practices for Supporting and Expanding Undergraduate Research in Chemistry; Gourley, B. L.; Jones, R. M., Eds.; ACS Symposium Series; American Chemical Society: Washington, DC, 2018; Vol. 1275. Pagano, J. K.; Jaworski, L.; Lopatto, D.; Waterman, R. An Inorganic Chemistry Laboratory Course as Research. J. Chem. Educ. 2018, 95, 1520–1525. Ninfa, A. J.; Ballou, D. P.; Benore, M. Fundamental Laboratory Approaches for Biochemistry and Biotechnology; John Wiley & Sons: Hoboken, NJ, 2010. Ayella, A.; Beck, M. R. A Course-Based Undergraduate Research Experience Investigating the Consequences of Nonconserved Mutations in Lactate Dehydrogenase. BAMBED 2018, 46, 285–296. McDonough, J.; Goudsouzian, L. K.; Papaj, A.; Maceli, A. R.; Klepac-Ceraj, V.; Peterson, C. N. Stressing Escherichia Coli to Educate Students about Research: A CURE to Investigate Multiple Levels of Gene Regulation. BAMBED 2017, 45, 449–458. Kowalski, J. R.; Hoops, G. C.; Johnson, R. J. Implementation of a Collaborative Series of ClassroomBased Undergraduate Research Experiences Spanning Chemical Biology, Biochemistry, and Neurobiology. CBE—Life Sci. Educ. 2016, 15, ar55. CUREnet, Course-based Undergraduate Research Experience. https://serc.carleton.edu/curenet/index. html (accessed Aug 20, 2019). Irby, S. M.; Pelaez, N. J.; Anderson, T. R. Anticipated Learning Outcomes for a Biochemistry CourseBased Undergraduate Research Experience Aimed at Predicting Protein Function from Structure: Implications for Assessment Design. BAMED 2018, 46, 478–492. Shortlidge, E. E.; Brownell, S. E. How to Assess Your CURE: A Practical Guide for Instructors of CourseBased Undergraduate Research Experiences. J. Microbiol. Biol. Educ. 2016, 17, 399–408. Corwin, L. A.; Graham, M. J.; Dolan, E. L. Modeling Course-Based Undergraduate Research Experiences: An Agenda for Future Research and Evaluation. CBE—Life Sci. Educ. 2015, 14, es1.

170

71. Irby, S. M.; Pelaez, N. J.; Anderson, T. R. Anticipated Learning Outcomes for a Biochemistry CourseBased Undergraduate Research Experience Aimed at Predicting Protein Function from Structure: Implications for Assessment Design. BAMBED 2018, 46, 478–492. 72. Linn, M. C.; Palmer, E.; Baranger, A.; Gerard, E.; Stone, E. Undergraduate Research Experiences: Impacts and Opportunities. Science 2015, 347, 1261757–1261757. 73. Dolan, E. L. Course-Based Undergraduate Research Experiences: Current Knowledge and Future Directions. http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_177288.pdf (accessed June 4, 2019). 74. Dolan, E. L. Undergraduate Research as Curriculum. BAMBED 2017, 45, 293–298. 75. Corwin, L. A.; Runyon, C.; Robinson, A.; Dolan, E. L. The Laboratory Course Assessment Survey: A Tool to Measure Three Dimensions of Research-Course Design. CBE—Life Sci. Educ. 2015, 14, ar37. 76. Brownell, S. E.; Wenderoth, M. P.; Theobald, R.; Okoroafor, N.; Koval, M.; Freeman, S.; WalcherChevillet, C. L.; Crowe, A. J. How Students Think about Experimental Design: Novel Conceptions Revealed by in-Class Activities. Bioscience 2014, 64, 125–137. 77. Sirum, K.; Humburg, J. ERIC-EJ943887-The Experimental Design Ability Test (EDAT), Bioscene: Journal of College Biology Teaching, 2011-May. Bioscene J. Coll. Biol. Teach. 2011, 37, 8–16. 78. Hanauer, D. I.; Dolan, E. L. The Project Ownership Survey: Measuring Differences in Scientific Inquiry Experiences. CBE—Life Sci. Educ. 2014, 13, 149–158. 79. Hernandez, P. R.; Woodcock, A.; Estrada, M.; Schultz, P. W. Undergraduate Research Experiences Broaden Diversity in the Scientific Workforce. Bioscience 2018, 68, 204–211. 80. Corwin, L. A.; Runyon, C. R.; Ghanem, E.; Sandy, M.; Clark, G.; Palmer, G. C.; Reichler, S.; Rodenbusch, S. E.; Dolan, E. L. Effects of Discovery, Iteration, and Collaboration in Laboratory Courses on Undergraduates’ Research Career Intentions Fully Mediated by Student Ownership. CBE—Life Sci. Educ. 2018, 17, ar20. 81. Boyer, E. L. Scholarship Reconsidered: Priorities of the Professoriate; Carnegie Foundation for the Advancement of Teaching, Princeton University Press: Lawrenceville, NJ, 1990. 82. Moore, J. W. Scholarship in Chemical Education. J. Chem. Educ. 1997, 74, 741. 83. Dolan, E. L. Recent Research in Science Teaching and Learning. CBE—Life Sci. Educ. 2007, 6, 259–259. 84. Dolan, E. L. Grappling with the Literature of Education Research and Practice. CBE—Life Sci. Educ. 2007, 6, 289–296. 85. Bass, R. The Scholarship of Teaching: What’s the Problem? Inventio: Creative thinking about learning and teaching 1999, 1, 1–10. 86. National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. Facilitating Interdisciplinary Research; The National Academies Press: Washington, DC, 2005. https://doi.org/10. 17226/11153. 87. Bush, S. D.; Pelaez, N. J.; Rudd, J. A.; Stevens, M. T.; Williams, K. S.; Allen, D. E.; Tanner, K. D. On Hiring Science Faculty with Education Specialties for Your Science (Not Education) Department. CBE—Life Sci. Educ. 2006, 5, 297–305. 88. McTighe, J.; Wiggins, G. Understanding By Design Framework; Association for Supervision and Curriculum Development: Alexandria, VA, 2012. 89. Cooper, K. M.; Soneral, P. A. G.; Brownell, S. E. Define Your Goals Before You Design a CURE: A Call to Use Backward Design in Planning Course-Based Undergraduate Research Experiences. J. Microbiol. Biol. Educ. 2017, 18, 1–7.

171

Chapter 8

Lab eNotebooks Keri Colabroy*,1 and Jessica K. Bell*,2 1Chemistry Department, Muhlenberg College, 2400 W. Chew Street,

Allentown, Pennsylvania 18104, United States 2Department of Chemistry & Biochemistry, University of San Diego, 5998 Alcalá Park,

San Diego, California 92110, United States

*E-mail: [email protected] and [email protected].

The digital world is omnipresent. In the laboratory classroom, the sweep from analog to digital is nearly complete with the laboratory notebook the final hold out to this transition. With the rapidly growing use of electronic laboratory notebooks (ELNs) in industry and academic research enterprises, instructors are incorporating ELNs into their courses to ensure STEM-based digital literacy that will come to be expected from their students. This chapter will discuss the rise of the electronic laboratory notebook as well as primary considerations when bringing an ELN into the classroom. To provide information on choosing an ELN, both commercial and non-commercial vendors are discussed along with overall strengths and weaknesses of some exemplars. Finally, the impacts on teaching from the instructor and student perspectives are shared as well as the role of ELN usage in training students. The intention is to provide an introduction to ELNs, the variety of implementations available, and challenges and benefits of adopting an ELN into your classroom.

Introduction: Role of a Laboratory Notebook in a Course Lab Notebooks in the Literature – History and Role of Lab Notebooks The stereotypical scientist gazes intently at a sample, brow furrowed in concentration, then turns to scribble down observations in their trusty laboratory notebook. The laboratory notebook might be the most recognizable fixture of scientific discovery. Leonardo da Vinci kept a notebook with observations, sketches and scientific insights (1). The laboratory notebooks of Marie Curie – winner of Nobel Prizes in Chemistry and in Physics - are enshrined at the French national library (Bibliotheque Nationale) (2). NIST has digitized and displayed the actual page of Dan Shechtman’s notebook, winner of the 2011 Nobel Prize in Chemistry, when he first observed 10-fold symmetry in crystals of Aluminum and manganese (3). Oliver Smithies – geneticist, biochemist and Nobel © 2019 American Chemical Society

Laureate – displayed a lifetime of laboratory notebooks in his office (4). The Laboratory notebook is synonymous with both the thrill of discovery and the monotony of quiet persistence. Few objects could so reliably represent the purpose and the process of science, and we inflict laboratory notebook keeping on our students for this very reason. But, times are changing. The world of cloud-computing, digital access and open-source is revolutionizing how we do science and how we teach it. Instead of photographs of handwritten notebook pages, in 2012, the Nobel prize winning Beutler lab (5) shared a publicly available electronic laboratory notebook page, containing their datasets for anyone in the scientific community to download and use (6, 7). A Brief Overview of the Purpose of Notebooks in Laboratory Courses The stereotypical student stares at the vast blankness of their laboratory notebook, enters a bland descriptor of their current experiment, and wonders, “Why am I being tortured with this busy work?” Data. Data must be archived and reproducible. Laboratory notebooks are used to instill the habit and mantra of meticulous protocol writing, careful observation, and thorough, but objective, data interpretation. Foundational knowledge of the scientific process is laid bare by objective, purpose, dependent and independent variables, the two-columned protocols vs. observations, and data analyses. From a pedagogical standpoint, laboratory notebooks are the first formal means of scientific communication we teach our students and, in turn, a student’s first foray into scientific prose. As Bowers wrote in 1926, “The greatest benefit, however, is in the training of the student in the organization of his note material, so as to cultivate scientific thinking and the expression of scientific thoughts (8).” Assessment of this communication, as recommended by the NSTA (9), informs the instructor of the student’s conceptual understanding. More importantly, potential misconceptions of an experiment or the scientific process are disclosed, providing the instructor direct access to students’ progress in the course and topics or techniques ripe for revision. Although much maligned by student and grader alike, the time-honored requirement of laboratory notebooks serves to not only hone a necessary professional skill, but also delivers students' narratives of their scientific thought processes. Overview of Chapter In this chapter, the rise of the electronic laboratory notebook is discussed as well as primary considerations when bringing ELNs into the classroom. To provide information on choosing an ELN, both commercial and non-commercial vendors are discussed along with overall strengths and weaknesses of some exemplars. Finally, the impacts on teaching from the instructor and student perspectives are shared as well as the role of ELN usage in training students. The intention is to provide an introduction to ELNs, the variety of implementations available, and challenges and benefits of adopting an ELN into your classroom.

Electronic Laboratory Notebooks (ELNs): What and Why? Brief Introduction to Development of ELNs from Shared Computer Files or Drives to Formal Electronic Systems That Are Cloud-Based. For every discovery that is memorialized in a laboratory notebook, countless observations go unnoticed, unread or perhaps, even un-recorded! The traditional laboratory notebook was kept in ink, on paper, and in a bound volume. One PhD would require multiple notebooks, and – as in the case of Oliver Smithies – a lifetime of laboratory notebooks could require an entire room to 174

store. But for all its character and charm, the paper laboratory notebook has serious flaws. It can be damaged or lost, the data and observations within unsearchable for anyone who doesn’t want to read from cover to cover, and the very nature of paper makes data sharing nearly impossible. Digitally acquired spectra or other data must be printed onto the page or transcribed, and a separate digital copy maintained. The digital data are separated in form and purpose from the mental workspace where reasoning and insight are supposed to take place. In the teaching laboratory, the principles of notebook keeping are in the forefront. We want students to forecast a purpose for their work, to record observations and data fastidiously, and then analyze those data to produce an insightful conclusion. These are the core habits of science, well planned and well executed. And yet, requiring students to practice these habits in a traditional paper notebook means that in addition to the aforementioned flaws inherent to recording and transcribing digitally generated data, instructors are now victim to student handwriting, and the decline of 21st century-student handwriting is well-documented (10, 11). Personal computing and information systems began to impact the world of laboratory record keeping in the 1980’s and was confined mostly to the pharmaceutical industry (12). But nearly 40 years later, digital data collection, cloud-based computing and online databases are regular parts of the scientist’s work. The first custom electronic laboratory notebooks (ELNs) appeared in the 1990’s, but wider interest and adoption of electronic laboratory notebook platforms began in the early 2000’s after the Food and Drug Administration issued new regulations under which electronic records and signatures could be valued as equivalent to those on paper (13). Due to the special role laboratory notebooks have always played in supporting claims of invention under US Patent Law, the legitimacy of the electronic laboratory notebook was an important requirement for broader inclusion of ELNs in industry settings (13, 14). As the use of ELNs in industry, government and academic labs continues to grow, educators are beginning to incorporate electronic laboratory notebooks into the teaching laboratory. Those implementing ELNs in the classroom have different concerns about and requirements from an ELN compared to those from industrial, government laboratories and even graduate laboratories. The concerns of instructors regarding the teaching laboratory ELN can be summarized as follows: Cost – When available resources do not permit licensing of products at an institutional level, which is often the case, tools that are free to the students are essential. Absent an institutional site license, students must often pay per class to utilize the ELN software. With textbook costs and student anxiety on the rise (15), instructors are loathe to require additional software purchases from students. Collaboration – student work in teaching laboratories is often the result of collaboration, and instructors want tools that make collaboration between students easy. Accessibility – the accessibility of any tool or platform is tied to the concerns of cost; but nonetheless, in the course of a teaching laboratory students are using combinations of mobile devices, laptop and desktop computers to acquire data, record data and analyze data. These different devices come with different operating systems and storage capacities. Instructors want a tool that will be accessible from multiple devices and operating systems and will not enforce unreasonable storage limitations. Control – the teaching laboratory is fundamentally different from a research laboratory because of the relationship between instructor and student. Instructors choose the activities students will do, they assess those students as activities are performed, and ultimately a grade is given. Implicit in the literature surrounding ELNs in the teaching laboratory is the premise that instructors have 175

ultimate control over delivery, structure and access to content. Instructors want to be able to provide frameworks, such as templates or folder structures to guide students. Templates provide prompts or elements that the student should address (e.g. Background, Materials, Safety and Hazardous Waste, etc.) and can be set for a single entry (e.g. Lab 1) or provided as a scaffold for the entire semester. Instructors want to control when students can access content and when content is no longer editable, and they want to control and view student access to content over the duration of the project. Other concerns – Some issues did not appear to be a concern for a majority of instructors. For example, the issue of “digital hygiene” in the ELN was paramount for some (16). To practice good digital hygiene in an ELN, the native file formats from an experiment should be directly attached in their original form to the notebook “page” detailing the experiment. In this way, the data which define an experiment are digitally co-located with the design and analysis of the experiment. Data security is another concern that is particularly relevant in industry, but seems to be less of a concern for many instructors of undergraduate laboratories. Types of Electronic Laboratory Notebooks (ELNs) This section will discuss non-commercial and commercial software that can serve as a platform for an electronic laboratory notebook in a biochemistry lab. Other means to share data and work collaboratively exist such as data repositories, like Zenodo, and developer platforms, like Github, but were not included in this chapter as they lack features that would make them readily adaptable as an ELN in biochemistry. Although the examples discussed in this chapter are not exhaustive, they provide discussion points for features and needs that an instructor should evaluate when adopting an ELN for the biochemistry classroom. A summary of discussed ELNs is given in Table 1. The non-commercial examples were chosen based upon literature reference to their application as an ELN whereas the commercial software examples were chosen based upon targeting to the classroom setting, longevity in the market, free or low-cost options. More comprehensive listings of available software have been reviewed, for example by LIMSWiki.org and the Harvard Biomedical Data Management group (17). Non-commercial ELNs Electronic laboratory notebooks don’t have to be specialized. Implementing a piece of free or readily available software as a teaching laboratory ELN is a viable solution. While some have experimented with blog and wiki platforms for electronic laboratory notebook keeping (18–20), the literature of the teaching laboratory indicates that tools are prioritized which minimize or eliminate cost to the student, enable collaboration and broad accessibility, and allow for robust, instructordriven design and delivery of access and content. To this end, tools which are freely available in some form and have been successfully implemented with these foci as ELNs in the teaching laboratory are discussed this section, they include: Google Drive (21–23), Microsoft OneNote (24) and Evernote (16). Google Drive Google Drive is a collection of cloud-based computing tools including Docs, Sheets and Slides. Google Drive tools are designed to allow multiple users to edit a single file simultaneously, and changes to any item are saved in the revision history by time and by user. Google Drive is freely

176

available with a modest storage limit, but institutional adoption of G Suite for Education (25) means that Google Drive tools and unlimited storage are available to all with an institutional email address. Strengths – As an ELN, Google Drive tools provided seamless integration across applications (i.e. docs, sheets and slides) (23), were familiar to students, and free up to 15GB of storage. Because the free version is so widely used, the learning curve for students is perceived by instructors to be typically small. Indeed some have reported how easily students used the technology (22), and yet, some instructors still experienced improved implementation and less student resistance when they gave a brief introductory lecture on how to use Google Docs as the course ELN. Despite their familiarity with Google Docs, students did not know how to use some of the important features of the Google Doc ELN, such as links, headers, and table of contents (21). Google Drive tools do have apps for all types of mobile devices, and the cloud-based access means that the platform functions independently of operating system. Live integration of photos is possible into a Google Doc from a mobile device camera. Instructor driven design of content requires using folders within Drive and the consensus was that instructor design should take place prior to the start of the course, so the framework of the ELN is in place before students engage with it. Additional features that instructors found useful were the capacity for offline editing (22) and the Table of Contents feature makes navigation within an instructor-designed doc template easier (21). Finally, the signature strength of the Google Drive tools seems to be the robust, simultaneous editing by multiple users. This feature appears to set Google Drive tools apart from other commercial and non-commercial ELNs. Indeed, simultaneous editing was a central element of success when Google Drive tools were implemented in an organic chemistry laboratory exploring reaction optimization for the synthesis of ethyl L-lactate (23). Weaknesses – Control and delivery of content requires more effort on the part of the instructor in a Google ELN. The collection and storage of Docs and Sheets in a drive requires a “master organizer” of content and access that will inevitably be the instructor. There is no prescribed format or template of folder structure which provides seemingly limitless flexibility; and yet, with so many options, it can take a great deal of experimentation to derive a method of organization that works. The ability to share Google Docs, Sheets, etc. with other users enables robust collaboration, but the access by students to instructor created Google Drive materials also allows for the possibility that student will make copies to their own Google Drive account, which is a security risk from the perspective of intellectual property and/or academic integrity. However, this does not appear to be a notable concern of instructors publishing on Google Drive as an ELN tool, and in fact – some assert “no issues with privacy or security (22)”and found it “easier to spot plagiarism and other code of conduct violations (21).” Indeed, keeping track of edits by multiple users is possible, but not robust. Accessibility of Google tools from different devices and platforms is very strong, but at the practical level, integrating content (images, text, data) into a single “notebook” is a challenge. The sharing feature of Google tools and widespread use means that users typically have access to many of Docs, Sheets, folders, etc. Finding the right one at the right moment to record data or insert a photo is not trivial. Furthermore, the editing or annotating of images to highlight important features (much like using a pen to circle a salient data point in a paper notebook) is not as straightforward or intuitive in the Google Doc ELN (21). Finally, backing up the ELN and/or working offline on a computer requires installation of the Backup and Sync applications, which then necessarily imposes additional cost (free usage goes up to 15GB) and limits on storage and accessibility (26). Finally, issues of digital hygiene are a concern for some considering the Google ELN. Data files for an experiment can be “hyperlinked” from elsewhere in Drive into a single Google Doc for an 177

experiment, but this process, while possible, is not intuitive. Furthermore, the data files are only ever “linked”, not embedded in their original file format. OneNote OneNote software is a Microsoft tool, designed from the beginning as note-taking software. Because it is designed for note taking, there are features that feel very much like a traditional paper notebook that has been digitized. For example, OneNote is organized into “notebooks” with three levels of organization: section-lists, subsections, and pages (24), and OneNote was one of the first platforms to integrate free-hand drawing with typewritten text , images, and files on a single “page.” OneNote was originally a locally installed application for a desktop or laptop computer. These local, installed versions of OneNote have existed with the Microsoft Office Suite of tools (including Word, Excel, PowerPoint, etc.) since 2003 (27). However, OneNote now comes in both local and cloudbased versions. OneNote 2016 is the last version of the traditional, local, desktop application that Microsoft is supporting; it is no longer being updated (28). Instead, users are encouraged to choose OneNote for Windows 10 which can be installed on a Mac or PC and on tablet PC’s like the Surface Pro (27). Later versions of OneNote are integrated seamlessly with OneDrive, Microsoft’s cloudstorage platform. OneDrive allows users to access OneNote notebooks on multiple devices. The cost for OneNote is an issue of some confusion. Older versions of OneNote came with the purchase of the Microsoft Office Suite; however, since February of 2015, the Windows and Mac OS X versions of OneNote 2013, 2016 OneNote for Windows 10 have been free for download, regardless of whether or not the user purchased the Microsoft Office Suite (29, 30). Syncing notebooks across platforms and devices does require cloud storage, and the OneDrive cloud-storage platform that interfaces with Microsoft products is free up to 15GB of storage (29). Since some OneNote versions operate exclusively from notebooks stored in the cloud, OneDrive storage is essential for cross-device and multi-platform functionality. OneNote Class Notebook is an add-in to OneNote 2016 and earlier versions that comes standard with OneNote for Windows 10. OneNote Class Notebook allows the instructor to design, deliver, and later update structured content to student users, while also providing options for grading and integration with a Learning Management System (31). OneNote has widespread support for use as an ELN both in terms of peer-review publications (24, 32–35) and in terms of first-hand testimonials, tips and how-to resources provided by scientists blogging on their personal experiences with OneNote as an ELN (36–39). However, the use of OneNote in the teaching laboratory is less well documented in the literature. Some have surveyed and observed students or PhD students use and interactions with OneNote as an ELN, but the context was not an undergraduate teaching laboratory (24, 33). While OneNote with OneDrive have been explored as tools to enhance teaching in the traditional, undergraduate, chemistry classroom (40, 41), and OneNote as an ELN for the teaching laboratory has been proposed (42), its implementation has rarely been studied (34). Strengths – Users of OneNote praise its intuitive interface (34–36), capacity for seamless offline editing and cross-device/platform compatibility and access enabled by integrated syncing with OneDrive (36), seamless integration of freehand drawing, (33), features that enabled collaboration, such as color-coded entries per user and time-date stamped user edits, and the capacity for good digital hygiene – original data files can be readily attached in their native file formats to a single notebook entry and entries can be internally referenced by linking (24, 33, 34, 37). 178

OneNote is the only ELN tool in this category of “non-commercial” that allows for sharing, storage and backup on a private server, a feature which can address data security concerns (33). To accomplish this, locally installed OneNote must be paired with Microsoft SharePoint services, a server-based operating system that can integrate with the Office suite by acting as a host for documents, including OneNote notebooks. Expanding OneNote features using SharePoint does impose additional cost, but it does effectively allow notebooks to be a shared, stored and archived within the environment of a privately-owned server, which has made ELN adoption using OneNote possible in industry (35) and academic (24, 37) lab environments where security of data is a higher priority than is typical for the undergraduate teaching laboratory. When OneNote notebooks are stored via the cloud-based storage tool OneDrive, then sharing of notebooks does not require a private server; however, cloud-based storage does pose some concerns for data security and regulatory compliance (33). In the undergraduate teaching laboratory context, some of the cloudbased storage functions enabled by OneDrive are enhanced with OneNote Class Notebook which allows instructors to control content delivery and access, and for example, given live-updates to a protocol students are using (34). Weaknesses – The salient weakness of OneNote as an ELN is the confusion and compatibility issues created by legacy, local and cloud-based versions of the software, and their use with privateserver and/or cloud-based storage options. User forums report difficulties with cross-platform compatibility (Mac vs. Windows), especially regarding authentication with OneDrive, and crossdevice (computer vs. mobile app) compatibility, authentication and syncing. In some cases, syncing delays have inhibited efforts at collaboration. The use of OneDrive (free usage up to 15GB) or SharePoint imposes additional cost at some point, which is more relevant for instructors that may want to archive and store student ELNs over many years. Perhaps ironically, undergraduate student users testing OneNote in the teaching laboratory did not value the OneNote feature that enables seamless incorporation of free-hand drawing into a notebook entry. Despite the use of stylus and tablet interface, students reported that tasks involving writing – such as diagrams, equations and sketching – were not improved using OneNote when compared to paper notebooks (34). Indeed, as this author has observed, students are much more likely to photograph a paper containing handwritten notes and upload that to the notebook rather than attempting the embedded, digital sketch tools. Evernote Evernote is a web-based tool with robust mobile applications. Evernote was conceived as a piece of Windows software, but it had a relatively small following until it debuted as a free app for the iPhone in 2008. Only three years later, Evernote had over 1 million users, and today the company boasts over 225 million users in countries across the globe (43). Marketed as the app that can “organize your life,” Evernote helps its users to “save everything, sync everything and share everything (44).” As an ELN, Evernote is considered a viable option (32, 45) especially in academic and teaching contexts (13, 16), but has less support in industry and government labs (33). Evernote has free, premium and business plans (46). Paid versions provide additional functionality including PDF annotation, version history, and enhance integration. The business plan accommodates teams of users. Strengths – Users of Evernote ELNs value the easy interface and “minimal learning curve (13, 16),” its cross-device and cross-platform compatibility that enables offline use and syncing (13, 16), mobile and add-on features that enable seamless integration of free-hand drawing, photo and audio 179

capture, and the capacity for good digital hygiene – original data files can be readily attached in their native file formats to a single notebook entry and entries can be internally referenced by linking (13, 16). Instructors like that Evernote supports templates, student access can be controlled by setting permissions and a document scanner is part of the app (16). Evernote also allows for inline, embedded annotation of images, enabling users to highlight important features of dataset, for example (16). Those handwritten notes and photos of chalkboards? Evernote can interpret handwritten text and make it searchable – a signature feature of this tool. Finally, Evernote is the only ELN in this “non-commercial” group that has a live “chat” function that allows for real-time student to student and instructor to student collaboration (16), this feature mitigates some of the limitations created by Evernote’s inability to support simultaneous editing by users sharing a notebook. Weaknesses – The primary weaknesses cited by those considering or using Evernote as an ELN are frequent limitations of the free version. The free version of Evernote is limited to two devices (16) and limited to 60 MB of monthly uploads (13). Tracking revisions by users – a feature valued by some teaching laboratory instructors in courses where students collaborate – also requires the paid Premium Evernote option (16). For some, the inability to embed Microsoft files in any Evernote version in a way that allows for inline editing (33), and the inability of the free version to search attached files including PDFs, Microsoft or other text containing files (33, 36) were reasons to choose against Evernote as an ELN. Finally, while some claim that the encryption options available with the Premium or paid-version of Evernote were sufficient to address concerns of security with cloudbased storage (13), these features were not enough to achieve compatibility with regulatory compliance as viewed by industry users (33). Table 1. A Comparison of Discussed ELN Platforms ELN Tool

Google Drive

Web browser Device accessible & mobile compatibility applications

Storage space

Cost

OneNote Desktop, mobile, web browser accessible applications

LabArchives

RSpace

labfolder

Desktop, Web browser mobile, web accessible and Web browser Web browser browser mobile accessible accessible accessible applications applications

60 MB monthly uploads for free version

Free Per account - community Free: 25 MB; version has Classroom: 1 unlimited GB; storage and Professional: 10 MB limit/ 100 GB upload

Free up to 3 GB and 3 users

Requires $7.99/ OneDrive, cloud storage mo./user for Premium, (15 GB free, $14.99/ 50 GB $1.99/ mo./user for mo., $6.99/ Business mo. for 100 GB)

Community Assess per edition: free. student or Enterprise: through an $100/user/y enterprise or as site license to the license to institution institution

Classroom edition is $26/term, $39/y, or $79/4 y

Cloud storage with Free storage OneDrive up to 15 GB (free up to 15 per user GB), private server storage is an option Cloud storage is free up to 15 GB, $1.99/mo. for 100 GB, $2.99/mo. for 200 GB

Evernote

180

Table 1. (Continued). A Comparison of Discussed ELN Platforms ELN Tool

Google Drive

Sharing & editing privileges

Robust. Synchronous co-editing possible

Possible

Instructor created template

OneNote

Evernote

LabArchives

RSpace

labfolder

Possible

Yes

Yes – formation of groups and classes

Yes – limited ability by PI/ instructor to control sharing

Yes – simple to create & manage

Yes – with OneNote Class Notebook

Yes – shared templates

Yes – “master notebook”

Yes – shared templates

Yes shared templates

Extensive integrations with third party apps and tools

Yes – DropBox, Figshare, Sign and Witness, Messages, Task, Todos, XHTML Export, and Material Database

Integration Embedded with other photo capture tools

Other Microsoft tools

Yes – photo/ audio capture, document Extensive scanning, integrations interprets with third handwritten party apps text. and tools Integration with GDrive, Slack, etc.

Integrated assessment

No

Yes – with OneNote Class Notebook

No

Yes – including LMS integration

Limited - no LMS integration

Some – no LMS integration

Yes – in Google Docs

Yes – sketch only, equation functionality is limited

Yes – sketch only

Yes

Yes

No

No

Embedded equation builders & sketch tools

Synchronous chat

No

No

Yes

No

Integrates with chat platforms, e.g. Slack

Digital hygiene & embedded file editing

No

Yes – some limitations

Yes

Yes

Yes

Yes

ELN support

No

No

No

Yes

Yes

Yes

Commercial ELNs Specialized software for implementing an ELN is readily available. With the advent of Title 21 CFR Part 11 that established the FDA’s regulations on electronic records and signatures in 1997, ELN vendors proliferated to meet the needs of researchers. Originally, commercial ELNs focused 181

on research laboratories, academic and industry. Their products delivered a system to record, search, and share data often coupled to a laboratory information management system (LIMS) that tracked workflow and reagent usage to improve efficiencies in the research enterprise. Cross-fertilization into the teaching laboratories arose from educators wanting to train students on digital standards in industry as well as the graduate-level academic laboratory. With this slow migration from carboncopy paper laboratory notebooks to a digital format, some ELN vendors have started to address the needs of a teaching laboratory in their software features, such as ease of notebook frameworks to distribute to classes, tracking student usage, and grading features. The level of classroom support in ELN products varies. How to choose? Many options are available. The LiMSwiki.org site lists >30 vendors. To identify a product that will serve an individual’s needs, several recent reviews and posts provide comparisons of current products available (32, 47, 48). These listings compare user-friendliness, completeness, connectivity, security, support, and price, but do not specifically address utility of the ELN in a classroom setting. As with all things commercial, there is a cost. A few free options exist, but are usually limited by number of users and/or storage. Pricing structures range from individual to institutional (site) licenses. In this overview, three commercial ELNs are discussed that use different formats, use SaaS (Software as a Service, also known as web-based software) software, and have features compatible with utilizing within a classroom. As web-based software, access through desktop, tablet, or mobile device is possible, but for the purpose of this discussion features are presented as experienced via a desktop/laptop experience. Mobile app availability is noted in Table 1. These are not the only ELNs that have these features, but the goal of this limited overview is to provide a frame of reference to discuss ELN features. LabArchives LabArchives is a cloud-based data management platform that was established in 2009 by founders experienced in the academic publishing world. The ELN is accessible from a web browser on a variety of devices (desktop, tablet, mobile) and through iOS or Android-based apps. For education applications, a Classroom Edition is available and integrates with learning management systems (Canvas, Moodle, BlackBoard) with TurnItIn integration on the horizon. The cost for LabArchives is assessed per student or through an enterprise license to the institution. In either case, instructor(s) and teaching assistant(s) can freely access the software. According to the company website, LabArchives is used by 2,200+ instructors with 317,000+ activated student notebooks. The layout of the notebook is based upon a folder tree. Customizable notebook templates enable the instructor to scaffold a course structure onto the folder tree and push updates to the notebook at any time during the course – this is known as the “master notebook.” The instructor can use one master notebook for all classes/section or customize the master notebook to each section/class. Each notebook incorporates a left hand vertical folder tree “Table of Contents” and a larger right hand window featuring the entry page. On a notebook page, standard entries include text editors (rich or plain), headings, attachments, Office documents, import a CSV file, mathematical equation builder, and sketch. In addition, LabArchives has several unique entry options: The PubMed Reference entry opens a search window linked to PubMed. Students can embed all or a subset of the results onto their notebook page that provides hyperlinks back to PubMed, the publishing journal, and PubMed Central, if applicable. The Widget entry opens a robust palette of utilities including calculators, database templates, and Google-based documents. The Assignment entry allows the instructor to

182

embed a formative assessment with assignment name, points, and instructions. The Classroom version allows 1Gb of data storage per student. Expected features are present such as time-stamping of entries, revision history, and archiving notebooks in PDF format. Instructors can choose to allow students to share their notebooks and this feature can be toggled to adjust for individual versus group work. To provide support for implementation, a knowledge-base, quick-start guides, and video tutorials are available for instructor, teaching assistant, and student. Content building and consultation services to assist in converting existing content or creation of a master notebook are support features available to instructors. The company is responsive to community feedback as noted by Pucinelli et al., “Desired features that were lacking or any concerns were directly communicated to the LabArchives development team and these features were implemented or addressed quickly. An example feature that was missing in LabArchives and requested by the students was the ‘autosave’ feature. The LabArchives development team implemented this feature into the ELN on request (49).” LabArchives has built integration partnerships with other scientific software to offer seamless access to software specific files inside the ELN. Google Docs and Office were mentioned above. Additional partners include, but are not limited to, GraphPad, SnapGene, FlowJo, iChemlabs, and Vernier as well as traditional laboratory education publishers such as OpenStax and Current Protocols. For pay to play software, this assumes the user has a valid license for the partner software. Strengths – LabArchives Classroom Edition was built for the educational setting. As such, course setup features enable intuitive linking of course name, instructor-designed notebook template, and enrolling students. In the student roster, instructors can assess student’s last activity, and link to each student’s notebook, assignments, and comments made by the student. Instructors can also add comments to any notebook entry. These comments are visible to the student, but when the notebook is downloaded as a PDF document, these comments are not a part of the permanent record. As noted above, Assignment entries enable “in notebook” assessments. The feature assigns a task, asks for students to submit work, and provides an entry point for grading. Linking to an LMS further streamlines instructor-student communication. Another strength of LabArchives is the “on the fly” updates that can be pushed out to student notebooks from the instructor’s “Master Notebook.” Who hasn’t had to alter a scheduled lab experiment due to reagent availability, fire alarm, or weatherrelated loss of a lab period? Equally likely, the optimistically ideal notebook structure at the beginning of the semester is not the bastion of perfection three weeks into the course. If students have modified notebook components involved in the update, the student entry will not be changed. This feature alleviates potential conflicts between updating notebook structure and loss of student data. The large number of integration partners is another strength of LabArchives. Although subject to software licensing, the ability to open and edit/update information in primary data files from the ELN rather than downloading/opening/editing/uploading reduces the time needed to maintain a laboratory notebook. This streamlined editing is available for Office documents via Office Online. Outside the ELN, the ability to save files directly to an ELN notebook page from software, for instance, direct saving of SnapGene files from program to ELN, is another time saving feature. The digital hygiene enabled by direct editing of some file types is further supported by the ability to universally “attach” any file type, in its original file format, for later download and analysis by specialized software. When data are embedded as an image, such as a jpeg, primary data analysis is possible with image annotation tools. This feature allows a user to add text, circle or otherwise annotate a feature of the image. The original image is preserved for download, and the annotated image is visible within the field of the notebook entry.

183

Weaknesses – As compared to paper notebooks, students noted the equation builder and sketch features as more difficult tasks (50). The Mathematical Equation Entry Editor requires formulae be entered in the TeX language. Translating an equation into the TeX format is the underlying source of student frustration. Instead, students take a photo of a handwritten equation and upload the photo to their notebook rather than utilize the equation editor. To facilitate equation entry, the editor has preloaded equation components that students can select and then modify the translated code to fit their needs. A “How to TeX” is included in the knowledgebase as well as a link to WikiBooks LaTeX/ Mathematics for additional assistance in entering an equation. For sketches, students noted difficulty in creating quick sketches (50). This comment may be due, in part, to the device dependent utility of the Sketch entry. On a desktop, mouse-enabled free draw line/arrow, straight line/arrow, and defined or free drawn shapes are available with color choice of outline and fill whereas the tablet app allows stylus-enabled (e.g. apple pencil) free draw line sketches. The desktop has more options but the tablet with stylus offers an experience more similar to traditional, paper notebook sketches. RSpace RSpace is an electronic laboratory notebook and document management system provided by Lab-Ally. Initially developed as eCAT in 2003 by a team from the University of Edinburgh to enhance internet-based collaboration and communication at the individual lab level, RSpace scales the eCAT experience to support multiple groups, including large-scale collaboration and group sharing. The software is accessed via a web browser making it compatible across platforms and devices. For educational purposes, a free version of the software is available as the RSpace Community edition. According to the company website, RSpace is used by 100+ universities and 5,000+ researchers. The layout of the notebook consists of a primary page with a horizontal set of tabs to access information. The Workspace tab is the primary entry point for laboratory data. A list of files and folders or a folder tree presents the contents of the notebook. Students can add folders or files such as a Basic Document (rich text editor), or from a Form, Template, Word, or Evernote file to customize their notebook. In addition to standard rich text options, a series of Science Tools (Dilution calculator, PCR Master Mix calculator, etc.) are available as well as an Online Tools section that opens a new web browser tab to links such as additional calculators, PubMed, and ScienceGateway.org. The Gallery tab holds files (documents, images, data files) that can then be uploaded to a notebook page or exported. The Messaging tab provides a communications hub within RSpace. The Apps tab lists integrated software, which currently includes OneDrive, Microsoft Teams, Google Drive, Google Hangouts Chat, box, DropBox, ownCloud, figshare, GitHub, Evernote, slack, Mendeley, Dataverse, Egnyte, ChemAxon, protocols.io, and eCat. Software can be disabled or enabled (with appropriate licenses, as needed). The My RSpace tab lists the user profile and options to create a lab group, view a directory, create a form, manage shared documents, audit notebook activity, and export/import folders and files from/to RSpace. The export feature allows selection of the entire ELN or a subset of entries. The export formats include zipped .HTML (for viewing in a web browser) or .XML (for transfer between RSpace servers), PDF, or Word .DOC (only for single documents). The RSpace Community edition has unlimited data storage, but a 10MB limit on individual uploads. Support for RSpace is provided in the form of video tutorials, written documentation and an FAQ through the company’s website. In the RSpace notebook, a chat feature is also available that directly connects to the RSpace Support team. Strengths – The RSpace interface is simple and intuitive. The layout capitalizes on users experience with tab-based separation of information similar to a web browser interface and universal 184

icons to relay basic information. Within an entry, selections are menu- or icon-driven giving a word processor feel to the environment. Original files such as spreadsheets, images, videos, etc. can be stored in the Gallery, and inserted onto notebook pages as an attachment. Other entries in the notebook, shared documents, or external sites can be referenced through embedded links within an entry. In terms of course design and setup, an instructor can create template pages that convey course information, formats for entries into the notebook, etc. and then disseminate these templates as shared documents. To create a class or section, the LabGroups feature provides a community where the designated PI (e.g. instructor) can invite others (students) to a named LabGroup. In the LabGroup, the designated PI can read the data of all group members, but individual members can also share data (read only or edit privileges) with none, any, or all members of the group. For extended collaboration, each entry can also be shared outside the LabGroup by providing another LabGroup name or email address. Weaknesses –An instructor acting as PI of a LabGroup cannot disseminate a “Master Notebook” to student group members. Rather, individuals choose the format of their notebook, which can only be influenced by instructor guidelines and shared template pages. The instructor also cannot control sharing of data or notebook pages, which is set by individual users. To facilitate assessment of student notebooks, RSpace has limited features, including time-stamped entries, auditing of user activity, and ability to export each group member’s work in an XML format. No options for grading, assignments, or linking to an LMS are available. With respect to features, although files (≤10 MB) can be uploaded to the Gallery space, common formats cannot be edited online (e.g. Office documents in Office online) but are converted to a PDF format for view only or must be downloaded to edit. Similar to LabArchives, the entry of mathematical equations utilizes the TeX language, but, unlike LabArchives, support for equation entry is limited to a link to the LaTeX Wikipedia page. labfolder labfolder is a digital hub for connecting the laboratory ecosystem. The core of the platform is the electronic laboratory notebook, but the system is built to integrate the ELN component with a Laboratory Information Management System (LIMS), and laboratory devices and associated software. Launched in 2012, labfolder was founded in Berlin by two scientists. As a web-based application, the software is accessible across operating systems and devices. Android and iOS-based apps are also available. A free version of the ELN allows up to 3 users to share a notebook with 3GB of storage per user (25 MB limit/file). A classroom edition is also available that is billed per user (instructor, student) at either the term, y, or 4y interval. The ELN is separated into Notebook, Manager, and Dashboard windows. The Notebook landing page has a classic graph paper background onto which entries are added. Entries are collapsible windows that contain a running dialog of the entry blocks that may consist of a rich text editor, sketch, table, uploaded file, or data element. Each window features a ribbon heading (to which the window can be collapsed) that provides pertinent information about the entry such as a title and associated project, created and most recent modified dates, tags (search terms associated with window), and a settings gearbox. Basic features are included in the rich text editor and sketch. The table block inserts a spreadsheet that carries basic Excel-like features including mathematical and logic formulae and graphing capabilities and can be downloaded in an Excel format. Files can be uploaded to an entry. Natively supported formats can be previewed and/or extracted. For example, PDFs can be previewed, Excel and Word can be previewed and extracted, and pixel-based image files (TIFF, PNG, JPEG) can be displayed in-line. Extracted contents are appended to the entry as 185

editable components (text, table, image). Under the Manage window, Project, Templates, Exports, Groups, and Apps are controlled. Groups are built by email invitation and hierarchical groups and sub-groups determine sharing privileges. Apps that integrate with labfolder include DropBox, Figshare, Sign And Witness, Messages, Task, Todos, XHTML Export, and Material Database. Apps can be activated or deactivated. The Dashboard integrates communication (Comments, To Do lists, Tasks, and Messages) within the ELN. Support for labfolder implementation includes video tutorials, a User manual, and online guides. A chat feature within labfolder invites users to submit feedback to the support team and provides a link to an online HelpDesk. A white paper from the company steps instructors through utilizing labfolder in the classroom setting. Strengths - The primary strength of labfolder is the straightforward interface. The notebook entry is a single page that is further simplified by collapsible windows and a selectivity filter to limit displayed entries by project, author, tag, and/or date. Choices in the text editor are streamlined to give an overall impression of simplicity. Links to internal entries open a separate window allowing a user to maintain their current location while accessing previous data/protocols. Blocks within a window can be rearranged by drag-n-drop in a single or 2-column format, which is handy for comparing data. Groups are easy to create and can be altered by “drag-n-drop” rearrangement of group members in a folder tree hierarchy. In terms of course design and deployment, shared template entries can provide a scaffold for student notebooks. Students can maintain an independent project to record notebook entries and/or be part of a group project where all members can submit entries. For a course section, an instructor creates a Group and invites students to join via email. To share between notebooks, students must be in the same group. Sharing is limited by dividing students into subgroups that are populated by a limited number to a single student. The Dashboard in combination with the Tasks app allows an instructor to create assignments for a subset or all students. Using a shared template to delineate the assessment, the instructor can assign a Task and track the student’s status on the Task. Comments made on student ELN entries are consolidated in the Dashboard by date and project. Selecting a comment takes the student to the entry where a reply can be posted. Weaknesses – In delivering a streamlined interface, labfolder gives up some functionality. The must-have options are all available but the nice-to-have components are somewhat limited. No built-in calculators, mathematical equation editor, or inserting mixed media into a stream of text. In terms of classroom use, a “Master Notebook” cannot be scaffolded onto student ELNs although shared templates can guide student notebook entries. To facilitate assessment of student notebooks, auditing of entries and an export feature are available. The company’s White Paper “Practical Lab Courses” offers witnessing a completed entry, commenting on student entries, and utilizing the search feature filtered by tags or structured Data Element blocks to navigate student ELNs as means to evaluate student entries. No options for grading or linking to an LMS are available.

Impact on Teaching and Training Teaching Instructor Perspective Instructors of laboratory classes balance conveying basic techniques and technical skills while continuously updating their courses to keep abreast of advances in their respective fields. The “omics” era has brought BIG DATA to the doorstep of the laboratory class as well as the digitally 186

connected laboratory. To prepare students for expectations in this STEM landscape, digital literacy is a critical component. The transition from PLN to ELN is one part of increasing digital literacy. Reports from faculty making this transition provide insight into the benefits and challenges from the ground floor, especially pertaining to advancing collaboration, communication, and assessment. From the literature, instructor perspectives are as follows: Digital Literacy – The American Library Association’s Task Force on Digital Literacy defines it as the “ability to use information and communication technologies to find, evaluate, create and communicate information, requiring both cognitive and technical skills.” Through ELN implementation, instructors are challenging their students to apply their technological savvy to their science. The American Chemical Society’s Committee on Professional Training has included in their Undergraduate Professional Education guidelines for bachelor degree programs instruction “in data management and archiving, record keeping (electronic and otherwise)… this includes notebooks, data storage, and information management (51).” In some cases, the shift to a paperless lab course ensures that students “gain experience in using electronic means of recording and presenting the results of their experiments (22).” As a part of digital literacy, the ELN provides a platform for students to integrate rich media, instrument use and data processing while realizing the efficiencies afforded by the digital venue. The format lends itself to an electronic portfolio of a student’s laboratory experiences that can be used for formative and summative assessment of a student’s technical skill set, but also by the student as a component of a digital résumé. A challenge, and perhaps aspiration, is to realize the full potential of digitally communicating science (34). Collaboration – In research, rarely do scientists work in isolation, but rather in teams where data are shared and discussed. An advantage of an ELN is the ability to aggregate data via cloud storage as well as sharing of data and subsequent analyses between students or classes (49, 50, 52). As noted by Riley, ELNs easily support data sharing between teams doing distinct, but complementary, experiments that leaves an audit trail (52). As most ELNs provide some control or at least monitoring of shared entries, instructors retain oversight on collaborative processes. Using an ELN has made my life much easier in many ways. The students seem to like it, which means they use it. It’s easy for me to check and comment on their entries. It’s also great to have all their experimental work protocols and data in one easily accessible location- easy to grade and saves lots of time! ~USD faculty using LabArchives in capstone Biochemistry lab. Course preparation – Each course requires an investment of instructor time to prepare materials and resources for students. Implementing an ELN may require an upfront outlay of time for training on the chosen software tool, but subsequent setup of the ELN template did “not add significantly to the overall setup time for the lab (22).” Key benefits of an ELN over a PLN are the capacity to archive data and searching the notebook contents (16, 34) – both of which are only realized after initial input. As more programs move to research-based curricula in their laboratory courses, the ELN’s archiving and search capabilities of data collected over successive terms will support and facilitate these pedagogical changes. Communication – Faculty felt that the ELN facilitated higher quality, immediate, and more frequent communication with their students (16, 52, 53). Bromfield Lee noted that, “Students will often ask questions in their notebooks ahead of time. In this way, students who prepare their notebooks ahead of time can get assistance earlier without sending emails or coming to office hours while directing an instructor specifically to the aspect of their notebook for which they need help (21).” The ability to access a student’s notebook remotely allowed instructors to more easily track a 187

student’s progress and offer guidance or feedback (50). This continuous access to student notebooks simplified consultation and assessment by the instructor.

Table 2. Example ELN Assessment Rubric for Learning Objective: Keep an Accurate Laboratory Notebook That Allows Others to Interpret and Reproduce Reported Experiments Criteria

Excellent

Good

Acceptable

Unacceptable

General Clarity

Writing and organization allows reader to easily follow flow, understand and find information.

Writing and organization allows reader to follow flow, understand and find information with a small amount of effort.

Writing not always clear, or poorer organization requires more effort to follow flow, understand and find information.

Poorly organized, requires significant effort to follow flow, understand and find information.

Required elements present*1 and notebook current.

All required elements All items present but present and up-to-date some elements (e.g. data at submission. analysis/interpretation) incomplete for most recent experiment (within 1-2 lab periods of submission*2).

Significant items absent from experiments performed within 2 – 3 lab periods of submission*2.

Substantial required elements absent or incomplete for experiments >1 week old.

Procedures

Clearly and concisely describe steps of experiment, showing set up calculations, preps (e.g. buffer prep), setup or instrument images and all necessary information to repeat.

Some information absent or incomplete but generally possible for a trained biochemist to attempt to repeat experiment.

Significant omission of information making it difficult for a trained biochemist to repeat experiment with certainty.

A trained biochemist could not repeat the experiment.

Data/ Observations

Complete, related to procedures and necessary information/ images given (e.g. units). Good organization of data in tables or other ways to present clearly. Tables/ Graph/Data embedded or included as attachments.

Complete, but not always related to procedures or some necessary information absent (e.g. units). Tables/Graph/Data embedded or included as attachments.

Not complete (e.g. Figures not embedded or not included as attachments.) or not always related to procedures or significant necessary information absent (e.g. units).

Significant data and observations or necessary information absent.

188

Table 2. (Continued). Example ELN Assessment Rubric for Learning Objective: Keep an Accurate Laboratory Notebook That Allows Others to Interpret and Reproduce Reported Experiments Criteria

Excellent

Good

Acceptable

Unacceptable

Results (Data Analysis)

Calculations and analysis using data are complete and logical. Figures, tables, or other graphics clearly present results or data with titles, figure legends, annotation that add appropriate context and necessary expteriment parameters.

Calculations and analysis complete and logical. Figures, tables, or other graphics clearly present results or data with titles, figure legends, annotation that add appropriate context but lack some minor information necessary to interpret them.

Calculations or analysis not complete. Figures, tables or other graphics present results or data but contain errors or lack significant information necessary to interpret them.

Necessary calculation or analysis absent. Data not presented as figures/tables or other graphics where appropriate.

Discussion/ Conclusion (Data Interpretation)

Logical conclusion(s) based on goals; Methods, observations and interpretations are evaluated and placed into context; thoughts and consideration of broader consequences of result(s).

Logical conclusion(s) based on goals; Methods, observations and interpretations are evaluated but not placed in context or less consideration of broader consequences of result(s).

Only portions of the results are evaluated, or contain unsupported assertions or fewer attempts to consider broader consequences.

Little or no evaluation or interpretations of results are presented. Broader context may be missing.

elements: Each expt: Title, Purpose, Procedures, Data & Observations, Results, Discussion/ Conclusions. *2 Definition of items that may be incomplete and acceptable lab periods covered depends on current lab activities/analysis required and/or any technical issues known to instructor. *1 Required

Assessment, Plagiarism – Two positive benefits loom large in terms of ELN assessment: no interpretation of student handwriting and no physical notebooks to collect or return. On a more serious note, does implementing an ELN alter instructor assessment? The use and maintenance of a laboratory notebook is a technical skill that is essential in the training of a scientist. Use of an ELN doesn’t alter this objective but offers an opportunity to extend a student’s training. Sadly, “going digital” will not automatically result in pristine experimental documentation that could withstand the scrutiny of a patent agent. Rather, clearly communicating expectations of ELN documentation through benchmarks for the student and points of assessment for the instructor will cultivate acceptable experimental documentation. A typical learning objective associated with laboratory documentation may read, “Students who succeed in the course should be able to explain the importance of and keep an accurate laboratory notebook that allows others to interpret and reproduce their experiments.” An ELN would allow the instructor to extend this objective to include principles of digital literacy. For example, are original data and analyses files embedded or included as attachments? Are observations recorded as annotated images? An example rubric for evaluating student ELNs is given in Table 2. The inclusion of the electronic aspect of assessment is subtle as the primary focus of maintaining a laboratory is unchanged. Other benefits noted by faculty included that providing a template meant a consistent layout which facilitated rapid assessment and forced students to include elements that they may not have 189

intuitively included such as formatting or tables (21) Overall, ELNs were more complete than comparable PLNs. To facilitate feedback, instructors reported using a library of comments to annotate student ELN entries as needed. This approach eased the grading burden while still providing critiques to the student (53). Faculty also noted that the ELN’s timestamp feature and instructor’s ability to remotely access the ELN meant that they could readily assess if the notebook was being used to record an experiment or an “after the fact” record (50). In terms of plagiarism, faculty noted that issues of academic integrity and notebook originality were more easily assessed (21, 22), although Cooke et al. noted that the barrier to “copy and paste” in an ELN is much lower than for a PLN (34). Student Perspective Students in the modern undergraduate teaching laboratory are more familiar with technology than ever before; and yet, implementing technology such as ELNs to accomplish learning outcomes is not without some challenge. The literature reports that while reservations exist, students are still, largely amenable to ELNs in the teaching laboratory. Examination of these reports reveals student perspectives on ELNs can be characterized as follows: Access and Collaboration – Students appreciated the digital access to pre-laboratory material that supported their learning (34), and being able to access the notebook from multiple devices and at any time was also a favorite feature (16, 21). “The most valuable component of ELNs is that they are accessible anywhere (this is at least true for the ELN our institution uses), making it easy to record thoughts and view or analyze data at your convenience.” (student ELN user, LabArchives ELN tool, Muhlenberg ’21). Since ELN collaboration tools remove the need to always have to meet in person, students also find it easier to collaborate on laboratory reports, and especially when prompted by instructors, they collaborated more thoroughly and effectively (21). We have seen the same effects in our own teaching laboratories where LabArchives is the ELN tool, for example, as one Muhlenberg student puts it: “I used paper notebooks in gen chem and organic and an ELN for research, experimental biochemistry, and physical chemistry. It is much easier to edit your entries, keep a neat record of data, and study from an electronic lab notebook. For group projects it is also very convenient to be able to share data through the electronic lab notebook that can be accessed from any device.” Van Dyke also observes the benefit to student collaboration was significant, especially for commuter students who have traditionally found it hard to meet with lab group members outside of class (16). I find maintaining a complete and accurate record much easier with ELNs because it is easier both to include supplemental material (images, graphs, data sheets, etc.) and to organize that data. It’s also very helpful to have access to other lab member’s notes and data. For me, though, the most valuable component of ELNs is that they are accessible anywhere (this is at least true for the ELN our institution uses), making it easy to record thoughts and view or analyze data at your convenience. The only issue I’ve had with ELN use is that it requires bringing computers into environments with hazardous chemicals or other contaminants, but when proper precautions are followed, I find this concern to be perfectly manageable. ~Muhlenberg student, Class of 2021 on the LabArchives ELN tool as used in 1 y of organic chemistry laboratory and independent research. Organization and Non-linear workflow – Students appreciate that with an ELN they do not have to figure out how many pages to set aside or organize in a paper notebook for a single experiment in 190

the teaching laboratory. Since the digital page is “infinitely scalable,” students could add information anywhere along the page and at any time during the laboratory (16). Furthermore, the ability to link to previous content and embed later data analysis emphasized the connections in the experiments while allowing for a non-linear workflow (34). The author’s own students also reflect similar observations: “I appreciate using an electronic lab notebook for its ability to insert graphs and pictures directly into the notebook. If using paper, this would not be possible and anything you would want to insert you would have to print out. Then, there is the possibility of it falling out of the paper notebook. Also, the electronic notebook allows you to easily copy and paste procedures, without rewriting it all over again.” Media enhancements to observations and data analysis – Students report that importing data including images, audio and other files types into an ELN is easier, neater and more organized (16, 21, 34). The multimedia options can improve and enhance the record of observation – after all, “a picture is worth a thousand words” – but, students acknowledge that they need to be prompted and reminded to incorporate media to enhance their recording of observations in an ELN (34). The embedding of data files within a notebook page also saved the students time in collecting all the parts of an experiment together, and it created a continuity of data acquisition and analysis that enhanced learning (16, 34). Our own student ELN users offer observations consistent with these, stating “ELNs are great since most of the data that I collect is digital (it’s 2019 after all), so being able to just attach the file to an ELN is really convenient, whereas it seems antediluvian and tedious to have to print out pictures of data and tape them into a traditional notebook.” Technology anxiety vs. valuable skill – While we may be teaching the “mobile generation,” familiarity with technology is not equivalent to facility with that same technology. For example, students reported not knowing how to use features of Google tools, despite indicating higher familiarity with Google tools over other options (21). Furthermore, students can be surprisingly anxious about the use of technology in the classroom, and report frustrations over loss of an internet connection, software glitches and crashes, in addition to learning how to use new software features (21–23, 34). And yet, a majority of students also believe – apparently, in spite of these anxieties and frustrations – that learning to use ELNs is beneficial to their future (16, 21). Device availability and viability – It goes without saying that ELNs require access to the Internet and to devices. Some instructors have students “bring your own device (16),” which operates on the assumption that 1) all students have reliable access to a device, which is not always the case (21) and 2) all students are comfortable using a personal device in the laboratory where spills and contamination are possible. As this author’s students have observed, “The only downfall of an electronic notebook is keeping a laptop or computer near you in the lab, risking ruining the device or bringing contaminants outside of the lab.” By purchasing relatively inexpensive Chromebooks that are kept in the Organic teaching lab, this author’s institution has mitigated issues of student access and device viability such that these concerns do not inhibit learning in the teaching laboratory. Training As noted earlier in this chapter, keeping and maintaining a laboratory notebook instills good laboratory practices as well as ensures the integrity of the data collected. Nussbeck et al. make a compelling argument that an ELN approach sets the standard for good scientific practice as i) an ELN allows seamless documentation of digital data, ii) an ELN fosters standardized protocols and data formats, and iii) an ELN automatically archives data (54). Beyond this foundational training that is enhanced by an ELN approach, what additional skills are provided to students by using an ELN 191

over a PLN? In this Age of Information where social, economic, and political processes are driven by technology, training students in STEM cannot be left to languish in analog mode. Adoption of ELNs, although approaches and formats vary widely, prepares students to smoothly transition to professional settings, e.g. biotechnology industry and academic research enterprises, where ELN usage is rapidly and widely expanding (50). Beyond bench side applications, ELN training and usage provides students with transferable, technical skills. Through collecting data and making ELN entries on smartphones, tablets, laptops, and desktops that consist of rich text, images, video, and data files, students are learning to integrate rich media as a part of documentation. For students pursuing healthcare careers, the electronic system of record keeping is applicable to their future profession’s documentation system, electronic healthcare records (53). Electronic laboratory notebooks better prepare students for careers. Training undergraduates to expect and excel in ELN usage creates a culture where digital documentation is the expected standard within the STEM workforce.

References 1. 2. 3. 4. 5. 6.

7.

8. 9. 10. 11.

12.

13.

da Vinci, L. The Leonardo Notebook. http://www.bl.uk/turning-the-pages/?id=cb4c06b902f4-49af-80ce-540836464a46&type=book (accessed Feb 27, 2019). Concasty, M.-L. Pierre and Marie Curie (Exposition) [A catalogue with portraits and a facsimile]; Bibliothèque Nationale: Paris, France, 1967. Hernandez, P. The Nobel Moment: Dan Shechtman. https://www.nist.gov/content/nist-andnobel/nobel-moment-dan-shechtman (accessed Feb 27, 2019). Gitschier, J. The Whole of a Scientific Career: An Interview with Oliver Smithies. PLoS Genet. 2015, 11. Volchenkov, R.; Sprater, F.; Vogelsang, P.; Appel, S. The 2011 Nobel Prize in Physiology or Medicine. Scand. J. Immunol. 2012, 75, 1–4. Wipperman, A. Nobel Prize-Winning Lab Releases Mutant Mouse Data to the Public Domain. https://blogs.biomedcentral.com/bmcblog/2012/10/24/nobel-prize-winning-lab-releasesmutant-mouse-data-to-the-public-domain/ (accessed Feb 27, 2019). Arnold, C. N.; Barnes, M. J.; Berger, M.; Blasius, A. L.; Brandl, K.; Croker, B.; Crozat, K.; Du, X.; Eidenschenk, C.; Georgel, P.; et al. ENU-Induced Phenovariance in Mice: Inferences from 587 Mutations. BMC Research Notes 2012, 5, 577. Bowers, W. G. Note-Books in Laboratory Instruction. J. Chem. Ed. 1926, 3, 419–424. National Science Teachers Association. NSTA Position Statement: The Integral Role of Laboratory Investigations in Science Instruction; 2007. Hosie, E. The Uncertain Future of Handwriting. http://www.bbc.com/future/story/20171108the-uncertain-future-of-handwriting (accessed March 5, 2019). Trubek, A. Opinion|Handwriting Just Doesn’t Matter. The New York Times; August 20, 2016. https://www.nytimes.com/2016/08/21/opinion/handwriting-just-doesnt-matter.html (accessed Aug 19, 2019). Computer Applications in Pharmaceutical Research and Development: Ekins/Computer Applications in Pharmaceutical Research and Development; Ekins, S., Ed.; John Wiley & Sons, Inc.: Hoboken, NJ, 2006. Walsh, E.; Cho, I. Using Evernote as an Electronic Lab Notebook in a Translational Science Laboratory. J Lab Autom. 2013, 18, 229–234.

192

14. Taylor, K. T. The Status of Electronic Laboratory Notebooks for Chemistry and Biology. Curr. Opin. Drug Discov. Devel. 2006, 9, 348–353. 15. Martin, M. T.; Belikov, O. M.; Hilton, J.; Wiley, D.; Fischer, L. Analysis of Student and Faculty Perceptions of Textbook Costs in Higher Education. Open Praxis 2017, 9, 79–91. 16. Van Dyke, A. R.; Smith-Carpenter, J. Bring Your Own Device: A Digital Notebook for Undergraduate Biochemistry Laboratory Using a Free, Cross-Platform Application. J. Chem. Educ. 2017, 94, 656–661. 17. HMS Data Management Working Group. Blog: Best Practices - Electronic Laboratory Notebooks. https://datamanagement.hms.harvard.edu/electronic-lab-notebooks (accessed Aug 19, 2019). 18. Milsted, A. J.; Hale, J. R.; Frey, J. G.; Neylon, C. LabTrove: A Lightweight, Web Based, Laboratory “Blog” as a Route towards a Marked Up Record of Work in a Bioscience Research Laboratory. PLoS ONE 2013, 8, e67460. 19. Lawrie, G. A.; Grøndahl, L.; Boman, S.; Andrews, T. Wiki Laboratory Notebooks: Supporting Student Learning in Collaborative Inquiry-Based Laboratory Experiments. J. Sci. Educ. Technol. 2016, 25, 394–409. 20. Johnston, J.; Kant, S.; Gysbers, V.; Hancock, D.; Denyer, G. Using an EPortfolio System as an Electronic Laboratory Notebook in Undergraduate Biochemistry and Molecular Biology Practical Classes. Biochemistry and Molecular Biology Education 2014, 42, 50–57. 21. Bromfield Lee, D. Implementation and Student Perceptions on Google Docs as an Electronic Laboratory Notebook in Organic Chemistry. J. Chem. Educ. 2018, 95, 1102–1111. 22. Weibel, J. D. Working toward a Paperless Undergraduate Physical Chemistry Teaching Laboratory. J. Chem. Educ. 2016, 93, 781–784. 23. Bennett, J.; Pence, H. E. Managing Laboratory Data Using Cloud Computing as an Organizational Tool. J. Chem. Educ. 2011, 88, 761–763. 24. Oleksik, G.; Milic-Frayling, N.; Jones, R. Study of Electronic Lab Notebook Design and Practices That Emerged in a Collaborative Scientific Environment. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing; CSCW 2014; ACM: New York, NY, 2014; pp 120–133. 25. Solutions Built for Teachers and Students. https://edu.google.com/ (accessed March 5, 2019). 26. Use Google Drive files offline - Computer - Google Drive Help. https://support.google.com/drive/ answer/2375012?co=GENIE.Platform%3DDesktop&hl=en&oco=1 (accessed March 5, 2019). 27. OneNote Versions Compared. OneNote for Beginners; 2018. http://onenote-for-beginners. com/onenote-versions-compared (accessed Aug 19, 2019). 28. What’s the difference between OneNote and OneNote 2016?. https://support.office.com/en-us/ article/what-s-the-difference-between-onenote-and-onenote-2016-a624e692-b78b-4c09b07f-46181958118f (accessed March 6, 2019). 29. Microsoft Makes OneNote for Windows Completely Free by Removing All Feature Restrictions. VentureBeat; 2015. https://venturebeat.com/2015/02/13/microsoft-makesonenote-for-windows-completely-free-by-removing-all-feature-restrictions/ (accessed Aug 19, 2019). 30. Download OneNote. http://www.onenote.com/download (accessed March 6, 2019). 31. OneNote Class Notebook. https://www.onenote.com/classnotebook (accessed March 6, 2019). 193

32. Kwok, R. How to Pick an Electronic Laboratory Notebook. Nature 2018, 560, 269. 33. Guerrero, S.; Dujardin, G.; Cabrera-Andrade, A.; Paz-y-Miño, C.; Indacochea, A.; InglésFerrándiz, M.; Nadimpalli, H. P.; Collu, N.; Dublanche, Y.; De Mingo, I.; Camargo, D. Analysis and Implementation of an Electronic Laboratory Notebook in a Biomedical Research Institute. PLoS One 2016, 11. 34. Cooke, N. J.; Robbins, P.; Lodge, J.; Shannon, I.; Hawwash, K. Recommendations for Electronic Laboratory Notebooks in Undergraduate Engineering Faculty; 2017; p 8. 35. Barber, C. G.; Haque, N.; Gardner, B. ‘OnePoint’ – Combining OneNote and SharePoint to Facilitate Knowledge Transfer. Drug Discovery Today 2009, 14, 845–850. 36. Blog: How to Use Onenote as Your Electronic Lab Book – Dr Martin Engel. http://martinengel.net/ 2015/12/how-to-use-onenote-as-your-electronic-notebook/ (accessed Aug 19, 2019). 37. Gotthardt, M. A Flexible Electronic Lab Notebook (ELN) with Microsoft OneNote. The LabOMator; http://lab-o-mator.blogspot.com/2015/06/a-flexible-electronic-lab-notebookeln.html (accessed Aug 19, 2019). 38. Harvard Biomedical Data Management, Electronic Lab Notebooks: OneNote. https://datamanagement.hms.harvard.edu/electronic-lab-notebooks-onenote (accessed March 5, 2019). 39. Polka, J. 11 Tricks for Using OneNote as Your Lab Notebook. ASCB 2015. 40. Urban, S. Pen-Enabled, Real-Time Student Engagement for Teaching in STEM Subjects. J. Chem. Educ. 2017, 94, 1051–1059. 41. Tofan, D. C. Using a Tablet PC and OneNote 2007 To Teach Chemistry. J. Chem. Educ. 2010, 87, 47–48. 42. Pence, H. E. Moving Chemical Education into the Cloud(s). J. Chem. Educ. 2016, 93, 1969–1971. 43. McCracken, H. Inside Evernote’s Brain. https://www.fastcompany.com/90216018/insideevernotes-brain (accessed March 7, 2019). 44. Focus on What Matters Most to You. https://evernote.com/why-evernote (accessed March 7, 2019). 45. ELN Features Matrix. https://docs.google.com/spreadsheets/d/1ar8fgwagOh30E31EAPLGorwn_g6XNf81g3VDQnQ_I8/edit?usp=sharing&usp=embed_facebook (accessed March 5, 2019). 46. Compare Plans and Get Started for Free. https://evernote.com/compare-plans (accessed March 7, 2019). 47. Marmillod, V. Blog: 2019 Review of the Best Electronic Laboratory Notebooks; 2019. https://www.labsexplorer.com/c/2019-review-of-the-best-electronic-laboratorynotebooks_197 (accessed Aug 19, 2019). 48. The Gurdon Institute, University of Cambridge. Blog: Electronic Lab Notebooks-for Prospective Users. https://www.gurdon.cam.ac.uk/institute-life/computing/elnguidance (accessed Aug 19, 2019). 49. Puccinelli, J. P.; Nimunkar, A. J. An Experience with Electronic Laboratory Notebooks in Real-World, Client-Based BME Design Courses. In ASEE Annual Conference & Exposition, Indianapolis, Indiana. Retrieved from https://peer. asee.org/20047; 2014.

194

50. Eblen-Zayas, M. Comparing Electronic and Traditional Lab Notebooks in the Advanced Lab. In Conference on Laboratory Instruction: Beyond the First Year of College, College Park, MD. Retrieved from https://www.compadre.org/advlabs/items/detail.cfm?ID=13799; 2015. 51. ACS Guidelines and Evaluation Procedures for Bachelor’s Degree Programs. https://www.acs.org/ content/acs/en/about/governance/committees/training/acs-guidelines-supplements.html (accessed Aug 19, 2019). 52. Riley, E. M.; Hattaway, H. Z.; Felse, P. A. Implementation and Use of Cloud-Based Electronic Lab Notebook in a Bioprocess Engineering Teaching Laboratory. Journal of Biological Engineering 2017, 11, 40. 53. Dood, A. J.; Johnson, L. M.; Shorb, J. M. Electronic Laboratory Notebooks Allow for Modifications in a General, Organic, and Biochemistry Chemistry Laboratory to Increase Authenticity of the Student Experience. J. Chem. Ed. 2018, 95, 1922–1928. 54. Nussbeck, S. Y.; Weil, P.; Menzel, J.; Marzec, B.; Lorberg, K.; Schwappach, B. The Laboratory Notebook in the 21st Century. EMBO Reports 2014, 15, 631–634.

195

Chapter 9

Formative Assessment to Improve Student Learning in Biochemistry Erika G. Offerdahl* and Jessie B. Arneson School of Molecular Biosciences, Washington State University, PO Box 647520, Pullman, Washington 99164, United States *E-mail: [email protected].

Assessment is the process by which biochemistry instructors gain insight into the effectiveness of instruction. As such, assessment is the bridge between teaching and learning. Formative assessment is particularly important for improving student learning. While much is known about the critical components of effective formative assessment, the practice of formative assessment within the context of an biochemistry requires careful consideration. In this chapter, a broad overview of the many functions of assessment is presented, followed by a discussion of the mechanisms by which formative assessment supports student learning. The chapter draws on the learning sciences to explain how the nature of biochemical knowledge presents unique challenges for the assessment of student understanding and concludes by presenting promising practices for assessment for learning. This chapter will serve as a resource for anyone interested in understanding the role of assessment in transforming undergraduate biochemistry education.

Introduction In the United States, it has been deemed a national imperative to increase both the number of STEM majors (1) and quality of the undergraduate educational experience (2). The problems facing our world today are increasingly multi-faceted, complex and capacious, often necessitating collaboration of interdisciplinary teams in order to generate effective solutions. This rapidly evolving landscape makes it difficult to predict all that STEM undergraduates will need in future jobs that have not yet been created. Solving modern problems will require a workforce with the skills to leverage core scientific concepts and cross-cutting ideas. Curricula focusing on knowledge acquisition and confirmatory laboratory experiences are no longer sufficient. This is particularly true for biochemistry undergraduates who, because of the interdisciplinary nature of the discipline, are likely to be recruited to play significant roles in interdisciplinary teams. In response, the life sciences community © 2019 American Chemical Society

has rearticulated undergraduate learning targets to focus, in equal measure, on understanding crosscutting concepts and developing competency with essential skills (1, 3–5). Establishing a shared understanding of the intended learning outcomes for biochemistry is necessary but insufficient for transforming undergraduate biochemistry education. Achieving these outcomes will require a change not only in how we teach, but also in how we assess student learning. It is well accepted that effective instruction is produced through the application of sound instructional design principles; the design process begins with clear and explicit learning outcomes to which all instruction and assessments are aligned (6, 7). Instructional designs that are student-centered and actively engage students in meaningful and challenging activities are more effective than lectureintensive designs (8). Similarly, instruction should incorporate ongoing assessment and feedback to support student learning (1, 9). While the biochemistry community has made significant progress in terms of articulating learning outcomes (10) and instructional approaches to achieve them (e.g. (11, 12)), there is a need for more examples and tools for assessment (5, 13). This chapter is designed as a resource for biochemistry instructors interested in understanding the role of assessment in transforming undergraduate biochemistry teaching and learning. We begin with a broad overview of the functions of assessment followed by a more in-depth description of the mechanisms by which formative assessment supports learning. We then discuss unique challenges for assessing student understanding within the context of biochemistry, and conclude with key considerations and promising practices. While the chapter is aimed at the practitioner, we believe the following discussion and examples will also be of relevance to biochemistry education researchers interested in bridging the gap between research and practice.

The Role of Assessment in Transforming Biochemistry Teaching and Learning Assessment is the bridge between teaching and learning. It is the process by which we gain insight into the overall effectiveness of instruction at achieving the intended student learning outcomes. Assessment has become increasingly important in undergraduate education not only to monitor student learning within the classroom, but also for accreditation and accountability purposes. To understand the role of assessment in transforming biochemistry teaching and learning, it is often helpful to explicitly identify the “who”, “what” and “how” of an assessment; that is, about who is the assessment providing insight, what does the assessment intend to measure, and how are the assessment data to be used? For biochemistry instructors, the most common “who” of an assessment is the individual student. Every assignment, quiz, and exam is an assessment of individual student learning, and therefore provides information about how well an intended learning outcome has been achieved. But when these same artifacts are examined in the aggregate, the “who” of the assessment shifts and provides insight into the learning of groups of students (e.g., course sections, lab groups, different semesters of same course), thereby facilitating comparison between or across groups of students. Academic coordinators and department chairs, while often instructors themselves, have the additional responsibility of being attentive to the degree to which program-level learning outcomes are met. For them, the “who” of assessment is likely to be the entire academic program or clusters of courses (e.g., gateway courses, service courses). As instructors, we spend a lot of time considering the content of our assessments - what have our students learned? It is therefore convenient for us to think of the “what” of assessment simply as the learning goals and objectives of instruction. For biochemistry and molecular biology, undergraduate learning goals and objectives have been defined both in terms of foundational concepts and skills 198

(3, 4). While much of the “what” of assessment is by design discipline-specific (we are, after all, interested in the teaching and learning of biochemistry), many concepts and skills bridge the disciplines. Across all STEM, and particularly for an inherently interdisciplinary field such as biochemistry, it is critical for students to recognize and leverage these crosscutting concepts (14). Similarly, many skills are ubiquitous to the scientific enterprise (e.g., data analysis, scientific argumentation) but for which assessing within the disciplinary context is important (15, 16). Assessments must therefore be designed in terms of what it means to demonstrate mastery of the disciplinary ways of knowing in biochemistry (16–18). Biochemistry learning can be transformed by recognizing that the “what” of assessment must include the central biochemical concepts as well as ideas common to other disciplines, and allow students to demonstrate what they can do with their knowledge (4, 14). In addition to “who” or “what” of an assessment, the “how” or the purpose of the assessment is also important to make explicit. For example, a common function of assessment is to evaluate results or outcomes, to make judgments about the degree to which or how much something has been achieved. In this regard, one well-known purpose of assessment is to demonstrate accountability (i.e., to what degree do students demonstrate mastery of the principles of thermodynamics?). Assessment used to evaluate the outcomes or products of learning are often called summative assessments. Yet assessments are also used to examine the processes of teaching and learning, as opposed to their products. Assessments used for the purpose of monitoring, improving, or to inform changes in the teaching and learning process are often referred to as formative assessments. The same assessment tool can be used both summatively and formatively. For example, exams are often used to assign grades that then represent the degree to which a student has achieved the desired outcomes of a course. Alternatively, an exam might be used to support student reflection on learning, resulting in changes in student motivation or approach to learning. Making explicit to students the purpose of the assessment tool (e.g., quiz, portfolio) may in fact facilitate learning because it allows the student to adjust performance and reflection accordingly. Attending to the “who”, “what”, and “how” of assessment makes apparent how a single assessment instrument can be used in a multitude of ways. For example, when the “who” is an undergraduate program, an assessment will likely be administered to individual students (e.g., accreditation exam) but the information will be used to assess programmatic (not individual) learning outcomes. Yet that same assessment can be used by an instructor interested in how instruction is working in the context of his or her own course to diagnose individual student learning. In either case, the assessment can address a different “how”. To continue our example, how might accreditation exam results be used? They could be used both as a summative assessment to draw conclusions about the learning outcomes of the graduating seniors in a program as well as formatively to inform curricular or course revisions. Clearly, the “who”, “what”, and “how” are interrelated. Yet, attending to each in turn allows instructors to recognize how each of their assessment choices contributes to larger transformation efforts in biochemistry. For instance, a single instructor may already administer a concept inventory at the beginning of a class to assess where students are and adjust instruction accordingly. If the concept inventory is administered again at the end of the course, the instructor can determine how well the intended learning outcomes have been achieved. And if the instructor repeats the use of this assessment each time the course is offered, those data can then be used to address broader curricular questions that will support scaffolding learning throughout an undergraduate program. Similarly, an instructor that attends to the “what” is more likely to recognize that a concept inventory generally provides insight into only a single dimension of student learning (e.g. conceptual understanding), 199

and therefore seek additional assessments to provide insight into other dimensions (e.g. disciplinary ways of knowing). We use the “who”, “what”, and “how” of assessment to draw the boundaries of discussion for the remainder of this chapter. This chapter is designed as a resource for biochemistry instructors, therefore the focus will be predominantly on formative assessment of individual student learning in biochemistry. In the next section, we present a brief overview of the proposed mechanisms by which formative assessment improves student learning, followed by sections that identify common difficulties in assessing student learning in biochemistry and promising practices.

Formative Assessment and Feedback Support Learning There is widespread agreement that formative assessment is a high-impact instructional practice that can significantly improve learning of STEM concepts and skills (e.g. (9, 19)). Formative assessment is often described as an iterative cycle through which instructors gather evidence of student thinking to reflect on students’ in-progress learning (e.g., (20–23)). Formative assessment can take on many forms (e.g., minute papers, muddiest point, clicker questions) depending on the instructor and course context. Critical to this process is the generation of actionable feedback for both the instructor and the learner that can be used to achieve an intended learning outcome (23). As such, formative assessment is a dynamic conversation between the instructor (or instructional team) and the learner that works to improve learning (24). Formative assessment begins with identifying the intended learning outcomes, and defining clear criteria for determining success (e.g., (6, 7, 22)). As a conversation, the process of formative assessment is focused on “asking” the students where they are in the journey toward the intended outcome, and providing feedback to help them get there. The conversation most often begins with the “asking”; an instructor prompts (e.g., verbal question, worksheet, clicker question) the student for information. Less often the conversation begins with a student asking a question. Whether by answering an instructor-generated prompt or by posing a question, students are making their thinking explicit to the instructor and to themselves. Both cases present an opportunity to learn about the current state of student understanding in relation to an intended learning outcome. Once made explicit, recognition of the current state of student understanding can in turn stimulate action by the student, a member of the instructional team, or both. By engaging metacognitive processes, students actively reflect on and evaluate their current level of understanding and generate internal feedback about how to make further progress through subsequent actions (e.g., adopting alternative study strategies, seeking more information from textbook). In the absence of well-developed metacognitive processes, students are unlikely to be able to effectively evaluate their current level of understanding relative to desired learning outcomes, or generate the kind of internal feedback that would be necessary to take further action (23, 25). Instructors can similarly use the evidence of student understanding once it has been made explicit. If the gap between the students’ current and desired level of understanding is too large, the instructor may decide to “go back to the drawing board” and revise the lesson or create a new one. Ideally, the instructor will provide feedback that can prompt the student to take further action (e.g. engaging metacognitive processes, revising study strategies). Feedback is critical to the efficacy of formative assessment at improving student learning. A meta-analysis of feedback efficacy studies has documented an overall average positive effect of feedback (d = 0.41) on student learning (26). Feedback is most effective when it is timely, relevant (27), and actionable (28). Feedback should contain information to help students see how they are 200

doing (i.e., current state) relevant to where they are going (i.e., intended outcome) and include guidance for how to proceed (28). The provision of high-quality feedback is indeed important, but equally so is the ability of the learner to recognize and act on that feedback to improve learning (22). There is both theoretical and empirical support that formative assessment and feedback can improve student learning (e.g. (20–23, 29)). Yet effect sizes associated with formative assessment vary greatly (30), with some meta-analyses reporting values as low as 0.2. Similarly, while feedback studies generally report an average overall positive effect, a significant number report negative effects (26). It has been argued previously that the variability in effect sizes may not be due to the efficacy of the practice itself, but rather to variations in how the critical components are enacted (23). For example, formative assessment requires eliciting and responding to evidence of student understanding with feedback to improve learning. But if a formative assessment prompt does not provide sufficient evidence, either because the prompt is not well aligned with the intended learning outcome (23) or does not reveal the range and extent of student thinking (31), it will be difficult for the instructor to generate relevant and actionable feedback. Even when all of the key components of formative assessment are implemented optimally for student learning, it is still possible for formative assessment to fail to reach its full potential. Described above is the general process of how formative assessment and feedback work to improve student learning, but the discussion above neglects the disciplinary context of biochemistry. As will be discussed in the next section, there are disciplinary nuances that can present challenges to formative assessment – namely that the disciplinary ways of knowing biochemistry are complex, thereby making it difficult to accurately reveal what students know, and don’t know, in biochemistry.

Assessment within the Disciplinary Context of Biochemistry Effective assessment design must take into account the nature of the discipline in order to provide students with opportunities to develop and demonstrate mastery of the disciplinary ways of knowing (13, 15, 17). Biochemistry is perhaps unique in that it stands at the crossroads of other scientific disciplines. To be knowledgeable in biochemistry requires leveraging cross-cutting concepts from physics and chemistry to explain phenomena that occur within biological systems. Integrating core principles across disciplinary boundaries also involves thinking across multiple levels as well as linking the invisible (e.g. energy, forces) to the submicroscopic (e.g. molecules) to the microscopic (e.g. organelles, cells) and the macro (e.g. organismal) levels (32–34). Development of biochemical expertise requires more than just building conceptual understanding. To achieve mastery in biochemistry, a learner must gain understanding of and fluency with all aspects of biochemical disciplinary discourse - the tools, activities, and representations utilized by practicing biochemists (17). For example, scientific visual literacy is a critical characteristic of biochemical discourse. Visual representations make apparent the connections between “unseen” biochemical processes with the observable, tangible world (15, 35). Biochemists frequently use visual representations to communicate their knowledge, make sense of data, propose hypotheses, and illustrate complex interactions between subcellular components (18, 36). Since a single visual representation cannot encapsulate all essential information for a given concept or process (37), biochemists must be able to interpret, create, and transition between multiple abstractions (e.g. graphs, schematics, cartoons), each of which comes with their own affordances and limitations (16, 36). Scientific visual literacy overlaps with conceptual understanding, but is clearly a distinct aspect of demonstrating biochemical expertise. 201

Other aspects of biochemical expertise that students must master include reasoning across levels of biological organization in order to build understanding about an abstract, complex world that largely cannot be seen. Students must also learn to navigate the verbal and visual conventions of biochemistry as disciplinary discourse includes the ability to represent data, hypotheses, and unobservable structures or processes through different abstractions (16, 37). Though students may have encountered certain abstractions (e.g. chemical formulae, equations, free energy diagrams) in other science courses, other types of representations (e.g. ribbon diagrams, Ramachandran plots) may be entirely novel. Even with abstractions that may be more familiar, the interpretation of scientific representations is not necessarily intuitive for students (38) and often requires both representational competency and conceptual understanding. To illustrate the many facets of knowledge that are in play for a student developing biochemical expertise, let’s look briefly at protein stability, a core biochemical concept at the heart of protein structure. The free energy change of folding is quite small under physiological conditions. Understanding this difference conceptually requires knowledge of thermodynamic principles and the nature of molecular forces from physics and chemistry. Students must then make sense of this knowledge within the unique constraints imposed by physiological conditions. In doing so, students must integrate knowledge of the subcellular environment in which proteins function (e.g. pH, temperature, aqueous) as well as the chemical properties of the amino acids that comprise the protein (e.g. pKa, hydrophobicity) to understand the discrete enthalpic and entropic contributions to overall stability. None of the molecular interactions and balancing forces are readily observable with the naked eye, so biochemists and students alike must make use of visual abstractions to represent and make sense of different aspects of protein stability. Included in Figure 1 are three such examples: a) a schematic demonstrating how entropic and enthalpic measures contribute to the overall free energy change, b) a cartoon depicting the protein folding process, and c) a graph characterizing how changing temperatures affects the free energy change of protein folding. Interpreting these images requires not only a foundational understanding of energetics but also the visual literacy skills necessary for extracting information from the representation. For instance, the first schematic and the graph both use symbolic language (e.g., ΔG, TΔS,) to represent thermodynamic principles and a pair of axes to diagram the relationship between variables. Important details can be contained within representational elements that may be overlooked or seem meaningless to novices. At a surface glance, Figure 1b merely demonstrates the different appearance of a protein in its denatured and folded states. However, a deeper examination of the image reveals additional information; the shading of the circles indicates polarity of amino acid side chains, providing clues about hydrophobic interactions, while the manner in which the arrows were drawn signals to the viewer which state is energetically favored. The multifaceted nature of biochemical knowledge can present complications with regard to assessment, even when “best practices” are implemented. Take, for instance, a study examining the development of visual literacy skills in an undergraduate biochemistry course (39). The instructional team used the Visualization Blooming Tool (VBT), which maps visual literacy skills on to Bloom’s taxonomy (40), to articulate clear learning outcomes that included scientific visual literacy skills. They also used the principles of backward design in concert with the VBT to generate formative and summative assessment items that aligned with the learning outcomes. Careful attention was paid to giving students practice developing visual literacy skills as well as opportunities to routinely test, through formative assessments, their understanding multiple times before each exam. 202

Figure 1. Visual representation is just one of the multiple facets of biochemical knowledge. All three figure panels demonstrate different ways of representing and making sense of protein stability. Despite increasing the amount of practice with visual representations, most notably on higherorder items, providing consistent levels of practice throughout the semester, and assuring alignment between practice and assessment on unit exams, overall student performance did not improve as compared to previous semesters (39). There was also no significant change in conceptual understanding, as measured by the Introductory Molecular and Cell Biology Assessment (41), between semesters before and after intervention, and students performed similarly on course assessments. The only noticeable difference occurred on the final exam, on which student performance actually decreased (39). How might this result be interpreted? It is possible that the multi-faceted nature of biochemical knowledge presents challenges to students in terms of information processing. Student difficulties have been well-documented in reasoning across biological levels (42, 43), as well as in interpreting visual representations (e.g. (36, 44)). As such, assessment items targeting multiple concepts, skills, and/or practices may be more cognitively demanding for students. The working memory holds a limited capacity for processing information, especially for learners who have not yet developed the sophisticated schema possessed by experts (45, 46). Increasing the number of elements with which students have to reason raises the likelihood that the cognitive load associated with the assessment will overwhelm working memory, resulting in lower performance. It can be difficult then as an instructor to determine whether students struggled on a question because the integration of multiple dimensions was beyond the capacity of their working memory, or if there was an underlying issue with the particular concept, visual representation, or terminology used within the question. The many aspects of biochemical knowledge presents some challenges to assessing student understanding in biochemistry. Given the common focus primarily on developing students’ conceptual understanding, it may be tempting to interpret poor assessment performance as indicative of incomplete biochemical understanding when instead the issue may actually be difficulties with other aspects of biochemical knowledge (i.e., representational competency) Given these potential 203

challenges, and what we know from educational psychology, cognitive science, and the learning sciences, we will present promising practices for leveraging the power of formative assessment to develop students’ biochemical expertise.

Promising Practices for Assessment for Biochemistry Learning The literature on formative assessment provides a generic blueprint for how well-crafted learning objectives can be used by instructors to iteratively probe and provide feedback to support student learning. Yet as discussed in the previous section, there are disciplinary and contextual nuances that complicate the actual practice of formative assessment. Certainly the multi-faceted and interdisciplinary nature of biochemical knowledge in and of itself presents a challenge. For example, assessments must be able to disentangle students’ understanding of cross-cutting concepts (i.e. thermodynamics) from the ability to leverage those concepts to explain biochemical phenomena (i.e. protein folding). The context of higher education in which biochemistry is taught may also present challenges. As an undergraduate program, biochemistry often finds its home with other disciplines in departments of chemistry, biophysics, and molecular biology, to name a few. Variations in institution type and size translate to diversity in biochemistry programs and biochemistry courses that will similarly vary greatly in size and student composition. At larger institutions, biochemistry is frequently taught as a service courses due to the number of majors requiring basic biochemical understanding. These service courses may have enrollment on par with large-lecture introductory chemistry or biology courses, despite the fact that they are often junior or senior-level. Implementing formative assessment and providing feedback on large scales can be difficult and time consuming. In the remainder of this section, we draw on literature from the learning sciences to identify promising practices for overcoming challenges associated with assessment for learning in biochemistry. Promising Practice #1: Optimize Cognitive Load to Facilitate Schema Development Expert knowledge differs from that of novices in a number of distinct ways. Experts are able to perceive meaningful patterns that are not readily noticed by novices (47). This ability is attributed to the development, over time, of highly connected conceptual structures (i.e., schema) that facilitate organization and easy retrieval of information (e.g. (48, 49)). These knowledge structures allow experts to interpret and solve problems efficiently and effectively. The process through which expertise is developed is one through which meaningful connections between key ideas are made and reinforced through routine practice (47). When connections between discrete cognitive structures are reinforced through practice, they are more likely to be retrieved together as a single cognitive element or “chunk”. As a result, another characteristic of expert knowledge is the flexible retrieval of key chunks of information with little effort (50). Instructors can accelerate the development of expert-like knowledge by creating opportunities to facilitate meaningful connection making through the process of formative assessment. These opportunities should be designed with an understanding of how humans process and encode information for long term storage. In order to properly encode and store information for later retrieval, humans must be paying attention. The “mental workspace” in which humans temporarily store and manipulate information is referred to as working memory (45, 51). Information is brought into the mental workspace either from an external stimulus (e.g. problem set, data figure, lecture) or through retrieval from long term storage. Discrete cognitive elements become connected when routinely manipulated together in working memory, thereby organizing the individual elements into 204

larger chunks (52–54). The more frequently the information is manipulated in working memory, the greater the likelihood that it will be transferred to long term storage (54). The capacity of working memory in humans is limited; students can manipulate only a finite number of cognitive elements at a time (52–55). The number of elements varies depending on the complexity of information; up to seven simple elements like strings of letters and numbers can be accommodated at once (55), whereas the limit is thought to be less than four for more complex ideas (52). The limited capacity of working memory has direct implications for how instructional opportunities should be designed for the development of biochemical expertise. Novice biochemistry students do not have well-developed schema that link the many facets of biochemical knowledge together. The opportunities for novices to practice must be intentionally crafted to increase the likelihood that novices will connect cognitive elements in a way that facilitates organizing ideas around core concepts and “big ideas” (2, 47), but without overloading working memory, and also accompanied by appropriate feedback. So how might biochemistry instructors use this information to craft formative assessments that support the development of expertise? Recall that formative assessments are effective because they allow students to test their understanding (thereby making it explicit) and receive feedback to shape learning. Let’s revisit the example of protein stability from the previous section. The novice student must demonstrate the ability to use cross-cutting concepts (i.e., thermodynamics) learned in chemistry and physics within a biochemical context (i.e., water as solvent, chemical properties of peptide side chains) to consider the enthalpic and entropic contributions to protein folding. The novice student must leverage cross-disciplinary competencies (i.e., graphing) with disciplinary conventions for representation (i.e. ribbon structures). Finally, all of these facets of understanding protein stability must be related to the overarching context of the foundational concepts of biochemistry and molecular biology [i.e., energy and matter transformation (4)]. The challenge for biochemistry instructors is determining how far to unpack expert schema into discrete cognitive elements that will be accessible to the novice. Depending on the course or the level of the student, each of the elements described above might need further unpacking. For example, in order to understand enthalpic contributions, students must be able to connect their ideas about enthalpy from chemistry to their ideas about the noncovalent and hydrophobic interactions of proteins. Similarly, some novices will need to connect symbolic formalisms (i.e., thermodynamic formulae) with which they might be more familiar from previous chemistry coursework with disciplinary representations (i.e., denaturing curves). The process of unpacking expert schema into smaller chunks is often challenging for experts because they forget what it was like to be a novice in the discipline (47). For this reason, biochemistry instructors should consult chemistry, biology, and biochemistry education research to garner insight into the progression of conceptual learning (see for example, (56–60)). Similarly, instructors might consider soliciting the input of advanced undergraduate or first-year graduate students to review materials and provide feedback about the appropriateness of the level at which an assessment is written. Unpacking expert schema into smaller, more accessible chunks is important for designing and implementing assessments that do not overtax students’ working memory. If too many cognitive elements are at play, student performance on an assessment will be low, especially if students are not allotted enough time to process or connect all of the elements. To the instructor, the interpretation of low student performance may be that the student doesn’t know a concept when low performance may actually just be indicative of working memory overload. Formative assessments can reinforce schema construction intentionally, yet gradually, by increasing the number of cognitive elements that students must manipulate in working memory. Let’s provide a practical example by continuing 205

our discussion of protein stability, in particular assessments that will reinforce students’ ability to consider the enthalpic and entropic contributions to protein folding (cognitive element #1) while also applying cross-disciplinary and discipline-specific representational competencies (cognitive element #2). Each of the panels in Figure 1 could be used to craft a series of formative assessment items that provide opportunities for students to connect these two cognitive elements by holding the conceptual element “constant” across all items but varying the representational competency targeted by the assessment. By using multiple assessment items in this way, the student is provided repeated practice and feedback on the connections between two cognitive elements (i.e., thermodynamic principle of protein stability versus representations of protein stability). Moreover, the instructor is provided with greater insight into the learner’s progression toward a desired outcome. If the student does poorly on one item, the difficulty may be with the student’s representational competency, not his understanding of the thermodynamics of protein folding. But if student performance is low on all three items, perhaps additional practice is needed with thermodynamic ideas before adding in the additional cognitive task of how to represent thermodynamics. Explicitly acknowledging the multiple facets of biochemical knowledge will be useful for instructors in terms of scaffolding formative assessment items that support student learning. In Figure 2, two formative assessment items probe the same concept, require similar levels of representational competency, and yet provide an opportunity for students to test and receive feedback on increasingly sophisticated cognitive skills. In Figure 3, the same question is posed to students (“How does the described mutation affect the stability of the protein?”), yet one of two prompts that employ different degrees of abstraction (i.e., graph versus symbolic) could be used, thereby allowing an instructor to determine if students are struggling with the concept of protein folding, or if the interpreting graphical representations is a skill that needs further attention.

Figure 2. Example of formative assessment prompts that can target the same biochemical principle and use the same level of visual abstraction, allowing more precise insights into students’ abilities to reason across cognitive levels. While multiple items may provide additional insight into which aspects are most difficult for students, instructors should also carefully consider the cognitive effort required to complete the entire assessment. This is especially important with regard to higher-stakes summative assessments where anxiety and time pressure can occupy a portion of the working memory’s limited capacity, increasing the overall cognitive load associated with the assessment (46, 61). Tasks that involve multiple dimensions are more cognitively demanding and often require more time to complete (62). Thus, it may at times be advisable to include items that only measure a single concept or skill in 206

addition to those that are more characteristic of the discipline. By no means should an assessment be devised predominantly of unidimensional or lower-order questions, but it is imperative to reflect on what balance will best suit the purpose of the assessment. A well-designed formative assessment plan can also help reduce some of the cognitive load associated with a summative assessment as it allows students to work through cognitively demanding tasks without added concerns of time, exam grades, or having multiple pages of items needing to be completed. Students can focus more attention on each of the task elements, supporting the formation of mental connections and cues that will ultimately reduce the cognitive effort required to complete a similar task encountered later.

Figure 3. Example of formative assessment prompts that assess students’ understanding of a concept using difference levels of visual abstraction (Prompt #1 vs. Prompt #2) while holding the cognitive skill level constant (“How does the described mutation affect the stability of the protein?). Since course learning outcomes, the multi-faceted nature of biochemistry, and cognitive architecture all need to be given due consideration, the process of designing effective assessments is, on its own, quite cognitively challenging. The increasing focus on good assessment practices, however, has led to the creation of tools and protocols that can help reduce the cognitive load associated with the design process experienced by instructors themselves. The Three-Dimensional Learning Assessment Protocol (3D-LAP), for instance, can assist instructors in crafting assessment items that measure across core biochemistry ideas, cross-cutting concepts from physics or chemistry, and biochemical practices (14). The BioCore Guide (63) can be similarly implemented to measure principles across levels of biological organization. The Biology Blooming Tool (BBT) is a tool that can be used to design assessment items that measure variations in students’ processing of content across cognitive levels (64). With regard to visual representations, instructors can use the Taxonomy of Visual Abstraction in Biochemistry (TVAB) to characterize the degree to which assessments provide students the opportunity to practice with the diverse abstractions frequently utilized by biochemists (16). The Visualization Blooming Tool (VBT), which provides descriptions of the skills and example tasks for each visual Bloom’s level, can then be used to establish explicit learning goals related to visual representation and design assessments that elicit evidence of biochemical visual learning skills (39). As an interdisciplinary science, it is often helpful to look to biology, chemistry, math, and physics to identify potential tools to support biochemistry teaching and learning. While certainly not a complete collection, Table 1 summarizes several currently available tools for biochemistry

207

instructors to use while designing assessments that attend to issues of optimizing cognitive load for student learning. Table 1. Tools for Helping Optimize Cognitive Load on Formative Assessment Items Tool

Biochemical Concepts

Skills and Practices

Three-Dimensional Learning Assessment Protocol (3D-LAP) (14)

X

X

BioCore Guide (63)

X

BioSkills Guide (3)

X

Taxonomy of Visual Abstraction (TVAB) (16)

X

Cognitive level

Biology Blooming Tool (BBT) (64)

X

Vizualization Blooming Tool (VBT) (39)

X

Marzano’s Taxonomy (65)

X

ICAP Framework (66)

X

Taxonomy of Biochemistry External Representations (TOBER) (34)

X

Molecular Biology Data Analysis Test (67)

X

The Graph Rubric (68)

X

Promising Practice #2: Use Assessments to Facilitate Retrieval of Relevant Information Learning requires students to encode information in working memory for eventual storage in long term memory in a way that is accessible for subsequent retrieval and use in both familiar and novel contexts. Information enters working memory from external sources after irrelevant or unimportant information is filtered out or through retrieval from long term storage. Not all information that is manipulated in working memory is transferred into long term memory; repeated interaction with and between cognitive elements increases the likelihood that they will be integrated into mental schema. The mechanics of how humans process information has important implications for assessments that support disciplinary ways of knowing. Formative assessments play a role in supporting students in developing biochemical expertise in at least two ways. First, formative assessments can trigger retrieval of relevant information from long term memory into working memory. For example, students learn how to represent molecules in general chemistry (e.g., chemical formulae, ball-andstick, line structures). Biochemists use an expanded repertoire (e.g., ribbon, backbone) for representing complex macromolecules. Formative assessments that require students to retrieve relevant information from prior chemistry experience and manipulate it with new information creates an opportunity for those cognitive elements to be connected and integrated into a more cohesive schema. Formative assessments can facilitate meaningful connection making between new and old material, which is particularly important for more novice students who are at early stages of constructing schema. As students learn more, they will integrate more information into their schema. Formative assessment and feedback will be useful for advanced biochemistry students because 208

repeated retrieval and manipulation of information in varied contexts and under different conditions can assist in reorganizing, restructuring, and pruning schema. Many evidence-based instructional practices (EBIPs) leverage the potential of formative assessment to facilitate retrieval of relevant information. For example, team-based learning (TBL) is an EBIP that has been implemented in biochemistry courses in medical and professional schools as well as university settings for more than 20 years (e.g., (69–71)). In TBL, students complete a series of formative assessments (i.e., individual readiness assurance test, team readiness assurance test) before they work collaboratively in teams to solve problems that require application of core concepts. The purpose of the readiness assurance tests is to “prime” students for deep learning by requiring them to retrieve key concepts that will be manipulated throughout the in-class problem solving session (72). Process-oriented guided inquiry learning (POGIL) is another EBIP that has been applied extensively in biochemistry (73, 74). POGIL activities are based on a learning cycle that guides the learner through exploration of a problem through direct questioning, followed by a concept invention phase where learners look for patterns and relationships in the information, and concludes with an application phase where learners apply their understanding in new situations. Much like TBL, POGIL relies heavily on formative assessments to facilitate retrieval of relevant information and opportunities for students to work with others in constructing increasingly sophisticated schema (73). While there are an increasing number of published EBIPs available (Table 2), biochemistry instructors should feel empowered to use the principles of how people learn to design formative assessments that meet the needs of their own students and unique context. Facilitating retrieval practice is just one example of a small adjustment that instructors can make in their assessment practice to support learning (75, 76). Table 2. Non-Exhaustive List of Instructional Practices That Facilitate Retrieval Practice Instructional Practice

Description

Team-Based Learning (TBL) (69–72)

Individual and team multiple-choice assessments to retrieve relevant information and rapid feedback prior to group problem solving.

Process-oriented Guided Inquiry Learning (POGIL) (73, 74)

Assessments trigger retrieval of contextualized information necessary for engaging in guided inquiry.

Just in Time Teaching (JiTT) (77, 78)

Individual assessments completed prior to class, requiring students to retrieve and connect ideas. Instructors provide feedback in class.

Peer Instruction (PI) (79, 80)

Short conceptual questions interspersed during lecture facilitate students retrieving information during lecture for immediate application.

Weekly Practice Exams (81)

Weekly practice exams, coupled with active learning in class, require students to interleave information and connect new content with previous content.

209

Promising Practice #3: Use a Range of Formative Assessment Prompts to Diagnose and Respond to Students’ In-Progress Learning The power of formative assessment is derived from the feedback provided to the learner about in-progress learning (e.g. (28, 29)). High-quality feedback should be attentive to the desired learning outcomes of instruction, clarifying to the learner what acceptable performance looks like and providing information about how to close the gap between the present and desired performance level (22). Both instructors and students alike recognize and appreciate the potential of feedback to shape learning (82). While the factors that most positively influence the likelihood that students will act on feedback remain to be fully elucidated (83), there is some evidence that they are more likely to act on feedback that is specific, detailed, and individualized (84). There are many resources to assist biochemistry instructors in the nuts and bolts of articulating clear and measurable learning outcomes (see for example, (6, 63, 85)). Though time consuming, the initial time investment required to concretely articulate the desired level of performance to be demonstrated by the student is worth it. The more explicit the learning outcome, the easier it is to evaluate evidence of student understanding to generate actionable feedback. Fortunately, the biochemistry community has articulated overarching learning goals (4), and generated sample learning objectives aligned with the learning goals (86). The task for instructors, then, is tailoring these examples for their own unique context. It is unlikely that a first (or second or third) time instructor will hit the mark the first time; it will likely take a few iterations of evaluating student assessment data and adjusting instruction and/or the targets to get them “just right”. With well-crafted learning outcomes identified, biochemistry instructors know where instruction is going and can use backward design to determine how students will get there – like having a roadmap with the destination, routes, and even alternate routes highlighted before setting out for a vacation destination. With roadmap in hand, instructors are poised to overcome two of the three hurdles to actionable feedback (specific and detailed) provided they know what the present state of the learner is. Imagine giving a friend directions to your favorite fishing spot in the presmartphone era. It would be difficult to help a lost friend meet up with you if you didn’t know where they started or where they are currently located. You might need to ask a series of questions to figure out where your friend took a wrong turn or to determine their present location before you can provide navigation instructions. Extending this metaphor to the classroom, a series of formative assessment prompts may be needed for an instructor to accurately pinpoint students’ current level of understanding. Once accomplished, the instructor can then provide specific and detailed feedback to support the learner in navigating to the “final destination”. Just as modern technology allows us to identify and monitor our location and navigate most anywhere in the world, instructors now have many resources available to help them diagnose student learning by crafting a wide variety of formative assessment prompts (e.g., (6, 85, 87, 88)). Early in instruction particularly with a newer topic, it will likely be appropriate to apply a more openended formative assessment prompt like a Muddiest Point or minute-paper (6, 87). These types of prompts ask students to reflect on instruction, putting to pen what they have taken away from the lesson and what they are still grappling with. The advantage of open-ended formative assessments is they reveal both the range and extent of how students are thinking about a concept; forcedresponse items constrain what can be gleaned about the diversity of student thinking within the class (31, 89). The disadvantage of some open-ended prompts is that they may not yield sufficient information about student thinking to generating very specific feedback; students sometimes have difficulty identifying and articulating what it is they don’t know. Sorting tasks can be used as a “happy 210

medium”, particularly if framed as an open task whereby students are not provided a priori sorting categories (90–93). In an open sorting task, students are provided a number of cards each with a different biochemical scenario and then instructed to construct their own categories into which the cards can be sorted. For instance, having students sort structural representations of amino acids into as many different groups as they can – beyond the common groupings they may have seen in their textbook. The categories that students construct, and the cards they assign to each group, will reveal important insight into how students are (or are not) making appropriate connections between the structural features and chemical properties of amino acids. While open-ended formative assessment prompts have the potential to reveal the diversity of student thinking in a class, forced-response prompts (i.e., clicker questions) may be more convenient later in instruction when students’ ideas become more refined and closer to the desired learning outcome,. By using a variety of formative assessment prompts, instructors can more deftly diagnose the present state of the learners in their classroom to generate detailed and specific feedback. Table 3. Concept Inventories That Measure Many of the Concepts Commonly Targeted by Biochemistry Curricula Can Be Used for Formative Assessment in Addition to Pre/Post Measures of Student Learning Concepts Measured

Concept Inventories

General biology and chemistry concepts

Expected Biology and Chemistry Foundational Concepts (95) Biology Concept Inventory (BCI) (96) Chemistry Concept Inventory (CCI) (97)

Molecular and cell biology concepts

Introductory Molecular and Cell Biology Assessment (IMCA) (41) Molecular Life Sciences Concept Inventory (MLSCI) (98) Molecular Biology Capstone Assessment (MCBA) (99)

Enzyme-Substrate interactions

Enzyme-Substrate Interactions Concept Inventory (ESICI) (100)

Energy and Metabolism

Diagnostic Question Clusters on Energy and Matter (101) Breathing and Respiration (102) Photosynthesis and Respiration (103)

Membrane Transport

Diffusion and Osmosis Diagnostic Test (DODT) (104) Osmosis and Diffusion Conceptual Assessment (ODCA) (105)

Visual Representations

Bonding Representations Inventory (BRI) (106)

The final characteristic of actionable feedback, individualized, is perhaps the most challenging, particularly for instructors of large-lecture courses and students with diverse academic preparation. In these settings, providing truly individualized feedback is difficult and time-consuming, if not impossible. Yet individualized feedback does not necessarily mean that a distinct feedback message is crafted for each individual learner. Rather, the receiver of the feedback message must perceive the information to be personally relevant to his/her individualized learning (83). A number of widespread and commonly held student reasoning difficulties have been documented as a result of discipline-based education research (19), including some that are specific to the teaching and learning of biochemistry (e.g., (94, 95)). Some of these difficulties are so common and consistent, that assessments have been developed to measure their prevalence (Table 3). While concept 211

inventories are often implemented as pre/post tests to measure progress over a semester (95), they can also lend themselves well as formative assessments, particularly in large-lecture settings. Concept inventories are comprised almost exclusively of multiple-choice questions that are carefully constructed based on research on students’ conceptual reasoning; the distractors represent commonly held incorrect ideas about a concept. For a given day of instruction, a subset of items from the inventory could be administered before or during class and tabulated for generation of proximal feedback. Since the items are constructed to measure ideas commonly held by students, the likelihood that one of the answer options will appeal to any single student is high. And the feedback generated, although presented to the whole class, will therefore seem more personalized and relevant to the learner. The provision of actionable feedback to students is a critical component of effective formative assessment (23). Instructors will find it easier to generate such feedback if they have a clear idea of where students are going, how to get there, and where they are in the moment. Articulating clear and explicit learning outcomes is the first step, but selecting a broad range of formative assessments will further facilitate instructors as they seek to make feedback that is specific, detailed, and individualized. In that regard, instructors must seek to diversify the arsenal of formative assessment prompts used to elicit student reasoning. And for those instructors who feel restricted by class size, strategic use of a variety of formative assessment prompts, as opposed to a single type of prompt, will provide flexibility in assessing student understanding while still maintaining a reasonable workload (31). Conclusion Assessment serves many educational purposes, from documenting program successes to monitoring student learning. As such, assessment is key to effective teaching and can unlock the door for transforming undergraduate biochemistry. With a renewed emphasis on developing our students’ mastery of foundational concepts and disciplinary practices (3–5), biochemistry instructors are tasked with supporting students as the gain fluency in biochemical discourse. In support of this endeavor, assessments should afford students opportunities to practice, reinforce, and receive feedback on the many facets of biochemical knowledge. As biochemists themselves, biochemistry instructors are well positioned to apply the principles of how people learn to scaffold opportunities that will ultimately lead to the development of expertise in biochemistry.

References 1.

2.

3.

Olson, S.; Riordan, D. G. Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics. Report to the President; Executive Office of the President: Washington, DC, 2012. American Association for the Advancement of Science. Vision and Change in Undergraduate Biology Education: A Call to Action; American Association for the Advancement of Science: Washington, DC, 2011. Clemmons, A.; Timbrook, J.; Herron, J.; Crowe, A. BioSkills Guide: Core Competencies for Undergraduate Biology, Version 2.0. QUBES Educational Resources. doi:10.25334/Q4KF37. Published online: June 21, 2019. http://qubeshub.org/qubesresources (Accessed July 29, 2019).

212

4.

5.

6. 7. 8.

9. 10.

11.

12.

13.

14.

15. 16. 17. 18.

19.

Tansey, J. T; Baird, T., Jr; Cox, M. M.; Fox, K. M.; Knight, J.; Sears, D.; Bell, E. Foundational concepts and underlying theories for majors in “biochemistry and molecular biology”. Biochem. Mol. Biol. Educ. 2013, 41, 289–296. White, H. B.; Benore, M. A.; Sumter, T. F.; Caldwell, B. D.; Bell, E. What skills should students of undergraduate biochemistry and molecular biology programs have upon graduation? Biochem. Mol. Biol. Educ. 2013, 41, 297–301. Handelsman, J.; Miller, S.; Pfund, C. Scientific Teaching; W.H. Freeman: New York, NY, 2007. Wiggins, G. P.; McTighe, J. Understanding by Design; Association for Supervision and Curriculum Development: Danvers, MA, 2005. Freeman, S.; Eddy, S. L.; McDonough, M.; Smith, M. K.; Okoroafor, N.; Jordt, H.; Wenderoth, M. P. Active learning increases student performance in science, engineering, and mathematics. Proc. Natl. Acad. Sci. U.S.A. 2014, 111, 8410–8415. National Academies of Sciences, Engineering, and Medicine (NAEM). Indicators for Monitoring Undergraduate STEM Education; National Academies Press: Washington, DC, 2018. Bell, E.; Bell, J. K.; Provost, J. Skills and Foundational Concepts for Biochemistry Students. In Biochemistry Education: From Theory to Practice; Bussey, T., Linenberger Cortes, K., Austin, R., Eds.; American Chemical Society: Washington, DC, 2019; Vol. 1337, Chapter 4. Ragan, E. J. Making a Switch to In-Class Activities in the Biochemistry Classroom. In Biochemistry Education: From Theory to Practice; Bussey, T., Linenberger Cortes, K., Austin, R., Eds.; American Chemical Society: Washington, DC, 2019; Vol. 1337, Chapter 13. Robinson, S. B.; Dolan, E.; Cornely, K.; Medlock, A.; Lee, J. K.; Lemons, P. P. The Development and Use of Case Studies. In Biochemistry Education: From Theory to Practice; Bussey, T., Linenberger Cortes, K., Austin, R., Eds.; American Chemical Society: Washington, DC, 2019; Vol. 1337, Chapter 6. Schönborn, K. J.; Anderson, T. R. Bridging the educational research‐teaching practice gap: Conceptual understanding, part 2: Assessing and developing student knowledge. Biochem. Mol. Biol. Educ. 2008, 36, 372–379. Laverty, J. T.; Underwood, S. M.; Matz, R. L.; Posey, L. A.; Carmel, J. H.; Caballero, M. D.; Fata-Hartley, C. L.; Ebert-May, D.; Jardeleza, S. E.; Cooper, M. M. Characterizing college science assessments: The three-dimensional learning assessment protocol. PLoS ONE 2016, 11, e0162333. Tibell, L. A. E.; Rundgren, C. J. Educational challenges of molecular life science: characteristics and implications for education and research. CBE-Life Sci. Educ. 2010, 9, 25–33. Offerdahl, E. G.; Arneson, J. B.; Byrne, N. Lighten the load: Scaffolding visual literacy in biochemistry and molecular biology. CBE-Life Sci. Educ. 2017, 16, es1. Airey, J.; Linder, C. A disciplinary discourse perspective on university science learning: Achieving fluency in a critical constellation of modes. J. Res. Sci. Teach. 2009, 46, 27–49. Jeffery, K. A.; Pelaez, N. J.; Anderson, T. R. Using expert data to inform the use of research methods and representations to enhance biochemistry instruction and textbook design. Biochem. Mol. Biol. Educ. 2019, DOI: 10.1002/bmb.21255. National Research Council. Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering; National Academies Press: Washington, DC, 2012. 213

20. Bennett, R. E. Formative assessment: A critical review. Assess. Educ. Princ. Policy Pract. 2011, 18, 5–25. 21. Sadler, D. R. Formative assessment: Revisiting the territory. Assess. Educ. Princ. Policy Pract. 1998, 5, 77–84. 22. Nicol, D. J.; Macfarlane‐Dick, D. Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Stud. Higher Educ. 2006, 31, 199–218. 23. Offerdahl, E. G.; McConnell, M.; Boyer, J. Can I Have Your Recipe? Using a Fidelity of Implementation (FOI) Framework to Identify the Key Ingredients of Formative Assessment for Learning. CBE-Life Sci. Educ. 2018, 17, es16. 24. Duschl, R. A.; Gitomer, D. H. Strategies and challenges to changing the focus of assessment and instruction in science classrooms. Educ. Assess. 1997, 4, 37–73. 25. Stanton, J. D.; Neider, X. N.; Gallegos, I. J.; Clark, N. C. Differences in metacognitive regulation in introductory biology students: when prompts are not enough. CBE-Life Sci. Educ. 2015, 14, ar15. 26. Kluger, A. N.; DeNisi, A. The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol. Bull. 1996, 119, 254. 27. Hepplestone, S.; Chikwa, G. Understanding how students process and use feedback to support their learning. Pract. Res. Higher Educ. 2014, 8, 41–53. 28. Hattie, J.; Timperley, H. The power of feedback. Rev. Educ. Res. 2007, 77, 81–112. 29. Ruiz-Primo, M.; Furtak, E. M. Informal formative assessment and scientific inquiry: Exploring teachers’ practices and student learning. Educ. Assess. 2006, 11, 237–263. 30. Kingston, N.; Nash, B. Formative assessment: A meta‐analysis and a call for research. Educ. Meas. Issues Pract. 2011, 30, 28–37. 31. Offerdahl, E. G.; Montplaisir, L. Student‐generated reading questions: Diagnosing student thinking with diverse formative assessments. Biochem. Mol. Biol. Educ. 2014, 42, 29–38. 32. Johnstone, A. H. Why is science difficult to learn? Things are seldom what they seem. J. Comput. Assist. Learn. 1991, 7, 75–83. 33. Talanquer, V. Macro, submicro, and symbolic: the many faces of the chemistry “triplet”. Int. J. Sci. Educ. 2011, 33, 179–195. 34. Towns, M. H.; Raker, J. R.; Becker, N.; Harle, M.; Sutcliffe, J. The biochemistry tetrahedron and the development of the Taxonomy of Biochemistry External Representations (TOBER). Chem. Educ. Res. Pract. 2012, 13, 296–306. 35. Gershon, N.; Eick, S. G.; Card, S. Information visualization. Interactions 1998, 5, 9–15. 36. Schönborn, K. J.; Anderson, T. R. The importance of visual literacy in the education of biochemists. Biochem. Mol. Biol. Educ. 2006, 34, 94–102. 37. Ainsworth, S. E. A functional taxonomy of multiple representations. Comput. Educ. 1999, 33, 131–152. 38. Schönborn, K. J.; Anderson, T. R. A model of factors determining students’ ability to interpret external representations in biochemistry. Int. J. Sci. Educ. 2009, 31, 193–232. 39. Arneson, J. B.; Offerdahl, E. G. Visual literacy in bloom: Using Bloom’s taxonomy to support visual learning skills. CBE-Life Sci. Educ. 2018, 17, ar7.

214

40. Anderson, L. W.; Krathwohl, D. R.; Bloom, B. S. A Taxonomy for Learning, Teaching, and Assessing a Revision of Bloom’s Taxonomy of Educational Objectives; Longman: New York, NY, 2001. 41. Shi, J.; Wood, W. B.; Martin, J. M.; Guild, N. A.; Vicens, Q.; Knight, J. K. A diagnostic assessment for introductory molecular and cell biology. CBE-Life Sci. Educ. 2010, 9, 453–461. 42. Duncan R. G.: Reiser, B. J. Reasoning across ontologically distinct levels: students’ understandings of molecular genetics. J. Res. Sci. Teach. 2007, 44, 938–959. 43. Marbach-Ad, G.; Stavy, R. Students’ cellular and molecular explanations of genetic phenomena. J. Biol. Educ. 2000, 34, 200–205. 44. Kozma, R. The material features of multiple representations and their cognitive and social affordances for science understanding. Learn. Instr. 2003, 13, 205–226. 45. Baddeley, A. D. Working memory. Science 1992, 255, 556–559. 46. Sweller, J.; van Merriënboer, J. J. G.; Paas, F. G. W. C. Cognitive architecture and instructional design. Educ. Psychol. Rev. 1998, 10, 251–296. 47. National Research Council. How People Learn: Brain, Mind, Experience, and School: Expanded Edition; National Academies Press: Washington, DC, 2000. 48. Chi, M. T.; Glaser, R.; Rees, E. Expertise in Problem Solving; Technical Report No. 5; University of Pittsburgh Learning Research and Development Center: Pittsburgh, PA, 1981. 49. Larkin, J.; McDermott, J.; Simon, D. P.; Simon, H. A. Expert and novice performance in solving physics problems. Science 1980, 208, 1335–1342. 50. Ericsson, K. A.; Kintsch, W. Long-term working memory. Psychol. Rev. 1995, 102, 211. 51. Alloway, T. P. How does working memory work in the classroom? Educ. Res. Rev. 2006, 1, 134–139. 52. Cowan, N. The magical number 4 in short term memory: a reconsideration of mental storage capacity. Behav. Brain Sci. 2001, 24, 87–114. 53. Halford, G. S.; Baker, R.; McCredden, J. E.; Bain, J. D. How many variables can humans process. Psychol. Sci. 2005, 16, 70–76. 54. Sweller, J. Cognitive load theory, learning difficulty, and instructional design. Learn. Instr. 1994, 4, 295–312. 55. Miller, G. A. The magical number seven plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 1956, 63, 81–97. 56. Cooper, M. M.; Underwood, S. M.; Hilley, C. Z.; Klymkowsky, M. W. Development and assessment of a molecular structure and properties learning progression. J. Chem. Educ. 2012, 89, 1351–1357. 57. Loertscher, J.; Green, D.; Lewis, J. E.; Lin, S.; Minderhout, V. Identification of threshold concepts for biochemistry. CBE—Life Sciences Education 2014, 13, 516–528. 58. Sevian, H.; Talanquer, V. Rethinking chemistry: a learning progression on chemical thinking. Chem. Educ. Res. Prac. 2014, 15, 10–23. 59. Schwarz, C. V.; Reiser, B. J.; Davis, E. A.; Kenyon, L.; Acher, A.; Fortus, D.; Shwartz, Y.; Hug, B.; Krajcik, J. Developing a learning progression for scientific modeling: Making scientific modeling accessible and meaningful for learners. J. Res. Sci. Teach. 2009, 46, 632–654. 60. Talanquer, V. Progressions in reasoning about structure–property relationships. Chem. Educ. Res. Prac. 2018, 19, 998–1009. 215

61. Ashcraft, M. H.; Kirk, E. P. The relationships among working memory, math anxiety, and performance. J. Exp. Pscyhol. 2001, 130, 224–237. 62. Chandler, P.; Sweller, J. Cognitive load theory and the format of instruction. Cognit. Instr. 1991, 8, 293–332. 63. Brownell, S. E.; Freeman, S.; Wenderoth, M. P.; Crowe, A. J. BioCore Guide: a tool for interpreting the core concepts of Vision and Change for biology majors. CBE-Life Sci. Educ. 2014, 13, 200–211. 64. Crowe, A.; Dirks, C.; Wenderoth, M. P. Biology in bloom: implementing Bloom’s taxonomy to enhance student learning in biology. CBE-Life Sci. Educ. 2008, 7, 368–381. 65. Toledo, S.; Dubas, J. M. Encouraging higher-order thinking in general chemistry by scaffolding student learning using Marzano’s taxonomy. J. Chem. Educ. 2015, 93, 64–69. 66. Chi, M. T. H.; Wylie, R. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educ. Phsychol. 2014, 49, 219–243. 67. Rybarczyk, B. J.; Walton, K. L.; Grillo, W. H. The development and implementation of an instrument to assess students’ data analysis skills in molecular biology. J. Microbiol. Biol. Educ. 2014, 15, 259–267. 68. Angra, A.; Gardner, S. M. The graph rubric: development of a teaching, learning, and research tool. CBE—Life Sci. Educ. 2018, 17, ar65. 69. Evans, H. G.; Heyl, D. L.; Liggit, P. Team-based learning, faculty research, and grant writing bring significant learning experiences to an undergraduate biochemistry laboratory course. J. Chem. Educ. 2016, 93, 1027–1033. 70. Frame, T. R.; Cailor, S. M.; Gryka, R. J.; Chen, A. M.; Kiersma, M. E.; Sheppard, L. Student perceptions of team-based learning vs traditional lecture-based learning. Am. J. Pharm. Educ. 2015, 79, 51. 71. Haidet, P.; Kubitz, K.; McCormack, W. T. Analysis of the team-based learning literature: TBL comes of age. J. Excellence Coll. Teach. 2014, 25, 303. 72. Michaelsen, L. K.; Davidson, N.; Major, C. H. Team-Based Learning Practices and Principles in Comparison With Cooperative Learning and Problem-Based Learning. J. Excellence Coll. Teach. 2014, 25. 73. Bailey, C. P.; Minderhout, V.; Loertscher, J. Learning transferable skills in large lecture halls: Implementing a POGIL approach in biochemistry. Biochem. Mol. Biol. Educ. 2012, 40, 1–7. 74. Loertscher, J.; Minderhout, V. Implementing Guided Inquiry in Biochemistry: Challenges and Opportunities. In Biochemistry Education: From Theory to Practice; Bussey, T., Linenberger Cortes, K., Austin, R., Eds.; American Chemical Society: Washington, DC, 2019; Vol. 1337, Chapter 5. 75. Lang, J. M. Small Teaching: Everyday Lessons from the Science of Learning; Jossey-Bass, A Wiley Brand: San Francisco, CA, 2016. 76. Roediger, H. L., III; Butler, A. C. The critical role of retrieval practice in long-term retention. Trends Cogn. Sci. 2011, 15, 20–27. 77. Novak, G. M; Patterson, E. T.; Gavrin, A. D.; Christian, W. Just in Time Teaching: Blending Active Learning with Web Technology; Prentice Hall: Upper Saddle River, NJ, 1999. 78. Simkins, S.; Maier, M. Just-in-Time Teaching: Across the Disciplines, across the Academy; Stylus Publishing, LLC: Sterling, VA, 2010. 216

79. Fagen, A. P.; Crouch, C. H.; Mazur, E. Peer instruction: Results from a range of classrooms. Phys. Teach. 2002, 40, 206–209. 80. Vickrey, T.; Rosploch, K.; Rahmanian, R.; Pilarz, M.; Stains, M. Research-based implementation of peer instruction: A literature review. CBE—Life Sci. Educ. 2015, 14, es3. 81. Haak, D. C.; HilleRisLambers, J.; Pitre, E.; Freeman, S. Increased structure and active learning reduce the achievement gap in introductory biology. Science 2011, 332, 1213–1216. 82. Carless, D. Differing perceptions in the feedback process. Stud. Higher Educ. 2006, 31, 219–233. 83. Winstone, N. E.; Nash, R. A.; Parker, M.; Rowntree, J. Supporting learners’ agentic engagement with feedback: A systematic review and a taxonomy of recipience processes. Educ. Psychol. 2017, 52, 17–37. 84. Jonsson, A. Facilitating productive use of feedback in higher education. Act. Learn. Higher Educ. 2013, 14, 63–76. 85. Barkley, E. F.; Major, C. H. Learning Assessment Techniques: A Handbook for College Faculty; Jossey-Bass, A Wiley Brand: San Francisco, CA, 2015. 86. American Society of Biochemistry and Molecular Biology Educational Strategies; http://www. asbmb.org/education/educationstrategies/foundationalconcepts/ matterenergytransformation/ (Accessed June 14, 2019). 87. Angelo, T. A.; Cross, K. P. Classroom Assessment Techniques: A Handbook for College Teachers; Jossey-Bass, a Wiley Brand: San Francisco, CA, 1993. 88. Hattie, J.; Clarke, S. Visible Learning: Feedback; Routledge: New York, 2018. 89. Smith, J. I.; Tanner, K. The problem of revealing how students think: Concept inventories and beyond. CBE—Life Sci. Educ. 2010, 9, 1–5. 90. Galloway, K. R.; Leung, M. W.; Flynn, A. B. Patterns of reactions: a card sort task to investigate students’ organization of organic chemistry reactions. Chem. Educ. Res. Pract. 2019, 20, 30–52. 91. Irby, S. M.; Phu, A. L.; Borda, E. J.; Haskell, T. R.; Steed, N.; Meyer, Z. Use of a card sort task to assess students’ ability to coordinate three levels of representation in chemistry. Chem. Educ. Res. Pract. 2016, 17, 337–352. 92. Krieter, F. E.; Julius, R. W.; Tanner, K. D.; Bush, S. D.; Scott, G. E. Thinking like a chemist: development of a chemistry card-sorting task to probe conceptual expertise. J. Chem. Educ. 2016, 93, 811–820. 93. Smith, J. I.; Combs, E. D.; Nagami, P. H.; Alto, V. M.; Goh, H. G.; Gourdet, M. A.; Hough, C. M.; Nickell, A. E.; Peer, A. G.; Coley, J. D.; Tanner, K. D. Development of the biology card sorting task to measure conceptual expertise in biology. CBE—Life Sci. Educ. 2013, 12, 628–644. 94. Robic, S. Mathematics, thermodynamics, and modeling to address ten common misconceptions about protein structure, folding, and stability. CBE-Life Sci. Educ. 2010, 9, 189–195. 95. Villafañe, S. M.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Uncovering students’ incorrect ideas about foundational concepts for biochemistry. Chem. Educ. Res. Pract. 2011, 12, 210–218.

217

96. Garvin-Doxas, K.; Klymkowsky, M. W. Understanding randomness and its impact on student learning: lessons learned from building the Biology Concept Inventory (BCI). CBE—Life Sci. Educ. 2008, 7, 227–233. 97. Krause, S.; Birk, J.; Bauer, R.; Jenkins, B.; Pavelich, M. J. Development, Testing, and Application of a Chemistry Concept Inventory; 34th ASEE/IEEE Frontiers in Education Conference, Savannah, GA, 2004. 98. Howitt, S.; Anderson, T.; Costa, M.; Hamilton, S.; Wright, T. A concept inventory for molecular life sciences: how will it help your teaching practice? Aust. Biochem. 2008, 39, 14–17. 99. Couch, B. A.; Wood, W. B.; Knight, J. K. The Molecular Biology Capstone Assessment: a concept assessment for upper-division molecular biology students. CBE—Life Sci. Educ. 2015, 14, ar10. 100. Bretz, S. L.; Linenberger, K. J. Development of the enzyme–substrate interactions concept inventory. Biochem. Mol. Biol. Educ. 2012, 40, 229–233. 101. Wilson, C. D.; Anderson, C. W.; Heidemann, M.; Merrill, J. E.; Merritt, B. W; Richmond, G.; Silbey, D. F.; Parker, J. M. Assessing students’ ability to trace matter in dynamic systems in cell biology. CBE—Life Sci. Educ. 2006, 5, 323–331. 102. Mann, M.; Treagust, D. F. A pencil and paper instrument to diagnose students’ conceptions of breathing, gas exchange and respiration. Aust. Sci. Teach. J. 1998, 44, 55–59. 103. Haslam, F.; Treagust, D. F. Diagnosing secondary students’ misconceptions of photosynthesis and respiration in plants using a two-tier multiple choice instrument. J. Biol. Educ. 1987, 21, 203–211. 104. Odom, A. L.; Barrow, L. H. The development and application of a two-tiered diagnostic test measuring college biology students’ understanding of diffusion and osmosis following a course of instruction. J. Res. Sci. Teach. 1995, 32, 45–61. 105. Fisher, K. M.; Williams, K. S.; Lineback, J. Osmosis and diffusion conceptual assessment. CBE—Life Sci. Educ. 2011, 10, 418–429. 106. Luxford, C. J.; Bretz, S. L. Development of the bonding representations inventory to identify student misconceptions about covalent and ionic bonding representations. J. Chem. Educ. 2014, 91, 312–320.

218

Chapter 10

Best Practices in Summative Assessment Heather L. Tienson-Tseng* Department of Chemistry and Biochemistry, University of California, Los Angeles, 607 Charles E. Young Drive East, Box 951569, Los Angeles, California 90095, United States *E-mail: [email protected].

In most of higher education, students are assigned grades based on an assessment of knowledge and skills they have after completing the course. This is known as summative assessment, and is different from formative assessment, which is used to provide feedback to students about their ongoing learning. Grades related to summative assessment are used to allow students to move on to higher level courses, as well as to evaluate the suitability of the student for graduate programs, professional schools, and/or employment. Thus, it is imperative that these assessments are as accurate, fair, and unbiased as possible. In order to accomplish this, the students must have a clear understanding of what is expected of them, the assessment must actually assess those skills, and the assignments must be evaluated consistently for every student. In addition to assigning grades, summative assessment can also be used to assess the overall effectiveness of instruction and therefore to inform curricular changes within the course or an entire program.

The Purposes of Assessment Assessments in the classroom can serve many purposes, such as assigning grades, adjusting the pace of the course, revising instruction, or developing curriculum. At their core, all of these purposes depend on accurately determining what students have learned. Broadly there are two forms of assessment, formative and summative. Formative assessments, discussed further in Chapter 9 of this book (1), are lower stakes assignments that provide students with the opportunity to practice, receive feedback on their progress and correct their mistakes. These assignments can inform both the instructor and the student of where students are in their learning. This feedback can be used immediately for students to evaluate study strategies or for instructors to adjust the pacing of a course by either reviewing concepts students need more time with or moving forward if students demonstrate sufficient learning. Summative Assessments are higher stakes assignments containing more difficult tasks for the students that are usually used for the purpose of assigning grades. Higher stakes assignments are usually longer and contribute more to determining grades (i.e. an exam worth 30% of a student’s grade versus a homework assignment, graded on completion, and worth 1 % © 2019 American Chemical Society

of the final grade). Students are usually provided their overall score on a summative assessment, but they are not always told where they made mistakes or how the overall score was determined. Summative assessments can also be used in a formative way when they are returned to students with explanations of their errors (2). While formative and summative assessment have different overall purposes, the assessment instruments are not clearly distinct. Most types of assignments can be used as either formative or summative assessments, or both, depending on the impact on grades and the type and timeliness of feedback. Quizzes and homework assignments are generally formative assessments. However, if they are a larger portion of the course assignments and evaluate deeper student learning they would also be appropriate summative assessments. While, exams, papers, or long term projects are generally used for summative assessment, midterm exams, for example, can also serve as feedback to the students and as an opportunity to learn from their mistakes. Instructors can highlight midterm exams as a formative assessment by giving opportunities to review and/or revise the exam for additional credit. Breaking up larger projects or papers into smaller components, such as an activity to learn a skill necessary for the final project, or rough drafts with extensive feedback, will make parts of those assignments formative as well. Many STEM courses used to have one or two midterms along with a final exam, thus basing the entire grade on two or three high stakes assignments with little to no formative assessment. However, research shows that more students succeed in mastering the course material if there is more structure, i.e. more types of assignments and multiple instances of each type of assignment. With this design, summative assignments earlier in the course take on a dimension of formative assessment as well. A course designed to encourage students to review repetitive types of assignments as well as learn from their mistakes can create valuable learning experiences for students, which in turn can help ease the anxiety students develop around higher stakes assignments (3). This chapter will focus on creating and evaluating summative assessments, including considerations for constructing assignments, evaluating student learning from them, and ultimately assigning a grade based on them. Course Revision In addition to assigning grades, the results of both formative or summative assessment can also be used after the course to revise instruction or to evaluate course and curriculum development efforts. Education is sometimes seen as a linear process: the instructor designs what the students will do in order to learn, the students learn something, they are assessed on that learning, and the perceived learning gains are used to grade the students. However, good instructors are constantly reflecting on the effectiveness of their teaching. Thus, courses should be constantly evolving based on feedback from multiple sources including student evaluations, classroom observations, formative assessments, and from student performance on summative assessment. The measured learning gains can be used to update the course content, activities, and the assessments themselves, if they are to be used in subsequent terms. Information on sustaining instructional practice reform can be found in Chapter 11 of this book (4). When reviewing assessment data for the purpose of evaluating and revising educational material, there are two things to consider. Is the assessment valid, i.e. does it accurately assesses what it is intended to assess? This is determined by actually asking students what they were thinking and why they chose to provide certain answers. Is the assessment reliable, i.e. will students with the same knowledge or understanding provide similar answers and receive similar scores? This is accomplished by repeating the assessment multiple times and assessing consistency. Evaluation of 220

reliability is therefore not possible on questions that are used only a single time. A test can be valid and not reliable, or reliable but not valid. For any given question there can be three general outcomes: most students correctly answer the question, most students answer the question incorrectly, or an intermediate percentage of students answer it correctly. As far as determining a grade, the first two possibilities are not very helpful. However, those questions can be quite helpful for course or curriculum revisions. If most of the students are answering a question correctly or incorrectly, reflect on the difficulty of the assessment (it may be too easy or too hard for the level of the course) and whether it is assessing what is intended. However, as long as the assessment is aligned with the learning objectives and the level of difficulty is what was intended (whether it is easier or more difficult), assignments related to those topics would not need to be revised or adjusted. Particularly difficult questions can be useful to evaluate the extent of students’ critical thinking skills. When fewer students answer a question correctly than is desired, the instruction for that topic should be revised or the question may need to be rewritten. Students may underperform on an assessment for a number of reasons: either the students did not understand the question or the topic, or the students interpreted the question differently than intended. If the students did not understand the question properly, the question should be revised if it is to be used again. In this case, it would be important to get feedback from students on what they were thinking when they read and answered the question, specifically to identify the parts of the problem that they were misinterpreting. Then the question should be clarified, either with additional instructions or by rewording the statements, but be careful to not be too leading with the revision. It should be noted that this advice assumes that the confusion does not arise from a lack of understanding of the concept itself, but rather from the format of the question. If students did not understand the concept well enough to answer the question properly or fully, reflect on how this concept is taught, what opportunities the students were given to practice, and if the assessment is at the appropriate level for the course. Once you think you have identified the problem, revise the content of the course to help the students better understand the topic. This may include introducing the concept in a different way, updating in-class activities, or adding more or different practice on the topic. Keep in mind that the next time the course is taught, the effectiveness of the updates should be evaluated using various forms of assessment and further reflection on the changes may be warranted. It is also possible that students came up with an alternative scientifically plausible interpretation of the question or answer to the question. In this case, a revision of the rubric may be warranted to encompass this alternate answer. If the alternate answer is not what was intended to be tested, the question should be revised to clarify or to exclude the alternate answer. Curriculum Development Departments can be motivated to change curriculum when students are unhappy with the course content, students are not effectively learning the prerequisite material for subsequent courses, or the content is outdated and needs to be revised. Regardless of the motivation for change, there should be some measure of whether the change is achieving the desired outcome, and what impact the change is having on student learning. In some cases, these two assessments will overlap, but not in all cases. Appropriate summative assessments at the end of courses can be used to evaluate the impact of curricular changes on students’ learning. In most instances, easily graded and analyzed questions involving mainly instructor generated responses will be the desired form of assessment. However, it is just as important to measure the impact on students’ critical thinking ability. A robust and 221

creative assessment program aligned throughout the courses will provide valuable data for informing department wide curricular change.

Establishing Student Expectations Given the various important uses of assessment data, it is imperative that the assessments of student learning are accurate. High quality assessments arise from clear learning expectations and instructional activities designed to support students in achieving those expectations. Clear expectations direct students to the importance of the lesson and communicate what they should be able to do and why it is important. Thus, if many of the students answer a question incorrectly it does not necessarily reflect their ability to learn. Students may have not understood the importance of the given topic or connections that were being taught; i.e. they didn’t focus on it while they were studying. The best, most widely accepted way to communicate to students what is expected of them is to share a set of course learning goals, learning objectives or student learning outcomes (SLOs). Learning goals are broader statements that communicate the larger importance of topics in the course as well as what main takeaways the instructor hopes the students will gain. Learning objectives and learning outcomes are sometimes used interchangeably, however, a distinction can be made between them. Learning objectives are generally more specific statements of learning goals, and communicate what the instructor intends for the student to get out of a specific lesson or activity. Student learning outcomes describe what the student should be able to do after a particular lesson or by the end of the course. Thus, learning objectives are more instructor centered while learning outcomes are more student centered. Learning outcomes are usually written as: After the class/lesson/activity students will be able to do “X”. The “X” begins with a verb reflecting various skills and abilities associated with the specific knowledge of the course. For the assessment of learning outcomes, it is important that these statements are specific, measurable, and attainable within the timeframe of the course. Table 1 contains examples of student learning outcomes that are not specific, measureable, or attainable, as well as examples of how to modify them to meet these criteria. Additional Resources There is extensive literature on writing and using high quality student learning outcomes. Some excellent resources are Biology in Bloom (5), Assessment in the College Classroom (2), and Teaching at its Best (6). The three-dimensional learning assessment protocol provides a method for aligning assessments with the three dimensional learning framework. This is useful for integrating scientific practices, scientific ideas that cut across disciplines, and discipline specific core ideas (7). An important consideration before writing learning outcomes is what content should be included in a course. For many of our classes there is more content that could be taught than students can master in the given term. Thus, the actual content of the course must be selected ahead of time. Writing a set of clear learning outcomes before the course begins allows the instructor to thoughtfully curate the content the students will learn. Deciding on the entire course content ahead of time can also ensure it is the appropriate difficulty and amount for the course. Even after the topics of the course have been decided instructors can choose to focus on different aspects of the same content. Thus, additional opinions on course content and important aspects of that content are useful for both new and long term instructors. Information on biochemistry course content and appropriate learning outcomes is available on the American Society for Biochemistry and Molecular Biology website 222

(8). Another perspective, on course content and how to focus a course, is thinking about threshold concepts for biochemistry (9). Threshold concepts are topics that are at the core of a discipline that students must understand in order to master the material, but tend to be particularly difficult for students to fully comprehend. Additional information on biochemistry specific learning goals and outcomes can also be found in Chapter 9 of this book (1). Table 1. Examples of Student Learning Outcome Characteristics SLOs should be: Specific

Measurable

Attainable

Which is:

Examples:

Clear statement of what the student must be able to do

Unclear

Understand the regulation of glycolysis

Specific

– Identify the regulated steps in glycolysis – Predict regulators within glycolysis

Uses a verb that can actually be measured by the instructor

Cannot be measured

Know the Michaelis-Menten kinetic parameters

Measurable

– Define the variables found in the Michaelis-Menten equation – Calculate the Michaelis- Menten parameters given data – Explain the function of an enzyme in terms of its kinetic parameters

The goal is reasonable for the students to achieve in the available time period.

unattainable

Design a research project to address a currently unanswered question in the field*

attainable

– Develop a research question based on currently available data – Design an experiment to test a hypothesis based on currently available data.

* This

may be attainable in a course where the full term is devoted to research design, but it would be better suited as a Learning Goal not a single learning objective.

Categorizing Learning Outcomes Within the scope of a course, instructors intend to assess students’ knowledge, skills, and ability to apply those skills in different contexts. Student learning outcomes should reflect the different cognitive levels required of students when demonstrating mastery of the material. There are various ways that learning has been categorized, all of which involve a hierarchy of categories that reflect increasing levels of complexity. The simplest classification is Lower Order Cognitive level (LOC) vs Higher Order Cognitive level (HOC). This divides any given assessment, either the whole assignment or individual parts, into two levels. LOC assessments are generally agreed to include remembering and understanding, and sometimes simpler application of information, whereas HOC assessments include analyzing information, evaluating provided information, or creating new information. Two additional classifications further separate LOC and HOC levels into more distinct categories that focus on different types of cognitive activities. Marzano’s taxonomy focuses on the amount of cognition necessary for students to solve the problem, specifically routine versus critical thinking (10). Bloom’s 223

taxonomy of learning focuses on the presumed difficulty of the task for the student (11). Table 2 compares the different classification systems. Table 2. Alignment of Learning Taxonomies

LOC

Marzano’s Taxonomy

Revised Bloom’s Taxonomy

Retrieval

Remember

Comprehension

Understand (comprehend) Apply

HOC

Analysis

Analyze

Knowledge Utilization

Evaluate Create

Table 3. Learning Outcomes from Table 1 Categorized by the Revised Bloom’s Taxonomy Remember (identify or recall) Factual (Basic Information; parts of a concept)

Conceptual (Relationships between the parts)

Understand (explain, interpret, compare)

Apply (Use )

Evaluate (checking, critiquing)

Create (generate, design, construct)

Define the variables found in the MichaelisMenten equation Explain the function of an enzyme in terms of its kinetic parameters

Predict regulators within glycolysis

Procedural (how to do something)

Metacognitive (problem solving strategies, evaluation of selfknowledge)

Analyze (differentiate or organize)

Calculate the Michaelis Menten parameters given data

Adapt study strategies based on performance on practice problems.*

* In typical STEM courses the metacognitive domain is often ignored despite its importance.

224

Develop a research question based on current available data Design an experiment to test a hypothesis based on current available data.

The Revised Bloom’s taxonomy is a two-dimensional grid consisting of both the cognitive dimension and the knowledge dimension. The cognitive dimension consists of the revised Bloom’s taxonomy shown Table 2, while the knowledge dimension divides each of those into four additional possibilities: factual knowledge, conceptual knowledge, procedural knowledge and metacognitive knowledge (11). Table 3 categorizes the sample learning outcomes from Table 1 using the twodimensional revised Bloom’s taxonomy. There are a number of resources that provide examples of activities and assignments that generally map to each level of the taxonomy. However, it is important to remember that the classification of a particular activity may be course specific. Take the question posed in Figure 1 as an example.

Figure 1. Sample summative assessment question. For an instructor who devotes a significant amount of class time and practice opportunities to having the students study the function and regulation of phosphofructokinase specifically, this question most likely is testing “remember” even though its intention may have been to probe “understand”. For another instructor who focuses on why and how pathways are regulated, but does not give specific information and practice on phosphofructokinase, this question may be targeting the level of analyze or evaluate.

225

Evaluation of Student Learning The first step of designing a new course or writing a new class activity is to write learning outcomes, i.e. decide what the students should be able to do. The instructor should then use the learning outcomes to decide on appropriate assessments (both summative and formative) and develop activities that will help the student achieve the learning outcomes. Informative assessments are aligned with the learning outcome and allow the instructor to identify known misconceptions or incomplete mental models the students may possess. A mental model is what a student sees happening in their head when they think about a concept or question (12). Their mental model can be complete, correct but lacking key pieces, or contain incorrect ideas. An effective way to assess what a student has learned is to evaluate the accuracy of their mental models. To do this the students must be asked to generate an answer from their mental model, not to choose from provided answers (13). For example, an assessment can make them predict outcomes due to changes in the system and explain their reasoning, or draw a diagram of a part of the concept. Instructors can identify misconceptions through their own students’ responses or through education research literature. In order for the evaluation to be reasonable, these instruments should be aligned with the learning outcomes communicated to the students. This is most directly achieved by backward design, which consists of instructors going through the following steps (11, 14): 1. Write SLOs for the desired topic. 2. Define how a student would demonstrate achievement of a given learning outcome. 3. Write an assessment instrument or question that requires the student to do Step 2. This is the summative assessment for this learning outcome. 4. Identify mechanisms to help the students gain and practice the knowledge and skills required for achieving this learning outcome. These include activities in and outside of class, as well as formative assessments to give the student feedback on whether they have mastered a topic. 5. Communicate and emphasize the SLOs to the students. Include SLOs in the syllabus and assignments throughout the course. Refer back to the SLOs when discussing topics and assignments with students. 6. Reflect on the validity of the assessment and the effectiveness of the activities. (Use assessment data to revise the course, as discussed earlier in the course revision section)

Summative Assessments Summative assessments come in a variety of shapes and sizes. The type of assignment and questions asked depend on the desired knowledge and skills to be tested. Additionally, there are various considerations to take into account when writing certain types of questions or assignments. This sections will detail those considerations and provide tips for addressing them.

Types of Questions Used for Summative Assessments There are many methods of summative assessment through which students can demonstrate their comprehension and critical thinking skills. Each type of question has advantages and disadvantages in terms of difficulty in construction, the depth of comprehension and skills that can be assessed, and the ease at which this assessment information can be obtained. This section will explore each of these aspects in greater depth. 226

Questions can broadly be categorized as open or closed and further divided by whether the answers are instructor generated or student generated. Closed questions have a single correct answer and are usually LOC, although they can be HOC as well. Open questions will have multiple answers that can be awarded full or partial credit. The variation in student responses can include the final answer itself or multiple paths to get to that final answer. These types of questions are generally more difficult for students to provide complete correct answers to and more difficult to grade given the variety of acceptable answers. As is implied by the terminology, instructor generated answers require students to choose from a set of possibilities provided by the instructor. These answers can make grading faster and, if crafted thoughtfully, can accurately identify students’ understandings and misunderstandings. Free response questions, where the students have to determine what type of information is necessary to answer the question, are the most straightforward way to fully assess student understanding. Two specific prompts that can lead to a wealth of information about how students understand and perceive a topic are “Explain your reasoning” or “Draw a representation of what is happening”. Both of these require a clear and organized mental model of a topic in order to give an accurate and coherent response. Ultimately open-ended questions that require the students to generate answers themselves will provide the richest information about a student’s understanding, thought process, and overall learning (2). Table 4 provides a summary of the characteristics of different question types. Table 4. Summary of Question Types Question type Multiple Choice

Open or closed

Who generates response

Ease of grading

Ease of writing

Cognitive level of question

Closed

Instructor

Easy

Difficult

LOC or HOC

Ordered Multiple Closed Choice

Instructor

Easy

Difficult

LOC and HOC

True/False

Closed (can be made open)

Instructor (can be modified for student response)

Easy

Moderate

LOC (modified for HOC)

Short Answer

Usually closed

Student

Easy

Moderate

Usually LOC

Long Answer

Closed or open

Student

Moderate

Easy*

LOC or HOC

Essay/Project

Open

Student

Difficult

Easy*

HOC

* While

the question itself may be easier to write, these types of questions require thoughtful scaffolding or instructions that may take more time to construct or may not seem necessary initially.

Multiple Choice Questions (MCQ) MCQs are by far the easiest question type to grade and potentially gather information and statistics about student learning. This is one reason they can be very useful for formative assessment. However, they may be one of the more difficult question types to write well given all the pitfalls that writers can fall into (15). The difficulty in writing MCQs is ensuring the student knew the correct answer before reading the options. There are a few characteristics that students can look for to narrow down their choices and ultimately make an educated guess as to the right answer, without actually knowing the answer. Test-wise students know to look for the following things: 227

• Correct answers tend to be longer more detailed answers choices • Absolutes are usually wrong • Answers in the middle are correct more often than the first or last answer provided. Here are some tips for avoiding the above pitfalls: • Make all answers similar in structure and length. • Avoid using absolutes, i.e. always, never, or all, especially in only a single answer choice. • Avoid obvious wrong answers. It is better to have fewer options than one that is obviously wrong. Three to five options are ideal (16). • Use student-generated answers as distractors. Use the phrasing from previous studentgenerated answers, to questions on the same topic to create distractors that probe a specific misconception. • Write answers and then randomize their order, unless there is an obvious order to list answers in. • For answers with obvious orders (i.e. numerical answers) make sure the correct answer is A, B, C, and D with the same frequency. • Check for grammatical errors, particularly in the grammatical agreement between the question stem and the answers. Correct answers will always be grammatically correct. Other common problems with MCQs lead to confusing questions, which unintentionally increase the difficulty of the question. These problems can include the use of negative statements, double negatives, or long and overly complicated phrasing. In the case of obvious clues or confusing questions, getting the correct answer may involve testing more than just the content knowledge intended or may require just plain luck. One final consideration is to always provide a single correct answer rather than possible combinations of choices that are all correct. While these options can stimulate further conversation during formative assessment, they are less informative for summative assessment. • If a set of statements (I-IV) are arranged in different groups (A-E) indicating the potentially correct statements (i.e. option A is both statement I and II, while option B is II and III), then it is impossible to determine which statements students would have grouped together. A better method is to use multiple true/false statements (discussed further in the next section). • If using all of the above, students who can determine that at least two statements are true know that all answers must be true. • If using none of the above, you will not know what the student thinks is the correct answer unless you ask a follow-up question. Multiple choice assessments naturally lend themselves to statistical analysis of the quality of the exam and student learning. They are evaluated using multiple metrics, but there are two metrics that are commonly included in any analysis provided by a scoring center: the difficulty and the discrimination index (17). These values can help evaluate the quality of each question and decide if a question should be included and/or updated, if it will be used on future assessments. The difficulty is essentially the percentage of students who answer the question correctly. The higher the percentage, the easier the question. Questions that have a high (greater than 0.8) or a low (lower than 0.3) difficulty factor may be too easy or too hard, respectively (17). 228

A second useful metric is the discrimination index. This is the correlation between a student getting the question correct and scoring high on the exam. In other words, how well does the question discriminate between higher performing students versus lower performing students. This will be somewhat linked to the difficulty of the question. If 100% or 0% of the students answer the question correctly, it cannot discriminate between students of different abilities. The discrimination index can range from -1 to +1. A more positive number reflects a stronger correlation between getting the answer correct and scoring higher on the overall exam. Lower positive numbers indicate little correlation, and may be related to items that are too easy or too hard and thus offer little discrimination. Items that have a negative discrimination index are problematic, as they indicate better performing students are less likely to get that question correct. To calculate a discrimination index, a question can only have one correct answer. Questions that are determined to be too easy, too hard, or have a low (or particularly a negative) discrimination index should be re-evaluated Potential problems with the question may be that the content was not properly aligned with the learning outcomes for the class, there was an obvious clue to the correct answer, or the question and/or answers were confusing or unclear. Another possibility for questions with a negative discrimination index is that the answer key was wrong. Ordered Multiple Choice Questions (OMC) OMCs are multiple choice questions that contain multiple correct answers that relate to increasing levels of understanding of the concept, from not knowing the concept, through a novice understanding, to an expert level of understanding. The levels usually reflect understanding expected at a point in the student’s education: before taking an introductory class, after an introductory course, after an advanced course, as an incoming graduate student, and PhD or expert level understanding. Thus the data collected indicates not only if the student understands the concept, but also the level of understanding of the student. The first step in writing OMCs is producing a construct map for the concept, which includes a description of what a student at each level would be expected to know. Next, each answer choice should reflect a distinct level of understanding (18). These questions offer a richer level of understanding of what mental models the students hold (18). However, these questions are more difficult and time consuming to write. In fact, they would most likely require a community of instructors to write and revise. At the very least, the construct map should be constructed collaboratively (19). True/False True or false questions require a student to evaluate multiple statements for their accuracy. These are easy to grade and can be a more valuable source of information than multiple choice questions. Students have to evaluate the veracity of each statement on its own as opposed to determining if it is more or less correct than other statements in a multiple choice question. Many of the same considerations given to multiple choice answers should also be applied to true/false statements. To assess a student’s ability to evaluate multiple statements about the same situation, multiple true/false statements are better than multiple correct answers to an MCQ. Write the multiple choice question stem and answer choices, then ask the students to evaluate whether each one is true or false. In terms of scoring, this would be reflected as multiple questions coming from the same stem. Another option for gaining more information about student understanding is to use true/false statements along with mistake correction. With this question format a student first determines whether the statement is true or false and then corrects the false statements to make them true. This 229

gives the question the power of having a student-generated response and eliminates the student’s ability to guess the correct answer on a multiple choice question. Short Answer Short Answer questions require the student to fill in a blank or answer a question in one sentence or less. These are generally closed questions that lend themselves more to LOC rather than HOC responses. They are easier to write than previously discussed questions and only slightly more time consuming to grade. Even though these are closed questions, with technically one correct answer, there may be a small range of acceptable answers, making it difficult to automate the grading. Overall, the advantage of a short answer question over a multiple choice question is that it requires the student to generate the response themselves. The incorrect responses can also be used to write distractors for future multiple choice questions. Long Answer These questions require students to generate a response that is equivalent to a few sentences in length or requires higher order cognitive skills. As such, they have a wide variety of student-generated response types. • • • • •

Describe a phenomena. Explain a procedure or a protocol (step-by-step process). Draw a conclusion and explain reasoning. Draw a diagram, representation, graph. Predict how a system would change under specific conditions.

Free response questions can be closed or open and LOC or HOC. These are relatively easier to write as the student’s response itself will show how they interpreted the question and thus alternative interpretations can be evaluated and considered in the grading. It is still important that the questions are clearly worded in order for students to know what is expected of them. However, the questions should not be so specifically worded that it gives hints to students who would not know the answer otherwise. A potential drawback is that these questions are more time consuming to grade based on the length of the answers and how many possible correct responses there can be. Accurate and fair grading of these questions requires a well-constructed rubric and properly trained graders, both of which will be discussed later in the section on Fairer Grading Practices. Overall, asking students to explain their reasoning or draw a representation of the phenomenon is an invaluable resource to uncover the mental models and misconceptions held by students (20). A final consideration is that students will require more practice and appropriate feedback in order to answer these questions well. They may need training on how to write coherent answers that include a specifically stated claim, evidence to support the claim, and reasoning that connects the evidence to the claim (21, 22). Essay (Large Project) Prompt Broader, open ended prompts lend themselves to lengthier assignments and projects. It can be easier to come up with a broad idea and then develop an appropriate prompt around that idea. While the question to address may be relatively easy to come up with, designing the necessary supporting 230

materials, developing each step of the project, and creating explicit instructions for the project will take more time. Additionally, these larger projects are generally more difficult to grade. They would also require detailed, well-constructed rubrics to grade consistently, especially for larger classes. Suggested Progressive Question Structure A single question can either be open or closed and can assess either LOC or HOC student skills. However, it is beneficial to assess both LOC and HOC for a given concept. It is useful to know if a given student has general knowledge about a topic as well as whether that understanding is deep enough to apply the ideas to new areas. A progressive question can be used to test both LOC and HOC on the same or related concepts. The whole question would start with a description of a scenario and any information the student would need to answer subsequent questions, which can be broken down as follows: First Part (LOC) Multiple choice or true/ false questions to assess the student’s basic knowledge of the concepts and understanding of the scenario described. Second Part (LOC) Short answer questions to either explain the choices in the first part or demonstrate comprehension of the concept. Third Part (HOC, Simple Application) A series of questions requiring student-generated responses that progress through a taxonomy of cognition. For example: • Draw conclusions about data relating to the scenario. • Predict the consequences of a change in the scenario. • Design a situation to match certain parameters. The number of questions in this section will depend on the students’ allotted time to complete the entire question as well as the time allocated to grade the question. To accurately evaluate the current mental models the students hold, at least one or two of these questions should require the students to explain their reasoning or draw a representation to illustrate the concept (9, 13, 23, 24). Fourth Part (HOC) Another useful question is one that asks students to make generalizations or draw connections between the provided scenario and other similar situations. As this requires a more abstract and broad answer, it tests the students’ ability to connect specific examples to more general concepts.

Summative Assignments A summative assessment is larger in scope and generally made up of a combination of question types. While all question types can be used for any assignment, certain questions will be better suited for certain assignments. The goals for the assignment will determine what types of questions are 231

most appropriate to write. A quiz testing general knowledge would lend itself to multiple choice and very short answer questions, while a lab report demonstrating understanding and use of the scientific method would most likely require an essay prompt. There are two main considerations for any assignment. One is that students will take longer to complete it than the instructor anticipates, especially first time assignments. The second is whether students will interpret the questions the same way the instructor intended or if there are alternative interpretations that are reasonable. Both of these potential problems can be mitigated the same way. Have someone else read the assignment and provide feedback on how long it took them to complete and how they interpreted the questions. The closer the person is to the student population in knowledge, the more relevant their feedback will be. Other students who have recently completed the course are the best option, in terms of providing a population similar to your current students. However, there are potential problems with providing undergraduates with assignment questions (especially exams) prior to giving the assignment to the class. Even though they are not currently taking the class, they may have friends in the class. Giving them knowledge of the assignment ahead of time can put them in a difficult position. An appropriate substitute would be a first-time teaching assistant or first year graduate student. While they are more advanced than the students in the class, they have learned the material more recently and may still think about the material more similarly to students rather than to an instructor. Assume that students will still require more time for the assessment than a graduate student. A graduate student can also identify potential alternative interpretations of a question beforehand. The questions can be revised to eliminate the possibilities, or the alternative interpretations can be options within the rubric. Three typical summative assessment types are discussed below. The assignments are in order based on length of time it would typically take students to complete them. For each type of assignment, there is a discussion of how they are best used as well as what problems one should be aware of in designing them. Table 5 provides a summary of these characteristics. Quizzes Generally quizzes are thought of as shorter, less formal, and easier assessments. They can be used as formative assessments as well. The use of quizzes is very flexible. Quizzes can be taken during class time, discussion/recitation sections (if available), or online, and can take advantage of any type of question depending on time available for students to take the quiz and for the instructors to grade them. To be most useful as a form of summative assessment, quizzes should contain a mixture of LOC and HOC questions and should be administered after the students have had an opportunity for learning, rather than only after an introduction to a concept or directions for memorization. Another option for making use of the advantages of quizzes is to have frequent quizzes with HOC questions on them that replace the exams in the class. These may take up more overall class time, however, if implemented thoughtfully this will provide additional practice and timely feedback on their overall progress. While this is not the specific intention of summative assessment, this type of training and feedback within the context of summative assessment can also be beneficial for students. This is an example of how summative and formative assessment overlap and work together (2). Considerations While there is flexibility in the format and administration of quizzes there are disadvantages to consider for many options.

232

• Quizzes administered during class may take up valuable class time. • Students taking quizzes in discussion/recitation may take the quiz at different times relative to receiving instruction relevant to the quiz. • Students taking a quiz earlier can tell other students what is on the quiz, whether the quiz is administered in discussion/recitation or online. • Given the time limitations, it can be more difficult to test higher order cognitive skills on quizzes, unless these will be a large portion of the course, replacing exams. • Online quizzes: It should be assumed that students will use notes and the internet, and potentially other students, when taking online quizzes. For this reason, online quizzes are best used as pre-class assignments, essentially to ensure that students have a minimum amount of background information before coming to class. In many cases this is more aligned with formative assessment. They can also be summative if they are mainly assessing factual content knowledge. Whether or not fill in the blank or short answer questions are available will be determined by what the online platform can support. Table 5. Summary of Assignment Types Assignment

Ease of Use For Instructors

Benefits For Instructors

Considerations

Adaptation for Large Classes

Quiz

Easy

Many format options, faster grading, more timely feedback, additional practice for students.

Shorter time to test higher order skills, challenges related to when and how the quiz is administered.

Online quizzes can be used to save class time and for automated grading (requires MCQ or fill in the blank)

Exam

Moderate

Possible evaluation of HOC skills

Length of time to complete, length of time to grade

Maximize use of faster graded questions like MCQ and short answer, use long free response questions strategically to identify students’ mental models of a concept.

Iterative reinforcement and development of HOC skills

Frequent feedback is required, and a high level of scaffolding within the assignment

Have students complete in groups to decrease number of submissions, break up into smaller assignments that can be graded throughout the term that will ultimately build to the complete project

Large projects Difficult

Exams Exams have as much flexibility in their format and implementation as quizzes, but are better suited for evaluation of higher order cognitive skills. While the typical exam is printed on paper that students take individually in a given time period, there are many variations to that format.

233

• Digital exams, taken on a computer. The practicality of this option depends on class size and access to computers. There are options with a digital format to randomize questions or even make the exam adaptive. • Open notes/book or student-created note sheet. In this case the exam would not test factual knowledge the students can access, but would focus on application, synthesis and evaluation; answers students cannot look up in their notes. • Take Home exams. These are the most extreme case of open note exams, in which the instructor assumes the student has access to all the information on the internet. Thus, the exam should consist of almost entirely very high cognitive level questions. Depending on the size of the class, it may be impossible to eliminate cheating in this format. • Group exams. Classes that focus on collaborative group work can reinforce that by having a portion of the exam score come from a group exam. The group portion is generally taken after the individual portion. It can be the last quarter or third of the exam time or taken at another time. If the exam time itself is limited the discussion/recitation period can be used for the group exam, if available. The questions for the group exam can be repeating a portion of the exam, usually the more difficult higher order question, or can be an even more difficult extension of one of the individual exam questions (25–27). Considerations Regardless of which types of question are included on exams, writing well-constructed questions that test an intended outcome and can be completed by the students within the allotted time can be very challenging. • Decide on question types based on how long the students have to take the exam, the amount of content to be tested, and how much time is available to grade the exam. HOC questions will take longer for the students to complete than LOC. Free response answers longer than a sentence in length or drawing a representation will take longer to grade than instructor-generated short answer questions. • Have a colleague familiar with the course look over any questions. They can provide valuable feedback on how they would interpret the question, at what cognitive level they would judge the question, and whether it aligns with the intended learning outcomes. • Refine exam questions until they are testing the desired outcomes by using an iterative process. Review the student responses, identify unintended answers and interview students to get their perspective on the question. From this information, decide if the question should be thrown out, reworded for clarification, or kept as is. Keep in mind that not all unintended answers need to be addressed when revising the question. In some cases, they can be informative of whether or not the students can interpret the question and determine what information is relevant to answer it. It is particularly important to be sure that when a student provides a particular answer, whether from choosing options or generating one on their own, that the answer correctly corresponds to the mental model the student has surrounding that concept. This generally requires discussing the questions and answers with students. • If revising and reusing questions, be exceedingly careful about releasing assessments to students. This makes the process purely summative as the students will not get feedback on their answers and therefore cannot learn from their mistakes within the structure of the

234

class. Thus, questions that will be reused are better on a final exam that does not have to be returned to the students. Large Projects These are long term projects that are fairly open ended. They are higher order cognitive assignments that take significant time for students to complete. Thus, they require the whole term or a large portion of it to complete. Due to the nature of these assignments, time and effort can be devoted to developing skills that students largely struggle with but are central to being a scientist; reading a scientific paper, analyzing data, writing a lab report, giving an oral presentation, or summarizing and presenting information from the literature. The students will need to be trained throughout this process of developing the final results and have plenty of support with feedback to create a final draft that is truly indicative of their thought process and scientific abilities. Considerations Scaffold the assignment breaking it down into smaller components. Build up to a first draft with frequent feedback through smaller assignments that are most likely formative assessments. At least some points along the way should be higher stakes to keep students on pace and accountable. Detailed instruction, frequent feedback, and a clear rubric will greatly enhance the quality of the product and the summative information gained from it. Lab Reports/Papers Through writing lab reports and literature reviews students can demonstrate skills required to communicate with the scientific community. These assignments are challenging for students if they are unfamiliar with the implementation of the scientific method and proper analysis of data or information. Feedback on their progress can be done in multiple ways. Students can complete a full draft of the report or paper and be given extensive feedback. An alternative is for them to focus on different sections one at a time and then synthesize that into a single report later in the term. See Chapter 8 in this book (28), for more information on lab notebook resources. Presentations Oral presentations are a great way to assess student understanding and oral communication skills, yet they can be time consuming to generate the assignment, watch the presentations, and grade the final presentations. As students may not have received instruction regarding oral presentations in the past, many would benefit from detailed instructions on what to include in the presentation, how to develop visual aids, and how to deliver the information. While each student/group should be given a set amount of time for their presentation this will add up to a considerable amount of time once all the presentations are complete. One option is to have the presentations during discussion or lab sections. However, to get meaningful summative data, the instructor should attend each presentation. Quality feedback on oral presentations can also be relatively lengthy to put together. Students can also benefit immensely from this feedback, giving the assignment a formative aspect as well. Chapter 14 in this book (29) discusses the use of oral presentations as an assessment of learning from primary literature.

235

Posters Having students create conference style posters about a concept in the class is a creative way to assess their ability to understand a concept in depth and to synthesize and process information from different sources. In addition to the poster itself, having a poster session in class is a great way to share the information and assess individual student’s oral communication skills. This also becomes limiting and difficult in large classes. One way to adapt this is to have groups make a poster style brochure than can then be shared with classmates.

Concept Inventories While concept inventories are largely used for formative assessment, they can also be used as part of a summative assessment. Concept inventories are iteratively designed in collaboration with a community of instructors and evaluated for their validity and reliability. Concept inventories can be used as a pre-test to gauge students’ incoming knowledge (formative assessment), as a post test to see where students are after a course or a unit (summative assessment), or as a pre and post-test to measure learning gains within the class. These instruments are intentionally designed so that known student misconceptions map to the distractors, or incorrect answers, making them invaluable tools to uncover students’ actual understanding of the concepts. Thus, the instructor can measure not only how many of the students understand a concept, but which misconceptions are most prevalent in a given class. The instructor can then tailor activities to specifically address those identified misconceptions (30). For biochemistry, there are two validated concept inventories that are readily available from the writers. One is the Foundational Concepts in Biochemistry Inventory (FCBI), which consists of 21 multiple choice questions that measure student understanding of seven concepts from chemistry and biology that serve as a foundation for biochemistry: Equilibrium, Bond Energy, Free Energy, pKa, hydrogen bonding, Alpha helix structure, and the effects of protein mutations (23, 31). Detailed examples of the use of the FCBI can be found in Chapter 12 (32) and Chapter 11 (4), and (30). The second concept inventory is the Enzyme-Substrate Interactions Concept Inventory (ESICI), which contains 15 multiple choice questions measuring student understanding of the basis and effect of both substrate and inhibitor binding to enzymes; concepts that would mainly be learned in an upper division biochemistry course (24). There is a third molecular life sciences concept inventory (MLSCI) that contains multiple truefalse questions that can be used in an adaptive fashion. The MLSCI includes questions related to equilibrium, macromolecular structure, energy transformation and genetic information coding. However, the validation state of this instrument is unclear and access is not readily available (33).

Fair Grading Practices Grading is often a difficult, but very important aspect of being an instructor. While it can be tedious to carefully and consistently evaluate each assignment, it provides a wealth of information on which we base overall grades and make subsequent changes to our courses. It is imperative for all interested parties that the grading be done accurately and consistently. Grades can be unfairly assigned for a number of reasons (34). It is possible that the grader can have an implicit bias towards the student leading to an inflation or reduction of their grade. One way to combat implicit biases is to be aware of them and consciously look for bias in your actions. The Project Implicit website has

236

different tests for a number of known implicit biases. Ultimately, the best way to prevent biases from influencing grading is by anonymizing grading of individual assignments (35). Course Grades Assigning of grades in a course will require a scheme for relating the summative assessments of student learning to a specific grade, which typically resembles either norm-referenced grading (grading on a curve) or criterion-referenced grading (also known as standards-based grading or a straight scale). Educational research supports the use of criterion-referenced grading as a fairer way to evaluate students (36). Norm-referenced grading by definition limits the possible achievement of some of the class relative to the other students, creating competition between students. While some students may thrive in this situation, certain groups of students are known to withdraw and be less successful when faced with this type of grading situation (36). Criterion-referenced grading provides an opportunity for all students to earn the grade that they deserve and to foster a sense of collaboration within the course. When using criterion-referenced grading, students are assigned grades based on their level of achievement compared to set criterion. This can be relative to a total number of points or percentage of points, or relative to mastery of certain specifications. Determine what number or percentage of points equates to each grade and communicate this scale to the students. It may be difficult for a less experienced instructor to determine the exact scale that is appropriate for the class. Thus, it is useful to maintain some flexibility with the scale. Tell students that the scale will not increase, but it may decrease. It would be appropriate to lower the scale if an exam or assignment was more difficult than intended or an activity was not effective. An alternative form of criterion-referenced grading involves setting major and minor outcomes for the course, and establishing a system for students to demonstrate mastery of each outcome. Students must master each major outcome to pass the class and can increase their grade by mastering the minor outcomes (36–38). A final consideration is that students have different talents and having a single type of high stakes summative assessment may favor some students over others, regardless of content knowledge and ability. Thus, having multiple ways to demonstrate proficiency on given topics would be an ideal way to allow all students to thrive. There are a few different ways this can be implemented: they all involve having multiple types of assignments, with relatively equal weight, that can test higher order thinking. Potential assignments other than in-class exams include take home or oral exams, papers, presentations, or even writing their own exam questions and providing answers. One option for the grading scheme is that students can choose which assignments to complete. The point ranges designated for given grades are determined so that no one type of assignment is sufficient to pass, or get an A, but a student also does not have to complete all of the assignments to pass the class. Another possibility is to have multiple grading schemes, where different assignment types are given different weights. Students can be required to choose which scheme they want to use by a deadline within the term, or their grades can be calculated using each of the grading schemes and the highest grade is assigned (39). Question/Assignment Grading Grading considerations are different based on whether the responses are instructor generated or student generated. Questions with instructor-generated responses are easy and straight forward to grade. However, once that is done the results should be analyzed, as discussed for reflecting on the course and multiple choice questions, to be confident the question functioned as intended. 237

Grading student-generated answers presents entirely new problems and considerations. Most of the time these types of questions generate more complex answers, with more than one correct response. Scores should reflect the degree to which correct reasoning was used by the student. Since there is more than one way to express a correct idea, the grading becomes subjective. The ideal situation would be to have the same person grade a single assignment, or part of an assignment for all students, to reduce grading variability. In some cases, this is not possible, for example with lab reports in large classes. In this situation some training must be given to the graders and ideally the level of inter-grader reliability should be established (40). Here are some additional practical tips for grading student-generated responses fairly: • Have a clear, detailed rubric before you start grading. • If someone else is grading, make sure they fully understand how to apply the rubric. • Be flexible and willing to adjust the rubric based on the range and extent of student responses. • Remain firm on core expectations. • Make notes along the way to remember which phrasing or idea was awarded points and which were not, in order to maintain consistency. Designing and Using Good Rubrics Rubrics are a checklist to keep track of what a student has or has not done within the context of an assignment so as to assign a grade based on that work. Rubrics generally fall into two categories, either holistic or analytical. Holistic rubrics give a general description of what the student demonstrated in their work and has a grader award one score based on a combination of characteristics. An analytical rubric would state specifically what information or logic would be included in the answer for each point and each characteristic would be evaluated separately. Table 6 provides an example of holistic versus analytical rubric items for the same concept. The analytical row represents one of many criteria that would be in the rubric. Holistic rubrics can be applied to a variety of questions, while analytical rubrics are specific to the question or assignment at hand. A potential drawback of holistic rubrics is that students must meet multiple criteria for a given score. This may lead to subjectivity in applying the rubric when some, but not all, of the criteria are met. Analytical rubrics reduce the ambiguity and subjectivity of the grading by scoring each criteria individually. However, they must be created for each new assignment. Regardless of rubric type, the more detailed and specific the rubric is the more precisely it can be applied (41). Thoughtful rubric design along with thorough grader training can ensure that the rubric is applied as uniformly as possible. This is essential for fair and accurate grading. Training and Reviewing Graders Even the best rubrics will not be useful if they are applied unevenly or in a biased manner. Many instructors have teaching assistants or graders to help with grading of larger assignments. While many of them are very dedicated, knowledgeable, and conscientious, they do not always understand the topics at an expert level or the nuances that could be reflected in the answers. It is imperative that there is open communication between the instructor and the grader. First, review the rubric with the grader, preferably with examples of real student work to demonstrate how the rubric should be applied. Then the grader should grade a small set of answers by themselves. Next, check the grading and communicate any changes in applying the rubric to the grader. Finally, after that initial consultation and calibration, check in on the grading once half of the assignment has been graded to 238

ensure that is it being graded consistently and the rubric is being applied correctly. Much of this can be done by email or with short meetings if a platform such as Gradescope (see tips below) is being used. Table 6. Examples of Holistic versus Analytical Rubric Items for a Question Asking Students to Predict the Effect of a Ligand Binding to a Protein Rubric type

3

2

1

Holistic

Demonstrated complete understanding of the concept

Description of concept was incomplete or contained errors

Did not address the concept in the answer

Analytical (evaluating if protein structural changes were appropriately discussed)

Discussed correct intermolecular forces that would be present and clearly explained how the interactions could lead to protein structural changes

Explained that the conformation of the protein would change but limited or no explanation of how that would occur. Or type of structural change was incorrect

Did not include protein structural changes in the answer

Score

Tips for Grading Faster and More Accurately (i.e. for Large Classes) • Automate as much as possible. Online homeworks and quizzes are readily available. These can be used for assignments that require students to memorize information and focus on relatively closed LOC level questions. These generally use instructor (or program) generated choices and can be. • Scaffold questions as much as possible. Write questions that require shorter answers with less variability initially, while having one or two parts that will have to be carefully interpreted in order to grade properly. • Try to minimize the length of the answers. HOC questions can be constructed in such a way as to assess the students’ level of understanding in a relatively short answer. A word or sentence limit can be helpful, however, students can become more focused on the limit than the quality of their answer. Keep in mind students will need more space to answer the question than an expert. • Use an online platform, such as Gradescope, to manage assignments and grading. Gradescope is a website that allows for the development of online scoring guides that can be easily used to grade students’ work that is uploaded to the platform. The students’ work can then be graded anytime, and anywhere the grader has internet access. The development of the rubric and ease of application improves the consistency of grading within a large course. It also eliminates the need for score tallying and entering, minimizing the chance of human error. Changes to the scoring guide can be applied easier than grading paper assignments by hand. Additionally, each question can be tagged in various ways (LOC vs HOC, SLOs, or Concept) and the platform provides a statistical breakdown of each question, each rubric item or tag. This data can be invaluable for identifying concepts that need to be revisited or activities that should be revised. Additionally, identifying student information is not clearly displayed, potentially reducing the impact of implicit biases (42). 239

Considerations for Large Classes Using Higher Order Cognitive Level Assignments Despite large class sizes, we still want to challenge our students to demonstrate higher order skills. However, this is generally limited by the amount of feedback and grading that is associated with these assignments. The amount of grading that can be accomplished by one person in a term is finite. Thus, assignments that require more hours to grade should be limited to assignments where students can demonstrate higher order cognitive thinking. Choose the most valuable ways that students can demonstrate their learning and focus time and effort on those assignments, both in developing them and grading them. Exams With consideration for the time it takes to grade hundreds of students’ work a potential exam format for large classes will have a majority of well written multiple-choice questions and then add follow-up questions to some of them asking the students to explain their choice. With wellconstructed multiple-choice questions, you can gain valuable information about your students’ understanding of a topic and to some degree their cognitive abilities surrounding that topic. This information can be attained relatively quickly. Thoughtfully placed free response follow-up questions will uncover and/or confirm your students’ mental model of a concept. Another consideration for large courses can be validated multiple-choice exams such as the ACS exam or concept inventories. Logistical considerations for exams in large classes include where the students will be taking the exam and how close to each other they will be sitting. If students must sit next to each other in the exam room it might be necessary to write multiple versions of the exam. For questions that rely on calculations or are based on a particular molecule, the question itself can remain the same and the pertinent details can be changed. For other questions, an additional question testing the same concept at the same cognitive level should be written. Teaching assistants can help evaluate if two questions will be of similar difficulty for the students. However, if in the end there are discrepancies in the scores for the two questions, the students with the more difficult question should not be penalized. Use of the ACS Exam in Biochemistry as a validated and standardized assessment is discussed in multiple chapters of this book; Chapter 11 (4) and Chapter 13 (43). A specific example of using exams to assess student learning can be found in Chapter 6 of this book (44). Larger Projects To decrease the grading time required for these projects: • Break up the assignment into smaller parts that are due throughout the term. This spreads out the work for the students and graders, and provides excellent opportunities for feedback and revision. • Group projects reduce the number of submissions to be graded and provide an additional collaborative learning opportunity. • Peer review of some portions of the project supplies feedback to the students and an opportunity to evaluate other students’ work.

240

References 1.

2. 3.

4.

5. 6. 7.

8. 9. 10. 11. 12. 13. 14. 15.

16. 17.

Offerdahl, E.; Arneson, J. Formative Assessment to Improve Student Learning in Biochemistry. In Biochemistry Education: From Theory to Practice; Bussey, T. J., Linenberger Cortes, K. J., Austin, R., Eds.; ACS Symposium Series 1337; American Chemical Society: Washington, DC, 2019; Chapter 9. Dirks, C.; Wenderoth, M. P.; Withers, M. Assessment In the College Classroom; W.H. Freeman and Company: New York, NY, 2014. Carmichael, M. C.; St. Clair, C.; Edwards, A. M.; Barrett, P.; McFerrin, H.; Davenport, I.; Awad, M.; Kundu, A.; Ireland, S. K. Increasing URM Undergraduate Student Success through Assessment-Driven Interventions: A Multiyear Study Using Freshman-Level General Biology as a Model System. CBE—Life Sci. Educ. 2016, 15, ar38. Austin, R.; Murray, T. Don’t Go It Alone: The Importance of Community and Research in Implementing and Maintaining Innovative Pedagogy. In Biochemistry Education: From Theory to Practice; Bussey, T. J., Linenberger Cortes, K., Austin, R. , Eds.; ACS Symposium Series 1337; American Chemical Society: Washington, DC, 2019; Chapter 11. Crowe, A.; Dirks, C.; Wenderoth, M. P. Biology in Bloom: Implementing Bloom’s Taxonomy to Enhance Student Learning in Biology. CBE-Life Sci. Educ. 2008, 7, 36–381. Nilson, L. B. Teaching at Its Best: A Research-Based Resource for College Instructors; Jossey-Bass: San Francisco, CA, 2016. Laverty, J. T.; Underwood, S. M.; Matz, R. L.; Posey, L. A.; Carmel, J. H.; Caballero, M. D.; Fata-Hartley, C. L.; Ebert-May, D.; Jardeleza, S. E.; Cooper, M. M. Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol. PLoS One 2016, 11, 1–21. ASBMB Foundational Concepts. https://www.asbmb.org/education/teachingstrategies/ foundationalconcepts/ (Accessed Sept 9, 2019) Loertscher, J.; Green, D.; Lewis, J. E.; Lin, S.; Minderhout, V. Identification of Threshold Concepts for Biochemistry. CBE—Life Sci. Educ. 2014, 13, 516–528. Marzano, R. J.; Kendall, J. S. The New Taxonomy of Educational Objectives, 2nd ed.; Sage Publications: Thousand Oaks, CA, 2007. Krathwohl, D. R. A Revision of Bloom’s Taxonomy: An Overview. Theory Pract. 2002, 41, 212–218. Mental Models; Gentner, D., Stevens, A. L., Eds.; Psycology Press: New York, NY, and London, England, 1983. Redish, E. F. Implications of Cognitive Studies for Teaching Physics. Am. J. Phys. 1994, 62. Wiggins, G.; McTighe, J. Understanding by Design, Expanded 2; Association for Supervision and Curriculum Development: Alexandria, VA, 2005. Breakall, J.; Randles, C.; Tasker, R. Development and Use of a Multiple-Choice Item Writing Flaws Evaluation Instrument in the Context of General Chemistry. Chem. Educ. Res. Pract. 2019, 20, 369–382. Rodriguez, M. C. Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of Research. Educ. Meas. Issues Pract. 2005, 24, 3–13. Bodner, G. M. Statistical Analysis of Multiple-Choice Exams. J. Chem. Educ. 1980, 57, 188. 241

18. Briggs, D. C.; Alonzo, A. C.; Schwab, C.; Wilson, M. Diagnostic Assessment With Ordered Multiple- Choice Items Diagnostic Assessment With Ordered Multiple-Choice Items. Educ. Assess. 2006, 11, 33–63. 19. Hadenfeldt, J. C.; Bernholt, S.; Liu, X.; Neumann, K.; Parchmann, I. Using Ordered MultipleChoice Items to Assess Students’ Understanding of the Structure and Composition of Matter. J. Chem. Educ. 2013, 90, 1602–1608. 20. Cooper, M. M.; Williams, L. C.; Underwood, S. M. Student Understanding of Intermolecular Forces: A Multimodal Study. J. Chem. Educ. 2015, 92, 1288–1298. 21. Cooper, M. M.; Kouyoumdjian, H.; Underwood, S. M. Investigating Students’ Reasoning about Acid-Base Reactions. J. Chem. Educ. 2016, 93, 1703–1712. 22. Kararo, A. T.; Colvin, R. A.; Cooper, M. M.; Underwood, S. M. Predictions and Constructing Explanations: An Investigation into Introductory Chemistry Students’ Understanding of Structure-Property Relationships. Chem. Educ. Res. Pract. 2019, 20, 316–328. 23. Villafañe, S. M.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Uncovering Students’ Incorrect Ideas about Foundational Concepts for Biochemistry. Chem. Educ. Res. Pr. 2011, 12, 210–218. 24. Bretz, S. L.; Linenberger, K. J. Development of the Enzyme-Substrate Interactions Concept Inventory. Biochem. Mol. Biol. Educ. 2012, 40, 229–233. 25. Hodges, L. C. Group Exams in Science Courses. New Dir. Teach. Learn. 2004, 2004, 89–93. 26. Wieman, C. E.; Rieger, G. W.; Heiner, C. E. Physics Exams That Promote Collaborative Learning. Phys. Teach. 2014, 52, 51–53. 27. Gilley, B.; Clarkston, B. Research and Teaching: Collaborative Testing: Evidence of Learning in a Controlled In-Class Study of Undergraduate Students. J. Coll. Sci. Teach. 2014, 043, 83–91. 28. Colabroy, K.; Bell, J. Lab eNotebooks. In Biochemistry Education: From Theory to Practice; Bussey, T. J., Linenberger, K. J., Austin, R. , Eds.; ACS Symposium Series 1337; American Chemical Society: Washington, DC, 2019; Chapter 8. 29. Scott, R. J. Vignette #3: Developing Student Proficiency in Reading Biochemical Literature. In Biochemistry Education: From Theory to Practice; Bussey, T. J., Linenberger Cortes, K. J., Austin, R. , Eds.; ACS Symposium Series 1337; American Chemical Society: Washington, DC, 2019; Chapter 14. 30. Xu, X.; Lewis, J. E.; Loertscher, J.; Minderhout, V.; Tienson, H. L. Small Changes: Using Assessment to Direct Instructional Practices in Large-Enrollment Biochemistry Courses. CBE—Life Sci. Educ. 2017, 16, ar7. 31. Villafañe, S. M.; Bailey, C. P.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Development and Analysis of an Instrument to Assess Student Understanding of Foundational Concepts before Biochemistry Coursework. Biochem. Mol. Biol. Educ. 2011, 39, 102–109. 32. Kopecki-Fjetland, M. A. Vignette #1: Introducing Active Learning to Improve Student Performance on Threshold Concepts in Biochemistry. In Biochemistry Education: From Theory to Practice; Bussey, T. J., Linenberger Cortes, K. J., Austin, R., Eds.; ACS Symposium Series 1337; American Chemical Society: Washington, DC, 2019; Chapter 12. 33. Howitt, S.; Anderson, T.; Costa, M.; Hamilton, S.; Wright, T. A Concept Inventory for Molecular Life Sciences: How Will it Help Your Teaching Practice? Aust. Biochem. 2008, 39.

242

34. Malouff, J. M.; Thorsteinsson, E. B. Bias in Grading: A Meta-Analysis of Experimental Research Findings. Aust. J. Educ. 2016, 60, 245–256. 35. Project Implicit. https://implicit.harvard.edu/implicit/takeatest.html (Accessed Sept 6, 2019) 36. Schinske, J.; Tanner, K. Teaching More by Grading Less (or Differently). CBE—Life Sci. Educ. 2014, 13, 159–166. 37. Lalley, J. P.; Gentile, J. R. Classroom Assessment and Grading to Assure Mastery. Theory Pract. 2009, 48, 28–35. 38. Boesdorfer, S. B.; Baldwin, E.; Lieberum, K. A. Emphasizing Learning: Using Standards-Based Grading in a Large Nonmajors’ General Chemistry Survey Course. J. Chem. Educ. 2018, 95, 1291–1300. 39. Goodwin, J. A.; Gilbert, B. D. Cafeteria-Style Grading in General Chemistry. J. Chem. Educ. 2001, 78, 490–493. 40. Stemler, S. E.; Tsai, J. Best Practices in Interrater Reliability, Three Common Approaches. In Best Practices in Quantitative Methods; Sage Publications: Thousand Oaks, CA, 2008; pp 29–49. 41. Allen, D.; Tanner, K. Rubrics: Tools for Making Learning Goals and Evaluation Criteria Explicit for Both Teachers and Learners. CBE—Life Sci. Educ. 2006, 5, 197–203. 42. Singh, A.; Karayev, S.; Gutowski, K.; Abbeel, P. Gradescope: A Fast Flexible and Fair System for Scalable Assessment of Handwritten Work. In Proceedings of the Fourth ACM Conference on Learning @ Scale; 2017; pp 81–88. 43. Ragan, E. J. Vignette #2: Making a Switch to In-Class Activities in the Biochemistry Classroom. In Biochemistry Education: From Theory to Practice; Bussey, T. J., Linenberger Cortes, K. J., Austin, R. , Eds.; ACS Symposium Series 1337; American Chemical Society: Washington, DC, 2019; Chapter 13. 44. Bass Robinson, S.; Dolan, E.; Cornely, K.; Medlock, A.; Lee, J. K.; Lemons, P. P. The Development and Use of Case Studies. In Biochemistry Education: From Theory to Practice; Bussey, T. J., Linenberger Cortes, K., Austin, R., Eds.; ACS Symposium Series 1337; American Chemical Society: Washington, DC, 2019; Chapter 6.

243

Implementation

Chapter 11

Don’t Go It Alone: The Importance of Community and Research in Implementing and Maintaining Innovative Pedagogy Rodney C. Austin*,1 and Tracey Arnold Murray2 1Department of Chemistry, Mathematics, and Physics, Geneva College,

3200 College Avenue, Beaver Falls, Pennsylvania 15010, United States 2Department of Chemistry and Biochemistry, Capital University, One College and Main, Columbus, Ohio 43209, United States *E-mail: [email protected].

This chapter examines the importance of a supportive faculty community in implementing and maintaining curricular change. Faculty connected either through face-to-face or online communities routinely have greater success and confidence in adopting the new curriculum. Furthermore, individuals who share their implementation data and experiences at meetings such as the Biennial Conference on Chemical Education can reflect on their teaching while contributing to the improvement of the community.

Introduction Our experience in a faculty community of practice has been formative and vital for improving our instruction of biochemistry. Biochemistry instructors use concepts from biology and chemistry that many students find difficult. In addition to content knowledge, a significant portion of a biochemistry course requires students to analyze and evaluate models and data. Such instruction can be challenging even under the best of conditions. However, a small community of practitioners from around the country helped us navigate the difficult work of reflecting on and improving our instruction. Having been part of this community of practice for years, we find it hard to imagine making curricular changes in isolation. We are not alone in realizing the importance of community for curricular change. Hebert Simon, a Nobel Prize-winning economist, has famously stated that “Improvement in post-secondary education will require converting teaching from a solo sport to a community-based research activity (1).” His life and work inspired the Open Learning Initiative (OLI) at Carnegie Mellon University, which is a platform that facilitates course instruction and data gathering (2). OLI, along with the

© 2019 American Chemical Society

educational work of physics Nobel Prize winner Carl Weiman (3), highlights the need for communities of practice. We also need to study our approaches to education. Science advances through research, so it is appropriate that research now informs our teaching. In recent years, studies have looked at the importance of classroom structure (4) and active learning (5) on student learning. There has been work related to reducing the achievement gaps for minority, women, and first-generation studies (6, 7). Also, a recent study shows that faculty who believe ability is fixed have a more significant racial achievement gap (8). Such research highlights the need for continued investigation of our teaching practices. This chapter considers the impact of both a community of practice and research for improving teaching in biochemistry and beyond.

Communities of Practice Importance of Communities The concept of communities as positive forces for change and support has existed for some time. For instance, structured student living and learning communities have been used in colleges for nearly 100 years (9). These communities have fostered academic success by building relationships, generating a sense of belonging, increasing and reinforcing academic skills, and providing a supportive environment. Communities, too, are essential factors for faculty in developing as instructors and implementing curricular change. Faculty experience many of the same obstacles as students when they are growing as professors. It is not surprising, then, that one of the best ways to accomplish curricular change is through sustained, on-going communities of practice. A community of practice is a group of like-minded people who work together to learn how to do something new (10). People in the community will build relationships as they work together on shared activities or to create common resources. When used for pedagogical or curricular reform, participation in a community will give faculty a place to discuss and solve implementation issues, provide helpful resources, facilitate improvements to existing resources, and promote innovation. A community that highlights faculty development is the Broadening the Representation of Academic Investigators in NeuroScience (BRAINS) program (11). This program was developed to increase diversity in neuroscience, both academically and industrially, by providing a supportive community and developing skills that promote career success. Access to fellow early-career scientists, mentors, and resources enabled the participants to develop professionally. Initially, the cohort of scientists met for a four-day symposium and then continued to meet bi-weekly via conference calls. Some participants also took part in the career intervention program. The outcomes of this program were significant gains in self-efficacy, community, and belonging, which are essential for faculty development. Curricular and pedagogical changes in all areas of undergraduate education have accelerated in recent years. This is especially true of active learning strategies, which are being implemented at the national (12, 13) and institutional levels (14). Organizations and institutions have realized that support for faculty is necessary in order to make these curricular changes a reality. The most common methods for dissemination of active learning strategies to other faculty include presentations at conferences, publications in discipline-specific journals (Journal of Chemical Education, Biochemistry and Molecular Biology Education, CBE Life Sciences, etc), and hosting workshops. Of these, the workshop model provides faculty with the most likelihood of success, but the faculty member is 248

still mostly left to implement the pedagogical reform on his or her own, which is difficult and not consistent with continual improvement (15). Evidence from such dissemination workshop models shows an increased adoption rate in some cases (16, 17). However, the evidence provided in such studies was relatively short-term and did not involve additional follow up nor an ongoing community of practice. Furthermore, in a review of 191 research studies, Henderson, et al. found that many of these most common methods of dissemination did not work well (18). The “best practices” model where faculty share their method of pedagogical reform, through papers, presentations, or even workshops, did not show a significant impact on teaching practices. Another particularly ineffective model was the “top-down” model where an authority figure at an institution decided that faculty needed to implement a particular reform. This does not mean that all attempts at pedagogical reform are doomed to failure. Henderson et al. identified successful practices, which included long-term implementations with follow up, change that fit with faculty perceptions, and understanding the complex nature of universities (18). These approaches are observed by Kezar, et al., who studied four STEM education reform groups that had demonstrated sustained pedagogical reform: Project Kaleidoscope (PKAL), The POGIL Project (POGIL = Process-Oriented Guided Inquiry Learning), the BioQuest Curriculum Consortium, and Science Education for New Civic Engagements and Responsibilities (SENCER) (19–21). Given what was seen in the Henderson et al. review of the literature, what was most striking is that all of these groups had intentionally or unintentionally formed communities that allowed for long-term implementation and follow-up and could change and adapt to fit faculty needs and participation in the communities. Other factors that appeared to affect the development of these communities was access to the leaders of the communities, valuing the leaders, and the culture the leaders fostered. Not surprisingly, faculty in these communities who attended regular events sponsored by the communities and had a stronger belief in the value of the reform reported more change in belief and pedagogy than those who participated less or were less passionate about the reform. An example of effective curricular change using a community of practice is from Iowa State University (22). Instructors there formed a faculty learning community (FLC) to support the reform of a massive biology course. For this three-semester study, faculty met to review active learning activities, review educational literature, and share challenges. Instructors involved in this work found interaction with fellow faculty helpful for implementing the course changes and found the associated data on student learning informative. Interestingly, even after the initial study was complete, all instructors continued to meet as an informal group. Core Collaborators Workshop The authors of this chapter participate in a faculty community of practice that began as the Core Collaborators Workshop (CCW) and developed into an on-going community of biochemistry educators. The Core Collaborators group was started by Jennifer Loertscher and Vicky Minderhout as part of a National Science Foundation grant and brought together faculty from across the country to develop a Process-Oriented Guided-Inquiry Learning (POGIL) curriculum for biochemistry (23, 24). The community continued to meet annually, except for one year, in a series of four-day workshops that occurred seven times over two grant periods developing a strong core of collaborators. Although some faculty participated in only one or two workshops, others participated in most workshops and three faculty members, in addition to the two founders of the group, attended all seven CCWs. Everyone who was involved has been invited to join an online community, which is still holding virtual events. These virtual meetings are loosely structured around a theme for the 249

session with all participants able to contribute. The informal sessions frequently lead to follow up discussions and posts on the community board. This natural interaction amongst participants was evident from the beginning. After two Core Collaborators Workshops, a crucial realization of participants was the need for on-going, long-term interaction of faculty committed to pedagogical change. A survey was conducted of those attending one or two of the first workshops. Details from that survey can be found in Murray et al. (24). Since the community has lasted for much longer and involved more people than that first study, we wanted to reassess the effectiveness of this community on encouraging and supporting pedagogical change. To assess the impact of the CCWs, we gave the participants a formative assessment survey to complete. In all, surveys were sent to 58 collaborators who attended at least one of the seven workshops. There were two additional collaborators who we did not have correct contact information for and were thus unable to include. Twenty of the 58 participants (34%) responded, but only 19 (33%) of the surveys could be connected to the number of workshops attended, so the following data is based those 19 responses. To see if there were correlations between the number of CCWs attended and responses, we grouped the respondents based on the number of CCWs they had attended. Six respondents had attended only one CCW, four had attended two or three CCWs, five had attended four CCWs (including one of the authors of this chapter), and four had attended five or more CCWs (including one of the authors of this chapter). The respondents were asked to consider how the CCWs influenced their teaching. They rated their current confidence in implementing active learning activities, the number of activities implemented, how often they used active learning, if they had convinced a colleague to try active learning, and if they had presented a talk or published an article related to active learning. Table 1. Participants in the Core Collaborator Workshops report confidence in using active learning in the classroom. Number of Core Collaborator Workshops Attended

Number of respondents (n)

1

2 or 3

4

5+

6

4

5

4

What is your current confidence level in teaching using active learning? Somewhat confident

-

-

2

-

Confident

3

1

1

-

Very Confident

3

3

2

4

Did your confidence in teaching using active learning increase after attending a CCW? None

-

1

1

-

Small gain

4

2

2

-

Large gain

2

1

2

2

The respondents reported increased confidence when they implemented activities as a result of participation in at least one CCW. Of the 19 respondents, 17 (89%) indicated that they were either confident or very confident in using active learning in the classroom (Table 1). The single largest response was very confident, chosen by 12 (63%) collaborators. To assess whether the current confidence level was a result of the CCW we asked how their confidence changed after attending a workshop. Seventeen of the 19 (89%) indicated at least a small confidence gain. The two individuals 250

indicating no confidence gain noted in the comments that they had previously used active learning and felt confident already. These two also commented that the workshops had been a favorable influence on their teaching in other ways. We also wanted to know how attending influenced participants’ use of active learning in biochemistry since this was the primary theme of the first three CCWs. The last four CCWs were directly related to the development of threshold concepts in Biochemistry, but active learning teaching styles were still strongly featured in discussions of how to help students develop those concepts and activities were designed during the CCWs. We asked the respondents to indicate if attending the workshop(s) increased their use of active learning (yes or no). If yes, the respondent was then asked to choose a range of increase that the CCW had on their use of active learning. Thirteen of 19 (68%) respondents indicated that attending the CCW(s) had increased their use of such activities with six collaborators (32%) indicating an increase of 1 to 5 activities per course and seven collaborators (42%) indicating an increase 6 or more activities (Figure 1). The remaining six respondents (32%) indicated there was not an increase, but they had already been using these strategies. In some cases, the respondents were already at full implementation of active learning (using it in nearly every class period), and thus an increase in implementation was not possible. Additionally, when asked how many times per semester active learning activities were being used, 17 of the 19 collaborators reported that they now used active learning 10 to 20 times per semester or nearly every class (Table 2).

Figure 1. Number of active learning activities implemented by workshop participants after attending at least one workshop. Participants are categorized by the number of workshops attended: one workshop (5 respondents), two or three (4 respondents), four (5 respondents), or more than five (4 respondents) CCWs attended. The number of responses in each category are indicated in the bar graphs as either, no increase (black), increase in 1 to 5 activities (gray), and increase in 6+ activities (hashed). There were a total of 7 CCW events held and 58 participants were sent surveys with 19 collaborators responding.

251

Table 2. Participants in the Core Collaborator Workshops implement active learning curriculum in the classroom and contribute to the teaching community. Number of Core Collaborator Workshops Attended

Number of respondents (n)

1

2 or 3

4

5+

6

4

5

4

How often do you use active learning in your Biochemistry course? 1-9 times

-

1

1

-

10-20 times

2

1

2

-

Most or all the time

4

2

2

4

In how many other classes do you use active learning? 0

-

-

3

-

1

2

1

1

1

2 or more

4

3

1

3

Have you convinced or influenced a colleague at your institution to use active learning? Yes

4

3

3

3

No

2

1

2

1

Have you published an article or presented on active learning? Yes

4

3

4

4

No

2

1

1

0

The founders of the CCW invited people to join the original community based on their use of active learning in Biochemistry. Since the focus changed after the first three CCWs, this was not a requirement for people invited to the last four CCWs, but most of the participants in this community already had an interest in using active learning, and many had already begun implementing before attending the CCW sessions. These survey data clearly represent people who have an interest in active learning. Some might argue, then, that most of these collaborators are using active learning because they already had an interest in the approach and the workshops were effective at creating change because the participants were primed for it. These workshops, the argument goes, are just appealing to the converts. Remember, however, that communities of practice naturally form to aid instructors who are interested in change (20) and are not for the sole purpose of convincing someone to change their teaching approach. However, participation in a community of practice can catalyze change in the broader academy. When we asked the respondents if they had convinced a colleague to use active learning, 13 (68%) responded that they had. One commented that he had been influenced to adopt active learning by a colleague, which is why he had joined the community in the first place. One last question on the survey dealt with presenting research related to active learning. Fifteen collaborators (79%) had published an article or given a talk related to active learning (Table 2). These last two questions highlight that participation in this community of practice gave the faculty the confidence to become advocates for change at their institutions and in the larger community of educators. 252

Online Communities The work highlighted in this chapter so far shows the need for communities of practice in order to make substantial curricular improvements. The importance of the community is to offer support, provide a sense of belonging, and remove feelings of isolation, all of which are barriers to professional development and change (11). Not all communities need to be face-to-face, however. While many faculty learning communities exist at a single institution or come together in person regularly to discuss change, examples of virtual communities exist. A group of biology professors from five different universities met virtually for multiple semesters to implement a single unit of a course (25). Other virtual communities exist to bring faculty with common interests together. The Online Network of Inorganic Chemists (IONiC) promotes collaboration in the broad field of inorganic chemistry (26). This community shares information through the Virtual Inorganic Pedagogical Educational Resource (VIPER) where instructional resources and forums are available. Also, the IONiC community hosts a traditional face-to-face workshop each year, which highlights the need for continued interaction. Biologists interact through the Society for the Advancement of Biology Education Research (SABER). This community, too, provides online resources and hosts a traditional meeting each year. Also, a group of Israeli chemistry teachers developed a virtual community using a wiki site to help with curricular change (27). Online communities have the advantage of bringing participants together frequently. The CCW community has mostly transformed from a face-to-face community into a virtual community. The interest is still in teaching biochemistry through the POGIL framework, but our meetings now happen throughout the academic year. These events occur once per month with discussion around a common theme. Such long-term interactions are common in communities of practice (20). Challenges and Ways to Overcome Them Communities, whether they meet online or face-to-face, encounter challenges. Differences in motivation, approach, and end goal can differ significantly among any group of faculty, even when they have the same purpose to start. There is a rich literature on forming and maintaining communities of practice that can help people trying to create a community (20, 28). Communities of practice are most successful when they are allowed to change and evolve naturally as the needs of the community change. Another critical feature is that members of the community can participate at different levels – some very active and others less so. It is also important that members can cut back on participation when busy with other projects and then rejoin the community at a later time. Lastly, some combination of familiarity and excitement is necessary to keep the community grounded, but open to change and allowing for the evolution mentioned above. Communities of practice go through natural growth cycles that can be nurtured (20). For instance, to improve the effectiveness of implementing biology curriculum, a group of educators in the discipline proposed the “Five C’s of Collaboration: Commitment, Collegiality, Communication, Consensus, and Continuity” (29). These C’s represent the collaborative principles for guiding the improvement of undergraduate biology education through the Advancing Competencies in Experimentation-Biology (ACE-Bio) Network. These principles were developed to improve collaboration rates by providing a common framework. While these principles are not fool proof, nonetheless, they provide a path for others.

253

Using Research to Improve Instruction and Communities to Foster Research Discipline-Based Educational-Research Discipline-Based Educational-Research studies (DBER) improve education in STEM fields. Studies have evaluated pedagogical approaches that increase classroom performance among STEM students (5, 22), especially among women, first-generation students, and minorities (30, 31). These DBER papers and others will force us all to scrutinize our teaching and learning approaches in ways that may be unexpected and unfamiliar. To improve DBER, Mulnix has argued that the scientific community needs to better connect cognitive sciences, education, psychology, and other areas to their work (32). This approach will help highlight the foundational learning principles that inform our teaching practice. One example of DBER that brings in research from other disciplines is the recent work related to supplemental instruction (7). In this research paper, the authors provided both exercises in the content area and evidence-based interventions to overcome psychological and emotional barriers to learning. Such barriers are higher for minorities, low-income, and first-generation students. The psychological and emotional issues are not as often targeted in STEM. This approach, however, considers learning holistically, which might not have occurred without a community of researchers probing how people learn. Also, this work helps to uncover issues that prevent students, particularly at-risk students, from achieving. Communities Foster Research One of the ways to foster DBER is through faculty communities. Communities of practice promote interaction between instructors, which can lead to collaborative research projects to assess curricular strategies (3, 33, 34). For instance, a conceptual inventory resulted when participants of the CCWs wanted to assess if the POGIL activities being developed were effective at improving student learning (35). This inventory focused on seven concepts that are generally taught in what are typically first-year chemistry and biology courses. The faculty participating in the CCWs selected these concepts through an iterative process at the annual workshops that included training in how to develop a reliable and valid inventory. The process of development was improved because the faculty could test the inventory at their home institutions and then share the data at the next workshop. In addition to developing a useful and robust conceptual inventory, this also allowed faculty to realize that their students were not the only ones lacking the expected foundational knowledge from the firstyear courses (36–41). One study that resulted directly from CCW participation used the conceptual inventory to probe the effectiveness of instruction (34). The primary investigator in this study was a member of the second grant CCWs and heard about the inventory from the members of the community that had participated in its development during the first grant. Drawing on the active learning community and conceptual inventory, the authors assessed the effectiveness of small changes in instruction on a crucial concept, the hydrogen bond. They showed that a small, targeted pedagogical change could double the number of students who correctly identified a hydrogen bond. Also, the authors of this chapter formed a collaboration after we met at a CCW. Our work has drawn upon the conceptual inventory, and “small changes” work to improve POGIL instruction (42, 43). We implement these changes in smaller classrooms (~20 to ~30 students) at two different mostly-undergraduate institutions. This approach permitted us to test the findings from studies such as Xu et al. 2017, which was gathered in a large classroom. We wanted to see if the work could be 254

repeated in classes that are smaller and contain more active learning. Our results (42) were mostly in line with Xu, where students showed significant improvement in understanding the hydrogen bonding concept. While our sample size was small and showed more variation, the results nonetheless were vital to us. Our work highlights the importance of gathering data even when the sample size is small, or the results may not be publishable. Such work is essential in checking one’s teaching to determine if the instruction is achieving the desired results (44). This assessment can be realized through the conceptual inventory mentioned here or standardized exams such as those produced by the American Chemical Society (ACS) or the American Society of Biochemistry and Molecular Biology (ASBMB). These exams permit an instructor to compare his or her students’ performance to a national group of students. Also, such work is valuable to convince colleagues that instructional approaches, especially non-traditional approaches such as active learning, are useful. This analysis is especially useful for early-career faculty who may need to provide objective support that their instructional method is effective. Even for seasoned faculty, changing the minds of peers can be a challenge that data can help overcome. Moving Forward There are many ways to get involved in the biochemistry community. For instance, a biochemistry community has formed around POGIL and contributes to meetings in many parts of the country throughout the year (pogil.org). Their work is dedicated to the POGIL approach to teaching and learning which appeals to many. The work of the POGIL community is presented at meetings of the ACS and the Biennial Conference on Chemical Education (BCCE). In biology, the SABER group is active with a web presence and holds annual meetings (https://saberbio.wildapricot.org). These meetings offer presentations about many aspects of teaching and learning in biochemistry and related subjects. Attending these meetings and presenting data and findings adds to the base of experiences in the community. For instance, in this book, authors have presented work at a BCCE symposium that was conducted in smaller classrooms but is essential given that many smaller institutions teach chemistry (41, 45, 46). Online resources for instructors exist to enable faculty to evaluate their instruction. The Carl Weiman Science Education Initiative (3) offers a variety of tools to evaluate instruction, implement new teaching techniques, and to use clickers in the classroom. The resources even include videos to aid instructors in implementing new strategies. The POGIL and SABER sites, among others, offer valuable resources. For instance, the live, online eSeminars by POGIL provide training in areas such as assessing process skills, implementing process skills, and evaluating teaching. The POGIL website offers additional curriculum and implementation resources. Locally, communities of practice (22) help faculty implement curricular change. Of necessity, these local communities are often multi-disciplinary which brings diversity to the discussions. Joining, or starting, a faculty group is an active step in improving instruction. The authors of this article are both actively working with faculty at our local institutions. Those connections provide meaningful conversation and feedback on teaching and the barriers to learning. Consequently, we have found joining a community of practice to be of significant value to our teaching. The interactions with other instructors have challenged us to try new activities and approaches in the classroom. Although we were all teaching biochemistry, some members of the group were from large institutions and teach large classes, while others were from small institutions. Many different department affiliations were represented – chemistry and biology, as well as chemical 255

education, biochemistry, genetics, and medicine. Community participants had different amounts of experience teaching and faculty rank. Each person made our experience, and the experiences of other participants, richer. Even though we had different backgrounds and individual situations, we all had the shared goal of improving biochemistry education, and this allowed us to work collaboratively and to learn from each other. Because the leaders created a community that was working to produce materials for use by all and incorporated suggestions and ideas from many different community members, we all felt valued and felt an ownership of the community. Our own experiences in this community and the general research on effective communities drive us to encourage the development of and participation in communities of practice whenever possible. We feel that this leads to the most effective implementation of curricular and pedagogical change.

References 1.

The Simon Initiative at Carnegie Mellon University Web site. https://www.cmu.edu/simon/whatis-simon/herbert-a-simon.html (accessed March 4, 2019). 2. Open Learning Initiative Home Page. https://oli.cmu.edu/ (accessed Feb 21, 2019). 3. Carl Wieman Science Education Initiative at the University of British Columbia Home Page. http://www.cwsei.ubc.ca/ (accessed March 4, 2019). 4. Eddy, S.; Hogan, K. Getting Under the Hood: How and for Whom Does Increasing Course Structure Work? CBE—Life Sciences Education 2014, 13, 453–468. 5. Stockwell, B.; Stockwell, M.; Cennamo, M.; Jiang, E. Blended Learning Improves Science Education. Cell 2015, 162, 933–936. 6. Haak, D. C.; Hillerislambers, J.; Pitre, E.; Freeman, S. Increased Structure and Active Learning Reduce the Achievement Gap in Introductory Biology. Science 2011, 332, 1213–1216. 7. Stanich, C.; Pelch, M.; Theobald, E.; Freeman, S. A new approach to supplementary instruction narrows achievement and affect gaps for underrepresented minorities, firstgeneration students, and women. Chemistry Education Research and Practice 2018, 19, 846–866. 8. Canning, E.; Muenks, K.; Green, D.; Murphy, M. STEM faculty who believe ability is fixed have larger racial achievement gaps and inspire less student motivation in their classes. Science Advances [Online] 2019, 5, Article 4734. https://doi.org/10.1126/sciadv.aau4734 (accessed March 14, 2019). 9. Fink, J. E.; Inkelas, K. K. A History of Learning Communities Within American Higher Education. New Directions for Student Services 2015, 2015, 5–15. 10. Wegner, E. Communities of Practice: Learning, Meaning, and Identity; Cambridge University Press: Cambridge, UK, 1999; pp 86−102 11. Margherio, C.; Horner-Devine, M.; Mizumori, S.; Yen, J. Learning to Thrive: Building Diverse Scientists’ Access to Community and Resources through the BRAINS Program. CBE—Life Sciences Education [Online] 2016, 15, Article 49. https://doi.org/10.1187/cbe.16-01-0058 (accessed March 14, 2019). 12. Austin, A. Vision & Change in Undergraduate Biology Education—Vision and Change: Unpacking a Movement and Sharing Lessons Learned; 2018. http://visionandchange.org/about-vcunpacking-a-movement-2018/ (accessed March 14, 2019).

256

13. Woodin, T.; Carter, V.; Fletcher, L. Vision and Change in Biology Undergraduate Education, A Call for Action—Initial Responses. CBE—Life Sciences Education 2010, 9, 71–73. 14. Aloi, D. Active Learning Initiative funds nine projects. Cornell Chronicle [Online] 2019. http://news.cornell.edu/stories/2019/02/active-learning-initiative-funds-nine-projects (accessed March 14, 2019). 15. Brownell, S.; Tanner, K. Barriers to Faculty Pedagogical Change: Lack of Training, Time, Incentives, and…Tensions with Professional Identity? CBE—Life Sciences Education 2012, 11, 339–346. 16. Gregg, C.; Ales, J.; Pomarico, S.; Wischusen, E.; Siebenaller, J. Scientific Teaching Targeting Faculty from Diverse Institutions. CBE—Life Sciences Education 2013, 12, 383–393. 17. Manduca, C.; Iverson, E.; McConnell, D.; Bruckner, M.; Greenseid, L.; Macdonald, H.; Tewksbury, B. Mogk, D. On the cutting edge: combining workshops and on-line resources to improve geoscience teaching. In GSA Annual Meeting [Online]; GSA, 2014. Available at https://gsa.confex.com/gsa/2014AM/finalprogram/abstract_248089.htm (accessed 26 Feb 2019). 18. Henderson, C.; Beach, A.; Finkelstein, N. Facilitating Change in Undergraduate STEM Instructional Practices: An Analytic Review of the Literature. Journal of Research in Science Teaching 2011, 48, 952–984. 19. Kezar, A.; Gehrke, S.; Bernstein-Sierra, S. Designing for Success in STEM Communities of Practice: Philosophy and Personal Interactions. The Review of Higher Education 2017, 40, 217–244. 20. Kezar, A.; Gehrke, S.; Bernstein-Sierra, S. Communities of Transformation: Creating Changes to Deeply Entrenched Issues. The Journal of Higher Education 2018, 89, 832–864. 21. Gehrke, S.; Kezar, A. Perceived Outcomes Associated with Engagement in and Design of Faculty Communities of Practice Focused on STEM Reform. Research in Higher Education [Online] 2019, 60 (6), 844. https://doi.org/10.1007/s11162-018-9534-y (accessed March 14, 2019). 22. Elliott, E.; Reason, R.; Coffman, C.; Gangloff, E.; Raker, J.; Powell-Coffman, J.; Ogilvie, C. Improved Student Learning through a Faculty Learning Community: How Faculty Collaboration Transformed a Large-Enrollment Course from Lecture to Student Centered. CBE—Life Sciences Education [Online] 2016, 15, Article 22. https://doi.org/10.1187/cbe.1407-0112 (accessed March 14, 2019). 23. Minderhout, V.; Loertscher, J. Lecture-free biochemistry. Biochemistry and Molecular Biology Education 2007, 35, 172–180. 24. Murray, T. A.; Higgins, P.; Minderhout, V.; Loertscher, J. Sustaining the development and implementation of student-centered teaching nationally: The importance of a community of practice. Biochemistry and Molecular Biology Education 2011, 39, 405–411. 25. Pelletreau, K.; Knight, J.; Lemons, P.; Mccourt, J.; Merrill, J.; Nehm, R.; Prevost, L.; UrbanLurain, M.; Smith, M. Faculty Professional Development Model That Improves Student Learning, Encourages Active-Learning Instructional Practices, and Works for Faculty at Multiple Institutions. CBE—Life Sciences Education [Online] 2018, 17, Essay 5. https://doi. org/10.1187/cbe.17-12-0260 (accessed March 14, 2019).

257

26. Jamieson, E.; Eppley, H.; Geselbracht, M.; Johnson, A.; Reisner, B.; Smith, S.; Stewart, J.; Watson, L.; Williams, B. Inorganic Chemistry and IONiC: An Online Community Bringing Cutting-Edge Research into the Classroom. Inorganic Chemistry 2011, 50, 5849–5854. 27. Shwartz, Y.; Katchevitch, D. Using Wiki to Create a Learning Community for Chemistry Teacher Leaders. Chem. Educ. Res. Pract. 2013, 14, 312–323. 28. Wenger, E.; McDermott, R.; Snyder, W. Cultivating Communities of Practice: A Guide to Managing Knowledge; Harvard Business School Press: Cambridge, MA, 2010. 29. Pelaez, N.; Anderson, T.; Gardner, S.; Yin, Y.; Abraham, J.; Bartlett, E.; Gormally, C.; Hurney, C.; Long, T.; Newman, D.; Sirum, K.; Stevens, M. A Community-Building Framework for Collaborative Research Coordination across the Education and Biology Research Disciplines. CBE—Life Sciences Education [Online] 2018, 17, essay 2. https://doi.org/10.1187/cbe.17-040060 (accessed March 14, 2019). 30. Freeman, S.; Eddy, S. L.; Mcdonough, M.; Smith, M. K.; Okoroafor, N.; Jordt, H.; Wenderoth, M. P. Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences 2014, 111, 8410–8415. 31. Neill, C.; Cotner, S.; Driessen, M.; Ballen, C. J. Structured learning environments are required to promote equitable participation. Chemistry Education Research and Practice 2019, 20, 197–203. 32. Mulnix, A. B. STEM Faculty as Learners in Pedagogical Reform and the Role of Research Articles as Professional Development Opportunities. CBE—Life Sciences Education [Online] 2016, 15, Essay 8. https://doi.org/10.1187/cbe.15-12-0251 (accessed March 14, 2019). 33. Ebert-May, D.; Derting, T.; Henkel, T.; Maher, J.; Momsen, J.; Arnold, B.; Passmore, H. Breaking the Cycle: Future Faculty Begin Teaching with Learner-Centered Strategies after Professional Development. CBE—Life Sciences Education [Online] 2015, 14, Article 22. https://doi.org/10.1187/cbe.14-12-0222 (accessed March 14, 2019). 34. Xu, X.; Lewis, J. E.; Loertscher, J.; Minderhout, V.; Tienson, H. L. Small Changes: Using Assessment to Direct Instructional Practices in Large-Enrollment Biochemistry Courses. CBE—Life Sciences Education [Online] 2017, 16, Article 7. https://doi.org/10.1187/cbe.1606-0191 (accessed March 14, 2019). 35. Villafañe, S. M.; Bailey, C. P.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Development and analysis of an instrument to assess student understanding of foundational concepts before biochemistry coursework. Biochemistry and Molecular Biology Education 2011, 39, 102–109. 36. Heyen, B.; Assessment of the Active Learning Biochemistry Classroom. Presented at the 2014 Biennial Conference on Chemical Education, Allendale, MI, August 3−4, 2014; P450. 37. Taylor, A.; Novak, W. Standing on the Shoulders: Integrating Pre-requisite Course Concepts into Teaching Biochemistry; Presented at the 2014 Biennial Conference on Chemical Education, Allendale, MI, August 3−4, 2014; P565. 38. Tienson, H.; Lewis, J.; Loertscher, J.; Minderhout, V.; Xu, X. Using Assessment to Direct Instructional Practices in Large-Enrollment Biochemistry Courses; Presented at the 2016 Biennial Conference on Chemical Education, Greeley, CO, July 31−August 4, 2016; P750. 39. Fletcher, H. Implementing Specifications Grading in Biochemistry; Presented at the 25th Biennial Conference on Chemical Education, South Bend, IN, July 29−August 2, 2018; BCCE 328.

258

40. Villafañe, S.; Loertscher, J.; Minderhout, V.; Lewis, J. Uncovering students’ incorrect ideas about foundational concepts for biochemistry. Chem. Educ. Res. Pract. 2011, 12, 210–218. 41. Kopecki-Fjetland, M. A. Vignette #1: Introducing Active Learning to Improve Student Performance on Threshold Concepts in Biochemistry. Biochemical Education: From Theory to Practice; ACS Symposium Series; American Chemical Society: Washington, DC, 2019; Vol. 1337, Chapter 12. 42. Austin, R.; Murray, T. Teaching and Assessing Conceptual Change in the Active Learning Biochemistry Classroom; Presented at 2017 Ohio-PKAL Annual Conference, Findlay, Ohio, May 20, 2017. 43. Austin, R.; Murray, T. Does Active Learning Lead to “Long-Term” Conceptual Change? Presented at 2018 Ohio-PKAL Annual Conference. Mount Union, Ohio, May 19, 2018. 44. Minderhout, V.; Loertscher, J. In Process-Oriented Guided Inquiry Learning (POGIL); Moog, R., Spencer, R., Eds.; ACS Symposium Series; American Chemical Society: Washington, DC, 2008; Vol. 994, pp 72−86. 45. Ragan, E. J. Vignette #2: Making a Switch to In-Class Activities in the Biochemistry Classroom. Biochemical Education: From Theory to Practice; ACS Symposium Series; American Chemical Society: Washington, DC, 2019; Vol. 1337, Chapter 13. 46. Scott, R. J. Vignette #3: Developing Student Proficiency in Reading Biochemical Literature. Biochemical Education: From Theory to Practice; ACS Symposium Series; American Chemical Society: Washington, DC, 2019; Vol. 1337, Chapter 14.

259

Chapter 12

Vignette #1: Introducing Active Learning to Improve Student Performance on Threshold Concepts in Biochemistry Mary A. Kopecki-Fjetland* Department of Chemistry, St. Edward’s University, 3001 South Congress Avenue, Austin, Texas 78704, United States *E-mail: [email protected].

Students entering a first semester biochemistry course are expected to possess knowledge of certain foundational concepts such as hydrogen bonding and bond energy from prerequisite coursework. These foundational concepts are essential for gaining mastery of threshold concepts in biochemistry such as the physical basis of interactions. Unfortunately, many students who enter a biochemistry course possess misconceptions or an incomplete understanding of these foundational concepts. This chapter describes an iterative process for introducing interventions to improve student foundational concept knowledge. The process includes pre-assessment of foundational concept knowledge, identification of targeted interventions, introduction of the intervention into the classroom, and re-assessment of student knowledge for potential learning gains. Utilization of this iterative process to guide implementation of active learning strategies such as problem based worksheets, tactile learning activities, and learning cycle activities will be discussed.

Introduction Biochemistry is an interdisciplinary upper division course built on the fundamental concepts of biology and chemistry. Student success is dependent on utilization of previous knowledge as a scaffold onto which to build new ideas. Most students encounter these foundational concepts in previous courses such as general chemistry, general biology and organic chemistry. However, upon entering a biochemistry course they often bring gaps in knowledge or misconceptions which hinder their learning of big ideas or threshold concepts. This chapter discusses an iterative process to design and implement active learning strategies in the classroom to improve student foundational concept knowledge which may then better position students to master threshold concepts.

© 2019 American Chemical Society

Bridging Threshold Concepts and Foundational Concepts In recent years a number of disciplines have made an effort to define and identify core concepts considered essential for students to master (1–3). These big ideas provide the basis of curricular design and assessment. In biochemistry these core concepts have been defined and identified with some variation. Loertscher described a core concept as a “threshold concept which represents a transformed understanding of a discipline, without which the learner cannot progress and therefore is pivotal in learning in a discipline” (4). Threshold concepts identified include: steady state, biochemical pathway dynamics and regulation, the physical basis of interactions, thermodynamics of macromolecular structure formation, and free energy (5). Rowland identified slightly different core concepts in biochemistry such as information transfer and storage (6) while Tansey identified core concepts in biochemistry and molecular biology such as evolution and information flow (7). Foundational concepts are concepts students encounter in pre-requisite courses and are expected to have mastered upon entering a biochemistry course. These concepts are typically encountered in courses such as general biology, general chemistry and organic chemistry. Similarly to threshold concepts, there is some distinction amongst identified foundational concepts for biochemistry. Foundational concepts from physics, chemistry and mathematics have been identified and were described as “necessary to both increase the depth of conceptual understanding in the field of Biochemistry and Molecular Biology and foster interdisciplinary thinking” (8). A few examples of these essential concepts include: Coulomb’s Law at work in the molecular life sciences, covalent bonds and polarity, hydrogen bonds and other non-covalent interactions, and rate laws and equilibria. Alternatively, a diverse community of biochemistry educators have identified seven foundational concepts as important for learning biochemistry (9). These foundational concepts were identified in conjunction with the development of a diagnostic instrument for foundational knowledge and include: hydrogen bonding, bond energy, pH/pKa, free energy, London dispersion forces, protein function and alpha helix structure. Xu later reported a slight variation of these seven identified foundational concepts substituting chemical equilibrium for London dispersion forces (10). In the biochemistry classroom students are expected to utilize foundational concept knowledge as a scaffold on which to build understanding of big ideas or threshold concepts. Relating existing knowledge to new knowledge is the basis for the constructivist model of learning (11). A student will utilize what they already know to create new meaning or new knowledge. According to Halpern, “the best predictor of what is learned at the completion of any lesson, course, or program of study is what the learner thinks and knows at the start of the experience” (12). Retention of content knowledge and the ability to transfer content knowledge to new contexts are enhanced by practicing retrieval of existing knowledge (13). Thus for a student in a biochemistry classroom the building of threshold concepts from foundational concepts reinforces their existing understanding of these fundamental ideas. Creation of connections between foundational concepts and threshold concepts is essential for student learning in biochemistry (4, 5). Knowledge of foundational concepts such as bond energy, free energy, and equilibrium are essential for better positioning students to grasp threshold concepts such as the thermodynamics of macromolecular structure formation and steady state. To instructors it seems plausible to anticipate that a student will be able to master the threshold concept of the physical basis of interactions as they already possess an understanding of hydrogen bonding and pH/ pKa and its relationship to charge. However, in reality many students fail to demonstrate retention 262

of these learned foundational concepts. A number of studies report inefficiencies when students were asked to demonstrate their prerequisite competency at the onset of a biochemistry course (9, 10, 14). Closer analysis of these results reveal that students often possess misconceptions or gaps in knowledge (15). These incorrect ideas can hinder learning in a new context (16–18). When identifying threshold concepts for biochemistry, Loertscher argues “we must develop methods that simultaneously teach new biochemistry concepts, and also help students refine and strengthen their understanding of foundational concepts” (5). As instructors it is well worth our time to reflect on how strengthening student knowledge of foundational concepts can better position our students to master threshold concepts thus transforming their understanding of biochemistry. One potential mechanism for strengthening foundational concepts is the introduction of active learning strategies in the classroom.

Introducing Active Learning to Overcome Foundational Concepts Deficiencies Overview of Active Learning in Biochemistry Active learning as a student-centered pedagogical approach to teaching and learning has become increasingly popular in chemistry classrooms. This cooperative learning structure aims to develop critical thinking, problem solving skills, teamwork and communication (19). In the biochemistry classroom, a number of different active learning models have been reported. In process oriented guided inquiry learning (POGIL) students initially explore a topic independently, work together in groups to construct and refine knowledge and eventually apply their knowledge to high level biochemical problems (17, 19). Other instructors have implemented the flipped classroom approach, where the traditional teacher-centered lecture is communicated to students outside of class via recorded lectures, leaving class time for activities that are normally assigned as homework (20). In addition, the case based learning (directed or problem-based) model aims at teaching content while actively engaging students in real-life case study scenarios, often borrowed from the literature (21–24). Regardless of the model you choose to explore, introduction of active learning into the biochemistry classroom creates a student centered environment for learning which allows students to hone collaboration skills and gain an awareness of their learning style. Additional information about this transition can be found in the chapter entitled “Making a Switch to In-Class Activities in the Biochemistry Classroom” by Emily Ragan (25). An Iterative Process for Introducing Active Learning An iterative process which can assess the impact of an intervention on students’ biochemical foundational concepts knowledge is an impactful mechanism any instructor can utilize to address students’ misconceptions or gaps in knowledge. This process includes multiple stages: an initial assessment of student foundational concept knowledge, review of the literature to identify an intervention that targets a specific foundational concept, introduction of the intervention into the classroom, assessment of learning gains, and revision and reintroduction of the intervention when applicable. This process has been utilized to introduce and assess diverse interventions focused on specific foundational concepts (10, 14). I will briefly describe how I utilized this iterative process to guide implementation of active learning strategies to address misconceptions or gaps in foundational concept knowledge.

263

In the first stage of the iterative process an instructor must review the literature to gain a more complete understanding of how a foundational concept is defined, identified and potentially assessed (9). Next the instructor can utilize a validated instrument to assess students’ knowledge of foundational concepts at the onset of the course. This allows the instructor to identify specific misconceptions and/or gaps in knowledge as well as identifying lower performing foundational concepts. In the second stage, the instructor reviews the literature for a specific intervention that addresses a deficient foundational concept as revealed by the initial assessment. These interventions may be specific for the biochemistry classroom or can be adapted from general chemistry, general biology or organic chemistry. In the third stage, the instructor introduces an active learning strategy to address specified deficiencies for an identified concept. This might include a published activity or one developed by the instructor. Utilization of an existing intervention allows the instructor additional opportunity to focus on implementation and adaptation of the activity into the classroom. Some instructors may choose not to implement a specific intervention but instead assess how current teaching practices impact learning gains. In the fourth stage, the assessment instrument is re-administered. Results can potentially reveal the overall effectiveness of the intervention as well as providing more specific feedback on which gaps or misconceptions may persist. If the desired learning gain is not achieved, the activity may be revised based on student feedback, instructor observations in the classroom and instructor selfreflection. Alternatively the instructor may choose to augment the existing activity with an additional activity. The revised activity or novel combination of activities can be implemented and assessed in a subsequent semester.

Design and Implementation of Active Learning Strategies to Address Foundational Concept Deficiencies Utilizing this iterative process, various active learning strategies were introduced to target four foundational concepts including: pH/pKa , hydrogen bonding, bond energy and chemical equilibrium. What follows is an account of the development and implementation of these strategies. Each strategy could be adapted to address any of the foundational concepts or any other biochemical concept the instructor deems particularly challenging. Literature Review and Pre-assessment of Foundational Concepts As described in stage one of the iterative process, a thorough review of foundational concepts in the literature was conducted. It was useful for the instructor to reflect on which foundational concepts had been identified and how student knowledge of these concepts could potentially be assessed using concept inventories. A number of different concept inventories are available for various subjects such as biology, molecular life sciences and general chemistry (26–28). In order to assess student knowledge of biochemistry foundational concepts, the Instrument of Foundational Concepts for Biochemistry (IFCB) first described by Villafane (9, 29) and subsequently modified by Xu (10) was utilized. The complete set of questions in this instrument has not been published thus it is suggested that instructors contact the original authors to request a copy. IFCB is a multiple choice instrument that measures knowledge of five concepts from general chemistry and two from general biology. Concepts measured include hydrogen bonding, pH/pKa and charge, chemical equilibrium, bond energy, free energy, alpha helix structure and protein function. Students answer three multiple choice 264

items for each of the seven concepts with the most common incorrect ideas serving as distractors. For example the three common incorrect ideas on hydrogen bonding include “all hydrogens are capable of hydrogen bonding”, “a covalent bond with a hydrogen in it is a hydrogen bond,” and “any polar molecule can make a hydrogen bond. This instructor administers the IFCB on the first day of class in each semester because student populations can vary, especially in courses offered in “off sequence” semesters. In addition the number of students enrolled in the course can vary from semester to semester which can potentially impact classroom dynamics. Students are not provided with advanced notice of this assessment and are told that their responses will help guide instruction in the course. In order to be consistent with the literature, student responses are scored based upon percent of students answering all three questions correctly for a given concept. Pre-assessment scores do not impact student grades in any fashion. Scores reveal students’ existing knowledge of foundational concepts and specifically lowest performing concepts while student responses reveal the most common misconceptions or gaps in student knowledge. Scores from the initial administration of the IFCB ranged from 29% correct (equilibrium and alpha helix) to 6% correct (hydrogen bonding). As indicated in Table 1, over the course of five semesters of administering the IFCB pre-assessment, pH/pKa and hydrogen bonding were consistently the two lowest performing concepts with only 3% of students on average answering all three hydrogen bonding questions correctly and 5% on average answering all three pH/pKa questions correctly. Bond energy was consistently one of the four lowest performing concepts with only 18% of students on average answering all three questions correctly. The two highest performing concepts included chemical equilibrium and alpha helix (30% correct and 27% correct respectively). For reasons related to post-assessment scores discussed later in this chapter, the instructor also decided to introduce activities to improve student knowledge of equilibrium. All assessment plans were approved by the Institutional Review Board at this instructor’s institution. Identification of Potential Concept Interventions Upon review of pre-assessment results, the instructor possesses a clearer understanding of what student deficiencies exist and can search the literature for a potential intervention to implement in the classroom. Diverse active learning strategies have been implemented in classrooms to address deficiencies in foundational concept knowledge. The Foundations of Biochemistry workbook authored by Jenny Loertscher, Vicky Minderhout, and Katherine Frato follows the process-oriented guided inquiry learning (POGIL) model of teaching (19, 30). It contains activities that focus on all seven foundational concepts assessed in the IFCB. Implementation of POGIL in the biochemistry classroom improved student performance on exam questions requiring complex cognitive skills and lowered the DFW percentage. An additional guided-inquiry activity developed by Werth explores noncovalent interactions utilizing the model of a neurotransmitter binding to its receptor (31). An extensive series of pre-class activities utilized in conjunction with in class reinforcement in the form of clicker questions or discussion questions have been described by Taylor. Select activities focus on pH/pKa, hydrogen bonding and bond energy. This type of intervention resulted in statistically significant improvement in learning gains for foundational concepts (14). Xu describes the implementation of a number of strategies including in class discussions, clicker questions, and an in class worksheet to address concepts such as pH/pKa, hydrogen bonding, and bond energy. These interventions resulted in improved learning gains when a pre and post-assessment of foundational concept knowledge was administered (10). Johnson describes a series of classroom exercises where students are asked to read a research article on HIV protease followed by in class small group discussion. They focus on topics such as thermodynamics, protein structure, ligand binding and 265

enzymatic catalysis. These exercises create opportunities for students to improve reading of the primary literature, data analysis, collaborative problem solving and scientific presentations (32). A number of tactile learning activities have been reported for biochemistry and related disciplines. Cooper describes the creation of 3D physical models to address student understanding of electrostatics and noncovalent interactions and their relationship to macromolecular structure (33) while a Foundations of Biochemistry activity has students build an alpha helix to illustrate structure (30, 34). Moreover, in Mathematics TACTivities were developed to introduce and review Calculus concepts (35, 36). Model based activities were shown to be more effective than non-model base activities for student understanding of the central dogma of molecular biology. Students showed higher learning gains on model-based questions when the Central Dogma Concept Inventory was administered (37). Introduction of the Intervention In the first semester of this work, the instructor reviewed the literature and implemented the Foundations of Biochemistry workbook. Because active learning was new to this instructor’s classroom, the workbook was selected to allow ample opportunity for focused implementation of activities. Because pre-assessment results indicated a sizable deficiency in student knowledge of pH/pKa and hydrogen bonding, problem-based worksheets were designed and implemented in the first semester as well. In an effort to further improve learning gains, a hydrogen bonding tactile activity and learning cycle activity were introduced in subsequent semesters. Bond energy was consistently one of the lowest scoring concepts as revealed by pre-assessment results, thus a learning cycle activity was designed and implemented to augment the workbook activity. Although pre-assessment scores for chemical equilibrium were consistently higher than other concepts, the normalized learning gain was consistently among the lowest hence a chemical equilibrium learning cycle activity was introduced in a subsequent semester. What follows is an account of the implementation of the workbook along with a description of three types of active learning strategies that were developed by this instructor and implemented in the classroom. Workbook For instructors wanting to strengthen student foundational concept knowledge and who possess very little or no experience implementing active learning into their biochemistry classroom, the Foundations of Biochemistry workbook is an excellent first step option (19, 30). Instructor support in the form of facilitation plans and answer keys for activities are available on the instructor’s website. This instructor had no prior experience with POGIL and found the activities could be adapted to the instructor’s implementation style. The workbook is implemented with each course offering. Prior to class, students were asked to read assigned textbook pages and answer all or part of the workbook pre-activity questions for a given activity. These questions review prerequisite knowledge and are based upon the reading. For example, when utilizing the activity related to pH/pKa, students were asked to define terms such as buffering region and equivalence point. In class students were provided with introductory information in the form of a short lecture and notes. Students then worked in small groups on the in class activity composed of critical thinking questions where they were asked to utilize pKa values to predict the predominant species of a weak acid at various pH values. This concept was then extended to an amino acid and a peptide including calculation of the isoelectric point. While students worked through these questions in small groups, the instructor moved about the room answering questions and gaining a clearer sense of misconceptions and 266

uncertainty. These questions provided an opportunity for students to further explore and develop the concepts. The class paused periodically to report out and discuss responses. Students consistently reported they valued these classroom discussions which either reassured their thinking was on the right track or revealed to them misconceptions or an incomplete understanding. This instructor found it was possible to start the activity in one class period and have students finish it in the next class period. Alternatively students were asked to complete the activity outside of class having the subsequent class begin by having students briefly discuss responses within their groups followed by a classroom discussion. Occasionally the post-activity questions were assigned as homework. One of the challenges of utilizing this approach is its reliance on student pre-class preparation. While lecture and notes help to minimize this expectation, it is beneficial in each class period to reemphasize to students the effectiveness of connecting with as much of the material as possible when first encountered in class. Problem-Based Worksheets Problem-based worksheets designed by the instructor were introduced to augment workbook activities. Initially these worksheets were designed to address extremely low scoring pre-assessment scores for both pH/pKa and hydrogen bonding. Each worksheet was designed to meet specific learning activity objectives associated with the most common misconceptions as revealed by preassessment responses. For example, in the case of pH/pKa students commonly confuse whether an ionizable group is protonated or deprotonated when pH is below pKa or when pH is above pKa. Each worksheet typically consisted of 1-5 short, free response questions where students are challenged to apply their knowledge to a new context. Answers are close-ended and prior knowledge is provided in lecture. For example, a problem based worksheet focusing on pH/pKa asked students to explore the titration curve of an amino acid, aspartic acid, and label the predominant species at each equivalence point and at each point where pH is equal to pKa. Students were also asked to calculate isoelectric point and provide the structure of the zwitterion. An additional worksheet was designed utilizing this same format but now challenging students to extend their knowledge to a tri- or tetra-peptide. These pH/pKa worksheets provide additional opportunities for students to retrieve existing knowledge and apply it in a different context. Likewise a problem-based worksheet for hydrogen bonding was also developed. Students were asked to predict whether a hydrogen bond could form between various organic molecules commonly encountered in organic chemistry and were provided with electronegativity values. The worksheet explored three different combinations of molecules: a molecule that could act as both a hydrogen bond donor and acceptor paired with another molecule that could also act as both a donor and acceptor (two alcohol molecules for example); a molecule that could act as an acceptor paired with another molecule that could act as an acceptor (two ketones for example); a molecule that could act as both a donor and acceptor paired with a molecule that cannot act as a donor or acceptor (alcohol and alkane for example). After the concept is introduced in lecture, student groups spend class time working through the worksheet, followed by a classroom discussion. On occasion, students were asked to complete the worksheet prior to class. This meant the next class began with group discussion of responses followed by classroom discussion that reinforced the concept. When this instructor was utilizing the traditional lecture-based format, a number of worksheet questions were originally assigned as practice outside of class. However, because they were ungraded assignments and because there was no in class discussion expectation, many students did not spend the time needed to master them or did not attempt them at all. For lecture-based instructors who have encountered the same challenge, setting aside in class time for these problems may be ideal. 267

Tactile Learning Activities Tactile learning activities are another active learning model that can be implemented to review and apply foundational concepts in the biochemistry classroom. They can focus on simple knowledge retrieval or more advanced application of concepts. For this activity the instructor poses a question and challenges students to make predictions using cards exhibiting specified information. Instructors can utilize this model to introduce a concept, review a concept or apply a concept already encountered in lecture. In an effort to address ongoing low scoring post-assessment results for hydrogen bonding, a tactile activity involving amino acids was developed. After briefly reviewing noncovalent interactions in lecture, each group of students was handed a series of cards each displaying an amino acid R group at a pH of 7.0. One amino acid per card. Students were asked to predict which amino acid R groups could potentially form a hydrogen bond and arrange these two cards side by side accordingly. Afterwards a classroom discussion ensued wherein each student group shared a prediction along with their justification. In addition to introducing students to amino acid R group structures, this activity contextualizes noncovalent interactions in a biological environment which for many students lends relevance to the concept. After discussing pH/pKa and its effect on charge in lecture, the instructor can once again pose these same two questions under varying pH conditions thus challenging students to connect two foundational concepts: hydrogen bonding and pH/pKa within the context of amino acids. This same format can be utilized to challenge students to predict which amino acid R groups could form a salt bridge under various pH conditions. Based on observations, tactile activities by far elicit the highest level of student interaction in my classroom and potentially could be developed to enhance any foundational concept. It is an excellent option to break up a lecture heavy class period. Learning Cycle Activity Activities following the learning cycle were developed as additional interventions in the classroom. Each learning cycle activity re-examines a foundational concept and demonstrates how it is applied in a biochemical system. Students often fail to recognize the relevance of a foundational concept until they observe its application within a biochemical context. The learning cycle is the basis for writing these POGIL-like activities and is rooted in three stages: exploration, concept invention and application (19). This activity was inspired by limited experience implementing POGIL activities in the Foundations of Biochemistry workbook. Much insight on writing a POGIL activity was gained by attending a regional POGIL workshop and is highly recommended if interested in crafting this type of activity (38). The learning cycle activity begins with an overarching biochemical question which piques student interest and can subsequently be answered upon mastering the activity. For example “What intermolecular force holds a DNA double helix together?” The question is followed by 23 specific learning objectives, some of which address common misconceptions revealed by pre and post- assessment responses. For example, the most common incorrect idea among my students was that all hydrogen atoms are capable of hydrogen bonding. Henceforth one of the learning objectives developed was for students to be able to identify a hydrogen bond as a special type of charge-charge interaction. As students work through the activity they explore, invent and apply the concepts of electronegativity and bond polarity which lay the foundation for hydrogen bonding. For instance, students explore a model depicting hydrogen bonding between DNA nitrogenous base pairs. Information including the definition of electronegativity, bond polarity and electronegativity values is shared. Students are then asked a series of questions wherein they predict whether a series of bonds are polar or nonpolar, labeling the polar bonds with appropriate partial charges. Armed with these 268

symbols they are asked to return to the nitrogenous base pair model and assign partial charges to the interactions between the base pairs. Information follows revealing this interaction is a hydrogen bond. Students then apply their knowledge of a hydrogen bond within a familiar context such as general or organic chemistry. Provided with a series of organic molecule pairs, students are asked to predict if a hydrogen bond can form and to label the atoms involved in forming the hydrogen bond with partial charges. If a hydrogen bond cannot form, they are asked to supply the rationale. Finally students extend their knowledge into a biochemical context using amino acids. Specifically, students are asked to predict whether a hydrogen bond can form between two amino acid side chains, again labeling atoms involved in forming this bond with partial charges. In an effort to minimize class time spent on review, each activity is completed as an ungraded pre-class activity. In the subsequent class meeting, students briefly discuss responses within their groups followed by a classroom discussion of key conceptual questions. Alternatively this type of activity can be performed in class followed by a group discussion thus more closely mirroring implementation of a traditional POGIL activity. Thus far, review activities focusing on hydrogen bonds, bond energy and chemical equilibrium have been implemented in my classroom. Re-assessment and Revision At the close of the course, the IFCB is re-administered to students. This post-assessment should be conducted for each class as student populations and course enrollment can vary. This instructor administers the assessment as part of the final examination so it does impact a student’s final exam grade. Students are not made aware that they will be retaking the exact assessment they encountered on day one of class. They are only told that the final examination format will consist of multiple choice and free response questions. Only scores from students completing both the pre and postassessment are examined. Student responses can reveal misconceptions which may persist even after an intervention is introduced. An instructor can utilize post-assessment scores to determine effectiveness of a particular intervention or combination of interventions for a particular foundational concept. Based on post-assessment gains, the instructor may choose to continue implementing the same intervention, introduce an alternate or additional intervention or revise an existing intervention. After the semester is completed, this instructor reflects on activities implemented and their impact on learning gains. Table 1 summarizes average pretest and posttest scores for the four foundational concepts discussed in this chapter: pH/pKa and charge, hydrogen bonds, bond energy and chemical equilibrium. This compilation reflects scores gathered over a five semester period and is reflective of overall trends in learning gains and normalized learning gains. In the first semester of this project, the Foundations of Biochemistry workbook along with multiple worksheets honing pH/pKa and one worksheet honing hydrogen bonding were implemented. This instruction resulted in improvements for all seven foundational concepts with learning gains ranging from 15% (chemical equilibrium) to 73% (pH/pKa). Because the pH/pKa learning gain was extremely encouraging to the instructor, no revision of activities occurred in subsequent semesters resulting in a normalized learning gain of 82% over a five semester period. The instructor attributes the higher scoring pH/pKa learning gain in part to the number of interventions introduced (workbook and multiple worksheets).

269

Table 1. Average Percentage of Students Providing the Correct Answer for Each Foundational Concept over a Five Semester Perioda Foundational Concept

Pretest (%)

Posttest (%)

Learning Gain (%)

Normalized Learning Gain (%)

pH/pKa

5

83

78

82

Hydrogen bonding

3

46

43

45

Bond energy

18

66

48

59

Chemical equilibrium

30

51

21

31

a Pretest

and posttest percentages represent students who answered all three questions for each concept correctly (N=121).

Although the learning gain for hydrogen bonding in the first semester was fairly high (35%), only 41% of students answered all three questions correctly. Because utilization of multiple pH/ pKa activities the first semester resulted in a high learning gain, the instructor wanted to ascertain if adding a hydrogen bonding activity could improve post-assessment scores. Thus a tactile learning activity was introduced in a subsequent semester resulting in a learning gain of 56% which was an improvement over previous semesters. A closer examination of student post-assessment responses indicated a misconception persisted thus although the tactile activity may have provided another opportunity to apply their knowledge within a biochemical context, it did not directly address this misconception. To address this persistent misconception, the hydrogen bonding worksheet was transformed into a learning cycle activity and introduced in addition to the workbook/tactile activity combination. This combination resulted in a lowered learning gain (42%) compared to the workbook/worksheet/tactile activity combination (56%) indicating the learning cycle activity may have detracted from student learning or the decline simply occurred due to variations in student population. Further reflection on this activity is required prior to revision and re-assessment. Overall, introduction of targeted interventions have resulted in an average normalized learning gain of 45% for hydrogen bonding thus indicating the impact of these activities on student learning. In an effort to improve the bond energy student learning gain, a learning cycle activity was introduced in addition to the workbook. This combination resulted in an improved learning gain of 53% on average compared to semesters where the workbook alone was implemented (42% average learning gain). The normalized learning gain for bond energy was 59% over the course of five semesters of intervention implementation. In a similar fashion, a chemical equilibrium learning cycle activity was introduced which resulted in an improved average learning gain of 33% compared to semesters wherein only the workbook was implemented (14% average learning gain). The normalized learning gain for chemical equilibrium over the course of five semesters was 31% indicating the current interventions are improving student learning. Overall the improvement in learning gains observed in this work correlates with trends described by Xu. Some variation exists as hydrogen bonding gains from this study were lower in comparison while bond energy and pH/pKa gains were slightly higher in comparison (10). When comparing learning gains reported by Taylor at a similarly sized private liberal arts college, learning gains for bond energy were strikingly similar, while both hydrogen bonding and pH/pKa learning gains from this study were both comparatively higher (14).

270

Challenges of Design and Implementation While introduction of active learning targeting foundational concepts may better position students to master threshold concepts, some challenges accompany its design and implementation. One challenge is the utilization of classroom time for review of concepts, detracting from time spent teaching advanced concepts. This instructor has reduced content covered in the course, recognizing that no instructor can cover everything but instead focusing on enriching student learning of threshold concepts. In addition, classroom time originally dedicated to discussion of certain concepts has been minimized and instead students are expected to conduct some review of these concepts outside of class. In order to accomplish this expectation, students are armed with additional resources such as lecture notes, study guides, textbook, and practice problems to aid in learning outside of the classroom. The instructor emphasizes that assessment will include content covered inside and outside of the classroom. Another challenge instructors may encounter is maintaining an engaging and stimulating environment for students who enter the course already having mastered foundational concepts. One potential strategy is to have students work through the activities in groups, creating an opportunity for those who have mastered the concepts to lead their peers in learning. This environment creates an opportunity to hone leadership skills and provides a sense of self-accomplishment. Another challenge faced by instructors may be maintaining student accountability for pre-class assignments. To combat this, an instructor may incorporate completion of each assignment as a component of the course grade or administer a brief online or in class quiz. Because students are working in groups, there is an expectation of peer preparedness which will motivate most students. This instructor repeatedly emphasizes the immense potential gain in classroom learning a student experiences due to pre-class vigilance.

Conclusion An iterative process which identifies students’ needs, introduces active learning interventions and assesses the impact of these interventions can be implemented into any instructor’s biochemistry classroom. This process can be utilized to model application of active learning strategies such as problem-based worksheets, tactile learning activities and learning cycle activities. Improvements in foundational concept knowledge may better position students to master threshold concepts thus transforming their understanding of biochemistry.

Acknowledgments The author would like to thank Sachel Villafane, Jennifer Loertscher, Vicky Minderhout, and Jennifer Lewis for the generous use of the pre/post assessment instrument.

References 1.

2.

Ross, P. M.; Taylor, C. E.; Hughes, C.; Kofod, M.; Withaker, N.; Lutze-Mann, L.; Tzioumis, V. Threshold Concepts and Transformational Learning, 1st ed.; Meyer, J. H. F., Land, R., Baillie, C. , Eds.; Educational Futures: Rethinking Theory and Practice; Sense Publishers: Rotterdam, The Netherlands, 2010; Vol. 42, pp 165−178. Cooper, M.; Posey, L.; Underwood, S. Core Ideas and Topics: Building Up or Drilling Down? J. Chem Educ. 2017, 94, 541–548.

271

3. 4. 5. 6.

7.

8.

9. 10.

11. 12.

13. 14.

15. 16. 17. 18. 19.

20.

Davies, P.; Managan, J. Threshold Concepts and the Integration of Understanding in Economics. Stud. High. Educ. 2007, 32, 711–726. Loertscher, J. Student Centered Education: Threshold Concepts in Biochemistry. Biochem. Mol. Biol. Educ. 2011, 39, 56–57. Loertscher, J.; Green, D.; Lewis, J.; Lin, S.; Minderhout, V. Identification of Threshold Concepts for Biochemistry. CBE-Life Sci. Educ. 2014, 13, 516–528. Rowland, S.; Smith, C.; Gillam, E.; Wright, T. The Concept Lens Diagram: A New Mechanism for Presenting Biochemistry Content in Terms of “Big Ideas”. Biochem. Mol. Biol. Educ. 2011, 39, 267–279. Tansey, J.; Baird, T.; Cox, M.; Fox, K.; Knight, J.; Sears, D.; Bell, E. Foundational Concepts and Underlying Theories for Majors in “Biochemistry and Molecular Biology”. Biochem. Mol. Biol. Educ. 2013, 41, 289–296. Wright, A.; Provost, J.; Roecklein-Canfield, J.; Bell, E. Essential Concepts and Underlying Theories from Physics, Chemistry, and Mathematics for “Biochemistry and Molecular Biology” Majors. Biochem. Mol. Biol. Educ. 2013, 41, 302–308. Villafane, S.; Loertscher, J.; Minderhout, V.; Lewis, J. Uncovering Students’ Incorrect Ideas about Foundational Concepts for Biochemistry. Chem. Educ. Res. Pract. 2011, 12, 210–218. Xu, X.; Lewis, J.; Loertscher, J.; Minderhout, V.; Tienson, H. Small Changes: Using Assessment to Direct Instructional Practices in Large-Enrollment Biochemistry Courses. CBELife Sci. Educ. 2017, 16, 1–13. Bodner, G. Constructivism: A Theory of Knowledge. J. Chem. Educ. 1986, 63, 873–878. Halpern, D.; Hakel, M. Applying the Science of Learning to the University and Beyond: Teaching for Long-Term Retention and Transfer. Change: The Magazine of Higher Learning 2003, 35, 36–41. Karpicke, J.; Roediger, H. The Critical Importance of Retrieval for Learning. Science 2008, 319, 966–968. Taylor, A.; Olofson, E.; Novak, W. Enhancing Student Retention of Prerequisite Knowledge Through Pre-Class Activities and In-Class Reinforcement. Biochem. Mol. Biol. Educ. 2017, 45, 97–104. Anderson, T.; Schӧnborn, K. Bridging the Gap: Bridging the Educational Research-Teaching Practice Gap. Biochem. Mol. Biol. Educ. 2008, 36, 309–315. Tanner, K.; Allen, D. Approaches to Biology Teaching and Learning: Understanding the Wrong Answers-Teaching toward Conceptual Change. Cell Biol. Educ. 2005, 4, 112–117. Minderhout, V.; Loertscher, J. Lecture-free Biochemistry: A process oriented guided inquiry approach. Biochem. Mol. Biol. Educ. 2007, 35, 172–180. Mayer, R. E. Information processing variables in learning to solve problems. Rev. Educ. Res. 1975, 45, 525–541. Loertscher, J.; Minderhout, V. Implementing Guided Inquiry in Biochemistry: Challenges and Opportunities. Biochemistry Education: From Theory to Practice; ACS Symposium Series; American Chemical Society: Washington, DC, 2019; Vol. 1337, Chapter 5. Ojennus, D. Assessment of Learning Gains in a Flipped Biochemistry Classroom. Biochem. Mol. Biol. Educ. 2016, 44, 20–27.

272

21. Kulak, V.; Newton, G. A Guide to Using Case-Based Learning in Biochemistry Education. Biochem. Mol. Biol. Educ. 2014, 42, 457–473. 22. Peters, A. Teaching Biochemistry at a Minority-Serving Institution: An Evaluation of the Role of Collaborative Learning as a Tool for Science Mastery. J. Chem. Educ. 2005, 82, 571–574. 23. Anderson, W.; Mitchell, S.; Osgood, M. Comparison of Student Performance in Cooperative Learning and Traditional Lecture-based Biochemistry Classes. Biochem. Mol. Biol. Educ. 2005, 33, 387–393. 24. Cornely, K. ConfChem Conference on Case-Based Studies in Chemical Education: The Use of Case Studies in an Introductory Biochemistry Course. J. Chem. Educ. 2012, 90, 258–259. 25. Ragan, E. Vignette #2: Making a Switch to In-Class Activities in the Biochemistry Classroom. Biochemistry Education: From Theory to Practice; ACS Symposium Series; American Chemical Society: Washington, DC, 2019; Vol. 1337, Chapter 13. 26. Garvin-Doxas, K.; Klymkowski, M. W. Understanding randomness and its impact on student learning: Lessons learned from building the Biology Concept Inventory (BCI). CBE-Life Sci. Educ. 2008, 7, 227–233. 27. Howitt, S.; Anderson, T. R.; Costa, M.; Hamilton, S.; Wright, T. A concept inventory for molecular life sciences: How will it help your teaching practice? Australian Biochemist. 2008, 39, 14–17. 28. Mulford, D. R.; Robinson, W. R. An inventory for alternate conceptions among first-semester general chemistry students. J. Chem. Educ. 2002, 79, 739–743. 29. Villafane, S.; Bailey, C.; Loertscher, J.; Minderhout, V.; Lewis, J. Development and Analysis of an Instrument to Assess Student Understanding of Foundational Concepts Before Biochemistry Coursework. Biochem. Mol. Biol. Educ. 2011, 39, 102–109. 30. Loertscher, J.; Minderhout, V.; Frato, K. Foundations of Biochemistry, 4th ed.; Pacific Crest: Hamptom, NH, 2015. 31. Werth, M. T. Serotonin in the Pocket: Non-covalent Interactions and Neurotransmitter Binding. CourseSource, 2017. https://doi.org/10.24918/cs.2017.14 (accessed March 10, 2019). 32. Johnson, R. J. Teaching Foundational Topics and Scientific Skills in Biochemistry Within the Conceptual Framework of HIV Protease. Biochem. Mol. Biol. Educ. 2014, 42, 299–304. 33. Cooper, A. K.; Oliver-Hoyo, M. T. Creating 3D Physical Models to Probe Student Understanding of Macromolecular Structure. Biochem. Mol. Biol. Educ. 2017, 45, 491–500. 34. Loertscher, J.; Villafane, S.; Lewis, J.; Minderhout, V. Probing and Improving Student’s Understanding of Protein α-Helix Structure Using Targeted Assessment and Classroom Interventions in Collaboration with a Faculty Community of Practice. Biochem. Mol. Biol. Educ. 2014, 42, 213–223. 35. Ernst, D.; Hodge, A.; Yoshinobu, S. What is Inquiry-Based Learning? Notices of the American Mathematical Society 2017, 64, 570–574. 36. Webb, D. In Actively Learning Mathematics (ALM); Proceedings of the Fifth Annual Mathematics Teacher Education Partnership Conference, Atlanta, GA, June 26−28, 2016; Lawler, B. R., Ronau, R. N., Mohr-Schroeder, M. J. Eds.; Association of Public Land-Grant Universities: Washington, DC, 2016; pp 61−67.

273

37. Newman, D.; Stefkovich, M.; Clasen, C.; Franzen, M.; Wright, L. K. Physical Models Can Provide Superior Learning Opportunities Beyond the Benefits of Active Engagements. Biochem. Mol. Biol. Educ. 2018, 46, 435–444. 38. POGIL Home Page. www.pogil.org (accessed March 10, 2019).

274

Chapter 13

Vignette #2: Making a Switch to In-Class Activities in the Biochemistry Classroom Emily J. Ragan* Department of Chemistry, Metropolitan State University of Denver, Denver, Colorado 80217, United States *E-mail: [email protected].

Active learning strategies aim to increase student critical thinking and engagement. In this article I describe my biochemistry classroom switch from lecture-only to half lecture and half in-class activities, inspired by Process Oriented Guided Inquiry Learning (POGIL). Students in a first semester biochemistry course maintained the same ACS exam scores at the end of the course, continued to rate the course and its instruction highly, and class attendance significantly increased after the change in pedagogy. This format was also implemented in a second semester biochemistry course during a course redesign. The flexibility of in-class activities allowed an iterative addition of a bioinformatics themed course-based undergraduate research experience (CURE). The Biochemistry II students report learning practical skills that are likely to benefit them in the future.

Introduction: Transition to Half Lecture and Half In-Class Activities Five years ago, after teaching in a primarily lecture-only format, I transitioned to a classroom structure of half lecture and half in-class activities. This switch was inspired by attending an intermediate-level workshop on Process Oriented Guided Inquiry Learning (POGIL). I continue to teach in this format because I see students make connections in the classroom, can more easily discern challenging areas for students, and I appreciate the flexibility of multiple teaching strategies. My experience with using in-class activities is that it requires an equivalent effort to lecturing but uses different skills due to the need to facilitate group work and develop activities that meet the learning objectives I use. Through facilitating in-class activities, I am more excited and engaged in the classroom and can more easily introduce new lecture innovations. For example, I recently introduced a course-based undergraduate research experience (CURE) during the first half of the semester in Biochemistry II.

© 2019 American Chemical Society

Evidence-Based Active Learning with a Focus on Biochemistry The Case against Straight Lecture Student success depends on active engagement, making connections between new content and prior knowledge, and developing sufficient understanding to apply material to new content areas (1–5). While lecture remains popular, there is increasing evidence that a lecture-only classroom has limited capacity to elicit skills that we want our students to develop, including accurate figure analysis, successful extraction of information from written passages, and an ability to make connections beyond specific examples (6–8). In addition, student attention lapses with increasing frequency during lecture. While inattention can be reduced by engaging practices such as demonstrations and real-time formative assessments such as classroom polling, these practices do not typically challenge students to transfer knowledge broadly (9). We hope students will retain key concepts for use in future courses and in their careers, but my experience in the biochemistry classroom shows I cannot assume that students will always successfully remember and apply general chemistry and organic chemistry concepts. Students also struggle with transfer of knowledge from the classroom to other domains. Hence, there is a call for educators in biochemistry and related fields to work collaboratively to improve our teaching and to rethink how we spend our valuable in-class time with our students (1, 10). Active Learning Pedagogies Active learning pedagogies, which include POGIL, problem-based learning (PBL), and peer-led team learning (PLTL), aim to create opportunities for students to engage with the course material in a supportive environment where they can take advantage of interacting with their peers while under the guidance of an experienced instructor (4). Active learning was an important element of a highly-structured course design that decreased the achievement gap in introductory biology (5); another study looking at an introductory biology course found gains were especially pronounced among students in the lower end of the grade distribution (11). Over eighty percent of biochemistry students in a large class reported in-class POGIL activities helped them learn (12). Group work may lead to learning gains due to increased peer and student-teacher interactions, and structured small groups and varied active learning strategies are proposed to promote equity and inclusion (13). It also provides an opportunity to target specific biochemistry concepts, as Mary Kopecki-Fjetland details in the chapter entitled “Introducing Active Learning to Improve Student Performance on Threshold Concepts in Biochemistry (14).” A recent study found increased course motivation in students taking biochemistry in a small group and discussion format (15). Other research has shown value in using significant amounts of class time for students to interpret textbook figures, first independently and then in groups (8). The skills that are important for the acquisition of new knowledge include visual literacy, reading and understanding biochemical literature, and deep (rather than surface) learning, which can be practiced through in-class activities (6–8). Other goals for active learning may include communication skill development, enhanced ability to work in groups, improved attitudes, increased motivation, higher retention rates and a decrease in the achievement gap (5, 11, 15–18). For example, a meta-analysis examining problem-based learning at a Dutch medical school found improved interpersonal skills (19). Students using PBL completed the program sooner and with a higher retention rate, with a smaller, positive effect for acquiring medical knowledge (19). 276

Recently there has been increased interest in course-based undergraduate research experiences (CUREs), which involve all students in the course in addressing an authentic research question (20, 21). Elements of a CURE include using scientific techniques, making discoveries that fit into broader scientific endeavors, collaboration, iteration, and authentic product production (20, 21). CUREs have been proposed to have a myriad of benefits for students, including increasing persistence in science majors, and offering the experience to all students in a course rather than a self-selected group of students who volunteer for other research experiences (20). Here I share my journey of introducing in-class activities in biochemistry lecture courses over the past five years. While detailed descriptions of implementing POGIL in full-class periods have been given elsewhere (17, 22, 23), here I share my experience with a hybrid approach in Biochemistry I, a move from full-class lecture to half lecture and half in-class activities, and include impacts on broad measures of student success and satisfaction. This foundation influenced the way I developed Biochemistry II during a course redesign. I initially mimicked what I did in Biochemistry I but evolved to link individual in-class activities and develop a CURE. After sharing my specific experiences, I discuss common concerns about making a switch to active learning and provide specific recommendations for making a smooth transition.

Active Learning Modifications to Biochemistry I In Fall 2014 I introduced in-class activities in Biochemistry I, the first semester of a two semester sequence, by adding guided-inquiry worksheets. There are well-regarded published POGIL activities for Biochemistry which I used throughout the Fall 2014 semester (24). I ultimately decided to write my own activities, typically using at least one image from the textbook, so I could specifically target my learning objectives with in-class activities reinforcing and expanding upon content initially introduced in lecture. At Metropolitan State University of Denver (MSU Denver), Biochemistry I is offered twice per week in 110 min blocks. Class formats before and after the introduction of active-learning segments are shown in Table 1. Groups of three to four students were formed on the basis of proximity in the classroom. This strategy worked in a classroom with movable, individual desks as well as in a classroom where students shared long tables, in which some students would turn around to work with students behind them. Other class formats were unchanged, including the learning objectives, weekly online homework assignments, weekly short (approximately 10 minutes long) quizzes, and clicker questions in lecture portion. I continued to give four midterm exams and the 2012 ACS Biochemistry exam (40 core questions) as the final exam. One slight change is that I shifted from giving 1 point/day for attendance (4% of total grade) before in-class activities to 2 points/day with in-class activities (7% of total grade). In both cases the lowest two attendance scores were dropped from the students’ grades. Table 1. Class Format before and after Addition of In-Class Activities Before in-class activities (F13, S14)

After in-class activities (F14, S15, F15, S16)

10 min quiz (one day a week)

10 min quiz (one day a week)

45-55 min lecture

45-55 min lecture

5 min break

5 min break

50 min lecture

35 min in-class activity 15 min reporting out

277

Fractional attendance (classes attended/total classes), final grades and ACS exam scores were compared for students who completed the whole semester, as measured by taking the ACS exam, from before and after the shift to POGIL instruction using unpaired, two-tailed t-tests (F13, S14, n = 59; F14, S15, F15, S16, n = 122). Pearson correlation coefficients were computed for class attendance (number of classes attended during the semester) and ACS Biochemistry exam score as well as for class attendance and final grade in course. All statistical analyses were performed using GraphPad Prism. Impact on Attendance In my Biochemistry I course there was a significant correlation between attendance and ACS final exam score, as well as attendance and final grade in the course, by Pearson correlation (Table 2). This correlation was observed before the introduction of in-class activities semesters (F13, S14) and after the introduction of in-class activities (F14-S16). These results are consistent with prior research, which shows class attendance in college is positively correlated with student grades (25–27). In addition, the meta-analysis shows that class attendance is a better predictor of college grades than SAT scores, high school GPA, study habits and study skills (25). Table 2. Correlation between Attendance and Student Performance F13-S14 Attendance F13-S14 Attendance F14-S16 Attendance F14-S16 Attendance vs. ACS exam score vs. Grade % vs. ACS exam score vs. Grade % Pearson r

0.2799

0.6896

0.3023

0.4622

95% confidence interval

0.02566 to 0.5001

0.5265 to 0.8038

0.1316 to 0.4556

0.3099 to 0.5914

R squared

0.07834

0.4756

0.09138

0.2136

P value

0.0318