Addressing Inequities in Modern Educational Assessment: Bridging the Gap [1st ed. 2023] 303145801X, 9783031458019

Citation preview

Jorge Sainz Ismael Sanz   Editors

Addressing Inequities in Modern Educational Assessment Bridging the Gap

Addressing Inequities in Modern Educational Assessment

Jorge Sainz · Ismael Sanz Editors

Addressing Inequities in Modern Educational Assessment Bridging the Gap

Editors Jorge Sainz Rey Juan Carlos University Madrid, Spain

Ismael Sanz Rey Juan Carlos University Madrid, Spain

Institute for Policy Research University of Bath Bath, UK

ISBN 978-3-031-45801-9 ISBN 978-3-031-45802-6 (eBook) https://doi.org/10.1007/978-3-031-45802-6 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Paper in this product is recyclable.

Introduction

During the pandemic, schools have made an enormous effort to continue providing virtual education to students. This modality can complement face-to-face education and, between the two, achieve a better outcome through hybrid systems. However, face-to-face education is needed more than online education, especially for lagging behind students. Moreover, the online education provided to learners during the pandemic was no more than a temporary emergency offering. Evidence shows that distance education has failed to replace face-to-face education during confinement. Many families report little learning has occurred in the UK and the Netherlands (Andrew et al., 2020; Bol, 2020) and throughout Ibero-America (OEI, 2021). As David Deming (2020) points out in an interesting article in the New York Times, an essential part of the work in teaching involves personalization through tutoring, feedback, or individualized follow-up that is impossible to scale. There is simply no technological substitute for teachers. Teachers significantly impact students’ life choices and career success and have no technological substitute. Learning loss will also be highly unequal, intensifying the skills gap by socioeconomic status. Access to broadband Internet service is different in all households. In the best of cases, young people from disadvantaged households have had to share a computer among all family members. They needed more support from their parents in their homework because of their parents’ obligation to work outside the home or their lower cognitive ability. In Latin American countries, more than 40% of households have no connectivity or devices. University of Michigan’s Prof. Susan Dynarski already pointed out in another article in the New York Times, before the health crisis, that even if Internet access was similar among households, online education harms the group of students lagging who are precisely those who most need the support and reinforcement of teachers in the classroom (Dynarski, 2018). Following virtual classes without a teacher requires high motivation, self-regulation, discipline, and organization. For advanced learners, online learning opens up new opportunities.

v

vi

Introduction

What is worse is that these asymmetries of virtual learning about face-to-face learning are much more pronounced when students come from disadvantaged socioeconomic backgrounds; even within the same socio-economic background, students who lag are the most disadvantaged when abandoning face-to-face learning for online learning. Heppen et al. (2017) already noted, in the case of 17 Chicago schools and the subject of Algebra, that when students in the same class are randomly assigned to a virtual or face-to-face format, the former score 20% of a standard deviation below that of the latter, along with a lower probability of passing (66% versus 78%). In 2022, Brown University’s Annenberg Education Research Center published a paper showing that the shift from face-to-face to virtual learning in the spring of 2020 due to the pandemic reduced college completion rates by three to six percentage points and also has moderately adverse effects on students’ relationships with their peers and their teachers (Bird et al., 2022). The results of this research suggest that faculty experience teaching a given course online does not mitigate these consequences for students who switch from face-to-face to online instruction. During the pandemic, there has been a 10–25% dropout rate in Latin American countries among higher education students. This dropout rate is higher among families with lower incomes and fewer technological resources. The above evidence does not indicate that technological tools cannot enhance and complement education but cannot substitute face-to-face teaching. The paper led by Stanford University Prof. Eric Bettinger (Bettinger et al., 2020) develops a randomized, controlled experiment with 6000 students in Russia varying the intensity of computer-assisted learning as a substitute for traditional learning. The study is entitled Does Educational Technology Substitute for Traditional Learning? Experimental Estimates of an Educational Production Function. The results show that academic performance is improved but that completely replacing face-to-face education is a mistake. At the baseline level, increases in computer use improve student outcomes as students become more engaged in learning through technology. However, if its use intensifies, the positive effects of traditional instruction are lost. The blended approach, i.e., hybrid systems, keeps students engaged while exposing students to the most beneficial learning methods. For example, the article led by Columbia University Prof. Maya Escueta in her review in the Journal of Economic Literature (2020) analyzes 30 randomized experiments and regression discontinuities. It concludes that computer-assisted learning programs, i.e., complementing face-to-face education, are a technological tool that provides outstanding educational outcomes. The project being carried out jointly by the OEI and the IDB, “Education for the 21st Century: Thriving, Competing and Innovating in the Digital Era”, aims to develop systematic, hybrid, and qualified models. A team of researchers from Harvard’s Center for Education Policy Research (CEPR) has used the results of a computer-based test (MAP) in the USA to study the possible learning loss during the COVID period (Goldhaber et al., 2022, https:// www.nber.org/papers/w30010). This year’s findings of this May study are that US students who followed most of the 2020–2021 course remotely lost about 50% of a

Introduction

vii

typical school year’s math learning during these last two years. In contrast, students who attended schools face-to-face for almost all of the 2020–2021 course lost the equivalent of 20% of learning. David Leonhardt’s New York Times article, also based on the CEPR researchers’ study, notes that this learning loss caused by the pandemic has been most intense for students from the most disadvantaged homes (https://www. nytimes.com/2022/05/05/briefing/school-closures-covid-learning-loss.html). Students with fewer resources have been more affected by their educational results for two reasons. The first is that in the schools that made the most use of remote education, a type of education that yielded worse results than face-to-face education in the pandemic, there is a higher proportion of students from disadvantaged backgrounds. The second reason is that even within the schools that used virtual education, learning was lower among students with fewer resources because, as mentioned above, they had worse Internet connections, fewer computers at their disposal, and more difficulties for their parents to help them with their homework. Another conclusion, therefore, of this Harvard CEPR analysis is that the pandemic has increased inequalities in education because the most vulnerable students have lost more learning, concerning the progress made in the grades prior to the pandemic, than the most advantaged. The study concludes by recalling the importance of promoting educational measures that allow students to recover lost learning and adopt measures to compensate for the effects and design and apply policies capable of implementing hybrid education systems, face-to-face and at a distance, for all equally. One of the initiatives that have shown the most significant effect on student learning has been small group tutoring with students who are lagging in reinforcing instrumental subjects such as mathematics and language, with teachers working with five or fewer students 3–5 times per week for at least 30 min. The conclusion of this study by CEPR researchers and the New York Times article coincides with the post that Almudena Sevilla (London School of Economics), and the coordinators of this book, Jorge Sainz and, Ismael Sanz made two years ago entitled “A proposal to avoid the negative effect of school closures on the future of young Spaniards” (https://nadaesgratis.es/admin/una-propuesta-para-evitar-el-efecto-negativo-de-loscierres-de-los-centros-educativos-en-el-futuro-de-los-jovenes-espanoles). We must avoid falling into the inertia that would lead us to a return to the time before the pandemic as if it were a return to full normality; the crisis had shown that education, before 2019, suffered from a lack of quality, inequity, and low capacity for inclusion: the educational future has to be hybrid, innovative, and transformative. Mariano Jabonero Secretary General of the OEI

References Andrew, A., Catta, S., Costas-Dias, M., Farquharson, C., Kraftman, L., Krutikova, S., et al. (2020). Learning during the lockdown: Real-time data on children’s experiences during home learning. IFS briefing note BN288. London: Institute for Fiscal Studies.

viii

Introduction

Bettinger, E., Kardanova, E., Fairlie, R., Loyalka, P., Kapuza, A., & Zakharov, A. (2020). Does EdTech substitute for traditional learning? Experimental estimates of the educational production function. Stanford Institute for Economic Policy Research (SIEPR). Working paper No. 20–010. Bird, K. A., Castleman, B. L. , & Gabrielle, L. (2022). Negative impacts from the shift to online learning during the COVID-19 crisis: Evidence from a statewide community college system. (EdWorkingPaper: 20–299). Retrieved from Annenberg Institute at Brown University: https:// doi.org/10.26300/gx68-rq13 Bol, T. (2020). Inequality in homeschooling during the corona crisis in the Netherlands. First results from the LISS panel. Working Paper. University of Amsterdam. Deming, D. (2020). Online learning should return to a supporting role. In The New York Times. https://www.nytimes.com/2020/04/09/business/online-learning-virus.html Dynarski, S. (2018). Online courses are harming the students who need it most. In The New York Times. https://www.nytimes.com/2018/01/19/business/online-courses-are-harming-the-stu dents-who-need-the-most-help.html Goldhaber, D., Kane, T., McEachin, A., Morton E., Patterson, T. y Staiger, D., (2022). The consequences of remote and hybrid instruction during the pandemic. Research Report. Cambridge, MA: Center for Education Policy Research, Harvard University Heppen, J. B., Sorensen, N., Allensworth, E., Walters, K., Rickles, J., Taylor, S. S. & Michelman, V. (2017). The struggle to pass algebra: Online versus face-to-face credit recovery for at-risk urban students. Journal of Research on Educational Effectiveness, 10(2), 272–296 Leonhardt, D. (2020). Not good for learning. In The New York Times. https://www.nytimes.com/ 2022/05/05/briefing/school-closures-covid-learning-loss.html Organización de Estados Iberoamericanos. (2021). COVID-19: Efectos en la educación un año después. Organización de Estados Iberoamericanos: Madrid. Sevilla, A., Sainz, J. & Sanz, I. (2020). Una propuesta para evitar el efecto negativo de los cierres de los centros educativos en el futuro de los jóvenes españoles. In Nada es Gratis. https://nadaesgratis.es/admin/una-propuesta-para-evitar-el-efecto-negativo-de-loscierres-de-los-centros-educativos-en-el-futuro-de-los-jovenes-espanoles

Contents

A Post-Covid Comparative Analysis of Digital Skills in Vocational Education Teachers for 4 Latin American Countries . . . . . . . . . . . . . . . . . . Marta Martín-Llaguno, Eva María Fernández-González, and Jorge Sainz

1

The Impact of the COVID-19 Pandemic on Education Learning . . . . . . . Ismael Sanz and J. D. Tena

15

The Role of International Education Agencies After the Pandemic . . . . . Ana Capilla

37

Tutoring and Its Effects on Academic Achievement: A Policy Evaluation with Machine Learning Methods . . . . . . . . . . . . . . . . . . . . . . . . . María Teresa Ballestar, María Teresa Freire-Rubio, and Arturo Ortigosa-Blanch

53

The Impact of the School Closures on Bullying and Cyberbullying in Spain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Miguel Cuerdo-Mir and Luis Miguel Doncel-Pedrera

71

Virtual or Face-to-Face Education: What Have We Learned from the years of the Pandemic? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pedro Adalid Ruíz and Jesús García Laborda

89

Opportunity Costs, Covid-19, and Early Dropout Rate . . . . . . . . . . . . . . . . 103 Luis Pires Random Experiment on Relative Performance Feedback in Higher Education at URJC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Cristian Macías Domínguez Gender Gap in STEM Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Rosa Belén Castro Núñez and Rosa Santero-Sánchez

ix

x

Contents

Digital Adoption in Times of Crisis: A Study for the European Countries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Aida Garcia-Lazaro A Review of Social Conditions During the Quarantine Period-Covid-19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Taraneh Shahin Bridging the Gap: Addressing Inequities in Modern Education Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

A Post-Covid Comparative Analysis of Digital Skills in Vocational Education Teachers for 4 Latin American Countries Marta Martín-Llaguno, Eva María Fernández-González, and Jorge Sainz

1 Introduction The irruption of COVID-19 pushed educators’ need for digital skills to the forefront of the learning debate as authorities closed schools and universities, in some cases for months, leaving virtual education as one of the few alternatives to knowledge dissemination (Sanz et al., 2020). These skills involve using digital technologies to enhance teaching, improve professional interactions with and between administrators and families, improve professional development, and promote continuous innovation in schools (Redecker & Punie, 2017). The development of ICT technologies sparks new opportunities for exchange and cooperation in education and helps to accommodate the learning needs of the new generations (millennials, Z…) of digital natives with the acquisition of new and traditional competencies (Sainz & Sandoval-Hernández, 2020). In this article on digital competence, we understand the knowledge, skills, abilities, and attitudes that allow the learner to perform predetermined tasks through digital technologies (OECD, 2021). The pandemic has made it clear that in training, digitalization establishes two functions: the development of their digital skills and the development of the minimum set of skills for students to advance in a technological environment. The COVID-19 pandemic and the rapid transition to online learning around the world have changed the acquisition of competences by students at an unprecedented scale. The question arose about the readiness of teachers for such changes, the digital competence of participants in the educational process, the emotional state of teachers and students, and the long-term effects on earnings of first, the closures and then the M. Martín-Llaguno · E. M. Fernández-González Department of Communication and Advertising, Universidad de Alicante, Alicante, Spain J. Sainz (B) Department of Applied Economics, Universidad Rey Juan Carlos, Madrid, Spain e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_1

1

2

M. Martín-Llaguno et al.

equivalence of physical and online education (Kuhfeld et al., 2022; Lindfors et al., 2021; Lockee, 2021). The difference in future income introduces distributional aspects on who and how online classes were taught and the teachers’ digital competences. Training in digital skills by teachers implies the appropriation and experimentation of technological resources that allow them to select, organize, and analyze information and share and disseminate the findings of their digital experiences. Consequently, the primary purpose of this research is to know and compare the acquisition of digital skills by Vocational Education Teachers in four Latin American countries after the pandemic. Our descriptive analysis focuses on the relevance of digital skills on teaching effectiveness and opens new insights into the transformation of teacher capabilities, especially for low-income and rural students (Gegenfurtner et al., 2020; Kuhfeld et al., 2022; Ballestar et al., 2022).

2 Theoretical Background The Covid crisis created a sudden rush to develop broad competences for the teachers, including pedagogical strategies, social competencies, and motivation for learning about digital participation that most of the time was reduced to the use, sometimes for the first time, of digital tools (Capilla et al., 2021; Sanz et al., 2020). Johannesen et al. (2014) introduce the difference between digital literacy, understood just as mastering the instrumental skills, and a more comprehensive concept where those abilities are accompanied by the potential to evaluate and use information critically in a context where technology includes aspects related to the socioeconomic context of the use of technology. That it is, digital skills go further than the utilitarian view of using any piece of software, which includes “using computers and software to download and upload different types of information, knowing how to search for information, being able to navigate within, classify, integrate and evaluate various types of information, communicating and expressing oneself through different meditational means, using digital tools for collaboration, and finally, being able to create and design complex digital material.” but also the ability to produce, create and, disseminate knowledge. Çebi and Reiso˘glu (2022a, 2022b) find that the previous definition is equally present in the new generations of teachers, who consider digital competence at all levels as a mix of knowledge on information and data, communication, safety, problem-solving, and productivity that can be transmitted to students thanks to personal abilities as well equipped, versatility, valuable, using technology effectively, creative, innovative, and querent. Although future teachers know what it takes to be a digitally competent teacher, and digital competences have increased over the years for future teachers at primary school, they are still in the middle range of competences and specifically below on content creation in most OECD countries, regardless of gender and education (Galindo-Domínguez & Bezanilla, 2021).

A Post-Covid Comparative Analysis of Digital Skills in Vocational …

3

Cattaneo et al. (2022) analyze if the same digital competences are sufficient across non-university educators. Their findings are especially relevant for Vocational Education Teachers (VET). Their findings, using multivariate regression, show they acknowledge a degree of digital competence analogous among educational levels. The main difference comes from personal factors and mindsets towards technology and the frequency of use of digital devices. As in many other aspects, Covid 19 has been a transformational event in vocational education. Still, the analysis of its impact on the digital skills of VET is an open question. In recent investigations, like Rauseo et al. (2022) and Lindfors et al. (2021), teachers find difficult the acquisition of digital abilities, the relevance of reskilling, and its support in educational policy. Experience shows that many teachers were forced to move their teaching online during the pandemic without proper preparation and support, which, in turn, was a source of frustration and stress. The pressuring needs forced school management to make decisions based on students’ and schools’ technological limitations, socioeconomic levels, and teachers’ digital competences, requirements, and opportunities, although they needed to be more optimal from a pedagogical point of view. To reduce this gap in the literature, this paper presents a systematic analysis of the adaptation of the “Digital Competence for Educators” self-assessment instrument developed by the European Commission Joint Research Centre of digital competency using it as an instrument to identify the level a sample of teachers who teach Technical Vocational Education in the Pacific Alliance countries: Chile, Colombia, Mexico, and Peru following the methodology presented by Çebi and Reiso˘glu (2022a) for Turkey. Our analysis identifies for each country the level of digital competence of their teachers, allowing the comparison of the competency levels among them. The third aim is to find whether the competency level of VET in the Pacific Alliance countries depends on gender and if there is a gender gap in any of the countries.

3 Methodology 3.1 The Digital Competences Framework The European Framework for the Digital Competence of Educators (DigCompEdu) responds to the conviction of almost all European Member States that educators need a set of digital competencies specific to their profession to take advantage of the potential of digital technologies to improve and innovate in education. It aims to collect and describe these specific digital competencies for educators by proposing twenty-two elementary competencies organized in six areas (Redecker & Punie, 2017). It is aimed at educators at all educational levels, from early childhood education to higher and adult education, including general and vocational training, care for students with special educational needs, and any other non-formal learning context.

4

M. Martín-Llaguno et al.

It aims to provide a general reference framework for developers of digital competence models, whether they are Member States, regional governments, national or regional agencies, educational organizations, or any entity, public or private, dedicated to training (INTEF, 2017).

3.1.1

Areas of Competences

The goal of the teaching task is to transmit to students the knowledge and skills necessary for them to develop their capabilities to the maximum and become digitally competent. To meet this end, teachers must possess the skills, abilities, and aptitudes that will enable them to be digitally competent in order to be able to transmit them to their students (Røkenes & Krumsvik, 2014). In this direction, the DigComEdu framework is composed of six areas in which the professional and pedagogical competencies needed by educators intertwine with the competencies to be acquired by students. The framework focuses on areas 2 to 5, where the digital pedagogical competence of educators is captured. These areas identify the digital competencies “educators need to adopt efficient, inclusive and innovative teaching and learning strategies.” The teaching process is covered in areas 2, 3, and 4, focusing on how to “make efficient and innovative use of digital technologies when programming (area 2), implementing (area 3) and evaluating (area 4) teaching and learning”. In contrast, area 5 is dedicated to the “potential of digital technologies in learner-centered teaching and learning strategies” and includes a series of guiding principles dealt with transversally in areas 2, 3, and 4. Area 1. Professional engagement. They are oriented to the broader professional environment, i.e., the use of digital technologies by educators in professional interactions with colleagues, students, parents, and other stakeholders, for their professional development and the collective good of the organization. Area 2. Digital content. This area examines the competencies required to use, create and share digital content related to learning effectively and responsibly. Area 3. Teaching and learning. This area focuses on the management and coordination of the use of digital technologies in teaching and learning. Area 4. Evaluation and feedback. Addresses the use of digital strategies to improve evaluation. Area 5. Student empowerment. It addresses the potential of digital technologies for student-centered teaching and learning strategies. Area 6. Development of students’ digital competence. This area details the specific pedagogical competencies needed to facilitate students’ digital competence acquisition and development. In turn, the analysis of the six competence levels goes from A1 to C2, moving successively through levels A2, B1, B2, and C1. The competencies of each area

A Post-Covid Comparative Analysis of Digital Skills in Vocational …

5

are defined by descriptors so that the progression between levels is continuous and cumulative; that is, each descriptor of a higher level includes all the descriptors of the lower levels except A1, which is the starting level.

3.1.2

Competence Levels

They describe five areas of competence defined by six layers defined by the (i) Model of Educational Digital Transformation, participation, and governance, (ii). Digital cultural literacy and inclusion of digital competencies in the education and training offer, (iii) Management and organization of centers, (iv) Teacher training, updating and certification of teachers’ digital competence, (v) Assurance of equipment, connectivity, and necessary software and (vi) Digital educational research and innovation. To measure it, we adapted the instrumental developed by the Spanish Ministry of Education (INTEF, 2017), which is widely used in Latin America and has exciting properties (Hernández, 2018). The questionary is defined as a 5-point Likert scale, where value one referred to “no use” and value five corresponded to “high use.” Teachers’ Digital Skills at the entry-level are A1 and A2. At the same time, the second layer (value 2) is level B1 (a person has an intermediate level if they can solve simple problems on their own using essential ICT resources for research, such as web browsers or scientific databases). Three would be assigned to level B2 (a person has an intermediate level if they are able to address their needs and solve research problems using specialized software). Value 4 would be assigned to level C1 (a person has an advanced level in using resources and software for research and is able to guide others in developing research skills). And finally, value 5 would be associated with level C2, or very advanced level, when an individual can use these ICT resources according to her needs and those of others in various complex contexts (Guillén-Gámez et al., 2020).

3.2 Data The Pacific Alliance, through the Technical Education Group (GTE), has been a decisive point of support for its collaboration and promotion of the work to be carried out. It has selected and provided the educational centers’ contacts and information and clarifications on infrastructure and educational policies implemented in different countries. It has also taken an interest in the progress being made. The representatives of the Technical Education Group of the Pacific Alliance have been responsible for the selection of the collaborating centers in each country; each center has selected the participant teachers too. Among the premises for the selection was that the center must teach TVE, regardless of the educational level. Being centers participating in projects of teaching innovation and internationalization of teaching or that the center was collaborating with entities and companies of digital projection have also been taken into account. A total of 265 teachers from 17 centers were

6

M. Martín-Llaguno et al.

Table 1 Data sample Country

Schools

Teachers

Females

Males

Chile

5

49

18

31

Colombia

4

54

26

28

México

5

93

39

54

Perú

3

69

21

48

Total

17

265

104

161

involved: 5 from Chile, four from Colombia, five from Mexico, and three from Peru (Table 1). The documentary analysis was based on previous studies within the Pacific Alliance: the project Strengthening Public Policies for Technical Vocational Education in the countries of the Pacific Alliance: Chile, Colombia, Mexico, and Peru (2020–2021) and the diagnostic phase of the project TVET Teacher Training in Pacific Alliance countries (2022–2023). From these studies, both the analysis of weaknesses, threats, strengths, and opportunities, as well as the proposals contained in each axis of the roadmap designed for each of the countries, have been considered. Data collection was carried out through documentary analysis, questionnaires, and interviews. Each teacher in the sample completed a questionnaire. Like most qualification frameworks, a framework of closed questions was used to determine the taxonomy of knowledge of the individuals surveyed, i.e., it made it possible to obtain the qualitative level of each of the individuals accurately, thus providing an in-depth view of their digital competence. After analyzing the data collected through the teacher questionnaires, personalized interviews were conducted with each of the representatives of the educational centers or institutions. Since the interviews were individualized, they were personalized by country and the educational center. The representatives of the centers were able to explain and qualify the effects resulting from the questionnaires, which allowed a more concise view of the state of the situation to be obtained when making the training proposal. Fourteen sample schools participated in the interviews: 4 from Chile, Colombia, and Mexico and 2 from Peru. Each center had the most suitable person to respond to the interviews, most of them being the directors of the centers. However, academic secretaries, department heads, technology coordinators, and a rector participated.

4 Results Globally, we found statistically significant differences from one country to another, placing the digital competence level of teachers at B1, integrator level, in Chile and Mexico with more than half of the teachers included in this study compared to the

A Post-Covid Comparative Analysis of Digital Skills in Vocational …

7

B2 level, expert, detected in Colombia or Peru, although it should be noted that with a lower percentage level (43% for both countries) (Table 2). At the global level, not differentiating between men and women, the level of digital competence is clearly at the B1 level, with 53% of respondents: saying they are “integrative” teachers who use digital technologies creatively, try them out in different contexts and are keen to broaden their application and understand which tools are more appropriate to different pedagogical strategies and methods. At the bottom levels, up to B1, women outnumber men in percentage, but from B2 onwards, men outnumber women, so that in the final balance men and women are equal in terms of teaching digital competence (Table 3). In Chile, 27% of teachers have acquired sufficient confidence in using technologies; they know how to identify the most appropriate ones according to their use. They have thus acquired the B2 level of expertise. They are also eager to broaden Table 2 Global results Chile (%) Competences

Colombia (%)

Mexico (%)

Peru (%)

A1

0

0

2

0

A2

12

11

11

19

B1

53

31

51

38

B2

27

43

32

43

C1

8

15

4

0

Table 3 Results for Chile Area 1 (%) Global

Mujeres

Hombres

A1

0

Area 2 (%) 2

Area 3 (%) 0

Area 4 (%) 10

Area 5 (%) 2

Area 6 (%) 4

Overall (%) 0

A2

22

14

18

16

24

16

12

B1

45

45

53

31

45

47

53

B2

12

16

12

16

6

12

27

C1

20

22

16

27

22

20

8

A1

0

6

0

22

6

11

0

A2

22

6

22

28

39

22

17

B1

56

61

72

22

39

56

61

B2

11

22

0

17

11

6

22

C1

11

6

6

11

6

6

0

A1

0

0

0

3

0

0

0

A2

23

19

16

10

16

13

10

B1

39

35

42

35

48

42

48

B2

13

13

19

16

3

16

29

C1

26

32

23

35

32

29

13

8

M. Martín-Llaguno et al.

their knowledge and discover new technological applications and tools. These experts progress to C1 (8% of the sample). At this level, the teacher has a consistent and comprehensive approach to digital technologies; they know many digital strategies and how to choose the most appropriate one in each context. Teachers with less technological skill go directly to A2, with 12% of the sample, while A1 is empty. These teachers need a boost and guidance to acquire more extraordinary skills and overcome the level. Regarding the competency levels by gender, in the explorer and integrator levels, A2 and B1, women lead significantly with up to 13 points of difference in the B1 level. In contrast, at the expert level, B2, men slightly outnumber women, with a more significant margin at the leader level, C1, where women are absent. Therefore, a digital gender gap is detected at the most advanced levels of technological use. In addition, women stand out in area 3, with 72% in competency level B1. This area is related to the teaching and learning process, with items such as programming and implementing digital devices and resources; guiding and supporting learning; using technologies to promote and improve student collaboration; and encouraging self-regulated learning. A higher and more homogeneous level of competence is also reached in area 2, identifying a low percentage of users at the A1 and A2 levels, reaching 61% at the B1 level. Therefore, there is an acceptable level of women who know how to select, create and modify digital resources and organize, protect, manage, and exchange digital content. For men, the level of competence by area is homogeneous, with no area having a higher level of competence than another. The weakest area for men is area 4, with 22% of men at the A1 level and 28% at the A2 level. In the case of Colombia, the joint processing of data, men and women, a high competence level can be deduced; 43% of the sample is at the B2 level, experts: they “use various digital technologies confidently, creatively and critically to improve their professional activities.” They know how to apply the appropriate tool to each situation and are open to continuous learning through experiences. In certain areas, the level reached by almost half of the sample is high, level C1 leader; therefore, in addition to having a vast repertoire of technologies, they also reflect on the practice they make of these tools and continue to develop them (Table 4). Analyzing the competency level by gender, in the lower levels, novice (A1) and explorer (A2), there are more men than women, but not in the integrator (B1), expert (B2) and leader (C1) levels, where women perform better. This comparison affirms that women in Colombia have a higher level of competence than men. The most significant gap is at the B2 level, where women outperform men by 29 points. In the breakdown by areas, both men and women, homogeneity prevails, with no area standing out over another. In area 3, there are more women with a high level of competence than men, a difference of 10 points in the area that covers the teaching and learning process, which implies that there are more women than men who program and implement digital devices and resources in the teaching process, which adequately manage and coordinate general didactic interventions and who experiment with new formats and

A Post-Covid Comparative Analysis of Digital Skills in Vocational …

9

Table 4 Results for Colombia Area 1 (%) Global

Mujeres

Hombres

Area 2 (%)

Area 3 (%)

Area 4 (%)

Area 5 (%)

Area 6 (%)

Overall (%)

A1

0

0

0

6

2

2

0

A2

2

15

6

7

15

15

11

B1

31

19

39

22

35

31

31

B2

31

26

19

15

19

19

43

C1

35

41

37

50

30

33

15

A1

0

0

0

4

0

4

0

A2

0

12

4

12

15

12

8

B1

27

15

35

27

31

19

23

B2

38

31

19

12

15

27

58 12

C1

35

42

42

46

38

38

A1

0

0

0

7

4

0

0

A2

4

18

7

4

14

18

14

B1

36

21

43

18

39

43

39

B2

25

21

18

18

21

11

29

C1

36

39

32

54

21

29

18

pedagogical teaching methods. There are also more women than men who use digital technologies to enhance individual and collective learner interaction and provide support and guidance. In area 2, related to digital content, there is also an increase in women versus men. More women locate, evaluate, and select digital resources for teaching and learning, considering the learning objective, context, and pedagogical approach. More women are modifying and adapting openly licensed resources and creating new resources from existing ones. In the area, men outnumber women regarding high competency levels (C1). Area 4 focuses on evaluation and feedback, with evaluation strategies and learning analytics or feedback, programming, and decision-making. There are, therefore, more men (8 points difference) who use digital technologies for formative and summative assessment. More men generate, select, analyze, and critically interpret digital statistics on learner activity, performance, and progress. Also, more males than females are using digital technologies to provide targeted and timely feedback or empower learners to use these technologies as decision aids. In Mexico, the primary and more advanced levels go unnoticed, with 2% and 11% for the basic, A1, and A2, respectively, and 4% for the advanced, C1. The most generalized level is the integrating teacher, B1, corresponding to 51% of the sample, followed by B2 with 32%. The analysis by areas of these percentages leads us to consider how there are teachers who, in certain areas, reach a high level of competence, even C1, but in other areas, they have a deficit; hence in the total computation of competence, level B1 is the most widespread. When analyzing the level of competence by gender, the

10

M. Martín-Llaguno et al.

difference is insignificant, and the level of competence is balanced. 13% of women have an A2 level of competence, compared to 54% at the B1 level or 28% at B2 and even 5% at C1 (Table 5). By areas, 49% of women have a B2 or C1 level in area 4: they are experts or leaders in assessment strategies and learning analytics while using digital technologies to provide targeted and timely feedback to learners to assist them in decision-making. There are also a considerable number of women with competency levels for “learner empowerment.” These women ensure accessibility and inclusion of all learners; they use digital technologies to cater to diversity and engage learners in their learning. In the computation by gender, the basic A1 level is present in 4% of the sample of men; the same occurs with the advanced C1 level pioneers. The bulk of the sample concentrates in level B1, integrator, with 48% of the sample, and in level C1, leader, with 35% of the sample. The analysis shows that in all areas, a considerable number of men with high competence levels, B2 and C1. However, the number of men reaching the C1 level is insignificant globally. This contrast is due to the disparity of knowledge and skills that the same teacher possesses between areas. The same teacher may have novice knowledge in each area, and expert or leading knowledge in another; this results in the teacher being placed at a medium level of competence, despite being a pioneer in some areas. Finally, in identifying digital competencies in Peru at the global level, considering both men and women, it is worth highlighting the absence of values at the extremes, both at the lower level, A1, and at the higher level, C1. At the basic level A2, there Table 5 Results for Mexico Area 1 (%) Global

Females

Males

A1

3

Area 2 (%) 1

Area 3 (%) 0

Area 4 (%) 2

Area 5 (%) 3

Area 6 (%) 3

Overall (%) 2

A2

16

19

26

11

24

17

11

B1

51

37

38

40

30

46

51

B2

18

12

14

16

13

17

32

C1

12

31

23

31

30

16

4

A1

3

0

0

0

0

3

0

A2

18

23

28

13

28

15

13

B1

51

41

41

38

36

56

54

B2

18

8

8

23

10

13

28

C1

10

28

28

26

26

13

5

A1

4

2

0

4

6

4

4

A2

15

17

24

9

20

19

9

B1

50

33

35

41

26

39

48

B2

19

15

15

11

15

20

35

C1

13

33

26

35

33

19

4

A Post-Covid Comparative Analysis of Digital Skills in Vocational …

11

is a considerable volume of teachers, 19%, while the bulk is located at level B1 with 38%, surpassed by the expert level B2 which reaches 43%. In level C1, teachers exceed 30% of the sample in areas 3, 4, and 5, while, when considering all areas, there are no teachers identified with level C1. The reason is that teachers with a C1 level in a given area are at an A2 or B1 level in another area, which implies that globally they have deficiencies and cannot be awarded the high level (Table 6). Regarding gender distribution at the advanced expert level, B2, women and men are practically in the same position, but not at the integrator level, B1, where women outnumber men, the reverse being valid at the basic explorer level, A2. In the analysis by areas and gender, considering women, the B1 competence level stands out with 48% of the sample, followed very closely by the B2 level with 43%. Ninety-one percent of women have a medium–high level of digital competence. They try digital technologies in various contexts and for different purposes; they strive to advance in improving their use of digital technologies. The most skilled gain confidence and use them creatively and critically. The percentages of women in each area are similar. Considering men, level B2 is the most significant, with 44% of the sample. Level B1 is also essential, with 33%, and even A2, with 23%. In the breakdown by area, the high number of men whose level is A2 contrasts with those who have C1. As has been pointed out, this gap results in a considerable percentage of men at intermediate levels and none at high levels. We can see that literacy, digital culture and the incorporation of digital competencies into the educational offer and the promotion of digital culture in institutions is an Table 6 Results for Peru Area 1 (%)

Peru Global

Females

Males

A1

0

Area 2 (%) 0

Area 3 (%) 0

Area 4 (%) 0

Area 5 (%) 0

Area 6 (%) 0

Overall (%) 0

A2

23

28

26

33

32

25

19

B1

39

38

29

17

20

55

38

B2

25

28

14

10

16

14

43

C1

13

7

30

39

32

6

0

A1

0

0

0

0

0

0

0

A2

29

24

19

29

24

29

24

B1

29

52

33

29

29

38

48

B2

29

19

19

14

19

24

43

C1

14

5

29

38

29

10

0

A1

0

0

0

0

0

0

0

A2

21

29

29

35

35

23

23

B1

44

31

27

17

17

63

33

B2

23

31

13

8

15

10

44

C1

13

8

31

40

33

4

0

12

M. Martín-Llaguno et al.

ongoing process in the four countries of the Pacific Alliance, but that it is advancing slowly, while at the same time it points out a greater effort in digital training in secondary education than in higher education. Teachers in the four countries assume training in digital transformation as part of their professional practice, although it is from the administration or management of the centers where they are assigned. Other results warn that collaboration and exchange of good practices are still unresolved issues in practically all countries; likewise, the most repeated slogan indicates that it is not a matter of having knowledge of digitization and digital tools, but of knowing how to apply it to teaching practice. Finally, it is highlighted the need to promote partnerships with the private sector to obtain advantageous conditions for equipment, software, and connectivity, as well as to acquire other devices in addition to computers and tablets in educational centers.

5 Conclusions and Recommendations The descriptive study has made it possible to identify the level of digital competence in a significant sample of teachers who teach Technical Professional Education in the countries of the Pacific Alliance, specifically Chile, Colombia, Mexico, and Peru—two hundred and sixty-five teachers from 17 centers. Our results show that the level of competence of the sample analyzed is practically between levels B1 and B2, corresponding to the profile of integrator and expert of the DigComEdu digital competence framework. The entry competency level A1 is exceeded in all four countries, being present only in Mexico with only 2% of the sample. The A2 competency level, the one that fits explorer teachers, can be considered to be exceeded, having minimal relevance in the selected sample; Peru is the country where most teachers are at this level, with 19% of the sample, compared to 12% in Chile or 11% in Colombia and Mexico. Concerning levels B1, integrator, and B2, expert, these are the competency levels that most teachers have, exceeding 80% of the sample in all countries in the sum of both levels. Thus, more than half of the sample in Chile and Mexico is at level B1, 53% and 51%, respectively, with high percentages, although lower, for Colombia (31%) and Peru (38%). At the B2 competency level, Colombia and Peru, with 43% of the sample, are above Chile and Mexico, where this level is representative of 27% and 32%, respectively. On the other hand, there are also teachers with competence level C1, leader, with Colombia standing out with 15% of the sample compared to 8% in Chile or 4% in Mexico and the absence in Peru. Teachers with competence level C2, pioneers, are practically nonexistent in our sample. There are no significant differences in the level of teachers’ digital competence by gender; in some countries, women are slightly above men at higher levels, and in others, the opposite is true. Competence levels B1 and B2, where the bulk of the sample is located, Chile, Mexico, and Peru follow the same pattern, with more women than men in the three countries at level B1. However, the figures are reversed at level B2, which allows us to

A Post-Covid Comparative Analysis of Digital Skills in Vocational …

13

affirm that the number of women and men is balanced at the intermediate, integrative, and expert levels. In Chile, there is a 13-point difference in favor of women at the B1 level, 6 points in Mexico, and 15 in Peru, compared to a 7-point difference in favor of men at the B2 level, 7 points in Mexico, and only 1 point difference in Peru. In Colombia, at the B1 level, men outnumbered women by 16 points. In contrast, at the B2 level, it was considerably reversed, and the number of women practically doubled the number of men, with 58% of women compared to 29% of men. The Organization of Ibero-American States for Education (OEI) has launched a training program in digital skills for teachers in the technical-professional modality in the Pacific Alliance countries. The program, which is based in our findings, aims to promote the skills, abilities, and digital competencies of participating teachers in their teaching practice. The program is based on modules on basic computer science notions, the use of Information and Communication Technologies (ICT) in the classroom, the use of digital platforms, and the creation of educational resources. Participating teachers will also develop pedagogical techniques related to new technologies. It is expected to improve the digital skills of teachers in the technical-professional modality, which will enable them to better prepare their students for the twenty-first century workforce. The program will also help to promote the use of ICT in education in the Pacific Alliance countries and is being implemented in collaboration with the Ministry of Education of each participating country.

References Ballestar, M., Sainz, J., & Sanz, I. (2022). An economic evaluation of educational interventions in the LOMLOE: Proposals for improvement with Artificial Intelligence. Revista Espanola De Pedagogia, 80(281), 133–154. Capilla, A., Sainz, J., & Sanz, I. (2021). COVID 19: Efectos en la educación un año después. Organización de Estados Iberoamericanos. Cattaneo, A. A. P., Antonietti, C., & Rauseo, M. (2022). How digitalized are vocational teachers? Assessing digital competence in vocational education and looking at its underlying factors. Computers and Education, 176(March 2021), 104358. https://doi.org/10.1016/j.compedu.2021. 104358 Çebi, A., & Reiso˘glu, ˙I. (2022a). Adaptation of self-assessment instrument for educators’ digital competence into Turkish culture: A study on reliability and validity. Technology, Knowledge, and Learning. https://doi.org/10.1007/S10758-021-09589-0 Çebi, A., & Reiso˘glu, ˙I. (2022b). Defining “digitally competent teacher”: Examining pre-service teachers’ metaphor. Journal of Digital Learning in Teacher Education, 38(4), 185–198. https:// doi.org/10.1080/21532974.2022.2098210 Galindo-Domínguez, H., & Bezanilla, M. J. (2021). Digital competence in the training of preservice teachers: Perceptions of students in the degrees of early childhood education and primary education. Journal of Digital Learning in Teacher Education, 37(4), 262–278. https://doi.org/ 10.1080/21532974.2021.1934757 Gegenfurtner, A., Schmidt-Hertha, B., & Lewis, P. (2020). Digital technologies in training and adult education. International Journal of Training and Development, 24(1), 1–4.

14

M. Martín-Llaguno et al.

Guillén-Gámez, F. D., Ruiz-Palmero, J., Sánchez-Rivas, E., & Colomo-Magaña, E. (2020). ICT resources for research: An ANOVA analysis on the digital research skills of higher education teachers comparing the areas of knowledge within each gender. Education and Information Technologies, 25(5), 4575–4589. https://doi.org/10.1007/s10639-020-10176-6 Hernández, S. M. B. (2018). Marco común de competencia digital docente. Revista Iberoamericana De Educación a Distancia, 21(1), 369–370. INTEF. (2017). Marco Común de Competencia Digital Docente No Title. MECD. http://educalab.es/ documents/10180/12809/Marco+competencia+digital+docente+2017/afb07987-1ad6-4b2dbdc8-58e9faeeccea Johannesen, M., Øgrim, L., & Giæver, T. H. (2014). Notion in motion: Teachers’ digital competence. Nordic Journal of Digital Literacy, 2014(4), 300–312. https://doi.org/10.18261/ISSN1891943X-2014-04-05 Kuhfeld, M., Soland, J., Lewis, K., Ruzek, E., & Johnson, A. (2022). The COVID-19 school year: Learning and recovery across 2020–2021. Aera Open, 8, 23328584221099304. Lindfors, M., Pettersson, F., & Olofsson, A. D. (2021). Conditions for professional digital competence: The teacher educators’ view. Education Inquiry, 12(4), 390–409. https://doi.org/10.1080/ 20004508.2021.1890936 Lockee, B. B. (2021). Online education in the post-COVID era. Nature Electronics, 4(1), 5–6. https://doi.org/10.1038/s41928-020-00534-0 OECD. (2021). The state of school education (Issue March). https://www.oecd-ilibrary.org/content/ publication/201dde84-en Rauseo, M., Harder, A., Glassey-Previdoli, D., Cattaneo, A., Schumann, S., & Imboden, S. (2022). Same, but different? Digital transformation in Swiss vocational schools from the perspectives of school management and teachers. Technology, Knowledge and Learning. https://doi.org/10. 1007/s10758-022-09631-9 Redecker, C., & Punie, Y. (2017). European framework for the digital competence of educators— DigCompEdu. In Y. Punie (Ed.). European Commission, Joint Research Centre, Publications Office. https://data.europa.eu/doi/10.2760/159770 Røkenes, F. M., & Krumsvik, R. J. (2014). Development of student teachers’ digital competence in teacher education—A literature review. Nordic Journal of Digital Literacy, 9(4), 250–280. https://doi.org/10.18261/ISSN1891-943X-2014-04-03 Sainz, J., & Sandoval-Hernández, A. (2020). Nuevos datos, nuevos retos: Iberoamérica en las últimas evaluaciones educativas. Revista Iberoamericana de Educación, 84(1 SE-Editorial). https://doi. org/10.35362/rie8414121 Sanz, I., Sáinz, J., & Capilla, A. (2020). Effects of the coronavirus crisis on education. Organization of Ibero-American States for Education. Science and Culture (OEI).

The Impact of the COVID-19 Pandemic on Education Learning Ismael Sanz and J. D. Tena

1 Introduction The COVID-19 pandemic has caused a global disruption in everyday life, including education, since March 2020. The closure of schools due to COVID-19 has resulted in a break in education provision that has produced long-lasting learning losses (Patrinos et al., 2022). Furthermore, school closures may increase educational inequality. Although online education can substitute for in-person learning, it is an imperfect replacement. Some students experienced more significant learning losses than others due to a lack of remote training for students, unequal access to broadband internet service in different households, and the general difficulty for working parents to support their children with homework due to their jobs or possible low cognitive ability. Schools enable children from various socio-economic backgrounds to interact, and these peer effects disappear when schools are closed (Goldhaber et al., 2022). In this chapter, we examine the literature on the impact of COVID-19 on student achievement. Section 2 discusses prior studies at the beginning of the pandemic, in the spring of 2020. These papers were based on forecasts and simulations using previous data collected from school breaks due to educational reforms (Pischke, 2007), weather (Goodman, 2014), teacher strikes (Jaume & Willén, 2019), or international comparisons of hours of instruction (Woessmann, 2003). Section 3 reviews the early analysis of the learning impact of lockdowns and school closures using actual data. Section 4 summarizes the recent evidence of the impact of COVID-19 on student achievement following the survey by Patrinos et al. (2022). Section 5 focuses on the I. Sanz (B) Universidad Rey Juan Carlos, Madrid, Spain e-mail: [email protected] J. D. Tena University of Liverpool, Liverpool, England Università Degli Studi di Sassari, Sassari, Italy © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_2

15

16

I. Sanz and J. D. Tena

effects of school lockdowns on education inequality, whereas Section 6 provides the first meta-analysis, as far as we know, of the effect of the pandemic on school closures. Finally, Section 7 concludes with the education measures that governments can implement to reduce the learning loss caused by the pandemic.

2 The Impact of the COVID-19 Pandemic on Student Learning Burgess and Sievertsen (2020) reported on April 1st, 2020, at the beginning of the Covid-19 pandemic, that the closure of schools in the last term of the 2019–2020 school year would negatively affect student learning by 6% of the standard deviation (equivalent to what is learned in 2 months) and reduce future salaries of current students by 1%. Their conclusions were based on previous studies analyzing the effect of time instruction, or class interruptions, on student achievement. Carlsson et al. (2015) analyzed the results of young people in Sweden when taking cognitive tests for military service as a function of the number of days of preparation. These authors found a significant effect of studying days on performance in crystallized intelligence tests (synonymous with verbal ability and comprehension tests for technical problems) and a negligible impact on fluid intelligence tests (spatial and logical). Lavy (2015) analyzed data from the Organisation for Economic Cooperation and Development (OECD) on the Programme for International Student Assessment (PISA). Following an analysis of the results of 400,000 students from 50 countries, Lavy (2015) found that one more hour per week throughout the school year in core subjects (the same amount of school time as in the last term of the 2019/ 2020 school year) increases exam performance by around 6% of the standard deviation. The economic uncertainty of families was another potential source of unequal opportunities. A student’s likelihood of graduating from school is significantly lower when parents or guardians have temporary employment contracts (Ruiz-Valenzuela, 2020). Similarly, when unemployment rates increase, students’ performance in mathematics may decrease by 7.6% of the standard deviation in math scores (Ananat et al., 2011). The loss of learning experienced by students from disadvantaged backgrounds may result in increased repetition rates and worse graduation rates in the coming years. Pischke (2007) analyzed the effects of changes in the school calendar in some states of Federal Germany during the 1966–67 and 1967–68 academic years, which resulted in a loss of two-thirds of instruction during those years. This led to an increase in repetition rates by 0.9 to 1.1 percentage points, a considerable impact given that prior to the reform, they ranged between 2 and 5%. The reduction in classes also reduced the percentage of students who selected pathways to access higher education, although Pischke did not find evidence that this reduction had a long-term effect on wages or employment.

The Impact of the COVID-19 Pandemic on Education Learning

17

RAND (2014) evaluated the effects of a 5-week summer program on the performance of third-grade students beginning in the spring of 2013 and enrolled in a public school in one of five urban districts: Boston, Dallas, Duval County (Florida), Pittsburgh, or Rochester (NY). This program had an effect equivalent to 20% of the yearly progress in mathematics, with the benefits being particularly large for poor, minority, and lagging students. Schueler (2020) studied a summer program providing math instruction to struggling sixth and seventh graders in small groups of 10 students during the 2016 holidays in low-performing schools in Massachusetts. The math program increased the probability of students passing Common Core-aligned tests scoring proficient or higher by 10 percentage points, and reduced disciplinary suspensions after the summer. Schueler (2020) also found suggestive evidence of positive spillover effects on English Language Arts achievement and end-of-course grades in math and reading. The impact of the COVID pandemic will not be the same across ages and education levels as it has been for previous health crises. Meyers and Thomasson (2017) analyzed the effects in US states of the Polio Pandemic in 1916 and found that an increase of one standard deviation of more cases per 10,000 inhabitants resulted in approximately 0.07 fewer years of schooling for children between 14 and 17 years old, relative to the reference cohort of people aged 19–21. Children who were already 17 to 18 years old, or who were 10 to 14 years old, were not as affected. Jaume and Willén (2019) examined another type of class interruption and showed negative longterm effects of teacher strikes in Argentina. The reduction in the educational level of the students who experienced school closures increased their probability of being unemployed in their future and reduced the levels of qualification of the occupations in which they were employed with respect to other generations that did not experience those closures. Jaume and Willén (2019) concluded that 88 days without classes for primary education students in Argentina resulted in a decrease of 2.99% of wages when they reached an age between 30 and 40 years. In contrast, there are other factors that may have an opposite effect. According to the empirical evidence presented in two papers (Harris & Larsen, 2018; Maurin & McNally, 2008), the negative short-term impact of Covid-19 on student achievement can be reversed in the medium term. The expected increase in unemployment may reduce the opportunity cost of continuing education, making it more attractive than immediate job alternatives. Therefore, many young people may choose to continue their studies in the medium term despite the initial negative effect. Harris and Larsen (2018) provide evidence that after Hurricane Katrina hit New Orleans in August 2005, there was an initial drop in student learning. However, educational indicators improved significantly in 2014, with notable improvements in both academic achievement and graduation rates in post-compulsory secondary school and above. Similarly, Maurin and McNally (2008) analysed the “natural experiment” of the effect of the May 1968 mobilizations in France on the future of the student cohort preparing for the baccalauréat exam. Due to the reduced demand for the exam that year, the pass rate increased significantly, allowing a larger proportion of students born in 1948 and 1949 to achieve more years of schooling. The marginal students who entered the University in that year, who may not have achieved higher education

18

I. Sanz and J. D. Tena

in other circumstances, largely completed their studies and achieved better salaries and positions of responsibility in their professional careers than if they had not passed the baccalauréat. This progress in education for the young people of these cohorts has also been passed down to their children. Although these studies demonstrate the flexibility of access to education in a single year, it remains unclear whether a reduction in the levels of permanent demand over time, as in the case of COVID-19, could have negative effects on student dedication and effort. Additionally, Goodman (2014) shows that the academic effects of school closures can be mitigated if there is a coordinated response and it is not prolonged over time. However, if some students are absent due to bad weather while others are not, the resulting disruptions to instructional time may reduce math achievement by 0.05 standard deviations per absence. Teachers seem to handle coordinated disruptions of instructional time, like snow days, well but struggle with disruptions that affect different students at different times (Table 1).

3 Impact Evaluations of Learning Loss with Actual Data The predictions presented thus far are based on previously collected data on school breaks or international comparisons of hours of instruction that placed the impact of the Coronavirus at a learning loss of 6% of the standard deviation. In the summer of 2020, when it became clear that the impact of the pandemic on education provision would last longer than initially expected, new analyses increased the forecast of the learning and economic losses produced by school lockdowns. Maldonado and De Witte (2020, 2022) analyzed the tests carried out by 6th-grade students in the period 2015–2020 in the network of Catholic Flemish schools in Belgium, including the tests conducted in June 2020, after the emergence of Covid-19. Their analysis revealed that students in the 2020 cohort had experienced significant learning loss in all subjects tested, with a loss of 18.6% of the standard deviation in Mathematics and 28.6% in Language. The impact of 18.6% in Mathematics is more than half of what is learned in an academic year and is higher than the estimated impact of 6% of the standard deviation from simulations. Only the World Bank and Doepke and Zilibotti (2020, Psychology Today) had suggested an impact of COVID-19 on student learning as high as half a school year. Doepke and Zilibotti (2020) also noted that the loss of the last quarter of the 2019–20 academic year should be added to the usual loss of skills that students suffer during the summer, which could have a negative impact on learning up to half of what was learned during the course. Grewenig et al. ( HYPERLINK "SPS:refid::bib45|bib46" ) used a survey of parents of school-age children to demonstrate that in Germany, the time children spent on school activities during the lockdown decreased from 7.4 to 3.6 h, while the number of hours devoted to television, video games, and mobile phones increased. Only 6% of German students had daily online group lessons, and more than half had these lessons less than once a week. Hanushek and Woessmann (2020) prepared a report for the OECD in September 2020, in which they estimated that given that in most European

The Impact of the COVID-19 Pandemic on Education Learning

19

Table 1 Papers used for the analysis of possible learning loss at the beginning of the Covid-19 pandemic Study

Sample

Conclusions

Woessmann (2003)

The TIMSS 1994–95 (Trends in International Mathematics and Science Study) conducted tests in math and science in 39 countries, including the United States, Japan, Germany, Spain and many others

The data from the TIMSS 1994–95 suggest that reducing 10% of instruction (equivalent to 17.5 days of class in the case of Spain) results in a 1.5% decrease in student learning

Pischke (2007)

He analyzed the effects of the change in the school calendar in some states of West Germany during the academic years 1966–67 and 1967–68. The aim was to align the calendar with the rest of European countries, but this meant that two-thirds of instruction was lost during those two years

The study found that repetition rates increased between 0.9 and 1.1 percentage points, which is a considerable impact since before the reform they ranged between 2 and 5 points. The study also found that the reduction in classes resulted in a decrease in the percentage of students who selected paths leading to higher education, even though there were no effects on wages and employment

Maurin and Mcnally In 1968, the baccalaureate—an (2008) important exam whose success guarantees access to university—only involved oral tests. Researchers found that the lowering of thresholds at an early and highly selective stage of the higher education system allowed a significant number of people born between 1947 and 1950, particularly in 1948 and 1949, to pursue more years of higher education than they would have been able to otherwise

Students who accessed higher education in France in 1968 experienced a significant increase in their subsequent wages and occupational attainment, particularly those who came from a middle-class family background. For those who were on the verge of passing their examinations, additional years of higher education increased their future wages and occupational levels. The authors of the study found that each additional year spent in higher education increases wages by about 14%

Ananat et al. (2011) Effects of job losses on student achievement at the state level in the United States between 1996 and 2009

Layoffs of more than 1% of the working-age population reduce student performance in mathematics by 7.6% of the standard deviation (continued)

20

I. Sanz and J. D. Tena

Table 1 (continued) Study

Sample

Conclusions

Goodman (2014)

He analyses student-level data on attendance and achievement in the 2003–2010 school years from the Massachusetts Department of Elementary and Secondary Education (DESE) and combined it with district-level data on the number of school closures reported between 2003 and 2010. Additionally, we collected weather data from the National Oceanic and Atmospheric Administration’s (NOAA) Climate Data Online to investigate the effects of weather-related closures on student outcomes

Each absence induced by bad weather reduces math achievement by 0.05 standard deviations, suggesting that attendance can account for up to one-fourth of the achievement gap by income

RAND (2014)

The study evaluates the effects of summer programs on the academic performance of third-grade students who began attending public schools in five urban districts (Boston, Dallas, Duval County in Florida, Pittsburgh, and Rochester) in the spring of 2013

The 5-week summer training program had a significant impact on the mathematics performance of students in third grade beginning in the spring of 2013 and enrolled in a public school in five urban districts: Boston, Dallas, Duval County (Florida), Pittsburgh, or Rochester (New York). Specifically, the program resulted in an increase in performance equivalent to 20% of the yearly progress in mathematics, with the greatest benefits observed among poor, minority, and lagging students

Carlsson et al. (2015)

The researchers exploit conditionally random variation in the date when Swedish men take cognitive tests in preparation for military service. They use administrative record data obtained from the Swedish National Service Administration, which includes information on every individual who enlisted in the military between 1980 and 1994

An additional 10 days of school instruction increases about 1% of a standard deviation in scores on tests of crystallized intelligence (synonyms and technical comprehension tests). Fluid intelligence test scores (spatial and logical tests) do not increase with additional days of schooling

(continued)

The Impact of the COVID-19 Pandemic on Education Learning

21

Table 1 (continued) Study

Sample

Conclusions

Lavy (2015)

The study used data from the PISA 2006 assessment, which included over 400,000 students from 57 countries

“Research shows that instructional time has a positive and significant effect on academic achievement. Moreover, evidence suggests that the productivity of instructional time is higher in countries that have implemented accountability measures or granted autonomy to schools. Specifically, an extra hour per week dedicated to core subjects during the school year (equivalent to the last term of the course) can increase test scores by approximately 6% of a standard deviation”

Meyers and Thomasson (2017)

The researchers analyzed the effect of the closure of educational centers on the educational level during the 1916 polio pandemic in the United States, which occurred at the beginning of the school year

The study finds that the 1916 polio pandemic in the United States, which led to the closure of educational centers at the beginning of the school year, had a significant impact on the educational level of children. Specifically, an increase of one standard deviation in the number of polio cases per 10,000 population resulted in a reduction of approximately 0.07 years of schooling for children aged 14–17, compared to the reference cohort of persons aged 19–21. The effect was less pronounced for children who were already 17–18 years old or who were 10–14 years old at the time of the closure

Harris and Larsen (2018)

Longitudinally linked data at the student level for publicly funded schools in Louisiana for the years 2001–2014, before and after Hurricane Katrina

Based on longitudinally linked data at the student level for publicly funded schools in Louisiana for the years 2001–2014, reforms implemented after Hurricane Katrina resulted in a more than 3 percentage point increase in high school and college graduation rates. Additionally, the reforms helped to narrow the educational gap between advantaged and disadvantaged groups within the district (continued)

22

I. Sanz and J. D. Tena

Table 1 (continued) Study

Sample

Conclusions

Jaume and Willén (2019)

The researchers use data on exposure to teacher strikes during primary school for children born between 1971 and 1985 in Argentina. They analyze the impact of teacher strikes on educational and labor market outcomes using data from household surveys conducted between 2003 and 2015. The surveys cover 91% of the urban population in Argentina and include information on outcomes such as years of schooling, educational attainment, and employment status

The study found that exposure to primary school closures due to teacher strikes in Argentina led to a reduction in future earnings for both men and women, with a reduction of up to 3.2% for men and 1.9% for women. Additionally, the study found that the educational level of students who experienced school closures was reduced, leading to lower skill levels in their future occupations compared to other generations who did not experience such closures

Ruiz-Valenzuela (2020)

The data are from the fourth quarter of the Labour Force Survey in Spain between 2000 and 2004

Students whose parents have a permanent rather than a temporary contract are 7.8 percentage points more likely to graduate on time. They are also 2.5 percentage points less likely to drop out of school at sixteen

Schueler (2020)

They analyzed a mathematics program taught to groups of 10 students during the holidays of 2016 in a group of low-performing schools in Massachusetts

The math program increased the probability of students passing Common Core-aligned tests by 10 percentage points and reduced disciplinary suspensions after the summer

countries, the closure of schools lasted up to a third of the course, the negative effect would be 11% of the standard deviation, based on the fact that progress in a school year is about 33% of the standard deviation. Based on the relationship between the level of skills and wages extracted from PIAAC (an OECD evaluation similar to PISA but applied to the population between 15 and 65 years of age), Hanushek and Woessmann (2020) further concluded that current students would lose 2.6% of earnings throughout their professional career due to the learning loss. From an aggregate macroeconomic perspective, a less well-qualified workforce will lead to lower economic growth, and these authors estimate that the skill shortfalls could lead to a 3.8% reduction in the level of gross domestic product (GDP) by 2100. Fuchs-Schündeln et al. (2020) used a heterogeneous agent partial equilibrium model based on the human capital models of Cunha et al. (2010), including public expenditure on education and the financial and time investment by parents in their children, calibrated with data taken from U.S. students aged 4 to 14 years. The cumulative nature of the human capital production function means that the achievement

The Impact of the COVID-19 Pandemic on Education Learning

23

of skills at an earlier stage of the life cycle increases skills at later stages. Therefore, younger children are likely to be more adversely affected by school closures compared to older children. However, it should be noted that at critical stages, such as at the end of compulsory secondary education or post-secondary education, the impact on older children can also be significant and may determine the possibility of continuing their studies or the type of training they can pursue. The negative impact of school closures can be mitigated to some extent by parents investing more time and resources in their children’s education. Calibrating their model for the US, FuchsSchündeln et al. (2020), found that there would be a 3.8% reduction in the high school graduation rate among children between the ages of 4 and 14, and a decrease of 2.7% in the percentage of young people with higher education. The negative impact of the Covid-19 crisis on the well-being of children is particularly severe for those with low-educated and low-active parents. School closures themselves are primarily responsible for the negative impact of the pandemic on children’s long-term well-being, explaining 87% of the impact, while pandemic-induced income declines play a secondary role. This is because every year of schooling raises earnings by 8 to 10% a year, according to Psacharopoulos et al. (2021) (Table 2).

4 Recent Analysis of the Impact of the Pandemic in Student Achievement In 2020–2021, real-world data on the impact of school closures during the pandemic began to emerge. Patrinos et al. (2022) conducted a comprehensive analysis of the recorded learning loss in countries where students experienced school closures. They surveyed studies examining the impact of Covid-19 from the beginning of the school closures in March 2020 to March 2022. Most studies found significant learning losses for students around the world, but there were also outliers, countries that managed to limit the amount of loss. Patrinos et al. (2022) focused on learning loss among school-age children between ages 5 and 18 across all subjects, with particular attention to math and reading. The average result of 35 studies analyzed by Patrinos et al. (2022) is that primary and secondary students lost learning equivalent to 17% of a standard deviation due to the pandemic, which is between one third and half of an academic year. Most studies found learning losses ranging from 12 to 25% of the standard deviation. These findings confirm that learning loss was higher than the initially estimated 6% when the Covid pandemic was expected to affect only the last trimester of the 2019–20 academic year and fade out during 2020–21. Table 3, extracted from Patrinos et al. (2022), shows that there is some correlation between the duration of school closures and the impact on learning. Not all studies found a negative effect of school lockdown. Sweden opened pre-schools, primary schools, and lower secondary schools very early in the pandemic, but student and teacher absence and pandemic-related stress could have still affected

24

I. Sanz and J. D. Tena

Table 2 Summary of the first papers analysing the impact of the school lockdown Study

Sample

Conclusions

Bacher-Hicks et al. (2021)

The intensity of the use of educational online resources can be assessed through Google Trends. Approximately 55 million students up to 12 years old in the US

Higher growth in use of online education resources in counties with higher incomes and located in urban areas

Psacharopoulos et al. (2020) 1.500 billion students in 192 countries. School years, income based on United Nations

The impact level of a 4-month school lockdown on future GDP is estimated to be a reduction of 15%

Burgess and Sievertsen (2020)

50 countries. 400.000 students in the PISA data base

Learning loss could be recovered with small group tutoring that provides an additional hour per day for 12 weeks

Hanushek and Woessmann (2020)

Students from grades 1–12. Loss of a 3.9% future earnings Learning and economic growth in the OECD

Fuchs-Schündeln et al. (2020)

Students from 4 to 14 years in US. Enrolment rate by student levels

The COVID-19 pandemic has resulted in a 2.7% decrease in higher education enrollment

Woessmann et al. (2020)

German schools. School achievement and the use of social digital media

Students coming from poor households are the most negatively affected

Chetty et al. (2020)

US schools. The use of the math platform Zearn

There has been a 60% reduction in the progress of academic achievement for students in the bottom income quartile

Maldonado and De Witte (2020)

Flemmish Catholic schools (Belgium). External and Standardized test

The school lockdown resulted in a loss of academic learning ranging from 19 to 29%, according to various studies

academic achievement negatively. However, Hallin et al. (2022) found no COVID19 related learning loss in Sweden and no increase in inequalities. Their results showed that word decoding and reading comprehension scores were not lower during the pandemic compared to before the pandemic. Furthermore, students from disadvantaged socio-economic backgrounds were not more negatively affected, and the proportion of students with weak reading skills did not increase during the pandemic (Hallin et al., 2022).

The Impact of the COVID-19 Pandemic on Education Learning

25

Table 3 Final database of robust studies of learning loss due to COVID-19 (Patrinos et al. 2022) Study

Country

Closure length in weeks

Average learning losses (SD)

Equity impact

Sample size

1

Ardington et al. (2021)

South Africa

22

− 0.22

Yes

5810

2

Arenas and Gortazar (2022)

Spain

− 0.05

Yes

41,000

3

Asakawa and Japan Ohtake (2021)

Yes

5400

4

Bielinski et al. United States (2021)

− 0.14

N

Not reported

5

Birkelund and Karlson (2021)

0

Yes

200,000

6

Borgonovi and Italy Ferrara (2022)

Yes

850,000

7

Chen et al. (2022)

United Arab Emirates

N/A

1920

8

Clark et al. (2021)

China

7

− 0.22

N/A

1835

9

Contini et al. (2021)

Italy

15

− 0.19

Yes

1539

10

Domingue et al. (2021)

United States

− 0.05

N

100,000

11

Engzell et al. (2021)

Netherlands

− 0.08

Yes

350,000

12

EPI (2021)

England

− 0.09

Yes

180,000

13

Gambi and De Belgium Witte (2021)

− 0.23

Yes

213,000

14

Gore et al. (2021)

Australia

8

0

N

4800

15

Haelermans et al. (2021)

Netherlands

10

− 0.17

Yes

201,819

16

Halloran et al. (2021)

United States

Yes

11,700

17

Hevia et al. (2021)

Mexico

Yes

3161

18

Hicks and Faulk (2022)

United States

Yes

Not reported

19

Jakubowski et al. (2022)

Poland

− 0.30

N/A

4581

20

Kogan and United States Lavertu (2021)

− 0.23

Yes

124,700

Denmark

11

8

8

48

29

− 0.55

(continued)

26

I. Sanz and J. D. Tena

Table 3 (continued) Study

Country

Closure length in weeks

Average learning losses (SD)

Equity impact

Sample size

21

Korbel and Czech Prokop (2021) Republic

− 0.11

N/A

2234

22

Kuhfeld et al. (2021)

United States

− 0.19

Yes

5,400,000

23

Lichand et al. (2021)

Brazil

− 0.32

Yes

7,000,000

24

Locke et al. (2021)

United States

− 0.15

No

55,000

25

Ludewig et al. (2022)

Germany

− 0.14

N/A

4290

26

Maldonado and Witte (2022)

Belgium

− 0.18

Yes

1300

27

Philadelphia (2021)

United States

N/A

65,000

28

Pier et al. (2021)

United States

− 0.08

N/A

100,000

29

Schult et al. (2021)

Germany

10

− 0.08

N/A

80,000

30

Schuurman et al. (2021)

Netherlands

8

− 0.09

Yes

886

31

Skar et al. (2021)

Norway

7

− 0.24

N/A

2453

32

Spitzer and Musslick (2020)

Germany

8

N/A

2500

33

Tomasik et al. (2020)

Switzerland

8

N/A

28,685

34

van der Velde et al. (2021)

Netherlands

8

N/A

133,450

35

Chaban et al. (2022)

Russian Federation

8–20

No

165,740

9

− 0.20

− 0.27

Source Patrinos et al. (2022)

5 The Impact of the Covid Pandemic in Education Equity Doepke and Zilibotti (2020) had already predicted in a post on April 1 that the Coronavirus crisis would widen the academic gap by socioeconomic level. In addition to the loss of the last quarter of the 2019–20 course, students also experienced a loss of skills during the summer, especially in language and mathematics, which could

The Impact of the COVID-19 Pandemic on Education Learning

27

reach up to half of what was learned during the course. This setback was even more pronounced in students from disadvantaged backgrounds. While schools have provided online resources for students to continue learning at home, the results indicate that online training cannot replace face-to-face education, especially for lagging students. In some countries, initial results indicate that online teaching has been a poor substitute for face-to-face teaching, with teachers being revealed as key figures in students’ future decision making. Heppen et al. (2017) reported that when students in the same class were randomly assigned to either an online or a face-to-face format, the results of the former were 0.2 standard deviations below those of the latter, and they had a lower probability of passing (66% compared with 78%). Bettinger et al. (2020) developed a randomized, controlled experiment with 6,000 students in Russia that varied the intensity of computer-assisted learning as a substitute for traditional learning. The results showed that academic performance was improved, but that completely replacing face-to-face education was a mistake. The blended approach kept students engaged while also exposing them to the most beneficial face-to-face learning methods. Access to broadband internet service is not the same in all homes, which further intensifies the skills gap by socio-economic level. Parents’ ability to support their children’s learning also depends on their own knowledge and on whether they can work from home during the crisis. Children from low-income families are affected by a decline in positive peer spillovers. There is evidence that online training has failed to replace in-person education during the lockdown, with many families reporting that little learning has taken place. The time spent by pupils on school activities has also decreased in many countries, while the time spent watching television, playing video games, or using mobile phones has increased. Only 6% of German students have group lessons online daily, and more than half had them less than once a week. As David Deming (2020) points out, an important part of the work in teaching involves personalization through tutoring, feedback or individualized monitoring that cannot be scaled. Teachers have a major impact on students’ life choices and career success and have no technological substitute. In 2020, Chetty et al. (2020) conducted a study on the indicators of the Zearn digital mathematics program in the United States, which was used by many schools before and after COVID-19 school closures. The results of the study were disheartening, as the academic progress of students in the lowest income quartile districts fell by 60%, while in the highest quartile, it fell by 20%. Similarly, a study by Maldonado and De Witte in Belgium from 2015 to 2020 reported an increasing trend in inequality within and between schools. Agostinelli et al. (2022) found that high school students in the lowest 20th percentile of the income distribution suffered a learning loss of 0.4 standard deviation after a one-year school closure, while students in high-income neighborhoods initially remained unaffected. The authors calibrated a model that predicts that the effect of school closures on educational inequality will be persistent, and only half of the gap will be closed by the end of high school. The lost of peer effects due to school lockdowns explained 60% of the increase in skill inequality.

28

I. Sanz and J. D. Tena

Patrinos et al. (2022) examined 20 studies on learning loss by socio-economic status and found that 15 studies reported greater learning loss among students or schools with lower socio-economic status. Eleven studies concluded that the pandemic has led to greater learning losses for students at lower levels of academic achievement. However, three studies found a higher impact for prior higher achievers. Recently, Goldhaber et al. (2022) analyzed the impact of the pandemic on student achievement, differentiating between students who attended in-person, hybrid, or online classes during 2020–21. They found that students who attended face-to-face classes for nearly all of 2020–21 lost about 20 percent of a typical school year’s math learning since the pandemic began in March 2020. Within school districts that followed virtual classes for most of 2020–21, students from high-poverty schools lost learning equivalent to 0.46 standard deviations, while students from low-poverty schools lost 0.30 standard deviations. It is possible that family stress in districts that remained remote could have caused the decline in achievement and driven school officials to keep school buildings closed. In areas where schools remained open in the academic year 2020–21, there was no widening of gaps between high and low-poverty schools in math, and even less in reading.

6 Meta-Analysis of the Impact of the School Closures in Student Achievement To conduct the meta-analysis, we collected the standardized impacts and their associated standard errors from the studies listed in Table 4. Standard errors were not always reported, but in some cases, they could be calculated using Cohen’s D and the number of observations for each group (Schult et al., 2021) or confidence intervals of the estimated parameters (Jakubowski et al., 2022). In many cases, we obtained the aggregate impact using estimates for different subjects. However, we were only able to recover estimated standard deviations of the impacts for 14 papers, which formed the sample we used for the meta-analysis (see Fig. 1). We began our study by testing the type of model that best fit the observed data. A fixed effect model assumes that a single homogeneous population generates the Table 4 Meta-analysis heterogeneity tests (14 studies) Statistic

Estimated value

95% confidence interval Lower bound

Upper bound

τ 2∗

0.008

0.0004

0.016

I 2**

46.65%

4.16%

63.80%

1.87

1.043

2.763

H 2*** Q (df =

13)****

31.505 (p-value = 0.003)

Notes ****Test for heterogeneity, ***Total variability/Sampling variability, **Total heterogeneity/ Total variability, *Estimated amount of total variability

The Impact of the COVID-19 Pandemic on Education Learning

29

Fig. 1 Forest plot of the 14 studies employed in the meta-analysis

estimated effects from the component studies, with the overall estimate being the average of the estimates from each study using measures of estimation precision as weights. In contrast, a random effect model assumes an error component associated with each study, such that we would observe a different output if that study were rerun. We denote the variance of that error by τ2 , such that a fixed effect model is a specific case of a random effect model that results when τ2 = 0. Table 4 reports the main results of the heterogeneity tests. We found that the estimated amount of total heterogeneity (τ2 ) was 0.008, and the ratio of total heterogeneity to total variability (I2 ) was 46.65%. The Q(df = 13) test of heterogeneity was 31.51, which was significant at conventional values. Therefore, there was significant heterogeneity, at least according to the Q test of heterogeneity, and a moderate amount of heterogeneity according to the estimated I2. Thus, the overall random effect estimated impact was -0.181, which was significant at conventional values (z-value = − 4.24). However, one caveat was that one of the studies (Lichand et al., 2021) was highly influential in this analysis, with an rstudent of − 5.04 and a dffits of − 11.81 (Viechtbauer and Cheung, 2010). This was not surprising given the high number of observations (7,000,000) used in the analysis by Lichand et al. (2021). When we removed this study, the estimated impact was − 0.126, which was also significant at conventional values (z-value = − 3.423). Figure 1 shows the forest plot of the 14 studies in the meta-analysis. All the studies show evidence of a negative impact of school closures on academic performance. It is also relevant that, despite the heterogeneity of results, we do not reject the null hypothesis of no publication bias in the sample (Z-Test for Funnel Plot Asymmetry is 0.223 with p-value = 0.824).

30

I. Sanz and J. D. Tena

7 Conclusions and Education Policy Implications The COVID-19 pandemic has led to worldwide school closures, with a risk of learning loss. The pandemic has brought about the largest disruption to children’s learning in many countries in generations. Learning losses, once accrued, are difficult to fully offset later on, suggesting that the current crisis will affect the economic opportunities of today’s children for decades to come. As a result of the first wave of COVID-19induced lockdowns and school closures beginning in March 2020, students have lost on average about one-third to one-half year’s worth of learning. We provide a meta-analysis of the impact of school closures on student achievement, which is the first meta-analysis as far as we know. Overall, the meta-analysis shows that school closures have a significantly negative impact on school learning, by around − 0.18 standard deviations. Even if we remove Lichand et al. (2021) from the meta-analysis due to its large weight in the estimations, the total effect is still significant at − 0.12, according to conventional values. These learning losses may impact a student’s educational trajectory, as the lost learning is likely to limit opportunities to advance to higher levels of schooling. Small group tutoring is one of the educational measures for which there is empirical evidence of effectiveness in rigorous research studies. The effectiveness of small group tutoring decreases as the group size increases, with a steeper drop when the group is larger than six students (Burgess, 2020). Studies reviewed by the Education Endowment Foundation show that half an hour of small group tutoring a day for 12 weeks produces an additional four months of progress in school, compensating for the loss of three months of schooling due to the closure of schools. The metaanalysis based on the study of 96 articles of randomized experiments of Nickow et al. (2020) shows that the effect of tutoring in small groups is important and significant (37% of the standard deviation). The effects for the reading and math interventions are similar, although reading tutoring tends to produce higher effect sizes in earlier grades, while math tutoring tends to produce higher effect sizes in later grades. High dosage tutoring can particularly increase cognitive outcomes for students from disadvantaged backgrounds (Dobbie & Fryer, 2013). In our proposal (Sevilla et al., 2020), we estimated a cost of 365 million euros for a national program of small group tutoring in Spain, benefiting 40% of pre-primary, primary, and secondary students who are lagging behind in academic performance. We based our assessment on the analysis of the Education Endowment Foundation, which estimated a cost of 770 euros per group for tutoring half an hour a day for 12 weeks in small groups of five students. The cost of the tutoring proposal in small groups in Spain is similar to the calculations made by Kraft & Goldstein (2020). This researcher calculates that there are 50 million non-university students in the US, and that the promotion of tutoring in small groups would cost 2700 million. Given that in Spain there are 8,276,528 non-university students in the 2019–20 academic year, one-sixth the size of the US educational system and with the current dollar/euro exchange rate, the cost of Kraft & Goldstein’s proposal would be 387 million euros transferred to the case of our country. Burgess’s proposal is 410 million pounds sterling, or 451 million

The Impact of the COVID-19 Pandemic on Education Learning

31

euros, somewhat higher than ours given that England has a larger school population than Spain. The cost of the proposal could be reduced by carrying out some of the online tutorials. The provision of online tutoring through tutors in the last years of higher education or recent graduates has already shown positive and significant effects (equivalent to what is learned in up to 5 months) in rigorous evaluations (Carlana & La Ferrara, 2021; Gortazar et al., 2022). Early data on High Dosage tutoring shows schools are sometimes finding it tough to deliver even low doses. Tennessee results for low-income children hint that tutoring benefits may be slow to emerge (Hechinger Report, 2023). More than two years after the pandemic shuttered schools, many programs are finding difficulties to hire enough qualified tutors. Halloran et al. (2023) underscores the multitude of implementation challenges involved in recovery plans for student learning loss due to the pandemic: the task of engaging the targeted students, staffing difficulties, and scheduling issues. These complications, as Halloran et al. (2023) observed, often resulted in interventions that did not achieve their intended scope and impact. The research also showed a higher rate of academic recovery in Mathematics compared to English Language Arts (ELA), with many districts still struggling to return to pre-pandemic performance levels. Importantly, the state in which a district is located emerged as a crucial factor in recovery rates, while other aspects such as district demographics, modes of schooling during 2020–21, and community COVID-19 transmission levels in 2021–22 appeared to have minimal influence. One of the more widely implemented recovery plans is small group tutoring. Kraft and Falken (2020) and Kraft et al. (2022) highlight potential barriers to success in these programs, such as when tutoring is provided by volunteers who may not commit to a full academic year, lack a binding contractual relationship, or lack specific teaching expertise, particularly for lagging students. As a result, these volunteers often struggle to keep students in need of support focused after regular classes, leading to low student participation rates and limited improvement in academic performance. Conversely, according to Kraft and Falken (2020) and Kraft et al. (2022), small group tutoring can yield promising academic outcomes when the following conditions are met: a rigorous selection process for tutors with higher education credentials, fulltime tutors working with the same group of students throughout the academic year, appropriate training and continual support for tutors, and coordinated efforts between teachers and tutors.

References Agostinelli, F., Doepke, M., Sorrenti, G., & Zilibotti, F. (2022). When the great equalizer shuts down: Schools, peers, and parents in pandemic times. Journal of Public Economics, 206, 104574. https://doi.org/10.1016/j.jpubeco.2021.104574 Ananat, E. O., Gassman-Pines, A., Francis, D. V., & Gibson-Davis, C. M. (2011). Children left behind: The effects of statewide job loss on student achievement. NBER Working Paper No. 17104. https://www.nber.org/papers/w17104

32

I. Sanz and J. D. Tena

Ardington, C., Willis, G., & Kotze, J. (2021). COVID-19 learning losses: Early grade reading in South Africa. International Journal of Educational Development. https://www.sciencedirect. com/science/article/pii/S0738059321001334 Arenas, A., & Gortazar, L. (2022). Learning loss one year after school closures: Evidence from the Basque country. IEB Working Paper 2022/3. Institut d’Economia de Barcelona. https://ieb.ub. edu/wp-content/uploads/2022/03/Doc2022-03.pdf Asakawa, S., & Ohtake, F. (2021). Impact of temporary school closure due to COVID-19 on the academic achievement of elementary school students. Discussion papers in economics and business 21–14, Osaka University, Graduate School of Economics. https://ideas.repec.org/p/ osk/wpaper/2114.html Bacher-Hicks, A., Goodman, J., & Mulhern, C. (2021). Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time. Journal of Public Economics, 193, 104345. Bettinger, E., & Fairlie, R., Kapuza, A., Kardanova, E., & Loyalka, P., & Zakharov, A. (2020). Does EdTech substitute for traditional learning? Experimental estimates of the educational production function. https://doi.org/10.3386/w26967 Bielinski et al. (2021). No longer a prediction: What new data tell us about the effects of 2020 learning disruptions. https://www.illuminateed.com/download/no-longer-a-predictio nwhat-new-data-tell-us-about-the-effects-of-2020-learning-disruptions/ Birkelund, J. F., & Karlson, K. B. (2021). No evidence of a major learning slide 14 months into the COVID-19 pandemic in Denmark. https://osf.io/md5zn/download Blainey, K., & Hannay, T. (2020). The impact of lockdown on children’s education: a nationwide analysis. RS Assessment, School Dash. https://www.risingstars-uk.com/media/RisingStars/Ass essment/Whitepapers/RS_Assessment_white_paper_1.pdf Borgonovi, F., & Ferrara, A. (2022). A longitudinal perspective on the effects of COVID-19 on students’ resilience. The effect of the pandemic on the reading and mathematics achievement of 8th and 5th graders in Italy. Available at: https://doi.org/10.2139/ssrn.4025865 Burgess, S. (2020). How we should deal with the lockdown learning loss in England’s schools. Voxeu (16 June). https://voxeu.org/article/how-we-should-deal-lockdown-learning-lossengland-s-sch ools Burgess, S., & Sievertsen, H. H. (2020) Schools, skills, and learning: The impact of COVID-19 on education. April 1, 2020. https://voxeu.org/article/impact-covid-19-education Carlana, M., & La Ferrara, E. (2021). Apart but connected: Online tutoring and student outcomes during the COVID-19 pandemic. HKS Faculty Research Working Paper Series RWP21-001. https://www.hks.harvard.edu/publications/apart-connected-online-tutoring-andstudent-outcomes-during-covid-19-pandemic Carlsson, M., Dahl, G. B., Öckert, B., & Rooth, D. (2015). The effect of schooling on cognitive skills. Review of Economics and Statistics, 97(3), 533–547. https://doi.org/10.1162/REST_a_ 00501 Chaban, T. Yu., Rameeva, R. S., Denisov, I. S., Kersha, Yu. D., & Zvyagintsev, R. S. (2022). [Russian Schools during the COVID-19 Pandemic: Impact of the First Two Waves on the Quality of Education]. Voprosy obrazovaniya/Educational Studies Moscow, 1, 160–188. https://doi.org/ 10.17323/1814-9545-2022-1-160-188 Chen, D. L., Ertac, S., Evgeniou, T., Miao, X., Nadaf, A., & Yilmaz, E. (2022). Grit and academic resilience during the Covid-19 pandemic. INSEAD Working Paper No. 2022/02/DSC. Available at SSRN: https://ssrn.com/abstract=4001431 or https://doi.org/10.2139/ssrn.4001431 Chetty, R., Friedman, J. N., Hendren, N., & Stepner, M. (2020). How did Covid-19 and stabilization policies affect spending and employment? A new real-time economic tracker based on private sector data. NBER Working Paper, n.º 27431. Clark, A. E., Nong, H., Zhu, H., & Zhu, R. (2021). Compensating for academic loss: Online learning and student performance during the COVID-19 pandemic. China Economic Review, 68, 101629. https://halshs.archives-ouvertes.fr/halshs-02901505/document

The Impact of the COVID-19 Pandemic on Education Learning

33

Contini, D., Di Tommaso, M. L., Muratori, C., Piazzalunga, D., & Schiavon, L. (2021). The COVID19 pandemic and school closure: Learning loss in mathematics in primary education (No. 14785). Institute of Labor Economics (IZA). https://ftp.iza.org/dp14785.pdf Cunha, F., Heckman, J. J., & Schennach, S. M. (2010). Estimating the technology of cognitive and noncognitive skill formation. Econometrica, 78(3), 883–931. Deming, D. (2020). Online learning should return to a supporting role. New York Times, April 9. https://www.nytimes.com/2020/04/09/business/online-learning-virus.html Dobbie, W., & Fryer, R. G. (2013). Getting beneath the veil of effective schools: Evidence from New York City. American Economic Journal: Applied Economics, 5(4), 28–60. Doepke, M., & Zilibotti, F. (2020). COVID-19 and children’s education. Psychology Today, April 1, 2020. https://www.psychologytoday.com/us/blog/love-money-and-parenting/202004/covid19-and-children-s-education Domingue, B. W., Hough, H. J., Lang, D., & Yeatman, J. (2021). Changing patterns of growth in oral reading fluency during the COVID-19 pandemic—Policy brief. Policy Analysis for California Education (PACE). https://edpolicyinca.org/publications/changing-patterns-growth oral-reading-fluency-during-covid-19-pandemic Engzell, P., Frey, A., & Verhagen, M. D. (2021). Learning loss due to school closures during the covid-19 pandemic. Proceedings of the National Academy of Sciences, 118(17). Fuchs-Schündeln, N., Krueger, D., Ludwig, A., & Popova, I. (2020). The long-term distributional and welfare effects of Covid-19 school closures. NBER Working Paper, 27773. Gambi, L., & de Witte, K. (2021). The resiliency of school outcomes after the COVID-19 pandemic. Standardised test scores and inequality one year after long term school closures. FEB. Research Report Department of Economics. https://lirias.kuleuven.be/3611628?limo=0 Goldhaber, D., Kane, T. J., McEachin, A., Morton, E., Patterson, T., & Staiger, D. O. (2022). The consequences of remote and hybrid instruction during the pandemic. National Bureau of Economic Research. Working Paper Series 30010. http://www.nber.org/papers/w30010 Goodman, J. (2014). Flaking out: Student absences and snow days as disruptions of instructional time. NBER Working Paper 20221. Gore, J., Fray, L., Miller, A., Harris, J., & Taggart, W. (2021). The impact of COVID-19 on student learning in New South Wales primary schools: an empirical study. Australian Educational Research, 48, 605–637. https://doi.org/10.1007/s13384-021-00436-w Gortazar, L., Hupkau, C., & Roldán, A. (2022). Online tutoring works: Experimental evidence from a program with vulnerable children. ESADE Working Paper 2. https://www.esade.edu/ecpol/wpcontent/uploads/2022/03/Working_paper_Menttores.pdf Grewenig, E., Lergetporer, P., Werner, K., Woessmann, L., & Zierow, L. (2020). COVID-19 and educational inequality: How school closures affect low- and high-achieving students. CESifo Working Paper 8648. Grewenig, E., Lergetporer, P., Werner, K., Woessmann, L., & Zierow, L. (2021). COVID-19 and educational inequality: How school closures affect low- and high-achieving students. European Economic Review, 140, 103920. Haelermans, C., Korthals, R., Jacobs, M., de Leeuw, S., Vermeulen, S., van Vugt, L., Aarts, B., Breuer, T., van der Velden, R., van Wetten, S., & de Wolf, I. (2021). Sharp increase in inequality in education in times of the COVID-19-pandemic. ROA. ROA Research Memoranda No. 010. https://doi.org/10.26481/umaror.2021010 Hallin, A., Danielsson, H., Nordström, T., & Fälth, L. (2022). No learning loss in Sweden during the pandemic: Evidence from primary school reading assessments. International Journal of Educational Research, 114, 102011. Halloran, C., Hug, C., Jack, R., & Oster, E. (2023). Post Covid-19 test score recovery: Initial evidence from state testing data (April 2023). NBER Working Paper No. w31113. Halloran, C., Jack, R., Okun, J., & Oster, E. (2021). Pandemic schooling mode and student test scores: Evidence from US states. National Bureau of Economic Research Working Paper Series No. 29497. http://www.nber.org/papers/w29497.pdf

34

I. Sanz and J. D. Tena

Hanushek, E. A., & Woessmann, L. (2020). The economic impacts of learning losses. OECD Education Working Papers, 225. Harris, D. N., & Larsen, M. F. (2018). The effects of the new Orleans post-Katrina marketbased school reforms on medium-term student outcomes. Technical Report Education Research Alliance for New Orleans. https://educationresearchalliancenola.org/files/publications/HarrisLarsen-Reform-Effects-2019-08-01.pdf Heppen, J. B., Sorensen, N., Allensworth, E., Walters, K., Rickles, J., Taylor, S. S., & Michelman, V. (2017). The struggle to pass algebra: Online vs. face-to-face credit recovery for at risk urban students. Journal of Research on Educational Effectiveness, 10(2), 272–296. Hechinger Report. (2023, February 27). Taking stock of tutoring. https://hechingerreport.org/proofpoints-taking-stock-of-tutoring/ Hevia, F. J., Vergara-Lope Tristan, S., Velásquez-Durán, A., & Calderón Martín del Campo, D. (2021). Estimation of the fundamental learning loss and learning poverty related to COVID-19 pandemic in Mexico. International Journal of Educational Development. https://www.scienc edirect.com/science/article/pii/S0738059321001681 Hicks, M. J., & Faulk, D. (2022). What contributed to COVID learning loss in Indiana’s schools? https://projects.cberdata.org/reports/CovidLearningLoss-20220118.pdf Jakubowski, M., Gajderowicz, T., & Wrona, S. (2022). Achievement of secondary school students after pandemic lockdown and structural reforms of education system results from TICKS 2021 assessment in Warsaw. Evidence Institute Policy Note 1, Warsaw. https://www.evidin.pl/wp-con tent/uploads/2022/01/POLICY-NOTE-1-2022-EN.pdf Jaume, D., & Willén, A. (2019). The long-run effects of teacher strikes: evidence from Argentina. Journal of Labor Economics, 37(4), 1097–1139. Kogan, V., & Lavertu, S. (2021). The COVID-19 pandemic and student achievement on Ohio’s thirdgrade English language arts assessment. Ohio State University. https://glenn.osu.edu/covid-19pandemic-and-student-achievement-ohios-third-grade-english-language-arts-assessment Korbel, V., & Prokop, D. (2021). Czech students lost 3 months of learning after a year of the COVID-19 pandemic. PAQ Research. https://bit.ly/3CKN2jg Kraft, M. A., & Falken, G. (2020). A blueprint for scaling tutoring across public schools, Ed Working Paper 21-335. Retrieved from Annenberg Institute at Brown University https://doi. org/10.26300/dkjh-s987. https://www.educationnext.org/case-for-national-student-tutoring-sys tem/ Kraft, M., & Goldstein, M. (2020). Getting tutoring right to reduce COVID-19 learning loss. Brookings Brown Center Chalkboard. https://www.brookings.edu/blog/brown-centerchalkb oard/2020/05/21/getting-tutoring-right-to-reduce-covid-19-learning-loss/ Kraft, M. A., List, J. A., Livingston, J. A., & Sadoff, S. (2022). Online tutoring by college volunteers: Experimental evidence from a pilot program. AEA Papers and Proceedings, 112, 614–618. Kuhfeld, M., Lewis, K., Ruzek, E., & McEachin, A. (2021). Learning during COVID-19: Reading and math achievement in the 2020–21 school year. NWEA https://www.nwea.org/content/upl oads/2021/07/Learning-during-COVID-19-Reading-and-mathachievement-in-the-2020-2021school-year.research-brief.pdf Lavy, V. (2015). Do differences in schools’ instruction time explain international achievement gaps? Evidence from developed and developing countries. The Economic Journal, 125(588), 397–424. Lichand, G., Dória, C., & Neto, O. L., et al. (2021). The impacts of remote learning in secondary education: Evidence from Brazil during the pandemic, 03 June 2021, PREPRINT (Version 1) available at Research Square. https://doi.org/10.21203/rs.3.rs-568605/v1 Locke, V. N., Patarapichayatham, C., & Lewis, S. (2021). Learning loss in reading and math in U.S. schools due to the COVID-19 pandemic. Istation. https://bit.ly/3egYlGq Ludewig, U., Kleinkorres, R., Schaufelberger, R., Schlitter, T., Lorenz, R., Koenig, C., Frey, A., & McElvany, N. (2022). COVID-19 pandemic and student reading achievement—Findings from a school panel study. PIRLS. https://doi.org/10.31234/osf.io/hrzae

The Impact of the COVID-19 Pandemic on Education Learning

35

Maldonado, J. E., & De Witte, K. (2020). The effect of school closures on standardised student test outcomes. Discussion Paper Series, DPS20,17. Leuven Economics of Education Research (LEER), KU Leuven Maldonado, J. E., & De Witte, K. (2022). The effect of school closures on standardised student test outcomes. British Educational Research Journal, 48(1), 49–94. Maurin, E., & McNally, S. (2008). Vive la Révolution! Long term returns of 1968 to the angry students. Journal of Labour Economics, 26, 1–33. Meyers, K., & Thomasson, M. A. (2017). Paralyzed by panic: Measuring the effect of school closures during the 1916 Polio pandemic on educational achievement. NBER Working Paper 23890. http://www.nber.org/papers/w23890 Nickow, A., Oreopoulos, P., & Quan, V. (2020). The impressive effects of tutoring on PreK-12 learning: A systematic review and meta-analysis of the experimental evidence. National Bureau of Economic Research, Working Paper 27476. https://www.nber.org/papers/w27476, https:// edworkingpapers.org/sites/default/files/ai20-267.pdf Psacharopoulos, G., Collis, V., Patrinos, H. A., & Vegas, E. (2020). Lost wages: The COVID-19 cost of school closures. World Bank Policy Research Working Paper, n.º 9246. SSRN: https:// ssrn.com/abstract=3601422 Psacharopoulos, G., Collis, V., Patrinos, H. A., & Vegas, E. (2021). The COVID-19 cost of school closures in earnings and income across the world. Comparative Education Review, 65(2), 271– 287. Patrinos, H. A., Vegas, E., Carter-Rau, R. (2022). An analysis of COVID-19 student learning loss. Policy Research Working Paper 10033. World Bank. Philadelphia. (2021). Assessing student performance before and during virtual learning: A cohort comparison of student performance on 2019–20 winter and 2020–21 fall plus and star assessments. School District of Philadelphia. https://www.philasd.org/research/wpcontent/uploads/ sites/90/2021/02/AimswebPlus-and-Star-Cohort-Study-Report-January-2021.pdf Pier, L., Christian, M., Tymeson, H., & Meyer, R. H. (2021). COVID-19 impacts on student learning: Evidence from interim assessments in California. Policy Analysis for California Education. https://edpolicyinca.org/publications/covid-19-impacts-student-learning Pischke, J.-S. (2007). The impact of length of the school year on student performance and earnings: Evidence from the German short school years. The Economic Journal, 117(523), 1216–1242. RAND. (2014). Ready for fall? Near-term effects of voluntary summer learning programs on lowincome students’ learning opportunities and outcomes. RAND Corporation. https://www.rand. org/pubs/research_reports/RR815.html Ruiz-Valenzuela, J. (2020). Intergenerational effects of employment protection reforms. Labour Economics, 62. https://www.sciencedirect.com/science/article/pii/S0927537119301101 Schueler, B. E. (2020). Making the most of school vacation: A field experiment of small group math instruction. Education Finance and Policy, 15(2), 310–331. https://doi.org/10.1162/edfp_ a_00269 Schult, J., Mahler, N., Fauth, B., & Lindner, M. (2021). Did students learn less during the COVID19 pandemic? Reading and math competencies before and after the first pandemic wave.https:// doi.org/10.31234/osf.io/pqtgf Schuurman, T. M., Henrichs, L. F., Schuurman, N. K., Polderdijk, S., & Hornstra, L. (2021). Learning loss in vulnerable student populations after the first Covid-19 school closure in the Netherlands. Scandinavian Journal of Educational Research. https://doi.org/10.1080/00313831. 2021.2006307 Sevilla, A., Sainz, J., & Sanz, I. (2020). Una propuesta para evitar el efecto negativo de los cierres de los centros educativos en el futuro de los jóvenes españoles. Nada es Gratis (31 July) https://nadaesgratis.es/admin/una-propuesta-para-evitar-el-efectonegativo-de-los-cierresde-los-centros-educativos-en-el-futuro-de-losjovenes-espanoles Skar, G. B. U., Graham, S., & Huebner, A. (2021). Learning loss during the COVID-19 pandemic and the impact of emergency remote instruction on first grade students’ writing: A natural

36

I. Sanz and J. D. Tena

experiment. Journal of Educational Psychology. Advance online publication. https://doi.org/ 10.1037/edu0000701 Spitzer, M., & Musslick, S. (2020). Academic performance of K-12 students in an online-learning environment for mathematics increased during the shutdown of schools in wake of the Covid-19 pandemic. EdArXiv. https://edarxiv.org/jncwt/ Tomasik, M., Helbling, L., & Moser, U. (2020). Educational gains of in-person vs. distance learning in primary and secondary schools: A natural experiment during the COVID-19 pandemic 29 school closures in Switzerland. International Journal of Psychology. https://doi.org/10.1002/ ijop.12728 Van der Velde, M., Florian, S., Rinske, S., Meeter, M., & van Rijn Hedderik. (2021). Lockdown learning: Changes in online foreign-language study activity and performance of Dutch secondary school students during the COVID-19 pandemic. Frontiers in Education, 6. https://doi.org/10. 3389/feduc.2021.712987 Viechtbauer, W., & Cheung, M. (2010). Outlier and influence diagnostics for meta-analysis. Research Synthesis Methods, 10, 112–125. Woessmann, L. (2003). Schooling resources, educational institutions and student performance: The international evidence. Oxford Bulletin of Economics and Statistics, 65, 117–170. https://pap ers.ssrn.com/sol3/papers.cfm?abstract_id=234820 Woessmann, L., Freundl, V., Grewenig, E., Lergetporer, P., Werner, K., & Zierow, L. (2020). Schulschließungen verbracht, und welche Bildungsmaßnahmen befürworten die Deutschen? [Education in the corona crisis: How did the schoolchildren spend the time the schools were closed and which educational measures do the Germans advocate?], n.º 73, ifo Schnelldienst.

The Role of International Education Agencies After the Pandemic Ana Capilla

This chapter describes the programs and initiatives an international education agency, the Organización de Estados Iberoamericanos para la Educación, la Ciencia y la Cultura (OEI), is taking into place in the Ibero-American countries to tackle the effects of the pandemic on their educative system, according to the mandate received by the national Education ministers. The region was already facing a learning crisis when COVID-19, which hit hardly the Latin-American countries, forced the educative authorities to suspend the face-to-face learning for almost two years. The first measures adopted to minimize the impact of the lock-down stopped being effective when this was prolonged in time and students continued at home. It is difficult to measure accurately the learning loss they have experienced after two years away from the educative centres because diagnosis standardized tests are more an exception than a rule for the Ibero-American countries. However, according to the first estimations 10 years may be necessary to recover the learning losses. Within this context, the OEI is proposing a completely new educative post-pandemic scenario, embracing digitalisation and longer school days as key elements to foster the learning recovery. The reforms promoted for the regional Higher Education Institutions (HEIs) are equally deep and substantial, assuming the digitalisation is also an opportunity to arrive to new students, to design a more attractive learning offer and to solve some of the main weaknesses the higher education dragged in terms of quality or mobility. The final goal is to increase the qualification of the graduates so they can boost the regional productivity and contribute to a long-term and stable economic and social growth for countries most of them caught up in the middle-income trap.

A. Capilla (B) Organización de Estados Iberoamericanos para la Educación, la Ciencia y la Cultura (OEI), Universidad Francisco de Vitoria (UFV), Madrid, Spain e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_3

37

38

A. Capilla

1 Introduction Education has always been at the core of international cooperation. The creation of the United Nations Educational, Scientific and Cultural Organization (UNESCO) at an early date, in 1945, is the best example. UNESCO is one of the specialized agencies founded concurrently with the United Nations at the end of World War II to promote “mutual understanding and dialogue between cultures” (UNESCO, 2022). Education is a powerful tool for peace, but also for boosting social and economic development. For that reason, the cooperation among the Ibero-American countries1 also started with education. The first Ibero-American organization for cooperation was the Organización de Estados Iberoamericanos para la Educación, la Ciencia y la Cultura (OEI), created in 1949. The OEI has its origin in the 1st Inter Ibero-American Congress of Education, held in Madrid in 1949 and with more than 200 participants (mainly schoolteachers and principals) from Argentina, Bolivia, Brazil, Colombia, Cuba, Chile, Equator, Guatemala, Panamá, Perú, Portugal, Uruguay, Venezuela and Mexico. It is especially significant that there were Mexican participants because at that time there were no diplomatic relations between Spain and Mexico.2 Since the very beginning we can find some of the main characteristics of the OEI, which make this organization quite peculiar. Firstly, it was not a governmental initiative, but it was created by what we call today “civil society”. The Spanish government provided an initial institutional infrastructure to guarantee the continuity of the initiative and the implementation of the agreements reached at the first Congress. The next one, celebrated in 1954 in Quito, decided to make of the OEI an international organization. For that reason, even today the OEI is very close to the Ibero-American educational community although its governing body, the Directive Board, is composed of the Education Ministers from the OEI Member States.3 The OEI is a common ground for the educational administration and community to work together on projects and programs with a highly technical content. That may explain why all the Member States are and have been involved and cooperating in different OEI initiatives despite the frequent, and sometimes irreconcilable, political disputes among them, like the above-mentioned case of Mexico and Spain back during the Francoist regime. The OEI is an international organization for cooperation, so the main goal is to implement programs and projects within the fields of education, culture and science aiming to support the development of one of the most unequal regions of the world. Even if the OEI publishes many reports, papers, and other academic work, this is 1

The Ibero-American region is composed of Portugal, Spain and all the Latin American countries with Spanish or Portuguese as official language. 2 Mexico did not recognize the Francoist regime after the Spanish Civil War and diplomatic relations were not re-established until 1977, after Franco’s death. 3 The OEI Member States today are: Argentina, Bolivia, Brazil, Colombia, Cuba, Chile, Ecuador, Guatemala, Panamá, Perú, Portugal, Uruguay, Venezuela, Mexico, El Salvador, Honduras, Nicaragua, Andorra, Costa Rica, Dominican Republic and Equatorial Guinea.

The Role of International Education Agencies After the Pandemic

39

not its main task. The OEI generates knowledge to share with the region and fosters better, evidence-based educational policies. And to guarantee the OEI cooperation projects to tackle in the best possible way the big challenges Ibero-American states face within the domain of education, culture, and science. OEI’s goal is to make the cooperation actually happen. The OEI currently has 19 national Offices in almost all the Member States and the General Secretariat, the headquarters, which is located in Madrid. This large network makes it possible to work on the ground and be very close to the needs of the countries and governments. Thus, the national Offices mainly implement cooperation programs demanded by the national authorities which need the assistance of the OEI owing to their lack of technical capabilities, know-how, and/or funding. Meanwhile, the General Secretariat launches regional programs to be implemented in all the Member States, or several of them, addressing common issues. This way Ibero-America, a region founded on a shared past and culture, can build a common future by facing together complex topics which may be more effectively addressed at a supranational level. During the last seven decades the OEI’s work has evolved according to what the region needed. Most of the Latin-American countries have become during that time middle-income countries. The so-called commodities price boom meant a decade (2003–2013) of economic growth for the region and resulted in a reduction in poverty, a growing middle-class and an increase in the number of students, especially in higher education. Numbers show that investment in R + D during this period increased too, but not as much as the growth rate (OCTS—OEI, 2020), so the region has been unable to capture the surplus of the period to boost a technological development leading to a more added-value industry. Instead, according to the UN Economic Commission for Latin America and the Caribbean (ECLAC), during that period there was a “reprimarization” or “re-commoditization” of the Latin American economies, meaning the share of natural resources goods in the export basket increased (Ocampo, 2017). ECLAC has identified the four development traps preventing Latin-American countries from reaching sustainable and inclusive development, the productivity trap being one of them. The region is characterised by persistent low productivity levels and poor productivity performance. One of the reasons is the alreadymentioned concentration of exports in primary and extractive sectors with low levels of sophistication (OECD, 2019). Low levels of technology and a lack of qualified human resources are other relevant factors explaining the decrease in Latin American productivity directly connected to education and science. Some numbers help to fully understand the effect the poor performance of the national education systems is having on the economic and social development of our countries: • 60% of young Latin American graduating from the education system lack sufficient skills and qualification (IIEYP—OEI & BCIE, 2022) • Only 40% of researchers and 13% of university teachers have a Ph.D. (OEI, 2022; OCTS—OEI, 2020)

40

A. Capilla

The first priorities for the OEI back in time were to guarantee universal access to basic education and eradicate illiteracy. Most recently, the efforts have been focused on early childhood because kindergarten education contributes to reducing inequalities; the acquisition of key skills, including digital ones; and improving quality education from elementary school to higher education. Equity and quality were the two main challenges the Ibero-American education was struggling with when the pandemic for COVID-19 arrived, especially the LatinAmerican education systems. Experts declared, according to the results of PISA 2018, that the region was facing a learning crisis. On average, 15-year-old students from the region were three years behind a student in an OECD country in reading, mathematics and science. The differences among the Latin-American countries were also outstanding: Chilean students perform on average three years ahead of the Dominican Republic students. The 2018 PISA results also confirmed the learning gaps between students from better-off and most disadvantaged backgrounds are very large, a gap that may be of four years in Brazil and Uruguay (Di Gropello et al., 2019). The education systems were clearly unprepared and the very long closure of schools, for nearly two years in most of the countries, may have produced serious outcomes that are yet to be revealed. Official data on student dropout rates are to be released soon but there are also difficulties in assessing the learning loss because standardized tests are not generalised in the region. The past two years have been also very challenging even for such an experienced organization as the OEI. The pandemic has been a stress test for all the education systems around the world, but also for all the international education agencies, which had to prove themselves to be useful in extraordinary and unprecedented circumstances.

2 The Shock of the Pandemia At the very beginning, when it seemed the lockdown would last a few weeks, the OEI uploaded multiple educational resources to provide teachers and schools with digital content they could use to continue teaching. Many of the cooperation priority working areas in all the projects focused on improving education quality. Therefore, the organization already had many digital resources in very different areas (Math, Science, Language, etc.) which could be used for online or distance learning. There were also special courses to improve teachers’ digital skills which had also a very high demand. On the 11th of March, the day the WHO declared the pandemic for COVID-19, the OEI had just closed an Ibero-American seminar on online and distance learning for the universities of the region in Loja (Equator). A few days after that event, we asked the online and Open universities participating in the seminar to prepare tutorials and to share with other Higher Education Institutions (HEIs) their experience in online learning, virtual labs, augmented reality, etc. One month later, in April 2020, the OEI launched 1000 grants for university teachers who wanted to learn

The Role of International Education Agencies After the Pandemic

41

the methodology of online learning and how to design virtual content. Since then, the OEI has offered more than 500 additional grants on those topics for university teachers and ministry staff thanks to the support of several universities which already had advanced digitalisation policies in place. This horizontal cooperation among countries or institutions from different IberoAmerican countries has been working in the region for a long time. It was one of the reasons why the OEI could provide a rapid and comprehensive answer to the first effects of COVID-19 on education. It has become a common saying to define the pandemic as an extremely disruptive event leading to major changes. However, those changes (a more intensive use of IT technologies in education, the extension of hybrid or online learning, advanced digital skills for students and teachers, etc.) were long overdue. OEI had in place different initiatives to promote them and some schools and HEIs were already applying them. Those changes were not disruptive at all. The disruption came because they had to be implemented within days and weeks of the arrival of the virus to the region, without clear guidance or previous experience. OEI presented its Higher Education and Science strategy, called Universidad Iberoamérica 2030, in February 2020, just a month before the worldwide spread of COVID-19. Not a single word of the strategy changed after March 2020. The pandemic has just sped up its implementation, and it is currently being updated because most of its goals have been already achieved. From this point of view, the pandemic has been an opportunity to embrace very quickly necessary reforms that had been delayed in the region for a long time. At the beginning, even rural schools were able to attend their students despite the connectivity breach. They have experience in attending their students during periods of absence due to adverse weather conditions or other circumstances by offline means. The rural schools had also the advantage the families were used to support their children educational process while they are at home. They were better able to attend their students during the quarantine from the first moment (Annessi & Acosta, 2021). The first shock of the pandemic for the education systems was, thus, not so negative. The deep negative impact education has suffered in Ibero-America has been the result of the longest and uninterrupted COVID-19 school closures in the world. Portugal and Spain were the exception because the closure lasted 11 weeks. Many efforts were made to make schools and universities safe, so the students could come back to them for the beginning of the new academic year in September 2020. As has been explained, a suspension of face-to-face education for a similar period in Latin American countries would have been acceptable. Even for the rural schools, which concentrate the most vulnerable students (low-income and indigenous students), because they had considerable expertise in pedagogical continuity when students cannot go to school. However, the Latin American students have been away from the classrooms for almost two years and the results are catastrophic for education systems already in crisis. The Secretary General of United Nations, Mr. Guterres, was the first one to use this term when in October 2020 he called for actions (reopen safe schools, learning recovery, educational innovation, etc.) to avert a “generational catastrophe”.

42

A. Capilla

The pandemic was a period of uncertainty given there was no precedent in the contemporary era. However, there was enough evidence to anticipate the negative educational impact that school closure was going to have. In April 2020 the OEI already published a report warning about the negative effects remote education may have (Sanz et al., 2020). Fifteen months later there was a second issue of the report confirming the worst scenario foreseen: a continued suspension of face-toface classes and no implementation of palliative initiatives to minimize the negative effects of remote education. By that time students were already several years behind with regard to the academic year they were supposed to be in (Sanz et al. 2021). Latin America has been one of the regions of the world most hit by the pandemic, which has killed around 1.5 million people according to different estimates. The lockdowns approved by governments were barely implemented in a region with a very high percentage of informal jobs. Those people had no savings or welfare network, so they needed to leave home every day to make their living. On the other hand, hospitals and sanitary facilities were overcrowded, unable to attend to all the infected who, thus, remained at home. Governments feared schools and universities may contribute to spreading the virus even more, although the experience in other countries, such as Spain, contradicted this argument. The education institutions may be safe if the right preventive health measures already known were taken (social distancing, hand washing, and wearing masks). Definitively they were safer than many students’ homes where they were exposed to physical or psychological violence, neglect, sexual violence, and online violence (CEPAL & UNICEF, 2020). Education administrations put in place different strategies to minimize the educational impact of school closures on students. Nonetheless, GDP fell 7.6% between 2019 and 2020, and scarce resources were allocated mainly to health and social policies. The estimation is that just 1% of the extra resources dedicated to COVID was invested in education. There was not funding enough for the strategies aiming at mitigating the suspension of face-to-face education, nor the necessary technological infrastructure. Although the percentage of the Latin American population with access to Internet was around 66% before the pandemic, there were important differences between the urban and rural population and the different economic contexts. Some 56% of urban homes were connected but only 30% of the rural ones were (OEI, 2022). The data from PISA 2018 also showed that most of the Latin American countries participating in this test had little chance of putting into place an online effective learning for the majority of their students. In Mexico, 94% of the 15-year-old students from better-off backgrounds have an Internet connection at home, as opposed to 29% of those from the most disadvantaged backgrounds. In Peru 88% of the students from elite schools have their own device at home to follow classes whereas only 17% of students from disadvantaged schools do not have to share the devices they have at home (Reimers & Schleicher, 2020). In Mexico, for instance, less than a third of the students had online learning with their teachers. And although this country has a long tradition of using television for remote education, fewer than 50% of students chose this method. Eighty percent of

The Role of International Education Agencies After the Pandemic

43

families worked with the exercises, books and material provided by the teachers by mail, on paper, or message Apps (MEJOREDU, 2020). The second key factor that would have helped to prevent students from lagging behind in knowledge during the pandemic was well-trained teachers in online methodology and with enough digital skills. The engagement shown by teachers during the pandemic has been more than outstanding and they have made the pedagogical continuity possible by keeping contact with their students by different means. They were also affected by the lack of connectivity or devices, and, in many cases, they were just basic ICT users. The Telecom International Unit (ITU) statistics warn that only 12% of the Chilean population and 7% of Spanish, Portuguese and Mexicans have advanced digital skills. Those numbers are even lower for the rest of the Ibero-American countries. Research on basic education teachers from five Brazilian regions revealed the biggest challenge was the use of technologies by unprepared teachers. They lack sufficient digital skills and did not know either how to play the role of learning mediator and get families involved and accompany the pedagogical continuity of their children (Araújo de Sousa et al., 2021). As has been said, the transition to remote education positioned families in the educational process and whereas most parents were able to help their children, 28% declared that they lacked sufficient knowledge to contribute to their children’s distance learning (Bonal & González, 2020). Evidence suggested also some formal and non-formal educational practices are associated with the social class of the families and thus reaffirm the educational inequality that is reproduced according to the economic resources a person has (Treviño et al., 2021). The official dropout figures for the pandemic will be released soon and then we will know how many students have been left behind during the last two years. Very likely most of them are from disadvantaged contexts because they were the ones with more difficulties to continue studying remotely. The greatest impact is expected in rural schools, so the OEI has opened a call for a monographic edition of its journal, Revista Iberoamericana de Educación, to provide more detailed analysis on the topic and figure out possible solutions to recover the students and educational lag during this period. Concerning higher education, experts estimate a dropout ranging between 10 and 25% that could have been partially compensated by a new profile of students seeking online education. The online offer before the pandemic was limited, only 17% of the studies were offered in this modality, so an increase in the offer may have led to an increase in students who for personal or professional reasons were unable to attend university (OEI, 2022). However, these new students are hardly going to make up for the ones who, due to economic difficulties, have left the university to enter the labour market. The number of HE students in the region increased by more than 15% between 2012 and 2017, more than the world average which was 10%. Many of those new students were the first ones in their families to go to college. They were the children of the middle class that had emerged as a result of the commodities boom and, probably, the ones behind the dropout numbers during the pandemic.

44

A. Capilla

HE enrolment in Ibero-American countries is concentrated in bachelor’s degrees, more than 80%, whereas master’s studies have been losing students over the last decade and the increase in PhD students is miniscule. Thus, bachelor level would have the highest dropout rates, but it may be very concerning if there is also an impact on postgraduate studies and if enrolment continues decreasing over the coming years. With fewer students achieving more qualifications, the chances of increasing productivity in the region fall dramatically. The pandemic revealed the low level of digitalisation of the Ibero-American universities. Most of them had a virtual campus and a tech platform for learning management but, as has been seen, virtual education represented only 17% of the qualifications offered. Despite some exceptions, there was not an intense use of technologies as part of the learning process. They were a mere support. For that reason, many of the HEI had to strengthen their technological infrastructure at the beginning of the pandemic because they were not ready to have all the university students simultaneously connected. During this time the universities have not been able to develop real virtual learning, but an emergency remote education, which is basically an adaptation of face-to-face classes to virtuality. It was necessary also to improve the digital skills of teachers and staff, because the daily management of HEIs has to be digitalised as well (OEI, 2022).

3 A New and Transformed Ibero-American Education The region was already in a learning crisis prior to the pandemic as has been explained, and the long closure of schools and HEIs, substituted by mostly ineffective remote learning, has exacerbated the already serious weaknesses of the education systems. A new report by the World Bank and UNICEF stated that four in five sixth graders in Latin America are expected to lack basic reading comprehension proficiency. Learning outcomes in the region may have been set back by more than a decade, which may cost today’s students in the region a 12% decrease in lifetime earnings (World Bank & UNICEF, 2022). Returning to the pre-pandemic situation is neither possible nor desirable. The only option is to establish the foundations of a new post-pandemic education for the region where technologies can finally be integrated into the learning process and used to repair the lag in learning and support the most vulnerable students. Technology also can help to mitigate the burden of education administrations which are managed and run like they were in the twentieth century, so resources may be focused where they are really necessary. For some experts, the pandemic has already started the fifth wave4 of transformation of the educational systems of the region, increasing the educational opportunity 4

The first wave started with the colonization; the second with the rise of the new republics and the third one is linked to the Human Rights Declaration, which recognizes education as a fundamental

The Role of International Education Agencies After the Pandemic

45

gaps between students of different social classes and nationalities, and thus increasing the need for greater relevance of the educational systems in addressing the social, political, and economic challenges in the region, which the pandemic has, in turn, complicated (Reimers, 2021). The OEI is already implementing the first generation of post-COVID projects, trying to use the lessons learnt from the pandemic to tackle the serious problems the Ibero-American education systems are facing and advancing towards new educational paradigms. The program “Education for the 21st Century in Latin America and the Caribbean: thriving, competing and innovating in the digital age” is being implemented in cooperation with the InterAmerican Development Bank (IDB) in nine countries: Argentina, Brazil, Colombia, Guatemala, Honduras, El Salvador, Mexico, Bolivia, and Equator. The goal of the program is to develop hybrid education models for those countries putting special emphasis on three key issues: the acquisition of skills and competences; the flexibility of the academic trajectories; and the use of innovative learning methodologies. The program has started with an inter-ministry dialogue to involve also the ministerial departments in charge of digital infrastructure in this initiative, because connectivity of schools and students is the premise for a hybrid education. The education community is also invited to these dialogues, where the testing programs of the second phase are being designed. Those testing programs will be focused on learning recovery or on educational transformation. Every country is choosing a different focus according to their most demanding needs, may be reading skills, Maths understanding or any other instrumental knowledge. It is, then, a flexible project and hopefully the education systems of the region are going to gain, thanks to the new technologies, flexibility in order to be able to better address the different needs of every student. An important part of the program is focused on teacher training, to improve their digital skills so they can use digital resources to improve the teaching experience in the classroom and to provide special support to those students in need. As has been said, there is still work to do to assess the impact of COVID-19 on Ibero-American education but according to the first estimations 10 years may be necessary to recover the learning losses. The emergent digitalisation started by the regional education systems during the pandemic represents an opportunity to start closing the gap, accelerate the learning recovery and reach more students. Hybrid education, however, is not just a conjunctural measure useful within the post-pandemic context but it may be incorporated into education systems. The last two years have proved the learning process is not limited to inside the classroom’s walls. The new technologies allow students to continue learning at home, after school, complementing, reviewing, or broadening what has been taught at school. A good model of hybrid education has the best of both worlds. The students learn and share

right. The fourth took place in the 1990s as an answer to the “lost decade” in Latin America and the democratization of several countries in the region.

46

A. Capilla

the knowledge with their classmates at school. At the same time, they can enjoy a tailor-made education according to their needs or interests. The priority during the pandemic was to reopen schools. Nonetheless, we should be aware that face-to-face education is not going to be able to recover the massive learning loss that has taken place over the last two years. Furthermore, it is no longer a guarantee of a quality education. Ibero-American countries have to transform their education systems and use the pandemic experience to implement hybrid education, which requires a serious plan of educational and school digitalisation and, obviously, a technological infrastructure providing, at least, connectivity for all students. Hybrid education and a more intensive digitalisation of regional education also demands new governance in education. This is the objective of another OEI program, implemented with the Development Bank of Latin America (CAF) to strengthen and support the digital transformation of the public education administrations. So far it is being implemented in certain territories of Argentina, Uruguay, and the Dominican Republic, where the OEI and CAF are supporting the technological modernization of the participating education administrations to create institutional and sustainable capabilities. A second component of the program refers to the digitalisation of administrative procedures that would make those procedures more efficient; allowing the automatization of certain tasks and the development of analytical capabilities that would help in decision-making. Lastly, it is necessary to promote a digital culture among the different actors taking part in those procedures, including the users and the final beneficiaries. This is one of the strongest lines of cooperation promoted by the OEI. Working on the production of effective digital contents, trainings on digital education’s methodology and systems to measure the effectiveness of the digital education. This is an important message: the digital education is useful if it gets learning results. The region assumed for years children’s assistance to schools was enough and a success. And, as PISA results have already warned, this is not enough. The learning results of the students in the region were very poor and the pandemic has finally revealed the serious weaknesses of the education systems. There is a risk that, as has actually happened already in the recent past, governments may be content with connectivity and the delivery of devices and then not pay enough attention to or invest in teachers’ training, or pedagogical contents. One of the lessons learnt from the pandemic is the need to focus on learning results and skills acquired and, thus, in a more generalised and systematic use of standardized testing for the diagnosis of the education system, but also for transparency and accountability. Pandemic has had a considerable impact on the learning of millions of students, but there are many other reasons for them to fall behind. We must build more resilient educative systems with many different resources at display so those students in trouble may find a safety network to keep them within the system. A more digital education presents many advantages, as has been explained and for that reason is a priority for the OEI. However, there are other instrumental and very practical measures already in place in the region that are proving very effective.

The Role of International Education Agencies After the Pandemic

47

One of them is the extension of the school day, a very effective tool to reduce the impact of the pandemic and increase the quality of education. There are already 17 Ibero-American countries applying longer school days, especially in those schools with disadvantaged students. The first results are promising because it is a chance to strengthen key skills and to design more personalised educative path. It contributes to create a closer link between the student and the school, decreasing the possibility of dropping out. Home is not a safe environment for many children in the region so to stay more time at school represents a benefit in itself, even more if the schools are able to offer during that extra time extra chances to improve their education. Thus, the OEI is studying the most successful experiences to share them with all the Member States in order to promote this initiative. Ibero-American universities are also called upon to face a major transformation. As happens in schools, universities have had a digital reaction to the pandemic but not an actual digital transformation. This is a work in progress for many HEIs in the region and, also, an opportunity to improve their performance. There was an important skill gap before the pandemic between the competences graduates acquired at universities and those demanded by the labour market. CEOs of large regional companies have declared that they have had difficulties finding suitable candidates for their vacancies. The candidates lacked key competences such as oral and written communication, flexibility, endurance, or advanced digital and technological skills (Ríos et al., 2020). However, with an increasing number of students year after year, HEIs had little incentives to change. The pandemic has decreased the number of students because many have had to support their families. It seems unlikely they can recover or maintain the pre-pandemic enrolment rates if the economic perspectives do not improve. On the other hand, the regional demographic bonus is about to end so in the medium-term the number of potential students will decrease. This means there are fewer students for a very high number of universities, because over the last decade many of them have been created to attend the increasing demand for Higher Education. They are mostly private universities, so today there are more students in private universities than in public ones in the following countries: Colombia, Costa Rica, Guatemala, Peru, Paraguay, El Salvador, Chile and Brazil (OEI, 2022). These private universities have also lost students because they have been unable to continue to pay their fees. Both public and private HEIs face a countercyclical period within a context of economic stress. The private ones, because they have less revenue coming from the students’ fees. The public ones, because the public budgets are giving priority to other areas. This may endanger the reforms they must undertake. Obviously, if there are fewer students, the competition among Ibero-American universities is going to increase. This ideally would lead to a competition based on prestige and quality. This is a powerful incentive for them to complete the digital transformation undertaken during the pandemic, so they can put into place an attractive offer of online or blended studies. To become a digital university demands investment and, at the same time, may be the only chance to survive in the complex panorama that has been explained.

48

A. Capilla

This is because a new profile of students has arisen during the pandemic. It is not especially new, since over the last decade the demand for online studies has grown more than 80% although the offer was very small. Prior to the pandemic, only 17%5 of degrees offered by Ibero-American universities were online. The pandemic forced the universities to transit all their studies to the virtual modality, and this meant an opportunity for all those who were unable to attend university. This profile of students is the one that may be increasing over the coming years. Virtual education is probably the only way to get back those students who have left the university during the pandemic. It is an attractive option also for those who are already in the labour market but need to reskill or upskill. This way, the number of students, including these new ones, may reach 45 million in the coming years, a considerable increase on the current 32 million students. The condition, nonetheless, is to complete the digital transformation and this may be a handicap for some universities, especially those lacking the necessary resources or experience to undertake this task properly. To help them to pass from remote emergency learning to true online learning, the OEI and RIACES6 have created the quality label “Kalos Virtual Iberoamérica (KVI)” for those online studies fulfilling a battery of quality criteria and indicators. This initiative comes from the Ibero-American Seminar on Quality of Learning Distance already mentioned, ending on March 11th 2020. It was the first implementation of the Universidad Iberoamérica 2030 strategy and had two main goals. The first was to encourage universities, especially traditional ones having exclusively face-to-face degrees, to offer online or blended studies. There was a greater interest in the region for them, as enrolment rates proved. However, there were many prejudices about this kind of education, still considered to be inferior when compared to the face-to-face modality. The event brought together, online, open and classic universities with quality assurance agencies, to eradicate those prejudices and foster online degrees. The limited offer was explained also because some national legislation and quality assurance agencies did not contemplate the online modality and had no procedures for evaluating them. The second goal of the seminar was to convince them of the benefits of a diversified offer of university degrees taught in different modalities. In May 2020, the OEI published a first quality standards guide for online studies thanks to the work done with the quality assurance agencies during the Seminar. This was a very useful tool for the HEIs, which were at that moment migrating to virtuality to guarantee learning continuity. That publication is the basis of the current KVI label Guide of evaluation, which contemplates five dimensions: academic processes, 5

Red IndicES (2022), Porcentaje de nuevos ingresos en condiciones de iniciar un programa por modalidad 2010–2019 [http://app.redindices.org/ui/v3/comparative.html?indicator=PCTENUEIN GRExMOD&family=ESUP&start_year=2010&end_year=2019]. 6 The Ibero-American Network for the Assurance of Quality in Higher Education (Red Iberoamericana para el Aseguramiento de la Calidad en la Educación Superior – RIACES) is an association which gathers the Ibero-American professionals working for the accreditation and certification of quality in Higher Education in order to contribute to quality assurance.

The Role of International Education Agencies After the Pandemic

49

academic staff, students, management and operations, and structure and technical support. KVI is a 100% Ibero-American label. All the experts and agencies participating in the design come from OEI Member States. They have been working for almost a year to agree on the set of standards included in the Guide and to design a common evaluation procedure. This procedure and this Guide are applied to all the IberoAmerican HEIs requesting the label, so for the first-time universities from Colombia, Chile, Spain or Peru are going to be subject to the same quality evaluation. This is a big step forward for the region and the idea of creating a common higher education area for Ibero-America. Finally, the evaluation committees are composed of Ibero-American evaluators. This is also an important asset because this will foster cooperation among the quality assurance agencies that are part of the label, encouraging them to use Ibero-American peers in other evaluation processes. The KVI label sets the quality standards for online studies, so it is an important guidance to transit from the emergency remote learning most universities have adopted during the pandemic towards true and quality virtual or blended studies. A quality virtual education is crucial to attract those new students but it’s not enough. It will be necessary to have a more flexible offer of studies, with traditional degrees but also other kinds of qualifications, such as micro-credentials. Universities need to respond to the needs of Ibero-American enterprises and industry much more quickly than in the past. This ought to be a critical shift for the regional productivity and, consequently, is a priority for the OEI. Digital transformation of HEIs may also have a deep impact on their governance and management. So far universities have adapted their administrative procedures to ITC technologies because of the COVID-19 but it is necessary to go further forward to become a digital university. Internationalization is one of the domains where digital transformation may also have a deep impact. On one hand, because during the pandemic some universities have been experimenting with virtual mobility, an interesting option for a region with a very low academic mobility. Going abroad has always been considered one of the best opportunities students have to enrich themselves academic and personally. In such a globalized world as the current one, mobility stands as an almost mandatory experience to have in higher education studies. Comprehensive benefits brought by those kinds of chances can easily lead to an improvement in the productivity of countries, being able to count on professionals with valuable skills acquired abroad. Nevertheless, as has already been shown, inequality is a serious problem for our countries. Traditionally, the majority of Ibero-American higher education students has not enjoyed a mobility experience; a large part of them due to economic problems or, for example, disabilities that impede travel. This situation can be a big challenge for overcoming inequalities. Currently, with the experience in virtual mobility brought by the pandemic, Ibero-America can recognize the chances hidden behind this modality as a possibility for widening access to these experiences.

50

A. Capilla

For that reason, during the pandemic the OIE published with the Spanish Universidad Nacional de Educación a Distancia (UNED) a Guide for the design, implementation and follow-up of virtual mobility actions. This is addressed to the HEI professionals in charge of mobility in the region of Latin America and the Caribbean so they can have tips about how to put into place a good-quality virtual mobility program. The goal of this guide is to promote internationalization and digital skills, by explaining what a virtual mobility is and what kind of activities may help the “incoming” students to have an international and intercultural experience from home. There is a risk that virtual mobility may become a second-class option for lowincome students, deepening the inequalities which are characteristic of the region. The best possible scenario is a hybrid model of mobility, reducing the time abroad and, thus, making mobility more affordable for more students. Furthermore, on the other hand, considering that economic and health problems are not the only challenges for mobility in the region, at the beginning of 2021 we joined a group formed by internationally recognized experts in higher education and mobility. We asked them to find a solution for the low level of academic mobility between Ibero-American countries. The first hypothesis was to design and implement a common credit system, as ECTS work in the European Union. However similar experiences in the past were not successful so it was necessary to figure out a more innovative solution. One of the main obstacles to student exchange is the validation and recognition of the studies carried out abroad. Students need the guarantee of this recognition from their university before going on an exchange. And the universities need a certain basic information about those studies to decide if they grant recognition or not. The OEI has published a technical report, The 2030 Ibero-American University in motion: a proposal for academic mobility, that proposes the creation of a platform which allows students, teachers and managers of both origin and destination universities to share information about subjects (called assessed learning units) and, thus, to enter into an academic agreement of recognition. This solution will also reduce the workload of the university International Relations Offices and the bureaucracy involved in academic exchanges, so they can reach more students. The proposal was checked with IR universities offices staff to improve the design and to complete a highly comprehensive consultation involving minister’s members but also potential platform users. This very open and consultative working methodology is an added value of the OEI and crucial for any international agency aiming to put effective measures into place. People affected by those measures, especially those directly concerned, may have the chance to participate in their design so they can feel they are their own. This way they will contribute to their success. This proposal of a digital platform for mobility is a good example of the benefits coming from the digitalisation of the university. Once again, this project is a proof that technology is a powerful ally in education and digitalisation must be a top priority for the Higher Education in Ibero-America. Quality, mobility, and digitalisation are the main challenges the Ibero-American Higher Education is facing in the post-pandemic context. As has been said, there is nothing new. When the OEI started working on the Universidad Iberoamérica 2030

The Role of International Education Agencies After the Pandemic

51

strategy, presented in February 2020, these were its main axes. The pandemic has made them even more urgent because the next few years are going to be extremely tought for many universities. They must undertake serious reforms, probably the deepest one since they were created and for some of them this happened centuries ago. They need to rethink how to fulfil their mission in a very different way from how they used to. A digital university means much more than virtual learning or the automatization of management. Artificial Intelligence (AI) may contribute to universities knowing their students better creating a tailor-made formative path for every one of them. This demands a highly flexible range of studies on offer, including traditional degrees and micro-credentials, and agile administration able to attend to so many different students’ trajectories. Digitalisation may have also a positive impact on research, which is one of IberoAmerica’s weakest points. Once more, it can create new opportunities, for instance, with the augmented reality laboratories. Most of the research is done in the universities and many of them lack the basic infrastructure, so technology may be helpful at this point and many others. The OEI may not only share this diagnosis with government and HEIs, but also to provide them with tools and solutions (KVI label, digital mobility platform, virtual mobility tips, etc.) for those serious challenges ahead. There are also plenty of opportunities for universities if they manage to embrace the reforms pandemic have made urgent but also high risks. Different kind of students are rising all over the region with different education demands, but also alternative providers of higher education such as the big tech companies. The universities, specially those more committed to quality in education and research, offer an added-value for those alternative students as long as they may be able to attend their demands concerning contents and format (virtual or hybrid learning).

References Annessi, G. J., & Acosta, J. I. (2021). La educación rural en tiempos de COVID-19. Experiencias de continuidad pedagógica en las escuelas primarias de Maipú, provincia de Buenos Aires, Argentina. Revista Iberoamericana de Educación, 86(1), 43–59. https://rieoei.org/RIE/article/ view/4145 Araújo de Sousa, L., Da Costa Souza, A., & Alves, A. (2021). Desafíos y aprendizajes con la enseñanza remota de los docentes de educación básica. Revista Iberoamericana de Educación, 86(1), 61–78. https://rieoei.org/RIE/article/view/4373 Bonal, X., & González, S. (2020). The impact of lockdown on the learning gap: Family and school divisions in times of crisis. International Review of Education, 66, 635–655. https://doi.org/10. 1007/s11159-020-09860-z CEPAL, & UNICEF. (2020). Violence against children and adolescents in the time of COVID-19. https://www.cepal.org/en/publications/46486-violence-against-children-and-ado lescents-time-covid-19

52

A. Capilla

Di Gropello, E., Vargas, M. J., & Yanez, M. (2019). What are the main lessons from the latest results from PISA 2018 for Latin America? World Bank Blog. https://blogs.worldbank.org/latinamer ica/what-are-the-main-results-pisa-2018-latin-america IIEYP—OEI, & BCIE. (2022). Empleo juvenil y emprendimiento en América Latina y el Caribe. https://oei.int/publicaciones/empleo-juvenil-y-emprendimiento-en-america-latina-y-el-caribe MEJOREDU. (2020). Experiencias de las comunidades educativas durante la contingencia sanitaria por Covid-19. https://editorial.mejoredu.gob.mx/Cuaderno-Educacion-a-distancia.pdf Ocampo, J. A. (2017). Commodity-led development in Latin America. In G. Carbonnier, H. Campodónico, & S. Tezanos (Eds.), Alternative pathways to sustainable development: Lessons from Latin America (p. 51–76). Brill. https://doi.org/10.4000/poldev.2354 OCTS—OEI. (2020). El Estado de la Ciencia. http://www.ricyt.org/wp-content/uploads/2021/02/ ElEstadoDeLaCiencia_2020.pdf OECD. (2019). Latin American economic outlook 2019: Development in transition. https://doi.org/ 10.1787/g2g9ff18-en OEI. (2022). Informe Diagnóstico sobre la educación superior y la ciencia post COVID-19 en Iberoamérica. Perspectivas y desafíos de futuro 2022. t.ly/OGgZ. Red IndicES. (2022). Porcentaje de nuevos ingresos en condiciones de iniciar un programa por modalidad 2010–2019. http://app.redindices.org/ui/v3/comparative.html?indicator=PCTENU EINGRExMOD&family=ESUP&start_year=2010&end_year=2019 Reimers, F. (2021). Oportunidades educativas y la pandemia de la COVID-19 en América Latina. Revista Iberoamericana de Educación, 86(1), 9–23. https://rieoei.org/RIE/article/view/4557/ 4177 Reimers, F. M., & Schleicher, A. (2020). A framework to guide an education response to the COVID-19 Pandemic of 2020. In OECD. Retrieved April 14, 2020. Ríos, G., Galán-Muros, V., & Bocanegra, K. (2020), Educación Superior, Productividad y Competitividad en Iberoamérica. t.ly/SvXE. Sanz, I., Sáinz, J., & Capilla, A. (2020). Efectos de la Crisis del Coronavirus en la Educación. https://oei.int/oficinas/secretaria-general/publicaciones/efectos-de-la-crisis-del-cor onavirus-en-la-educacion Sanz, I., Sáinz, J., & Capilla, A. (2021). Efectos en la Educación Iberoamericana: un año después de la COVID-19. https://oei.int/oficinas/secretaria-general/publicaciones/efectos-en-la-educac ion-iberoamericana-un-ano-despuesde-la-covid-19 Treviño, E., Miranda, C., Hernández, M., & Villalobos, C. (2021). Clase social y estrategias parentales de apoyo a los estudiantes en pandemia. Resultados para Chile del International COVID-19 Impact on Parental Engagement Study. Revista Iberoamericana de Educación, 86(1), 117–133. https://rieoei.org/RIE/article/view/4449 UNESCO. (2022). History of UNESCO. https://www.unesco.org/en/history World Bank, & UNICEF. (2022). Two years after saving a generation. https://documents1.wor ldbank.org/curated/en/099512306222222251/pdf/IDU00d6b64030a55e0423b0913f0f2ef3f76 4417.pdf

Tutoring and Its Effects on Academic Achievement: A Policy Evaluation with Machine Learning Methods María Teresa Ballestar, María Teresa Freire-Rubio, and Arturo Ortigosa-Blanch

1 Introduction When the COVID pandemic struck schools across the world, alarms started to flash as the overall impact on learning achievement and, specially, the effects on disadvantaged students became clear. Studies at the beginning on the pandemic forecasted learning losses of up to 25% per academic year concentrated among children of low income (Sanz et al., 2020). As actual data started to emerge the figures seemed to confirm those expectations. This data confirmed that the final effect was related to the time that schools were closed, the age of the students, the availability of virtual learning, family income and parents’ education (Capilla et al., 2021). As the expectation of a massive learning deficit became a reality, the worst fear was, and still is, that many of those disadvantaged students will become disincentivized to continue school, with a loss of future income of up to 10% for year of school lost (Donnelly et al., 2021; Patrinos & Psacharopoulos, 2020). Different policy options have been considered to help those disadvantaged pupils, but it seems that tutoring is the option yielding the best results mitigating learning losses (Sevilla et al., 2020). Among the factors that seem to ensure optimal educational outcomes, Kraft (2020) recommends that tutors engage for the whole academic year, are chosen after a rigorous selection process, are well trained in the special issues they coordinate and enjoy ongoing support and coordination with the teachers and the institutions. M. T. Ballestar (B) Universidad Rey Juan Carlos, Madrid, Spain e-mail: [email protected] M. T. Freire-Rubio · A. Ortigosa-Blanch ESIC University, Madrid, Spain e-mail: [email protected] A. Ortigosa-Blanch e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_4

53

54

M. T. Ballestar et al.

Finally, although there is ongoing discussion, it seems that online tutoring, while better than no tutoring at all, is not as productive as in person tutoring. Our goal in this chapter is to evaluate one of these tutoring interventions by using Tree Augmented Naïve Bayesian Networks (TAN) to predict the probability of academic success of the students engaged in presential tutoring. We look into the causality among variables to recommend the design and implementation of education support programs, identifying additional factors to enable maximizing the social return of the investment. The program we analyze is known as Programa para la Mejora del Éxito Educativo (Program for the improvement of academic success) of the Junta de Comunidades de Castilla y León, in Spain, 2020, after the closure of colleges due to the pandemic. The Junta made a call for financial aid in order to help schools with students with problems in the areas of Spanish language and literature, arithmetic, or English. The program yielded an improvement of 5% for those students who participated compared to those who had been in a position to take part but did not.

2 Theoretical Framework Burgess (2020) was among the earliest academics to call for measures during the pandemic to avoid irreversible educational losses. His recommendations warn of the relevance of an early intervention and, specifically, the relevance of extra tutoring. Fryer (2017) analyzes 196 randomized experiments and find that tutoring has impacts from 0.507 to 1.582 standard deviations, much higher that other interventions like poverty reduction or early childhood schooling. The results are appealing not only because of the large magnitudes but also due to the long-term impact effect of the treatment. All these experiments are evaluated through traditional econometric methods. As Athey and Imbens (2019) show, Machine Learning methods can yield similar, or even better, results to measure policy interventions, as shown in Ballestar et al. (2019). To further evaluate this point, we formulate our first hypothesis: Hypothesis 1: Tree Augmented Naïve Bayesian Networks (TAN) is a type of Machine Learning method that enriches the triangulation method applied by Ballestar et al. (2022a, 2022b). This method is able to predict the success ratio of a student in passing the 4th course of ESO, considering the characteristics of the academic programs, the students and their performance in the academic programs. In addition, the method depicts causality between these variables providing information to Public Administrations to enable them to design special education support programs that engage students and improve the success rate. Following Fryer (2017), Kraft (2020), Dynarski (2020) and Ballestar et al. (2022a, 2022b), we can reference some characteristics of efficient tutoring systems: it requires of a quick response once the need is detected; support courses should be taught by dedicated educators; there should be a strong coordination between support programs and the school; family involvement in the process as well as a commitment to attendance on the part of the student. They also give special relevance to the continuity

Tutoring and Its Effects on Academic Achievement: A Policy Evaluation …

55

of the program over time to ensure a lasting effect on the recovery of the academic performance. The above authors also point out the relevance of in person classes versus alternatives which include online classes. The latter represent an alternative to school closures but give lower academic results. The researchers also agree that a relevant issue is the selection of the tutors. The effects are better in professional mentoring programs than support provided by volunteers and parents because of the continuity and quality of the former. We continue to address these issues in other two hypotheses: Hypothesis 2: The characteristics of the special education support programs, such as length and timing have an impact on the student’s engagement and success rate in passing the 4th year of ESO. Public Administrations need both the information provided by analytical models and the resources to design and implement education support programs with that engage the students, maximizing the probability of success in passing the course. And: Hypothesis 3: Early identification of students at risk of school failure and enrolling them in a special education support program is crucial for avoiding course repetition and school drop-out. These two hypotheses have an important effect on the long term academic performance and wellbeing of children. If public administrations do not take steps to overcome the deficit, the education inequality in schools will probably increase, widening achievement gaps and raising school drop-out rates. This growing inequality will carry over into the workforce, where the well-educated few will have higher wages and the poorly educated will earn even less because their numbers have increased (Dynarski, 2020). Tutoring is one of the few effective solutions to avoid that a privileged minority of children are well educated, using private resources, while the bulk of the population risk their access to well paid jobs.

3 Data Collection The Program for the Improvement of Educational Success includes support for reading and writing in the 3rd year of primary education, for students in the 1st year of secondary education, and for students in the 4th year of secondary education both during the summer and during the academic year. The regional government selects the schools at the beginning of the academic year, taking into account educational indicators and the needs of the students and their families. The schools that provide extra support do so for their students and for pupils from other schools in the same province. They are grouped with a minimum of 7 students in rural areas and 10 in urban areas. The teaching team encourages these students to participate and requires the family’s authorization. The database in this research uses data of students in the 4th year of ESO who participated in the Programme for the Improvement of Educational Success in the 2019–2020 academic year. Our working sample contains 1739 records of students

56 Table 1 Academic programs and number of students per each one in the working sample

M. T. Ballestar et al.

Academic program

Number of students

Percentage (%)

C2

1328

76.37

C3

296

17.02

C2C3

115

6.61

Total

1739

100

in one of the three programmes implemented in the Autonomous Community of Castilla y León, of which 822 (47.27%) are girls and 917 (52.73%) are boys and 820 (47.15%) had already repeated a grade. Students who finished the Program for the Improvement of Educational Success had a success rate of 85.34% in the completion of the 4th year of ESO. The program aims to help students who require educational support and reinforcement of the learning process in instrumental areas, educational assistance, and extraordinary educational guidance to develop their abilities to the maximum, acquire the corresponding competencies, increase their school performance, and achieve the general objectives of each academic stage. There are three categories of programs in which students can be enrolled (Table 1): the C2 academic program support students during the academic year; the C3 academic program consists of extra classes during the summer in July; the C2C3 academic program is a combination of the two previous programs with classes for the whole academic year and in July. These programs were delivered in the nine provinces of Castilla y León, where 66.36% of the participants lived in provincial capitals and 33.64% lived in smaller localities. Ballestar et al. (2022a, 2022b) applied the triangulation methodology to this database to determine which variables are relevant to predict the success rate of the students who participated in each of the academic programmes. The triangulation method consists of the development of more than one quantitative method, applying different approaches, with the aim of not only enriching the results provided by each method but also confirming the results obtained (Ballestar et al., 2020), where the authors developed Machine Learning (ML) models based on CHAID (Chi-Square Automatic Interaction Detector) decision trees and Artificial Neural Networks Multilayer Perceptron (ANN-MLP). These two supervised ML methods describe and explain the underlying relationships between the input variables (characteristics of the academic programs, the students’ academic background and their performance in the academic programs) and the target variable (prediction of success of the students in passing the 4th year of ESO) (Maimon & Rokach, 2005). Hence, the social return of the investment made in the academic programs can be measured. In this research, we contribute to enriching these results by adding a new supervised ML method based on Bayesian Networks (BN) (Fitzek et al., 2020) to extend the triangulation method and depict causality between variables, including direct and indirect causality which were not addressed by the two previous ML methods. Bayesian Networks (BN), unlike other ML methods, are very effective not only for

Tutoring and Its Effects on Academic Achievement: A Policy Evaluation …

57

finding correlations between variables but also revealing causality among them (Zhu et al., 2020). Our aim is to identify the causal relationships among variables in the models identifying additional factors to consider in order to maximize the social return of the investment when designing an academic intervention.

3.1 Definition of the Variables in the ML Models This database is made up of 1739 records of students that were enrolled in one of the three academic programmes implemented by Castilla y Leon Autonomous Community in 2019–2020. It contains 21 variables that capture information with regard to the characteristics of the students and their performance and results both in the overall academic programme and the 4th year of the ESO. (Ballestar et al., 2022a, 2022b) identified the five variables which were statistically significant and relevant for the empirical analysis and development of the two Machine Learning Methods based on CHAID decision trees and ANN-MLP. We use these five variables to extend the triangulation method from Ballestar et al. (2022a, 2022b) with a new supervised ML model based on Bayesian Networks (BN) with the aim of explaining the causality among the input variables and its effects on the target variable. Finding causality between variables have been found very relevant in the development of successful and impactful public interventions (Heckman & Pinto, 2022). In the development of the Bayesian Network (BN) four of these variables act as input variables and one variable as target variable as described in Table 2. The target variable corresponds with the percentage of students who did not pass the 4th year of the ESO. This is an unbalanced sample, as 14.66% of the students did not pass the course, compared to 85.34% who passed it.

4 Method and Empirical Analysis In this phase, we developed a model consisting of a Tree Augmented Naïve Bayesian Network (TAN) to extend the triangulation method applied by Ballestar et al. (2022a, 2022b) to this database. The objective of this research is dual. The first objective is to enrich the existing results clarifying the relationship between the input variables that contribute to the ML model to predict the success rate of a student in passing the 4th course of ESO. The second objective consists of confirming the results obtained by Ballestar et al. (2022a, 2022b) and asses which of the three Machine Learning Methods is most convenient for the measurement of results in the implementation of public interventions. Tree-Augmented Naïve Bayesian Networks (TAN) is a restricted Bayesian Network (BN) where each predictor depends on another predictor besides the target variable (Shastri et al., 2017). This method has become very popular because of its

58

M. T. Ballestar et al.

Table 2 Description of the variables of the three ML Models: CHAID decision tree and multilayer perceptron artificial neural network (ANN-MLP) from Ballestar et al. (2022a, 2022b) and the new supervised ML model consisting of a Bayesian Network Input variables

Description

Values

Years_ repetition

Discrete numeric variable. Number of years the student has Value 0: 919 repeated. Its value will be 0 if the student has not repeated any year students (52.85%) Value 1: 576 students (33.12%) Value 2: 230 students (13.23%) Value 3: 14 students (0.81%)

Student_ repetition

Boolean variable. Its value is 1 if the student has repeated a course Value 0: 919 or value 0 if the student has not repeated a course students (52.85%) Value 1: 820 students (47.15%)

Kind_ program

Categorical variable that contains which of the three kinds of existing academic programs: C2; C3; C2C3 the student was enrolled in

Value C2: 1,328 students (76.37%) Value C3: 296 students (17.02%) Value C2C3: 115 students (6.61%)

Finish_ program

Boolean variable. Its value is 1 if the student has completed the academic support programme or value 0 if the student has not completed it

Value 0: 491 students (28.23%) Value 1: 1,248 students (71.77%)

Output variable

Description

Finish_ studies

Boolean variable. Its value is 1 if the student has passed the course Value 0: (4th year of ESO) or value 0 if the student was not able to pass it 14.66% of the sample Value 1: 85.34% of the sample

Tutoring and Its Effects on Academic Achievement: A Policy Evaluation …

59

simplicity and strong prediction accuracy (Shi & Li, 2006). Bayesian Networks (BN) are a type of probabilistic graphical model based on an intersection between statistics and machine learning (Bielza & Larrañaga, 2014; Shastri et al., 2017). These graphical models consist of directed acyclic graphs that link variables by conditional probabilities, where the outputs are probabilities calculated using the Bayes’ Theorem (Fenton & Neil, 2014; Koski & Noble, 2011; Marcot & Penman, 2019) which allow for decision making for risk analysis and management (Aalders, 2008; Farmani et al., 2012). This kind of models are very useful for finding cause-effect relationships between variables which make them widely adopted for real-data analysis in many fields (Hanea et al., 2012; Landuyt et al., 2013; Marcot & Penman, 2019). Based on the information provided by the input variables consisting of the characteristics of the student, the kind of the academic program taken by the student, and the student’s performance, our Tree Augmented Naïve Bayesian Network (TAN) model provides the following information. Firstly, a classification of each student from the database into two homogeneous groups depending on the student’s likelihood of passing the 4th year of the ESO. The algorithm identifies the rules that will be used in future for predicting academic success of new students who enrol in new academic support programs. Secondly, the identification of the causal relationships among the input variables and its effect on the student success in passing the 4th year of ESO. This is fundamental for developing impactful policy interventions to improve the success rate of the academic programs by type of student. In this research the Bayesian Network (BN) enable us to differentiate between whether an event or intervention causes a specific outcome or is merely a correlation effect (Lee & Lee, 2006). The development and training of the Tree Augmented Naïve Bayesian Network (TAN) was performed applying a training, testing and validation (TTV) method (Ballestar et al., 2022a, 2022b). This method consists of training the model with 70% (1203 records) of the sample, testing with 20% (367 records) of the sample and validating with 10% of the sample (169 records). Before the training stage of the model, it should be considered that the distribution of the target variable is unbalanced. 14.66% of the students that participated in the academic programs did not pass the 4th year of ESO, compared to 85.34% who passed it. As this bias could have a negative impact on the training stage of the model, we applied an oversampling method on the underrepresented group to balance the training sample up to 2098 records. Balancing the sample, we ensure that both groups of students will be equally represented when training the model. The testing and validation sample remain unbalanced (Ballestar et al., 2022a, 2022b).. Figure 1 shows the Tree Augmented Naïve Bayesian Network (TAN) obtained for the special education support programme success model.

60

M. T. Ballestar et al.

Fig. 1 Tree augmented Naïve Bayesian network (TAN)

4.1 Evaluation of the Model Classification accuracy, sensitivity, specificity, the area under the receiver operating characteristic (ROC) curve (AUC) (Fig. 2) and the Gini coefficient were the measures to evaluate the performance of the Tree Augmented Naïve Bayesian Network (TAN). In addition, the confusion matrix contains the percentage of cases classified both correctly and incorrectly for the two possible values of the target variable which are 1 if the student has passed the 4th year of ESO or 0 if this is not the case. These accuracy indicators and the confusion matrix are available in Table 3 and they have been calculated for the total sample and also for each of the subsamples of the model: training, test and validation. In Table 3 we observe that the overall classification accuracy of the Tree Augmented Naïve Bayesian Network (TAN) is 70.73%, which represents an error rate of 29.27%. In this research, the target variable is Boolean. The accuracy represents the percentage of success when predicting whether a student will pass the 4th year of the ESO or not. This classification accuracy indicator is very similar across the tree subsamples (training, testing and validation), confirming that the model was not overtrained. The percentage of true positive (TP), also called sensitivity, is 71.23%. This value denotes the percentage of students who pass the 4th year of the ESO and who have been correctly classified by the Tree Augmented Naïve Bayesian Network (TAN) based on the characteristics of the student, the kind of special education support programme, and their results of the student in this programme. The percentage of true negatives (TN), also called specificity, was 67.84%. This value refers to the

Tutoring and Its Effects on Academic Achievement: A Policy Evaluation …

61

Fig. 2 Receiver operating characteristic curves for the full sample

percentage of students who do not pass the 4th year of the ESO and were correctly classified by the model based on the same given input variables. At the same time, the complementary values are the percentage of false negatives (FN), which was 28.77% and which refers to the percentage of students who, having passed the course were classified by the model as not having passed it. Finally, the percentage of false positives (FP) was 32.16%, which denotes the percentage of students who did not pass the course but were classified by the model as having succeeded. Based on these results this model tends to be optimistic in its predictions of students’ success passing the 4th year of the ESO, because the false positives rate (32.16%) is larger than the false negative rate (28.77%) (Firth et al., 2020). We use the area under the curve (AUC) ROC indicator as the main measure of the accuracy of the Tree Augmented Naïve Bayesian Network (TAN) because it is more robust than the classification accuracy indicator when working with unbalanced samples as in this research (Chen et al., 2008; Dželihodži´c & Ðonko, 2016; Jensen, 1992; Yin et al., 2016). This evaluation of the model using the area under the ROC curve (AUC) was performed for the total sample (0.767) and for the training (0.776), test (0.751) and validation (0.744) subsamples getting very similar results. These AUCs are higher than 0.7, so the model was good (Hosmer et al., 2013). Complementary to this indicator, the GINI coefficient has also been calculated. The GINI coefficient is related to the AUC representing twice the area between the ROC curve and the diagonal (Fig. 2). The GINI coefficients were 0.535 for the total sample, 0.511 for the training sample, 0.502 for the test sample and 0.488 for the validation sample. These findings support H1 because the Tree Augmented Naïve Bayesian Network (TAN) is a robust model for predicting the success rate of a student in passing the

169

Validation

* Oversampled

68.12

367

Test

to balance the sample

70.41

71.07

2098

Subsample

Training*

70.73

0.744

0.751

0.776

0.767

0.488

0.502

0.551

0.535

Validation

Test

Training*

Total sample

1739

Total sample

GINI

Sample

AUC

Sample size

Sample

Percentage correct (%)

Confusion matrix

Model accuracy

17 41

1

97 0

30 1

289 0

756

1

427

0

173

1

0

102

9

220

20

735

318

1057

82

1

Sample size

Prediction

0

Observed

Table 3 Model accuracy and confusion matrix of the Tree Augmented Naïve Bayesian Network (TAN)

28.67

65.38

30.60

60.00

28.22

70.39

28.77

67.84

0

71.33

34.62

69.40

40.00

71.78

29.61

71.23

32.16

1

Percentage (%)

62 M. T. Ballestar et al.

Tutoring and Its Effects on Academic Achievement: A Policy Evaluation …

63

Fig. 3 Relative importance of input variables in the Tree Augmented Naïve Bayesian Network (TAN)

4th year of ESO, taking into account the characteristics of the academic programs, and the students and their performance. The relative importance of the variables provides information about the importance of each input variable in making the overall prediction in the Tree Augmented Naïve Bayesian Network (TAN) (Fig. 3), but they are not related to the model’s accuracy. As they are relative values, their sum is equal to 1. In this model, the variable that accumulates 36% of the relative importance is the type of program in which the student participated (kind_program). This finding highlights the importance of Public Administrations having the right tools to invest their resources in designing and implementing special education support programs with the right characteristics to boost student success (Ballestar et al., 2019) confirming Hypothesis 2. The variables with the second and third highest relative importance are the number of years that the student has repeated a course (years_repetition) with an importance of 34%, and the variable which indicates whether the student has repeated the course of not (student_repetition) with an importance of 30%. This finding confirms Hypothesis 3, highting the importance of enrolling the students in special education support programs as soon as possible to avoid early students failure (Ballestar et al., 2022a, 2022b). Finally, with 1% the variable that contains the information on whether the student completed the special education support program (finish_program).

5 Results and Discussion Besides the output prediction, Augmented Naïve Bayesian Network (TAN) method provides a detailed distribution view of the network with conditional probabilities allowing for the identification of dependencies between the variables (Chen & Pollino, 2012).

64

M. T. Ballestar et al.

In the first subsection the conditional probabilities of the target variable in the Tree Augmented Naïve Bayesian Network (TAN) are explained, while the conditional probabilities of the input variables are explained in the second subsection.

5.1 Conditional Probabilities of the Target Variable in the Tree Augmented Naïve Bayesian Network (TAN) In this section the conditional probabilities of the target variable in the Tree Augmented Naïve Bayesian Network (TAN) are explained. Table 4 shows the probability of the students finishing the 4th course of the ESO (finish_studies) conditioned to the kind of academic special education support program where they were enrolled (kind_program), the student’s academic background in terms of number of repeated courses (student_repetition; years_repetition) and their performance in the special education support program (finish_program) (Table 2). The academic special education support program which achieves the best results is C2, consisting of supporting students throughout the academic year. Reaching a success probability of passing the 4th course of ESO of 0.84 for the group of students that were engaged on the program and finished it and had not repeated any course before. These results indicate that this kind of long duration support programs are very successful for avoiding early student failure. However, it is very important to ensure that the student completes the program, otherwise the success probability decreases to 0.75. Having the mechanisms in place to identify the students at risk of school failure and enrolling them in a special education support program is crucial for avoiding course repetition and drop-out. The percentage of success of the programs decreases considerably when the student has repeated a course. In the case of the C2 program the percentages passing the 4th course of the ESO drop to 0.58, 0.42 and 0.47 for one, two- and three-years repetition respectively if the student finished the program. But if the student didn’t finish the program these percentages decrease even more to 0.36, 0.24 and 0.15 for one, two- and three-years repetition respectively. These findings also support Hypothesis 3 and highlight the relevance of enrolling the students in the support programs before repeating a course to avoid school failure in advance. The other two special education support programs C2C3 and C2 also contribute positively to increase the percentage of students that pass the 4th course of the ESO to 0.58 and 0.51 respectively, but they don’t reach as good results as the C2 program. In these programs it is also important to enrol the students before repeating any course, as their probability of passing the course deceases considerably when they have repeated a course before. These low probabilities of success go from 0.26 to 0, depending on the student’s performance in the program and the number of repeated courses. While in the programs C2 and C3 the engagement of the student in the program makes a difference, increasing the probability of passing the course, we observe that

Tutoring and Its Effects on Academic Achievement: A Policy Evaluation …

65

Table 4 Conditional probabilities of finish_studies Conditional Probabilities of finish_studies Parents

Probability

kind_program

finish_program

years_repetition

student_repetition

0

1

C2

0

0

0

0.25

0.75

C2

0

1

1

0.64

0.36

C2

0

2

1

0.76

0.24

C2

0

3

1

0.85

0.15

C2

1

0

0

0.16

0.84

C2

1

1

1

0.42

0.58

C2

1

2

1

0.58

0.42

C2

1

3

1

0.53

0.47

C2C3

0

0

0

0.42

0.58

C2C3

0

1

1

0.74

0.26

C2C3

0

2

1

0.85

0.15

C2C3

1

0

0

0.70

0.30

C2C3

1

1

1

0.84

0.16

C2C3

1

2

1

0.77

0.23

C3

0

0

0

0.62

0.38

C3

0

1

1

0.80

0.20

C3

0

2

1

0.90

0.10

C3

0

3

1

0.85

0.15

C3

1

0

0

0.49

0.51

C3

1

1

1

0.80

0.20

C3

1

2

1

0.82

0.18

C3

1

3

1

1.00

0.00

this is not the case in program C2C3. In program C2C3 students that didn’t repeat any course before and finished the program have a probability of success of 0.30 compared to 0.58 when they didn’t finish the program. Because the program C2C3 is the longest, a combination of programs C2 and C3, students that pass their exams in June abandon the program to spend the month of July on holiday with their families. On the other hand, students that fail in June stay engaged in the program in July, trying to extend the benefits of the program until its end. These findings are relevant for the policy makers to design the programs with the right length to avoid premature drop-offs even if the student got good results from the program in summertime, reinforcing the confirmation of Hypothesis 2.

66

M. T. Ballestar et al.

Table 5 Conditional probabilities of student_ repetition and finish_program

Conditional probabilities of student_repetition

Conditional probabilities of finish_program

Probability

Probability

0

1

0

1

0.42

0.58

0.34

0.66

5.2 Conditional Probabilities of the Input Variables in the Tree Augmented Naïve Bayesian Network (TAN) In this section the conditional probabilities of the input variables in the Tree Augmented Naïve Bayesian Network (TAN) are explained. Table 5 shows the probabilities of two input variables in the model that are not conditioned by other input variables in the model, but both have an effect on other input variables in the model. On the one hand, there is the conditional probability of the students that had repeated at least one course before joining the academic program (student_repetition) which is 0.58. On the other hand, there is the conditional probability of the students finishing the special education support programme (finish_ program) which is 0.66. Table 6 shows the probability of number of courses repeated by the students before joining the special education support program (years_repetition) conditioned to whether they had repeated a course before (student_repetition). If a student had repeated a course, there is a probability of 0.66 that it happened once, 0.32 twice and 0.02 three times. Table 7 shows the probability of having been enrolled in one of the three special education support programmes (kind_program) conditioned to have finished the program (finish_program) and the number of times the student repeated a course before (years_repetition). If the student has finished the program, the probability of having been enrolled in the C2 program is higher than the others (ranging from 0.65 to 0.74). Hence, designing these programs with suitable characteristics and duration is crucial to guarantee students’ engagement, reinforcing the confirmation of Hypothesis 2. If the student has not finished the program the probabilities of having been enrolled in the programs C2 or C3 are much more balanced, ranging from 0.5 to 0.57 for C2 and 0.38 to 0.5 for C3.

Table 6 Conditional probabilities of years_repetition Conditional probabilities of years_repetition Parents

Probability

student_repetition

0

1

2

3

0

1

0

0

0

1

0

0.66

0.32

0.02

Tutoring and Its Effects on Academic Achievement: A Policy Evaluation …

67

Table 7 Conditional probabilities of kind_program Conditional probabilities of kind_program Parents

Probability

finish_program

years_repetition

C2

C2C3

C3

0

0

0.52

0.05

0.42

0

1

0.57

0.05

0.38

0

2

0.51

0.1

0.39

0

3

0.5

0

0.5

1

0

0.74

0.1

0.16

1

1

0.71

0.14

0.16

1

2

0.71

0.18

0.11

1

3

0.65

0

0.35

6 Conclusion The intervention we just analyzed aims at 4th year of secondary school students in the Castilla y León region who require support and reinforcement of the learning process, preferably in instrumental areas and subjects, in order to develop their abilities to the maximum, acquire the corresponding competencies, increase their school performance and achieve the general objectives of each educational stage. The schools that provide these interventions do so for their own students, and also students from other schools in the same province. They are placed in groups with a minimum of 7 students in rural areas and 10 in urban areas. Our results show that those students that engage in the program increase their graduation rates regardless of their background and validate tutoring as an effective educational policy and are consistent with previous results like Fryer (2017), Kraft (2020), Dynarski (2020) and Ballestar et al. (2022a, 2022b). Through our research we have shown the need for an economic evaluation of the LOMLOE (Organic Law Amending the Organic Law of Education). The LOMLOE started with the same budget allocation as the LOMCE (The Organic Law for Improving the Educational Quality) for interventions such as the one analysed (45 million euros). The arrival of European funds opens up new possibilities not con templated in its initial design, which in turn raises the challenge of using this additional investment efficiently. Our research shows that Tree Augmented Naïve Bayesian Networks (TAN) are a type of Machine Learning method that complement the triangulation method applied by Ballestar et al. (2022a, 2022b) to predict the probability of at-risk students passing the 4th year of ESO and also confirms their research results, supporting Hypothesis 1. The overall classification accuracy of the Tree Augmented Naïve Bayesian Network (TAN) is 70.73% very similar to the 71.19% and 70.73% accuracy achieved by the artificial neural network multilayer perceptron (ANN-MLP) and CHAID decision tree in Ballestar et al. (2022a, 2022b).

68

M. T. Ballestar et al.

Our findings depict causality between variables, including direct and indirect causality which were not addressed by the two ML methods developed by Ballestar et al. (2022a, 2022b), contributing to differentiation between whether an event or public intervention causes a specific outcome, or whether these are merely correlation effects (Lee & Lee, 2006). These insights are crucial for Public Administrations to be able to develop impactful policy interventions which improve the success rate of the special education support programmes. These findings shed light on how the characteristics of the special education support program, such as length and calendar have an impact on the student’s engagement with it. Public Administrations need the right tools to invest their resources in designing and implementing education support programs which boost students’ loyalty and success, confirming Hypothesis 2. The degree of engagement of the students with the program is very important because those who finish the program have a higher probability of passing the 4th year of the ESO compared to those who did not finish the program. Our analysis shows that the probability of passing the 4th year of ESO is 0.84 for the group of students that were enrolled in the C2 program and finished it and had not repeated any course before. Nevertheless, this probability decreases to 0.75 (-10.7%) if these students did not finish the program. In addition to this, it is important to use these tools and resources to identify the students that are at risk of school failure, because the special education support programs get their best results with students that had not repeated any course before. These programs contribute to minimizing early school failure and drop-out in advance, confirming Hypothesis 3. The probability of passing the 4th year of the ESO for students that have repeated a course before and are enrolled in a course such as C2 can be − 80% less compared to students who did not repeat a course, dropping to probabilities of success of 0.15. Spain’s high early school dropout rate (EDR) presents a significant challenge to educational policymakers. However, our research suggests that relatively low-cost, high-impact educational policy options, like tutoring, may be effective in reducing EDR. For example, educational success plans (ESPs) have been shown to be more efficient than traditional policies that target students who have already repeated a grade. Additionally, artificial intelligence (AI) tools have the potential to be effective in education, as they can be used to identify students at risk of dropping out and provide them with targeted interventions.

References Aalders, I. (2008). Modeling land-use decision behavior with Bayesian belief networks. Ecology and Society, 13(1). Athey, S., & Imbens, G. W. (2019). Machine learning methods that economists should know about. Annual Review of Economics, 11(1), 685–725. https://doi.org/10.1146/annurev-economics-080 217-053433

Tutoring and Its Effects on Academic Achievement: A Policy Evaluation …

69

Ballestar, M. T., Díaz-Chao, Á., Sainz, J., & Torrent-Sellens, J. (2020). Knowledge, robots and productivity in SMEs: Explaining the second digital wave. Journal of Business Research, 108, 119–131. https://doi.org/10.1016/j.jbusres.2019.11.017 Ballestar, M. T., Doncel, L. M., Sainz, J., & Ortigosa-Blanch, A. (2019). A novel machine learning approach for evaluation of public policies: An application in relation to the performance of university researchers. Technological Forecasting and Social Change, 149. https://doi.org/10. 1016/j.techfore.2019.119756 Ballestar, M. T., García-Lazaro, A., Sainz, J., & Sanz, I. (2022a). Why is your company not robotic? The technology and human capital needed by firms to become robotic. Journal of Business Research, 142, 328–343. https://doi.org/10.1016/j.jbusres.2021.12.061 Ballestar, M. T., Sainz, J., & Sanz, I. (2022b). An economic evaluation of educational interventions in the LOMLOE: Proposals for improvement with Artificial Intelligence. Revista Espanola De Pedagogia, 80(281), 133–154. Bielza, C., & Larrañaga, P. (2014). Bayesian networks in neuroscience: A survey. Frontiers in Computational Neuroscience, 8, 131. Burgess, S. (2020). How we should deal with the lockdown learning loss in England’s schools. VOX Cepr Policy Portal. https://voxeu.org/article/how-we-should-deal Capilla, A., Sainz, J., & Sanz, I. (2021). COVID 19: Efectos en la educación un año después. Organización de Estados Iberoamericanos. Chen, S. H., & Pollino, C. A. (2012). Good practice in Bayesian network modelling. Environmental Modelling & Software, 37, 134–145. Chen, Y., Ghosh, A., Mcafee, R. P., & Pennock, D. (2008). Sharing online advertising revenue with consumers. In International workshop on internet and network economics, pp. 556–565. Donnelly, R., Harry, & Patrinos, A. (2021). Learning loss during Covid-19: An early systematic review. In PROSPECTS 2021, 1–9. https://doi.org/10.1007/S11125-021-09582-6 Dynarski, S. (2020). The school year really ended in March. The New York Times. Dželihodži´c, A., & Ðonko, D. (2016). Comparison of ensemble classification techniques and single classifiers performance for customer credit assessment. Modeling of Artificial Intelligence, 3, 140–150. Farmani, R., Henriksen, H. J., Savic, D., & Butler, D. (2012). An evolutionary Bayesian belief network methodology for participatory decision making under uncertainty: An application to groundwater management. Integrated Environmental Assessment and Management, 8(3), 456– 461. Fenton, N., & Neil, M. (2014). Decision support software for probabilistic risk assessment using Bayesian networks. IEEE Software. Firth, J. A., Hellewell, J., Klepac, P., Kissler, S., Kucharski, A. J., & Spurgin, L. G. (2020). Using a real-world network to model localized COVID-19 control strategies. Nature Medicine, 26(10), 1616–1622. Fitzek, F. H. P., Granelli, F., & Seeling, P. (2020). Computing in communication networks: From theory to practice. Academic Press. Fryer Jr, R. G. (2017). The production of human capital in developed countries: Evidence from 196 randomized field experiments. In Handbook of economic field experiments (Vol. 2, pp. 95–322). Elsevier. Hanea, D., Hanea, A., Ale, B., Sillem, S., Lin, P. H., Van Gulijk, C., & Hudson, P. (2012). Using dynamic Bayesian networks to implement feedback in a management risk model for the oil industry. In 11th International probabilistic safety assessment and management conference and the annual European safety and reliability conference 2012. Heckman, J. J., & Pinto, R. (2022). Causality and econometrics. National Bureau of Economic Research. Hosmer Jr, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (Vol. 398). John Wiley & Sons. Jensen, H. L. (1992). Using neural networks for credit scoring. Managerial Finance. Koski, T., & Noble, J. (2011). Bayesian networks: An introduction (Vol. 924). John Wiley & Sons.

70

M. T. Ballestar et al.

Kraft, M. (2020). What if every struggling student had a tutor? Education Week. https://www.edw eek.org/education/opinion-what-if-every-struggling-student-had-a-tutor/2015/08 Landuyt, D., Broekx, S., D’hondt, R., Engelen, G., Aertsens, J., & Goethals, P. L. M. (2013). A review of Bayesian belief networks in ecosystem service modelling. Environmental Modelling & Software, 46, 1–11. Lee, C.-J., & Lee, K. J. (2006). Application of Bayesian network to the probabilistic risk assessment of nuclear waste disposal. Reliability Engineering & System Safety, 91(5), 515–532. Maimon, O., & Rokach, L. (2005). Data mining and knowledge discovery handbook. Springer. Marcot, B. G., & Penman, T. D. (2019). Advances in Bayesian network modelling: Integration of modelling technologies. Environmental Modelling & Software, 111, 386–393. Patrinos, H. A., & Psacharopoulos, G. (2020). Chapter 4—Returns to education in developing countries (S. Bradley & C. B. T.-T. E. of E. (Second E. Green, Eds.; pp. 53–64). Academic Press. https://doi.org/10.1016/B978-0-12-815391-8.00004-5 Sanz, I., Sáinz, J., & Capilla, A. (2020). Efectos de la crisis del coronavirus sobre la educación. Organización de Estados Iberoamericanos Para La Educación, La Ciencia y La Cultura (OEI). Sevilla, A., Sainz, J., & Sanz, I. (2020). Una propuesta para evitar el efecto negativo de los cierres de los centros educativos en el futuro de los jóvenes españoles—Nada es Gratis. Nada Es Gratis. https://nadaesgratis.es/admin/una-propuesta-para-evitar-el-efecto-negativo-delos-cierres-de-los-centros-educativos-en-el-futuro-de-los-jovenes-espanoles Shastri, B. J., Tait, A. N., de Lima, T. F., Nahmias, M. A., Peng, H.-T., & Prucnal, P. R. (2017). Principles of neuromorphic photonics. ArXiv Preprint ArXiv:1801.00016 Shi, H., & Li, K. (2006). Comparison of two learning methods of the tree augmented Naïve Bayesian network classifier. International Conference on Machine Learning and Cybernetics, 2006, 1054– 1059. Yin, X., Li, K., & Zong, Z. (2016). Resolution enhancement of robust Bayesian pre-stack inversion in the frequency domain. Journal of Geophysics and Engineering, 13(5), 646–656. Zhu, Q.-X., Ding, W.-J., & He, Y.-L. (2020). Novel multimodule Bayesian network with cyclic structures for root cause analysis: Application to complex chemical processes. Industrial & Engineering Chemistry Research, 59(28), 12812–12821.

The Impact of the School Closures on Bullying and Cyberbullying in Spain Miguel Cuerdo-Mir and Luis Miguel Doncel-Pedrera

1 Introduction The government closure of schools in the spring of 2020 due to Covid-19 has had multiple effects on the school population. One of them has been the possible evolution of bullying and cyberbullying as a result of this school closure period. The issue is relevant since, as some analysts have suggested, the importance of these specific phenomena of violence and bullying in schools constitutes one of the relevant public health problems. As highlighted by a recent study by the US Department of Health and Human Services (Basile et al., 2020), which, using 2019 data from the US Youth Risk Behavior Survey, indicated that bullying in high schools is a phenomenon that leaves significant consequences for the health of those who suffer it and that, moreover, it occurs in 50% of the cases reported at ages under 18; in other words, at the age of school education. All this means not only an impoverishment of the health of those who suffer from it but also a lower degree of social interaction and integration, as well as direct and indirect costs to the economic system, as shown in an Australian study (Alannah & Madeline Foundation, 2018). It is true that, as Hymel and Swearer (2015) have pointed out, the phenomenon of bullying is not new and is already reflected in nineteenth-century Western literature, such as in Charles Dickens’ novel “Oliver Twist”. However, in recent decades it has taken on a new dimension and is becoming increasingly important in today’s society. In fact, since the pioneering analyses of Olweus (1978), more than forty L. M. Doncel-Pedrera—Honorary Associate while part of this research was carried out, University of Liverpool, Liverpool, United Kingdom. M. Cuerdo-Mir · L. M. Doncel-Pedrera (B) Universidad Rey Juan Carlos, Madrid, Spain e-mail: [email protected] Grupo de Innovación Docente en Economía y Finanzas (GIDEF), Grupo de Investigación Consolidado de Estudio y Evaluación de Políticas Económicas de La URJC, Madrid, Spain © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_5

71

72

M. Cuerdo-Mir and L. M. Doncel-Pedrera

years of studies have been ongoing in relation to a phenomenon, (Hymel & Swearer, 2015). It has not only become more evident as a major problem in today’s society but has taken on new dimensions with the emergence and spread of new information and communication technologies, which have led to the so-called cyberbullying. With words by Smith et al (2008), cyberbullying can be defined as “an aggressive, intentional act carried out by a group or individual, using electronic forms of contact, repeatedly and over time against a victim who cannot easily defend him or herself”. Since the first seminal studies, bullying has been defined as a category within the more general category of interpersonal violence characterized by an imbalance of power, intentionality, and repetition (Olweus, 1999). Although some researchers have pointed out that when schoolchildren are asked about this issue, they often do not include all these elements in their description (Vaillancourt et al., 2008). This has left an open debate to be considered since the assessment of such a complex phenomenon depends largely on how it is measured, and which variables are most relevant for both analysis and elimination. In any case, it is true that it is not the same to ask schoolchildren directly as to ask parents, teachers, or school principals. Information from schoolchildren themselves is probably the cheapest and most direct; however, it is heavily influenced by what victims and perpetrators say. On the other hand, if information comes from parents, they tend to have very limited information about what is happening in schools (Cornell & Brokenbrough, 2004). In the United States, the prevalence of bullying among schoolchildren has been measured and between 5 and 13% of schoolchildren themselves admit to bullying at some point, while between 10 and 33% admit to having been victims of bullying, according to different studies reported by Hymel and Swearer (2015). On the other hand, studies are not definitive about the evolution of bullying either. Currie et al. (2012) and Rigby and Smith (2011), using data from the World Health Organisation, point to slight decreases over time. Jones et al. (2013) assume these declines but qualify that they are accompanied by significant increases in so-called cyberbullying. In relation to this phenomenon of cyberbullying, the WHO data cited above suggest, Hamm et al. (2015), that it could be affecting between 15 and 25% of young people, underlining that both bullying and cyberbullying coexist. On another note, it is important to underline that the studies collected and analysed by Hymel and Swearer (2015) emphasize bullying as a “stable” experience, although it varies from one time to another, from one age to another, or also depending on the methodology used by the researcher. It tends to be longer lasting (between five months and 1 or 2 years) in middle-aged students (between 8 and 16 years) and shorter at younger ages. Furthermore, the aforementioned studies point to a decrease in physical bullying (more frequent in boys) and an increase in verbal and social bullying (more frequent in girls), due to the greater or lesser degree of identification of the aggressor in both cases. In addition, as they get older, students are less likely to talk about bullying and even to report it, depending very much on the degree of trust that adults, especially teachers and school leaders, inspire in them when it comes to prosecuting and punishing it effectively.

The Impact of the School Closures on Bullying and Cyberbullying in Spain

73

Finally, many of the studies cited by Hymel and Swearer (2015) have found that bullying activity is a ‘socially intelligent’ activity, which even gives bullies a certain status among their peers, which can make it difficult for bullies to stop the activity. Similarly, Basile et al. (2020) also note, in the context of American high schools, that there are “specific groups of students” at higher risk of experiencing certain types of interpersonal violence, such as LGBT (Lesbian, gay, bisexual and transgender) youth, which raises the need to know how it evolves in certain contexts and socio-economic realities. This cannot ignore some of the results of recent years. Thus, a meta-study conducted by Spanish doctors (Fraguas et al., 2021) has shown that there is a set of anti-bullying interventions that present universal characteristics and have proven to be effective. However, they point to the need for better designs and the specific characteristics in which these programs are applied, since bullying and cyberbullying also develop and are exacerbated in specific socio-economic and educational contexts and differentially affect high-risk students such as LGBTQ (Lesbian, gay, bisexual, transgender and queer) or those with some kind of disability. In this context, the COVID-19 pandemic brought about a very significant change in the lives of students with confinement, due to, among other issues, the closure of schools and the consequent period of social isolation. In addition, the impact of the economic crisis and instability in the labour market must be considered, which may have generated difficulties in their families and ended up becoming an additional source of stress and uncertainty. A very interesting analysis of this difficulty can be found in Shahin (2022), who analyses the effect on the well-being of Spanish parents and teachers during the period of school closures. Young people may have felt isolated from their friends, teachers, the school environment, and the impossibility of enjoying leisure time outside the home, together with the need to continue their education at home. In this sense, Racine et al. (2020) and Hawes et al. (2021) highlight the increase in mental health problems that these circumstances have caused in adolescents, including depression, anxiety and even cyberbullying (Orben et al., 2020). In fact, Duckworth et al. (2021) demonstrate, through a study conducted on high school students in the United States, that students who attended classes in the Fall of 2020 without being present in the classroom showed lower social and emotional wellbeing compared to students who had continued to attend classes in person. Scott et al. (2021) or Esposito et al. (2021), in this case, applied to middle and high school students in Italy, reach the same conclusions in terms of physical and mental health impairment or sadness due to the loss of daily relationships with friends. However, some young people may have benefited from the containment measures by spending more time with parents or caregivers and improving family relationships, or avoiding stressors related to the school environment (Raw et al., 2021). Lessard and Puhl (2021) show through a study with American adolescents that about 25% of students thought that bullying had decreased since the beginning of the pandemic. Along the same lines, Vaillancourt et al. (2021), after surveying more than 6500 Canadian students, found that before the pandemic, 60% of students reported being bullied at school, whereas during the pandemic this figure was 40%. In this regard,

74

M. Cuerdo-Mir and L. M. Doncel-Pedrera

Sainz et al. (2021) describe studies from the field of educational economics on the impact of school closures on student well-being. Bacher-Hicks et al. (2022) point out that in-person interaction is an important mechanism underlying not only bullying but also cyberbullying. Indeed, areas in the US where more schools restarted face-to-face classes earlier saw a greater return to pre-pandemic levels of searching for bullying-related terms. That is, school lockdown and closure did decrease the prevalence of bullying, predictably in the absence of physical interaction between students, but it is not clear what happened to cyberbullying. This is further evidence that the impact of the pandemic on student wellbeing, while negative overall, has not been linear and in some cases has benefited some students. At this point, it is worth noting that the particularity of the cyberbullying phenomenon lies in the anonymity of the perpetrator (Bonanno & Hymel, 2013), and that most cyberbullying victims also suffer from direct bullying (Olweus & Limber, 2018). Jain et al. (2020) show that in India around 80% of those students who had been cyberbullied during the pandemic had also been bullied face-to-face prior to the pandemic. Mota et al. (2021), in a study on the mental health of university students, indicate that the effect of isolation due to the disruption of face-to-face teaching and increased computer use due to the pandemic has increased the likelihood of being involved in cyberbullying. Barlett et al. (2021a) and Barlett et al. (2021b) suggest that there is a correlation between cyberbullying and Covid-19 and that attitudes, behaviour, and beliefs about cyberbullying have altered during the period of educational closure due to the pandemic. The anonymity afforded by the internet and social networks favours the phenomenon of cyberbullying. McHugh et al. (2019) conduct a study on the tremendous effect of cyberbullying on high school students through Twitter by considering Twitter as an “intentionally aggressive channel of communication”. Karmakar and Das (2020) and Babvey et al. (2021) conduct similar studies with the same platform and conclude that there has been an increase in aggression and hatred in the content of tweets once the educational shutdown occurred. In contrast, Perez and Karmakar (2022) study the phenomenon of cyberbullying through Twitter during the pandemic period using a natural language processing model that analyses one million tweets from the beginning of 2019 to the end of 2021 and whose content uses a term related to bullying. Their analysis does not detect a significant behavioural change in messages before and after the Covid-19 pandemic. A similar conclusion of not increasing in aggression during Covid-19 period was reached in a study by professors Bacher-Hicks et al. (2022), who analysed, based on Google searches for bullying-related terms, the evolution of bullying in the US before, during, and after confinement. According to their study, one in five American high school students reports having been bullied in school, a situation that increases the risk of mental health problems even in adulthood. The research plots, in an event study framework, monthly deviations of trends in the intensity of bullying searches, with February 2020 as a benchmark. According to the study, search intensity for both face-to-face and social media forms of bullying declined substantially in spring

The Impact of the School Closures on Bullying and Cyberbullying in Spain

75

2020, returned to or slightly above its usual low levels during the summer, and then fell again in autumn 2020. The reduction in bullying in the US, also found by Bacher-Hicks et al. (2022), during the partial return to face-to-face classes in the 2020–21 academic year, may be related to the substantially more structured hours and schedule than in previous years or to the implementation of public health measures, such as social distancing, face masks and attempts to create bubbles of students living together in different classrooms, which significantly restricted the number of interactions that students might otherwise have experienced. The reopening of schools also saw increased monitoring of student-to-student interactions by education professionals compared to the pre-pandemic period, including during lunch, recess, and movement between classrooms—times and spaces that are sometimes where students feel less safe and are more likely to experience harassment. Therefore, understanding how bullying and cyberbullying in Spain have evolved during the pandemic period, and comparing different periods before, during, and after the government closure of schools can provide useful guidelines for the design of the most efficient (in terms of costs) and effective (in terms of objectives) policies. So far, progress has been made on some specific aspects of the effects of Covid-19 on the school population. For example, in the case of Spain, a detailed examination has been carried out, by Sanz-Labrador et al. (2021), on the learning effort through digital resources in the period of school closure, based on data provided by Google Trends. The use of these digital resources was unparalleled in previous periods, although this level of practise of online resources was not maintained once schools opened their doors. In the same way, Brodeur et al. (2021) studied the impact of school closures on the well-being levels of schoolchildren based on Google Trends data. They observed that the search intensity of terms such as “boredom”, “loneliness”, “worry” or “sadness” shot up and showed for the first of these terms, with a Difference-in-Difference (DiD) method, increases of more than twice the standard deviation in Europe and more than one standard deviation in the case of the United States. With this study’s hypothesis about whether the results, for the Spanish case, are in line with the conclusions obtained for the USA by Bacher-Hicks et al. (2022), in the next section, Data and methods used in this research are described. Section 3 exposes the results of estimations whereas general conclusions are depicted in Sect. 4.

2 Methods and Data 2.1 Data The empirical analysis relies on the data provided by Google Trends, a free tool developed by Google Labs to identify search patterns. The firm makes available indicators for internet searches worldwide based on a sample of the searches conducted. Data

76

M. Cuerdo-Mir and L. M. Doncel-Pedrera

provided are anonymized, categorized, aggregated, and presented not in terms of gross volume of searches but relativized. In particular, Google Trends normalized search data by dividing each data point by the total searches of the geography and time range and represents to account for relative popularity, obtaining a sort of “search intensity”. Then, the resulting number is scaled on a range of 0 to 100 in terms of the proportion of the topic compared to all searches on all topics (Google, 2022). Thus 100 shows the highest proportion of searches for that topic relative to all searches, along the period of reference. The analysis of bullying and cyberbullying will be conducted using the results for the topic “Acoso escolar” (that could be translated as bullying in schools, although not necessarily within the school) and for the terms “Bullying” and “Cyberbullying”. While the results for “terms” show the searches for that specific word, the topic shows results for similar keywords in other languages. Using this strategy, we extract weekly results for the period September 2017-August 2022 for all the Spanish regions, except for the term “Cyberbullying” for which there are data for 14 of the 19 regions. The other 5 regions are the least populated regions in Spain, which has an impact on the availability of data (Cantabria, Navarra, La Rioja, Ceuta y Melilla). The use of the Google Trend intensity indicator provides detailed data and comparable data for regions, with high frequency since 2004. This allows the application of statistical strategies that improve the robustness of the conclusions. Moreover, as it is self-reported it does not suffer from interviewer biases. But it might suffer from a digital bias, as Google is a digital tool and thus only people with internet access and a device to conduct the search will be represented in the data. In this sense, the share of children aged 10–15 years that have used the internet in Spain has steadily increased from 76.9% in 2007 to 97.5% in 2021, being 92.9% in 2019 as a reference previous to the 19 Covid-pandemic (INE, 2022). Regarding access to the internet by the level of income, in 2021, 89% of the families with less than 900e have access to the internet, a percentage that reaches 100% for family incomes above 2,500e (INE, 2022). Thus, we can assume that in global terms people have access to search for information on the internet in the observed period (2017–2022). Finally, the available data only shows aggregate trends, without detail about the people that conduct the searches. This limitation eliminates the possibility of a deeper understanding of the profiles and patterns underneath the global results. A crucial assumption in the use of internet searches is that it constitutes a proxy for the bullying that people suffer. In this sense, Bacher-Hicks et al. (2022) analyse the predictive validity of internet search intensity for actual bullying using a survey focused on teenager students that includes bullying related questions. They observe that self-reported bullying rates are coherent with the online search intensity reported by Google trends. We will assume in this work that this relationship might be also valid for the Spanish case. Moreover, the stational behavior of internet searches in Spain matches the school calendar (Fig. 1). The Spanish academic year is divided into three terms (trimestres): from September to December, from January to March, and from April to June. The results for searches for “Acoso escolar” show two interesting global patterns. First, there is a clear change in the relative intensity starting with the closure of

The Impact of the School Closures on Bullying and Cyberbullying in Spain

77

Fig. 1 Nationwide weekly relative search intensity for “Acoso escolar”, “bullying” and “cyberbullying”. Note The vertical line shows the beginning of the closure of schools (March 13, 2020). The dashed lines at the first graph show the beginning of each academic year (beginning of September)

78

M. Cuerdo-Mir and L. M. Doncel-Pedrera

schools, but that affects also the academic year 2020–2021. Spanish schools reopened with a 100% onsite system in primary schools and a mixed system of classes in high schools that combine online and onsite classes in public schools, while in charted and, especially, in private schools there was a higher share of onsite classes. In all cases, students were organized in bubble groups. Second, a general view to the values for the searches supports the idea that the bullying among students decreased along the closure period and during the academic year 2020–2021, but the following year, when schools when back to fully onsite classes and relaxed the Covid protocol, search intensities experience higher relative values. In terms of seasonal patterns, search intensity values decrease along the summer periods and increase at the beginning of September, when classes start. The intensity search usually remains above average values along the first term until Christmas holidays (from December 22 to January 7). Curiously, as the return to classes comes closer, the intensity increases within the last week of Christmas holidays. There is also an Easter effect in Spain, which starting point match a point of lowest search intensity. After that holiday period, intensity peaks around the end of April-beginning of May, followed by a steady decrease to its lowest values in summer. Values for search intensities about “Bullying” show lower dispersity, but the patterns are similar to those in “acoso escolar”, with which it has a correlation coefficient of 0.96. Moreover, it also shows an increase in the dispersion along the academic year 2021–2022, especially in terms of the upper values. The term “Cyberbullying” does not seem to be a popular search in Spain, as there are several moments with “zero” values including summer periods, which means there are not enough data for that week. The stationarity and other patterns are less clear than for “Acoso escolar”.

2.2 Methods The study of how the closure of schools during covid-19 pandemic affected the patterns of bullying (by means of the results for the topic “acoso escolar” and the term “Bullying”) and cyberbullying among students is approached following the strategy used at Bacher-Hicks et al. (2022). Using event study techniques, we analyse changes in the relative search intensity before and after the closure of schools in Spain (March 11, 2020). We design a monthly event study strategy that provides an estimation of the average effect that the closure has on bullying relative search intensity in internet, our objective variable. The first step is to eliminate trend and seasonal patterns in the objective variable. In our previous section, we observed seasonal changes in holidays (summer, Christmas and Easter) as well as some positive peaks within the school terms. There seems to be also a slightly decreasing trend in internet searches on the analysed topic and terms. Equation 1 shows the regression of the logarithm of our objective variable (lsearchrt ) for each region (r) and week (t).

The Impact of the School Closures on Bullying and Cyberbullying in Spain

lsear ch r t = βyeark + γm M + δr R + εr t

79

(1)

where β captures a linear time trend in the years before COVID-19, γm captures the effect of a set of 12 monthly dummies (M) while δr does it for a set of dummies for each Spanish region (R). The term ε refers to the random error term of the equation. The prediction obtained is used to calculate a measure of the excess of relative search intensity, lsearch2rt , as the difference between the real and predicted logarithm of search intensity in a given region-week. This new measure shows by how much the relative search intensity deviates from the predicted value based on pre-closure time trends, and month and region effects. The second step uses the variable lsearch2rt to conduct an event study model with panel data, as described in Eq. 2. lsear ch2r t =

−1  m=−12

βm Be f or em +

12 

γm A f term + αm Other Y ear sm + δr R + εr t

m=1

(2) where as in Eq. 1, r indicates the region and t the week of the event, meaning the last week before the closure of schools. The equation includes three sets of temporal dummy variables. Beforem and Afterm are dummy variables for the week “t”, differentiating whether it is before or after the event and taking as a reference period 12 months before and after the event (thus, March 2020 is excluded). OtherYearsm includes 12 dummies for months different from the period analysed in Beforem and Afterm . Besides those time variables, the equation includes a set of regional dummies, Rr . Therefore, the coefficients for γm will capture the average effect (for the four weeks observed in each month) of the m-month deviation from the calendarpredicted relative search intensity for March 2020 in the region r. Moreover, that effect is net from other circumstances that could affect that month in previous years (as those effects will be included in other dummies) and region-specific factors, included in the regional dummies. The same logic applies to the coefficients for the months included in Beforem , βm . According to the literature we would expect to obtain negative values for the coefficients γm or if positive, lower than those obtained for the period before the closure of the schools (βm ).

3 Results Table 1 shows the results of the estimation in Eq. (1). We have data week by week for the 17 Regions (called Comunidades Autónomas) in Spain, plus Melilla and Ceuta. In the case of searches for the term Cyberbullying, we do not have information for 5 of the Spanish Regions (Cantabria, Navarra, La Rioja, Ceuta y Melilla) because the low number of searches leads to Google trends to avoid providing this data.

80 Table 1 Estimation of Eq. 1: Searches in Google Trends in Spain and its Regions of the terms “Acoso escolar”, “Bullying” and “Cyberbullying” between September 2017 and December 2019

M. Cuerdo-Mir and L. M. Doncel-Pedrera

Variables

Acoso escolar

Bullying

Cyber-bullying

February

1.215

− 0.364

1.763**

(0.812)

(0.840)

(0.855)

− 0.949

0.508

0.374

(0.789)

(0.817)

(0.831)

0.420

0.761

0.603

(0.789)

(0.817)

(0.831)

1.283

1.172

0.530

(0.812)

(0.840)

(0.855)

− 0.199

− 0.461

− 0.0881

(0.789)

(0.817)

(0.831)

− 2.551***

− 2.271***

− 0.933

(0.789)

(0.817)

(0.831)

− 2.677***

− 2.859***

− 0.663

(0.812)

(0.840)

(0.855)

− 0.744

− 0.0869

− 0.0218

(0.726)

(0.751)

(0.764)

0.371

0.315

0.915

(0.741)

(0.767)

(0.780)

1.326*

1.227

1.446*

(0.749)

(0.776)

(0.789)

− 0.516

− 0.0616

0.0412

(0.719)

(0.745)

(0.758)

− 0.204

0.146

0.0425

(0.223)

(0.231)

(0.235)

− 1.160

− 3.371***

− 8.388***

March April May June July August September October November December Trend Constant

(0.801)

(0.829)

(0.844)

Observations

2318

2318

1708

R-squared

0.030

0.022

0.013

Standard errors in parentheses. ***p < 0.01, **p < 0.05, *p < 0.1

Following Bacher-Hicks et al. (2022) estimation 1 includes only data for weeks before the year 2020. Thus, in the case of searches for the term “Acoso escolar” and Bullying we have 2 years (from September 2017 to September 2019) and the 18 weeks between September 2019 and the end of December 2020. That is, we have 52 weeks for two years (104 weeks) and 18 additional weeks of the period September 2019-December 2019. Overall, that makes 122 weeks multiplied by 19 Regions (17 Comunidades Autónomas plus Melilla and Ceuta) sums up to 2318 observations. In the case of Cyberbullying we have 14 Regions multiplied by 122 weeks makes 1708 observations. Results in Table 1 shows that July and August are the months in which

The Impact of the School Closures on Bullying and Cyberbullying in Spain

81

there are a lower number of searches for “Acoso escolar” and bullying with respect to January, the omitted month. Interestingly, July and August also have a negative coefficient associated in the case of searches in Google for Cyberbullying, but these coefficients are not significant. That is, there is a significant reduction in searches for “Acoso escolar” and Bullying in Spain in July and August. Cyberbullying also shows an important reduction which is according with the idea that Cyberbulling is influenced by physical presence as stated by Bonnano and Hymel (2013), Olweus and Limber (2018) or Bacher-Hicks et al. (2022). Among the school calendar year, it seems that November and February are the months with more searches for Cyberbullying and “Acoso escolar”. These are the coldest months in Spain and both of them are full in terms of teaching in the sense that there is not holiday’s period in them. There might be, though it needs much more research, an association between the weather and bullying in schools. Table 1 also shows that there is not any significant trend in searches for bullying in Spain related terms before the Covid. This is an important result, because it means that any change in the bullying trend after Covid should be related to this pandemic rather than to other factors taking place before March 2020. In addition, there is a significant decrease for “Acoso escolar” in summer. The data in Spain seem to point to a climate effect on bullying activity and also seem to be in line with Duckworth et al (2021) and Mota et al (2021) in the sense that greater socialisation of students reduces the phenomenon of Cyberbulling. Thus, in times of greater isolation as face-to-face interaction outside school hours decreases due to the cold weather, in November and February, cyberbullying significantly increases. It should be remembered that in the other two months with more adverse weather conditions due to lower temperatures, December and January, the existence of the Christmas holidays conditions the result of bullying as school activity ceases for a week or so. Likewise, this increase in cyberbullying seems to be accompanied by an increase in face-to-face “acoso escolar”, as this term shows remarkable values on the same dates. Table 2 shows the results of estimating Eq. (2). The dependent variable is the deviation of searches for bullying terms in Spain in a particular month with respect to the usual searches in this same month between September 2019 and December 2020. Consistently with the finding of Table 1, we do not discover any significant effect on the searchers for bullying in the 12 months before the pandemic. Between February 2019 and January 2020, the 12 months before the pandemic, there was not any difference in searches on Google for bullying than in the same month of previous years. That is, there was not any other factor affecting the trend in searches for “acoso escolar”, “bullying” or “cyberbullying” in Spain before the Covid 19. This feature will allow us to identify any changes after the pandemic as probably caused by the school lockdowns. In the 12 months after the Covid 19 surges, there is a negative and significant effect in searches for “acoso escolar” in Spain for the 3 months: May 2020, December 2020, and February 2021. As indicated in Sect. 2, these are periods, the ends of April, November, and January and the beginnings of May, December, and February, with a traditional effect in “Acoso escolar” and Cyberbullying searches.

82

M. Cuerdo-Mir and L. M. Doncel-Pedrera

On the contrary, July usually shows an important reduction in “Acoso escolar” and Cyberbullying. However, for the month of July 2020, there is also a positive impact. In this sense, it is relevant to indicate that whereas July was included in the summer holidays in other years in July 2020 due to the effect of Covid-19 and the delay in starting remote teaching in March 2020, there was an extension of two additional weeks in July 2020 especially aimed at stragglers students. Our evidence is close to Bacher-Hicks et al. (2022) findings for the US. There is a significant reduction in searches for “acoso escolar” after the pandemic. In fact, this impact is still in place in December 2020 and February 2021, almost one year later. The effect of the school lockdowns in reducing bullying in Spain is not fading out. On the contrary, the impact is lasting and it is, at least, of the same magnitude in December 2020 (− 1.687) and February 2021 (− 1.709) than in May 2020 (− 1.517) right after the surge of the Covid. We do not find any significant change for the searches for Bullying or Cyberbullying. These null effects for these two terms might be explained by the fact that these terms are in English, and students and their families might look for the Spanish term “acoso escolar”. We also find that the prevalence of bullying is quite different among Spanish Comunidades with most Region dummies being significant. In this sense, it is noteworthy that Andalusia, Catalonia, Madrid and Valencia, the four most populated regions of Spain in that order, show a similar pattern of behaviour concerning the terms “school bullying” and “bullying”. However, in terms of “cyberbullying”, while the first 3 regions continue with similar behaviour, in Valencia, a differentiated performance does appear.

4 Conclusion Bullying and Cyberbullying are two growing problems in today’s societies. This has important effects on public health in addition to the effects on the proper development of the school population. However, an event as unique and global in scale as Covid-19, which involved the government shutdown of schools, raises the question of whether there has been any transitory or permanent effect on these phenomena. The hypothesis put forward in this research has been to check whether the results, for the Spanish case, are in line with the conclusions obtained for the USA by Bacher-Hicks et al. (2022). Preliminary results show a drop in the relative intensity of searches for the concepts of “Acoso escolar”, “Bullying” and “Cyberbulling” in Spain, although this phenomenon of reduction appears to be transitory. When analysing the deviations from the existing patterns throughout the academic year, a decrease in search intensity was observed in the period after Covid-19. In the 20–21 academic year this phenomenon of reduction continued, although it should be remembered that in this academic year there was an overlap of centres with hybrid classes and other face-to-face classes. The variation in search intensity was recovered in the 21–22 academic year with the return to full face-to-face classes.

The Impact of the School Closures on Bullying and Cyberbullying in Spain

83

Table 2 Estimation of Eq. 2: Deviation in searches in Google Trends in Spain and its Regions of the terms “Acoso escolar”, “Bullying” and “Cyberbullying” between September 2017 and August 2022 Deviation in searches for “acoso escolar”

Deviation in searches for “Bullying”

Deviation in searches for “Cyberbullying”

− 0.940

− 0.00426

− 1.662

(0.967)

(1.008)

(1.205)

0.0615

0.407

− 1.303

(0.917)

(0.956)

(1.143)

April 2019

− 1.252

− 1.297

0.800

(0.967)

(1.008)

(1.205)

May 2019

− 0.303

0.496

− 0.760

(0.967)

(1.008)

(1.205)

− 0.149

0.372

− 1.496

(0.917)

(0.956)

(1.143)

July 2019

− 0.129

0.0273

− 1.166

(0.967)

(1.008)

(1.205)

August 2019

–

–

− 0.0930

Variables Before Covid February 2019 March 2019

June 2019

(1.205) − 0.976

0.248

1.415

(0.917)

(0.956)

(1.143)

October 2019

0.970

1.048

− 0.391

(0.967)

(1.008)

(1.205)

November 2019

0.663

0.402

− 0.0958

(0.967)

(1.008)

(1.205)

− 0.150

0.578

− 0.796

(0.917)

(0.956)

(1.143)

− 0.0734

0.963

0.534

(0.967)

(1.008)

(1.205)

1.380

0.877

− 1.443

(0.917)

(0.956)

(1.143)

April 2020

− 0.462

− 0.259

0.370

(0.967)

(1.008)

(1.205)

May 2020

− 1.517*

− 1.287

− 0.936

(0.917)

(0.956)

(1.143)

− 1.030

− 1.227

− 1.295

(0.967)

(1.008)

(1.205)

1.654*

− 0.825

− 0.152

September 2019

December 2019 January 2020 After Covid March 2020

June 2020 July 2020

(continued)

84

M. Cuerdo-Mir and L. M. Doncel-Pedrera

Table 2 (continued) Variables

Deviation in searches for “acoso escolar” (0.967)

(1.008)

(1.205)

August 2020

1.331

− 0.0540

0.472

(0.917)

(0.956)

(1.143)

0.906

− 0.891

September 2020

Deviation in searches for “Bullying”

Deviation in searches for “Cyberbullying”

(0.967)

(1.008)

October 2020

− 0.737

1.186

(0.967)

(1.008)

(1.205)

November 2020

− 0.603

− 0.300

− 1.190

(0.917)

(0.956)

(1.143)

− 1.687*

− 1.587

− 0.581

(0.967)

(1.008)

(1.205)

January 2021

− 0.953

− 0.579

1.677

(0.917)

(0.956)

(1.143)

February 2021

-1.709*

1.278

− 1.280

(0.967)

(1.008)

(1.205)

January 2018, 2019, 2022

0.119

0.334

− 0.344

(0.782)

(0.815)

(0.974)

February 2018 & 2022

1.179

2.444***

− 1.248

(0.789)

(0.823)

(0.984)

March 2018, 2021 & 2022

1.914**

0.318

− 0.770

(0.789)

(0.823)

(0.984)

April 2018, 2021 & 2022

0.832

0.327

− 0.558

(0.782)

(0.815)

(0.974)

May 2018, 2021 & 2022

0.221

− 0.150

− 0.460

(0.775)

(0.808)

(0.966)

June 2018, 2021 & 2022

− 0.135

0.252

− 0.0615

(0.789)

(0.823)

(0.984)

July 2018, 2021 & 2022

0.818

0.192

− 0.592

(0.775)

(0.808)

(0.966)

August 2018, 2021 & 2022

− 0.132

0.339

− 1.363

(0.782)

(0.815)

(0.974)

September 2017, 2018 & 2021

− 0.560

− 0.741

− 0.321

(0.782)

(0.815)

(0.974)

October 2017, 2018 & 2021

− 0.407

− 0.484

− 0.549

(0.775)

(0.808)

(0.966)

December 2020

− 0.119

(continued)

The Impact of the School Closures on Bullying and Cyberbullying in Spain

85

Table 2 (continued) Variables

Deviation in searches for “acoso escolar”

Deviation in searches for “Bullying”

Deviation in searches for “Cyberbullying”

November 2017, 2018 & 2021

− 0.503

0.204

− 0.886

(0.789)

(0.823)

(0.984)

December 2017, 2018 & 2021

− 0.0920

− 0.212

− 0.647

(0.775)

(0.808)

(0.966)

Aragón

− 5.821***

− 7.269***

− 2.957***

(0.522)

(0.544)

(0.558)

Asturias

− 6.851***

− 7.936***

− 3.542***

(0.522)

(0.544)

(0.558)

− 7.002***

− 7.414***

− 3.271***

(0.522)

(0.544)

(0.558)

Canarias

− 3.113***

− 4.427***

− 3.373***

(0.522)

(0.544)

(0.558)

Cantabria

− 9.044***

− 9.127***

(0.522)

(0.544)

− 3.498***

− 6.030***

(0.522)

(0.544)

(0.558)

Castilla León

− 2.695***

− 3.986***

− 3.056***

(0.522)

(0.544)

(0.558)

Cataluña

0.0362

0.255

− 0.991*

(0.522)

(0.544)

(0.558)

Ceuta

− 11.51***

− 11.12***

(0.522)

(0.544)

− 6.634***

− 8.588***

(0.522)

(0.544)

(0.558)

Galicia

− 2.392***

− 3.886***

− 3.113***

(0.522)

(0.544)

(0.558)

Madrid

− 0.0878

0.104

− 0.491

(0.522)

(0.544)

(0.558)

− 5.268***

− 6.773***

− 2.452***

(0.522)

(0.544)

(0.558)

Melilla

− 11.89***

− 11.10***

(0.522)

(0.544)

Navarra

− 8.400***

− 8.933***

(0.522)

(0.544)

Andalucía

Baleares

Castilla La Mancha

Extremadura

Murcia

− 3.341***

− 3.081***

(continued)

86

M. Cuerdo-Mir and L. M. Doncel-Pedrera

Table 2 (continued) Variables

Deviation in searches for “acoso escolar”

Deviation in searches for “Bullying”

Deviation in searches for “Cyberbullying”

País Vasco

− 2.801***

− 3.840***

− 2.121***

(0.522)

(0.544)

(0.558)

La Rioja

− 10.53***

− 10.56***

(0.522)

(0.544)

Valencia

− 0.477

− 0.477

− 2.384***

(0.522)

(0.544)

(0.558)

5.130***

5.471***

3.160***

(0.772)

(0.805)

(0.933)

Observations

4959

4959

3654

R-squared

0.303

0.276

0.038

Constant

Standard errors in parentheses ***p < 0.01, **p < 0.05, *p < 0.1

In relation to cyberbulling, our results are mostly in line with those expressed with Bacher-Hicks et al. (2022) in the sense that no change in the pattern of behaviour of this phenomenon can be seen, as well as the parallelism between Cyberbulling and physical presence in the educational centre. One aspect to highlight is the importance of socialisation and climatic conditions. Thus, less socialisation due to more adverse weather conditions seems to increase cases of cyberbulling and bullying. Finally, although at the beginning of the pandemic there was a common state regulation for all the Autonomous Communities, in the management of the return to the physical presence there was a diversity of behaviour in relation to factors such as, for example, the different regional regulations both in terms of the education sector and in other sectors of activity. This may have affected the socialization of students and thus the results with regard to bullying behaviour in the different Autonomous Communities. An interesting field of study is thus emerging, as is the influence that climatic conditions may have on the effect of “bullying” and cyberbullying.

References Alannah and Madeline Foundation. (2018). The economic cost of bullying in Australian schools. March 2018. https://www.amf.org/au/media/2505/amf-report-280218-final.pdf Babvey, P., Capela, F., Cappa, C., Lipizzi, C., Petrowski, N., & Ramirez-Marquez, J. (2021). Using social media data for assessing children’s exposure to violence during the COVID-19 pandemic. Child Abuse & Neglect, 116. https://doi.org/10.1016/j.chiabu.2020.104747 Bacher-Hicks, A., Goodman, J., Green, J. G., & Holt, M. K. (2022). The COVID-19 pandemic disrupted both school bullying and cyberbullying. American Economic Review: Insights, 4(3), 353–370. https://doi.org/10.1257/aeri.20210456

The Impact of the School Closures on Bullying and Cyberbullying in Spain

87

Barlett, C. P., Rinker, A., & Roth, B. (2021a). Cyberbullying perpetration in the COVID-19 era: An application of general strain theory. The Journal of Social Psychology, 161(4), 466–476. https:// doi.org/10.1080/00224545.2021.1883503 Barlett, C. P., Simmers, M. M., Roth, B., & Gentile, D. (2021b). Comparing cyberbullying prevalence and process before and during the COVID-19 pandemic. The Journal of Social Psychology, 161(4), 408–418. https://doi.org/10.1080/00224545.2021.1918619 Basile, K. C., Clayton H. B., DeGue S., & Gilford, J. W. (2020). Interpersonal violence victimization among high school students—Youth risk behavior survey, United States, 2019. US Department of Health and Human Services/Centers for Disease Control and Prevention, MMWR, August 21, 69(1), 28–37. Bonanno, R. A., & Hymel, S. (2013). Cyber bullying and internalizing difficulties: Above and beyond the impact of traditional forms of bullying. Journal of Youth Adolescence, 42, 685–697. https://doi.org/10.1007/s10964-013-9937-1 Brodeur, A., Clark, A. E., Fleche, S., & Powdthavee, N. (2021). Covid-19, lockdowns and wellbeing: Evidence from Google Trends. Journal of Public Economics, 193, 104346. Cornell, D. G., & Brokenbrough, K. (2004). Identification of bullies and victims: A comparison of methods. Journal of School Violence, 3, 63–87. Currie, C., Zanotti, C., Morgan, A., Currie, D., de Looze, M., Roberts, C., Samdal, O., Smith, O. R. F., & Barnekowet, V. (2012). Social determinants of health and well-being among young people. Health behaviour in school-aged children (HBSC) study: International report from 2009–2010 survey. Health Policy for Children and Adolescents, No. 6, Copenhagen: WHO Regional Office for Europe https://apps.who.int/iris/handle/10665/326406 Duckworth, A. L., Kautz, T., Defnet, A., Satlof-Bedrick, E., Talamas, S., Lira, B., & Steinberg, L. (2021). Students attending school remotely suffer socially, emotionally, and academically. Educational Researcher, 50(7), 479–482. https://doi.org/10.3102/0013189X211031551 Esposito, S., Giannitto, N., Squarcia, A., Neglia, C., Argentiero, A., Minichetti, P., Cotugno, N., & Principi, N. (2021). Development of psychological problems among adolescents during school closures because of the COVID-19 lockdown phase in Italy: A cross-sectional survey. Frontiers in Pediatrics, 8, 628072. https://doi.org/10.3389/fped.2020.628072 Fraguas, D., Díaz-Caneja, C. M., Ayora, Durán-Cutilla, M., Abregú-Crespo, R., Ezquiaga-Bravo, I., Martín-Babarro, J., & Arango, C. (2021). Assessment of school anti-bullying interventions. A meta-analysis of randomized clinical trials. JAMA Pediatrics, 175(1), 44–55. Google (2022). Google Trends. https://trends.google.com/. Accessed September 1, 2022. Hamm, M. P., Newton, A. S., Chisholm, A., Shulhan, J., Milne, A., Sundar, P., Ennis, H., Scott, S. D., & Hartling, L. (2015). Prevalence and effect of cyberbullying on children and young people: A scoping review of social media studies. JAMA Pediatrics, 169(9), 770–777. Hawes, M. T., Szenczy, A. K., Klein, D. N., Hajcak, G. & Nelson, B. D. (2021). Increases in depression and anxiety symptoms in adolescents and young adults during the COVID-19 pandemic. Psychological Medicine, 1–9. Advance online publication. https://doi.org/10.1017/S00332917 20005358 Hymel, S., & Swearer, S. M. (2015). Four decades of research on school bullying. American Psychologist, 70(4), 293–299. INE. (2022). Survey on equipment and use of information and communication technologies in households. Results for several years. https://www.ine.es/index.htm. Accessed September 13, 2022. Jain, O., Gupta, M., Satam, S., & Panda, S. (2020). Has the COVID-19 pandemic affected the susceptibility to cyberbullying in India? Computers in Human Behavior Reports, 2, 100029. Jones, L. M., Mitchell, K. J., & Finkelhor, D. (2013). Online harassment in context: Trends from three youth internet safety surveys (2000, 2005, 2010). Psychology of Violence, 3, 53–69. Karmakar, S., & Das, S. (2020). Evaluating the impact of covid-19 on cyberbullying through Bayesian trend analysis. In Proceedings of the European interdisciplinary cybersecurity conference, pp. 1–6. https://doi.org/10.1145/3424954.3424960

88

M. Cuerdo-Mir and L. M. Doncel-Pedrera

Lessard, L. M., & Puhl, R. M. (2021). Adolescent academic worries amid COVID-19 and perspectives on pandemic-related changes in teacher and peer relations. School Psychology, 36(5), 285–292. https://doi.org/10.1037/spq0000443 McHugh, M. C., Sandra, L., Saperstein, S. L., & Gold, R. S. (2019). OMG U #Cyberbully! An exploration of public discourse about cyberbullying on Twitter. Health Education & Behavior, 46(1), 97–105. https://doi.org/10.1177/1090198118788610 Mota, D. C. B., Silva, Y. V. D., Costa, T. A. F., Aguiar, M. H. D. C., Marques, M. E. D. M., & Monaquezi, R. M. (2021). Mental health and internet use by university students: Coping strategies in the context of COVID-19. Ciência & Saúde Coletiva, 26, 2159–2170. https://doi.org/10. 1590/1413-81232021266.44142020 Olweus, D. (1999). Sweden. In P. K. Smith, Y. Morita, J. Junger-Tas, D. Olweus, R. Catalano, & P. Slee (Eds.), The nature of school bullying: A cross-national perspective (pp. 7–27). Guilford Press. Olweus, D. (1978). Aggression in the schools: Bullies and whipping boys. Hemisphere. Olweus, D., & Limber, S. P. (2018). Some problems with cyberbullying research. Current Opinion in Psychology, 19, 139–143. Orben, A., Tomova, L., & Blakemore, S. J. (2020). The effects of social deprivation on adolescent development and mental health. The Lancet Child & Adolescent Health, 4(8), 634–640. https:// doi.org/10.1016/S2352-4642(20)30186-3 Perez, C., & Karmakar, S. (2022). An NLP-assisted Bayesian time series analysis for prevalence of twitter cyberbullying during the COVID-19 pandemic. arXiv preprint arXiv:2208.04980 Racine, N., Cooke, J. E., Eirich, R., Korczak, D. J., McArthur, B., & Madigan, S. (2020). Child and adolescent mental illness during COVID-19: A rapid review. Psychiatry Research, 292, 113307. https://doi.org/10.1016/j.psychres.2020.113307 Raw, J., Waite, P., Pearcey, S., Creswell, C., Shum, A., & Patalay, P. (2021). Examining changes in parent-reported child and adolescent mental health throughout the UK’s first COVID-19 national lockdown. PsyArXiv. Disponible en https://psyarxiv.com/exktj/ Rigby, K., & Smith, P. K. (2011). Is school bullying really on the rise? Social Psychology of Education, 14, 441–455. Sainz, J., Sanz, I., & Doncel, L. M. (2021). Abandono educativo, bienestar emocional y pandemia. In A. Blanco, A. M. Chueca, J. A. López, & S. Mora (Coord.), Informe España 2021, pp. 187–232. ISBN 978-84-8468-903-4 Sanz-Labrador, I., Cuerdo-Mir, M., & Doncel-Pedrera, L. M. (2021). The use of digital educational resources in times of COVID-19. Social Media + Society, 7(3). https://doi.org/10.1177/205630 51211049246 Scott, S. R., Rivera, K. M., Rushing, E., Manczak, E. M., Rozek, C. S., & Doom, J. R. (2021). “I Hate This”: A qualitative analysis of adolescents’ self-reported challenges during the COVID-19 pandemic. The Journal of Adolescent Health: Official Publication of the Society for Adolescent Medicine, 68(2), 262–269. https://doi.org/10.1016/j.jadohealth.2020.11.010 Shahin, T. (2022). The impact of the COVID-19 on the socio-emotional well being of students. In I. Sanz, & J. Sainz (Eds.), New trends in educational assessment in Iberoamerican countries: Reviewing the effect of gender, pandemics, poverty and new technologies in education. Springer, In press. Smith, P. K., Mahdavi, J., Carvalho, M., Fisher, S., Russell, S., & Tippett, N. (2008). Cyberbullying: Its nature and impact in secondary school pupils. Journal of Child Psychology and Psychiatry, 49(4), 376–385. Vaillancourt, T., Brittain, H., Krygsman, A., Farrell, A. H., Landon, S., & Pepler, D. (2021). School bullying before and during COVID-19: Results from a population-based randomized design. Aggresive Behavior, 47(5), 557–569. Vaillancourt, T., McDougall, P., Hymel, S., Krygsman, A., Miller, J., Stiver, K., & Davis, C. (2008). Bullying: Are researchers and children/youth talking about the same thing? International Journal of Behavioral Development, 32, 486–495.

Virtual or Face-to-Face Education: What Have We Learned from the years of the Pandemic? Pedro Adalid Ruíz and Jesús García Laborda

1 Introduction The COVID-19 pandemic brought forward several issues in education between 2020 and today that have been conveniently addressed in a large number of papers. Among those, one common issue has been the apparent shift to online environments during the first semester of 2020 due to emergency learning and teaching. Although, most studies have proved an apparent success in local cases with limited samples and usually in middle or middle-high schools and college, very few papers have addressed situations of at risk or even unsuccessful experiences. Therefore, in order to address this topic, it is necessary to assess the differences between benefits and drawbacks between in-person and virtual education, it is necessary to assume a neutral perspective. This chapter will intend to consider the differences between both approaches and emphasizing their mutual independence. Lafford et al. (2021) stress that virtual teaching and learning processes are completely different, and also suggest that online teaching is independent from the learning approaches used in face-to-face situations. This paper will first describe both contexts first to compare them afterwards. Comparisons like this are never final but tentative and depend very much of different features among which the individual cognitive styles is not minor.

P. Adalid Ruíz Universidad CEU - Cardenal Herrera, Madrid, Spain J. García Laborda (B) Instituto Franklin - Universidad de Alcalá, Madrid, Spain e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_6

89

90

P. Adalid Ruíz and J. García Laborda

2 Face-to-Face Education Educational processes have evolved over millennia. Education has always existed, from the prehistoric ages when hunting was taught from generation to generation to the current methods when socio-cultural adaptations are the current trend. Education, however, became necessary in the classical age (Oelkers, 2001), first in Greece when mathematics and philosophy had the lead, to Rome where Greek slaves taught languages to their masters for war and business purposes. Over the centuries, education evolved socially, and it was not considered a matter of the most refined society progressively. Eventually, education became popular and a way to democratize and strengthen the citizenry (Grossman, 2008). Today, when schooling is compulsory until the age of 16 in most OECD countries, more formal, questions have arisen, such as: How do students learn? When will they learn? How does learning happen? Where is the learning? Do students have all the facilities they need? What are the real benefits of education? Among others, which involve several fields such as philosophy, pedagogy, psychology, and even sociology. Research in these disciplines has generated several theories that attempt to define learning as a unidirectional process towards the fulfilment of the students’ needs. Currently, learning is understood as the process of acquiring specific skills, assimilating information and/or adopting new knowledge and behavior strategies, to cause a transformation in the subject and in the environment in which it develops. It is also important to keep in mind that the learning process goes beyond schooling. However, it is within the school context where students learn to relate to other social groups that have different traditions, customs, or beliefs (Saygin & Karakas, 2021). In this way, the school becomes the institution that facilitates the full development of the pupil until he reaches the ontological condition of becoming what each person potentially should be, in relation to the others. However, little is really known about how online or virtual students (for the purpose of this paper, both terms will be used indistinctly) interact or, even more importantly, whether there is a connection between the lack of development of social skills and individual virtual learning beyond the emotional support in learning difficulties (Pérez et al., 2022; Suprabha & Subramonian, 2021). The teaching and learning process in face-to-face settings allows the students to discover different realities. For example, the permanent presence of the mentor in the classroom enables the learner to recognize disorders that interfere with learning and recognize possible motor, mental, maturational, emotional and sociocultural causes that influence students. Similarly, during the international pandemic confinement in 2020, through a school guidance team it was possible to guide parents to provide efficient support and achieve adequate educational progress (Alsarayreh et al., 2022) but not always and, in some cases, parents were not able to cope with the stress produced by that situation (Nyanamba et al., 2022). The importance of face-to-face learning lies in the fact that the dynamics of development are two-way: you act in the world and the world acts towards you. The conditioning that occurs in the environment is related to the mutual interaction

Virtual or Face-to-Face Education: What Have We Learned …

91

(Vygotsky, 1978). This is because learners are not passive beings, but they continuously interchange their experiences by observing models that influence their motivations and emotions. Through this learning, students acquire certain social norms that will temper their behavior all through their lives. Another important aspect in the differentiation between virtual and face-to-face education is the capacity for communication among the educational stakeholders and subjects. This is because the communication process favors collaborative work. Inclassroom interaction favors an environment of trust and security that improves the communication process, but also allows linguistic confusion between senders and receivers to solve communication gaps. In fact, speech in the face-to-face context is not limited to words, but to non-verbal communication such as gestures, movements or body postures. Besides, socialization and social diversity make possible for students to develop social and moral values (Aririguzoh, 2022). Educational institutions offer physical spaces where people can establish relations of cooperation, solidarity, knowledge production and action through meaningful negotiations that consider the various points of view and create a democratic space where differences can be overcome.

3 Virtual Education The term virtual education does not have a terminological unanimity, so it is common to read that it is referred to as “training through the network”, tele-teaching”, “teletraining”, “e-learning” or “electronic learning”, “web-based learning or learning based on the WEB” or “online education”, all refer to non-face-to-face teaching, mediated by the use of information and communication technologies (ICT). Each of these concepts has different connotations and limitations. Thus, defining virtual learning will be related specifically to two ideas: delivery and learning design. For instance, many may consider that E-learning in virtual classrooms is used by teachers as an educational tool or a proper and distinctive way of teaching, or by students to achieve autonomous learning. However, problems arise in these new contexts when pedagogical phenomena related to school support arise, especially in students with learning difficulties in face-to-face schools. This might be so when teachers understand virtual learning as mere substitution or support of traditional in-person learning (Assi & Rashtchi, 2022) However, we consider that both virtual and face-to-face instruction are two different ecological systems. Obviously, the introduction of ICT in education has had an impact on the student’s learning process, on the teachers’ role, in the content of subjects and in assessment. The role of educational stakeholders in the production of significant learning has been in danger due to the lack of training in this field—more specifically during the confinement in 2020 (Flores et al., 2022), which has caused a great generational lag, and the digitally limited teachers face difficulties when teaching to their young techno-students of the twenty-first century. However, innovation in the educational system has been gradually implemented according to

92

P. Adalid Ruíz and J. García Laborda

each educational institution or individual practitioner’s resources (Núñez-Regueiro et al., 2022). Virtual learning through intelligent systems represents a change in the way information is obtained, and knowledge consolidated. This serves to describe the content of educational materials, monitor student check-ins, check-outs, and activities, and perform automated reviews of tests. IA has served to go a further step by serving to analyze deficiencies in education, special needs, distinctive education and more. However, the question of the role of the teacher in the technology-based classroom remains. This might be due to the fact that the goal of education is changing from support to full learning. The explanatory role of the teacher is now understood as that of a guide or materials designer or producer Kim et al. one thing is clear in this environment, autonomy and meaningful learning acquire new interpretations in virtual learning due to the proximity or more and more realistic materials. Virtual education benefits from the educational community by enabling the development of skills such as the organization of information, the management of new concepts and the expansion of language that favors communication and connectivity. Through virtual education, various units of information can be exchanged (including graphics, images, sound files, databases, among others). In this sense, the Internet offers infinite possibilities of connectionism and also allows for learning based on innovative methods adapted to the needs of the student. Online learning has a different way of socializing and motivating (Çoban & Göksu, 2022) through the use of resources such as blogs, wikis, Twitter, Facebook, WhatsApp, and others. Many of these forms have also been introduced into normal education through regular use such as WhatsApp (i.e. for pair/groupwork in virtual live classes) (Venturino & Hsu, 2022).

4 Distance Education Versus Face-to-Face Education from the Role of the Teacher Of course, face-to-face education is important for the first levels of schooling, for social relations and because motor skills are being worked on, which cannot be learned in a virtual classroom. This will also not be fully applicable for higher courses in which, in addition to theory, students need to master practical activities, such as nursing. Additionally, the role of the caretaker and the teacher are fundamental. It is true that from the beginning, the role of the teacher in face-to-face education has a direct and immediate effect, because apart from speech, non-verbal behavior is considered also another part of the teaching–learning interaction (for example through eye contact, posture, gestures and smiles with the students). Verbal actions such as talking about experiences outside of the classroom, use of humor, calling students by name, and praising student work and comments are included in this definition. However, most of these non-verbalized interactions seldom happen in virtual education.

Virtual or Face-to-Face Education: What Have We Learned …

93

The development of distance education has accelerated, but not all teachers, even some working in virtual education, have embraced the paradigm shift needed to successfully develop an online teaching skill. Since in virtual learning, students and teachers are separated in time and space, the online learning model has many challenges which include (but are not limited to) policy issues, process development, administration, technical support, technology access, legal issues, teacher preparation, performance, and student satisfaction. Motivation is probably one of the most significant ones. It is recognized that motivation in education is one of the most essential aspects that is hard but also necessary to transmit to the students. Many teachers on online courses are not well acknowledged by their students because although their materials can be excellent and the sessions adequate, students do not feel attached to their teachers, especially in places where both types of education are available. Motivation, personal interaction, and communication flow are thus three aspects that must be considered in virtual environments, as the demands on teachers using technology in the classroom will continue to grow. A fundamental point to try to help teachers overcome difficulties in using communication technologies, along with the possibility of providing training and support in the use of these technologies, is to address teacher motivation. Motivation is an expression of human behavior. Behaviors are functions determined by the interaction of people and situations, they occur because something is happening inside or outside the person, and that something is the reason. The behavior is a direct goal and is usually motivated by the desire to achieve a particular result, it is necessary to understand the motivations and needs that are driving certain behaviors at a given moment. Five basic motivational needs have been identified, including physiological needs (food, air, and water), safety needs (shelter), social needs, esteem needs, and selfactualization needs. Hierarchies exist on a continuous scale, and when people have at least partially met a need, they are motivated to work on the next need. Motivation is a factor that helps improve teaching performance when carrying out the teaching and learning process of distance education models. Teachers have to function in the digital world in the creation and distribution of content and resources, in a variety of situations and using a variety of devices. In this context, teaching in virtual classrooms is presented as a great challenge. Teachers who are trained and learn in face-to-face classes have little or no experience with virtual training and what it means to be a teacher in these virtual training rooms. The teacher who remains in constant training in ICT must be characterized by being oriented to the development of general knowledge and skills in ICT to integrate them into the educational process. The teacher must efficiently exploit ICT based on his instructional, research, and extension work, not limiting himself only to its use as a means of teaching, but as part of the information management strategies, dissemination of results, and the pedagogical process in the one found. Similarly, he must consider current issues emerging from his discipline, but without neglecting methodological and practical issues related to the process of student training. And he has to take into account the plurality of methods and the

94

P. Adalid Ruíz and J. García Laborda

various ways of teaching and learning, considering the role of problem-solving, self-learning, and cooperative work from the integration of ICT. By computerizing the training process, teachers must develop the following skills: verbal and written communication skills in a virtual environment; the design, selection, and production of digital content; and information management with the publication of results.

5 Distance Education Versus Face-to-Face Education from the Perspective of the Students The effectiveness of student performance in distance learning must consider four characteristics of distance learning: (a) the teaching and learning process, which includes activities where students are at a distance from their teachers; (b) the media, which include a combination of strategies, such as audio conferences, computers, fax, email, Internet, videotapes, among others: (c) knowledge and content, which are available from a variety of sources: (d) ubiquity, the delivery of the course can be offered at any time and place. Historically, distance learning played a complementary role to traditional modalities, but in recent times it has grown rapidly and has become an alternative ecosystem to school and university modalities, representing a training opportunity for those students who have little time to attend traditional courses or have few possibilities of accessibility to face-to-face education. Some of the features identified in distance learning students are: (a) the ability to work alone, (b) goal-oriented behavior, (c) good time management and (d) a high level of perseverance. Successful study requires the capacity for self-discipline and self-motivation. The lack of these skills can affect student satisfaction and reduce the likelihood of completing distance learning. However, although these characteristics are not exclusive to distance learning students, they can be identified as important prerequisites for choosing this modality. Students taking distance courses can be classified according to the way they approach learning the course content. The two approaches are called (a) the superficial approach and (b) the deep approach. The superficial approach to learning is a non-reflective methodology. Students who are unsure of what they are learning tend to focus on memorizing specific facts to supplement assignments and meet course requirements. While students using the deep approach focus on a deep understanding of the course content, they are more focused and selective in their learning. Students who have extensive and creative use of web tools tend to adapt better to the creative environments of the virtual classroom than to the traditional methods of face-to-face education.

Virtual or Face-to-Face Education: What Have We Learned …

95

6 Distance Education Versus Face-to-Face Education: Virtual Environments and Media Technical issues can have a negative impact on the success of students and teachers in online learning. Instructors need to pay more attention to keeping online students content by communicating more frequently and interacting with them to check their unique needs. It is important to choose virtual media carefully and conscientiously, according to the type of communication that you want to maintain based on the content to be developed. For example, if you want to create a sense of community, you can use chat rooms, which is the most common type of synchronous communication. The closest thing to a face-to-face class discussion is produced by synchronous communication, which can take place through online chat. However, being genuine in online conversations is a big challenge. The ideas and concepts offered by students can also be difficult to follow. As well as when some participants dominate the conversation and others may be distracted or want to go unnoticed during the chat. As can be seen, in synchronous media, there are some obstacles such as the ability to control the order in which participants read the messages they receive, the time it takes them to respond and the possible need to read a large number of messages and clarify what other participants are saying and thinking. Hence, students need opportunities to participate in synchronous and asynchronous communication tasks, for example, email, bulletin boards, or newsgroups, are within the type of asynchronous communication, in this, the participants will be able to follow more than a discussion and contribute to discussions without being interrupted. Likewise, virtual learning environments can be used, which allow students to access virtual information. Especially in the asynchronous mode of communication, students have more time to think about the material and respond meaningfully, just like in a traditional classroom. Asynchronous classes are independent of time and place and therefore can present their own pace and context for discussion. This improves student learning because it adheres to the principles of effective teaching, which are: 1. 2. 3. 4. 5. 6. 7.

Strengthen the connection between teachers and students Strengthen collaboration between students Strengthen active learning Provide quick feedback Prioritize task time Communicate with high expectations and Respect different talents and paths of knowledge.

Asynchronous learning environments, for example, access to times and periods that inhibit discussion, especially for students who travel or work full-time off campus, are not excluded from problems. Some students cannot ask questions about

96

P. Adalid Ruíz and J. García Laborda

how to proceed with the class or what they do not understand, because they cannot receive an answer right away. One advantage that the Internet provides is the perception of anonymity, which can be positive for those students who have physical or cultural limitations. And the can be a disadvantage, for those students who are prone to making negative comments or otherwise, since they behave without the restrictions of face-to-face communication, and without the non-verbal cues of body language, gestures, tone of voice, and facial expressions, students may misunderstand the comments and statements of their peers. Virtual or technology-mediated education, especially in virtual learning environments, marks the autonomy and independence of students in the learning process, their capacity for self-regulation, and the pace and time of learning. In this sense, students with low self-regulation capacity should pay special attention to the time they dedicate to study, their plans, and the proposed learning itineraries to achieve their learning objectives. The most successful students from a pedagogical and academic performance perspective in virtual environments are those who are more efficient and effective in their self-regulation processes. Going from face-to-face to virtual means adopting best practices in virtual learning environments. For this, the planning of curricular activities must be significant for the students, leading them towards participation, managing diversity, and the collaborative and supportive participation of all (Table 1). What does the research say about the differences between online learning and faceto-face learning The following diagram shows a specific comparison observed after the COVID-19 pandemic. In order to carry out this selection of research, we used the ERIC database since it is the most comprehensive database available. Initially, we used the terms “COVID-19”, and “virtual versus face-to-face education”. We obtained 50,352 hits. Although we considered it potentially risky, we reduced the search to just journals (8,886 hits) published in 2021 and 2022 (1952). Then we eliminated those that did not compare both ways of teaching. We also eliminated redundant results that actually reflected similar results. The results can be observed in the following table (Table 2). Table 1 Comparison of advantages and disadvantages of online education Advantages

Disadvantages

Promotes the use of ICT and development of new skills

You do not learn to socialize

Facilitates the creation and adoption of new content

There is no healthy competition

Adapts to individual time and pace of learning You need an Internet connection and electronic equipment Greater flexibility and lower cost

Preparation and training required

A variety of research resources are available

The student must have a lot of motivation and self-discipline

Virtual or Face-to-Face Education: What Have We Learned …

97

Table 2 Data obtained from the selection of sources Research

Topic

Gherhes et al. (2021)

Implementation during COVID-19 was inconsistent Students want to go back to college

Costado Dios and Piñero Charlo (2021)

Problems in implementation may lead students who are familiar with both types back to face-to-face

Abdullah et al. (2022)

Students usually respond to satisfaction which was hardly found during the confinement due to COVID-19

Beatty et al. (2022)

Students who study face-to-face outperform their colleagues in virtual learning in similar conditions

Caprara and Cataldo (2022)

Virtual learning requires an adequate environment; otherwise, it is likely to fail

Dervenis et al. (2022)

Teachers working in virtual environments require adequate preparation. Emergency teaching led to flaws due to lack of preparation

Freire and Rodríguez (2022)

When the appropriate conditions are met, changing both types of education delivery can be positive, especially in certain situations

Lin (2022)

Students learned more effectively when engaged with an instructor in a face-to-face classroom versus online learning although both methods led to similar results

Wahyuni et al. (2022)

Google Classroom as an effective way to replace in-classroom teaching

Ghosh et al. (2022)

No significant differences were found between virtual and face-to-face learning

Schulze et al. (2022)

Personality is the main trait that makes the difference among students especially in terms of acceptance

Carter and Youssef-Morgan (2022)

Notable advantages for online and micro-learning. Students did well in online learning

Valaitis et al. (2022)

The instructor’s attitude is among the most significant factors for virtual teaching to succeed

Khan et al. (2022)

Remote learning included more inclusivity, flexibility, availability of recorded sessions and time efficiency

Lee (2022)

The results indicated that the students perceived online learning as less effective than traditional face-to-face classes

Phillips (2022)

While students’ attitudes may not be positive, results in learning may still be positive

Zaghal et al. (2022)

Teachers believe that virtual learning is as effective as the face-to-face approach for novice learners. It is acceptable despite the challenges related to the remote learning of practical skills

Fouad et al. (2022)

Results indicate similar performance although students prefer face-to-face compared to online learning (continued)

98

P. Adalid Ruíz and J. García Laborda

Table 2 (continued) Research

Topic

Rice (2022)

Teachers facilitated the inclusive use of technologies by evaluating and modifying digital instructional materials. However, while some students appreciate the autonomy to choose technologies, they need to be adequately organized, sustained, and supported which may not be always the case

Ferrer et al. (2022)

Attitude to online learning mediated in both intrinsic motivation and extrinsic motivation. The design of online learning environments can play a role in enhancing learning experiences. Thus, it very much depends on the student’s personality and their acceptance of virtual learning

Alzamil (2021)

Participants: (a) had positive attitudes towards the importance of speaking English; (b) appreciated the benefits that online learning offers but felt it could not replace face-to-face learning

Lee (2022)

Students perceived online learning as less effective than traditional face-to-face classes overall but were satisfied with the customized instruction in terms of instructor’s feedback promptness, interaction among students, and effective design of tasks

7 Results Analysis Based on the above, it can be concluded that the strengths and weaknesses of both face-to-face and virtual education can complement each other. Most students, however, consider face-to-face education is fundamental during the first years of training because during this period the necessary skills must be developed, such as planning, organization, search capacity, information selection, research, written communication, reasoning, etc., so that later on they can be successfully incorporated into virtual learning spaces. Teachers at all levels must be aware of the role that technology currently plays in the teaching and learning process, so they must be trained to use it properly, having it as a support resource to facilitate educational activities, away from fear and resistance to change. According to our research, teacher preparation and attitude are a must for the success of virtual learning. Most of these researches also evidenced that teachers’ immediate feedback and volition were considered positively. In relation to the virtual classroom, peer interaction was considered valuable in the experience but a lack of cooperation was observed in many cases. Apparently, during the emergency situation and confinement, many teachers worked hard to prepare their classes and make them successful classes by adjusting instructional plans, curriculums, contents, and assessments.

Virtual or Face-to-Face Education: What Have We Learned …

99

8 Conclusion In general, most students stated that although virtual support during COVID-19 worked well so the possibility to complement future classes. As has been stated above, certain aspects are especially challenging, such as summative assessment despite the distance proctoring systems. Additionally, the selection of agreeable but instructional and accessible materials is the cornerstone of virtual education. The current study showed that although most research on students’ attitudes showed them to be generally satisfied with the class, most still missed traditional classes where active personal learnings are shared. Therefore a major issue in this comparison relates to the socio-affective aspects of education. Avoiding isolation, feelings of loneliness, and having support are a must. Teachers need to take a special interest in supporting virtual education learners’ needs. Our research also envisioned that many instructors require special training and adaptation for virtual learning but in the emergency teaching during the confinement in the pandemic it was difficult to find true specialists in virtual learning even among those who had previous, if limited, experience in virtual education. Thus, it is not surprising that students prefer face-to-face education since it is not technology that makes the difference but the classroom stakeholders. Certainly, students have been more empathic in synchronous classes but that still depends on each student’s personal learning style. This study has several limitations so our results cannot be taken as conclusive. Probably the most significant of these was pointed to in the introduction as the current research has obtained partial results that are mostly locally contextualized. There is also an evident interest in stressing the success of most classes and programs but, in fact, very little negative is presented in most research and thus the space for improvement is rather limited. Additionally, we believe that there is a need for more research to explore how students see the differences between distance and face-toface education over time and the impact in the medium and long term. Thus, future research is needed, particularly to address these limitations. The implication of our research has led to proposals for the need to transform and hybridize education, improve the training of teachers especially those with limited experience and skills in distance education, revise the virtual distance programs by adapting them to the local realities, and look carefully both at motivation and socioemotional skills. As stated above, further research should look not only at teaching but also at how students perceive the differences between distance and face-to-face education when time has passed, and feeling-free experiences are retrieved; as well as to address these and other potential limitations. Only then, will we be able to evaluate and assess properly whether virtual or in-person education may lead to obtain more conclusive results and to inform policy and practice in the field of education.

100

P. Adalid Ruíz and J. García Laborda

References Abdullah, S. I. N. W., Arokiyasamy, K., Goh, S. L., Culas, A. J., & Manaf, N. M. A. (2022). University students’ satisfaction and future outlook towards forced remote learning during a global pandemic. Smart Learning Environments, 9. https://doi.org/10.1186/s40561-022-00197-8 Alsarayreh, R. S., Al-Khasawneh, F., & Soub, T. F. A. (2022). Parental engagement in online teaching and learning during COVID-19 pandemic: Implications for sustainable education. Journal of Teacher Education for Sustainability, 24(1), 129–144. https://doi.org/10.2478/jtes-2022-0010 Alzamil, A. (2021). Teaching English Speaking Online versus Face-to-Face: Saudi Students’ Experience during the COVID-19 Pandemic. Arab World English Journal, 12(1), 19–27. https://www.proquest.com/scholarly-journals/teaching-english-speaking-online-versusface/docview/2550734422/se-2 Aririguzoh, S. (2022). Communication competencies, culture and SDGs: Effective processes to cross-cultural communication. Humanities & Social Sciences Communications, 9(1). https:// doi.org/10.1057/s41599-022-01109-4 Assi, E., & Rashtchi, M. (2022). Virtual classes during COVID-19 pandemic: Focus on university students’ affection, perceptions, and problems in the light of resiliency and self-image. AsianPacific Journal of Second and Foreign Language Education, 7(1)https://doi.org/10.1186/s40 862-022-00144-7 Beatty, A. E., Esco, A., Curtiss, A. B. C., & Ballen, C. J. (2022). Students who prefer face-to-face tests outperform their online peers in organic chemistry. Chemistry Education Research and Practice, 23(2), 464–474. https://doi.org/10.1039/d1rp00324k Caprara, L., & Cataldo, C. (2022). Effects of virtual learning environments: A scoping review of literature. Education and Information Technologies, 27(3), 3683–3722. https://doi.org/10.1007/ s10639-021-10768-w Carter, J. W., & Youssef-Morgan, C. (2022). Psychological capital development effectiveness of face-to-face, online, and micro-learning interventions. Education and Information Technologies, 27(5), 6553–6575. https://doi.org/10.1007/s10639-021-10824-5 Çoban, M., & Göksu, I. (2022). Using virtual reality learning environments to motivate and socialize undergraduates in distance learning. Participatory Educational Research, 9(2), 199–218. Costado Dios, M. T., Piñero Charlo, J. C. (2021). Face-to-face vs. e-learning models in the COVID19 era: Survey research in a Spanish university. Education Sciences, 11(6), 293. https://doi.org/ 10.3390/educsci11060293 Dervenis, C., Fitsilis, P., & Iatrellis, O. (2022). A review of research on teacher competencies in higher education. Quality Assurance in Education: An International Perspective, 30(2), 199– 220. https://doi.org/10.1108/QAE-08-2021-0126 Divjak, B., Rienties, B., Iniesto, F., Vondra, P., & Žižak, M. (2022). Flipped classrooms in higher education during the COVID-19 pandemic: Findings and future research recommendations. International Journal of Educational Technology in Higher Education, 19, paper 9. https://doi. org/10.1186/s41239-021-00316-4 Elena, D., Vlasios, K., Damianos, G., & Stella, S. (2022). Virtual reality as a communication medium: A comparative study of forced compliance in virtual reality versus physical world. Virtual Reality, 26(2), 737–757. https://doi.org/10.1007/s10055-021-00564-9 Ferrer, J., Ringer, A., Saville, K., Parris, M. A., & Kashi, K. (2022). Students’ motivation and engagement in higher education: The importance of attitude to online learning. Higher Education: THe International Journal of Higher Education Research, 83(2), 317–338. https://doi.org/ 10.1007/s10734-020-00657-5 Flores, M. A., Barros, A., Simão Ana, M. V., Pereira, D., Flores, P., Fernandes, E., Costa, L., & Ferreira, P. C. (2022). Portuguese higher education students’ adaptation to online teaching and learning in times of the COVID-19 pandemic: Personal and contextual factors. Higher Education, 83(6), 1389–1408. https://doi.org/10.1007/s10734-021-00748-x

Virtual or Face-to-Face Education: What Have We Learned …

101

Fouad, E., Austin, R., Kryger, R. A., Holton, D., Srinivasan, S., & Ngwa, W. (2022). Online versus face-to-face learning in introductory university physics following COVID-19. Physics Education, 57(2). https://doi.org/10.1088/1361-6552/ac3dc5 Freire, T., & Rodríguez, C. (2022). The transformation to an online course in higher education results in better student academic performance. Revista Iberoamericana De Educación a Distancia, 25(1), 299–322. https://doi.org/10.5944/ried.25.1.31465 Gherhes, V., Stoian, C. E., F˘arcas, iu, M. A., Stanici, M. (2021). E-learning vs. face-to-face learning: analyzing students’ preferences and behaviors. Sustainability, 13(8), 4381. https://doi.org/10. 3390/su13084381 Ghosh, S., Pulford, S., & Bloom, A. J. (2022). Remote learning slightly decreased student performance in an introductory undergraduate course on climate change. Communications Earth & Environment, 3(1), article 177. https://doi.org/10.1038/s43247-022-00506-6 Grossman, D. L. (2008). Democracy, citizenship education and inclusion: A multi-dimensional approach. Prospects: Quarterly Review of Comparative Education, 38(1), 35–46. Khan, S., Kambris, M. E. K., & Alfalahi, H. (2022). Perspectives of university students and faculty on remote education experiences during COVID-19–A qualitative study. Education and Information Technologies, 27(3), 4141–4169. https://doi.org/10.1007/s10639-021-10784-w Lafford, B. A., Wermers, J., & King Ramírez, C. (2021). Online world. Language instruction, training and assessment. An ecological approach. Georgetown Press. Lee, S. (2022). Factors affecting the quality of online learning in a task-based college course. Foreign Language Annals, 55(1), 116–134. https://doi.org/10.1111/flan.12572 Lin, T. (2022). Student learning performance and satisfaction with traditional face-to-face classroom versus online learning: Evidence from teaching statistics for business. E-Learning and Digital Media, 19(3), 340–360. https://doi.org/10.1177/20427530211059625 Núñez-Regueiro, F., Jamain, L., Laurent-Chevalier, M., & Nakhili, N. (2022). School engagement in times of confinement: A stress process approach. Journal of Youth and Adolescence, 51(7), 1257–1272. https://doi.org/10.1007/s10964-022-01621-5 Nyanamba, J. M., Liew, J., & Li, D. (2022). Parental burnout and remote learning at home during the COVID-19 pandemic: Parents’ motivations for involvement. School Psychology, 37(2), 160–172. https://doi.org/10.1037/spq0000483 Oelkers, J. (2001). History of pedagogical reform movement. In Neil J. Smelser, & Paul B. Baltes (Eds.). International encyclopedia of the social & behavioral sciences (pp. 11176–11179). Pergamon. Oye, N. D., Noorminshah, A. I., Madar, M. J., & Ab Rahim, N. (2012). The impact of e-learning on students performance in tertiary institutions. International Journal of Computer Networks and Wireless Communications, 2(2), 121–130. Pérez, F., Ledesma, E., Montero, J. M. C., Meléndez, L. V., & Navarro, E. R. (2022). Development of social skills of high school students on virtual platforms, 2021. World Journal on Educational Technology: Current Issues, 14(1), 231–242. Phillips, A. (2022). An uneasy, challenging, yet meaningful experience: A reflection on a face-toface to online synchronous COVID-19 course transition. Journal of Education for Library and Information Science, 63(2), 120–125. https://doi.org/10.3138/jelis-2020-0100 Rice, M. F. (2022). Special education teachers’ use of technologies during the COVID-19 Era (Spring 2020-Fall 2021). TechTrends: Linking Research and Practice to Improve Learning, 66(2), 310–326. https://doi.org/10.1007/s11528-022-00700-5 Salas-Pilco, S., Yang, Y., & Zhang, Z. (2022). Student engagement in online learning in Latin American higher education during the COVID-19 pandemic: A systematic review. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13190 Saygin, M., & Karakas, H. (2021). The effect of social skills-based activities on primary school students’ critical thinking and empathic tendencies. International Journal of Curriculum and Instruction, 13(3), 2789–2812.

102

P. Adalid Ruíz and J. García Laborda

Schulze, J., Zagorscak, P., West, S. G., Schultze, M., & Krumm, S. (2022). Mind the context—The relevance of personality for face-to-face and computer-mediated communication. PLoS One, 17(8). https://doi.org/10.1371/journal.pone.0272938 Suprabha, K., & Subramonian, G. (2021). Blended learning instructional strategy interventions for higher secondary school students: A focus on social skills. Journal of Educational Technology, 17(4), 55–64. Valaitis, K. L., Ellis, H., Hastings, N. B., & Havard, B. (2022). The influence of personal attitude on instructor intent to integrate online collaborative activities. Education and Information Technologies, 27(4), 5277–5299. https://doi.org/10.1007/s10639-021-10814-7 Venturino, M., & Hsu, Y. (2022). Using Whatsapp to enhance international distance education at the university of south Africa. TechTrends, 66(3), 401–404. https://doi.org/10.1007/s11528022-00718-9 Vygotsky, L. S. (1978). Mind in society: The development of higher mental processes. Harvard University Press. Wahyuni, S., Etfita, F., & Alkhaira, N. (2022). Students’ preferences and challenges in learning english fully online with google classroom. Journal of Education and Learning (EduLearn), 16(2), 244–253. https://www.proquest.com/scholarly-journals/students-prefer ences-challenges-learning-english/docview/2699637895/se-2 Zaghal, A., Marley, C., Rahhal, S., Hassanieh, J., Saadeh, R., El-Rifai, A., Qaraqe, T., Bejjani, M., Jaafar, R., & Hoballah, J. J. (2022). Face-to-face versus distance learning of basic suturing skills in novice learners: A quantitative prospective randomized trial. Research Square. https://doi. org/10.21203/rs.3.rs-1289770/v1

Opportunity Costs, Covid-19, and Early Dropout Rate Luis Pires

1 Introduction The rate of early leaving from education and training is the main indicator of the general situation of education in Spain. This article analyzes this long-term educational variable, its relationship with other educational variables and with the Covid-19 crisis. This analysis will show the state of the Spanish education system and its evolution over time, with its strengths and weaknesses. The following section explains the concept of early educational dropout and school failure, within the evaluations and statistics of education. Section 3 analyzes the evolution of early school dropout in the European Union and in Spain, relating it to other elements of the education system and paying special attention to the influence of the Covid-19 crisis on the evolution of these educational variables. The last section is the conclusions.

2 Early Educational Dropout and School Failure The evaluation of educational systems is a way of assessing the efficiency of the education received by citizens. This evaluation can be grouped into two types. The first is that of external or standardized evaluations, promoted by tests such as PISA of the Organization for Economic Cooperation and Development (OECD) or PIRLS and TIMSS of the International Association for the Evaluation of Educational Achievement (IEA), in addition to external evaluations developed by the educational authorities themselves. These evaluations are carried out on students and schools, since their objective is to analyze the achievement of certain standardized objectives for students and schools, mainly their knowledge and skills acquired during learning. L. Pires (B) Rey Juan Carlos University, Madrid, Spain e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_7

103

104

L. Pires

The analysis of the external evaluation of students and centers also serves to evaluate the performance of the educational system. In addition, through surveys carried out together with the knowledge tests themselves, these evaluations obtain very valuable information of the personal and economic-social characteristics of the students and their families, characteristics that educational studies consider key to explaining the academic results of students. The second type of evaluation is that of educational statistics. Most of these statistics follow a common criterion in many countries of the world, including all those of the European Union, which in this case implies the coordination of three international statistical organizations, UNESCO, the OECD, and Eurostat (in this compilation exercise, coordinated data is called UOE). These statistics include general descriptive data of the education system, such as the number of students, teachers, centers, and their characteristics (sex, age, ownership, etc.). Most of these statistics focus on the stage of compulsory schooling, which varies between countries but generally begins at six years of age (some European countries bring it forward until the last year of early childhood education, and some countries outside Europe delay it until seven years) up to 16 years, so there are 10 years of compulsory education (in Spain, six of primary education and four of secondary education). Apart from the previous descriptive statistics and other very specific elements (for example, the acquisition of foreign languages), educational statistics focuses its interest on the analysis of non-compulsory stages. On the one hand, early childhood education, which is of interest since the attendance of students at this stage prior to compulsory education can decisively influence educational performance during the compulsory stages. On the other hand, statistics show there is great interest in the stage after compulsory education. In addition to the description of the different options (higher professional training, university studies, lifelong adult learning), the analysis of this postcompulsory stage allows the quality of compulsory education and its link with the professional development of students to be evaluated. This quality can be measured directly with external evaluations (for example, PISA is carried out during the last year of compulsory education in most countries, when students are 15 years old), but also indirectly through the capacity of compulsory education to allow students to continue acquiring more education and greater skills and competencies for their professional life. In other words, it is understood that one of the functions of compulsory education is to facilitate and set the conditions for most students to continue educating themselves voluntarily. It is at this point that one of the most important educational statistics comes in, the early school dropouts otherwise referred to as early leavers from education and training. The National Institute of Statistics (INE) compiles the rate of early leavers from education and training based on a labor statistics source, the Active Population Survey (EPA, “Encuesta de Población Activa”). Specifically, this indicator shows the percentage of people aged 18–24 who have not completed second-stage secondary education and are not following any type of study or training in the four weeks prior to the interview. In other words, the two necessary conditions that a person must have for this statistic to be considered an early leaver is that her/his highest educational

Opportunity Costs, Covid-19, and Early Dropout Rate

105

level is (0–2) from the ISCED 2011 and that she/he has not received any education or training (either formal or non-formal). The first thing that stands out is that the early dropout rate is not an educational statistic, that is, in Spain, specifically, it is not carried out by the educational authorities, the Ministry of Education, which is responsible for preparing the country’s educational statistics. The EPA is a labor statistic, coordinated in Europe by Eurostat within the European Union Labor Force Survey (LFS). This is so because the early school dropout statistic tries to measure the transition of the population from the educational system to the labor system. As explained above, a good education system is one that makes it easier for most students to acquire the skills that allow them to obtain a quality education beyond the compulsory education stage and, thus, to be able to successfully handle their entry into the labor market. The early school dropout reflects, therefore, a weakness of the educational system by not managing to retain some of the students in education. This weakness is nothing more than the final manifestation of school failure. School failure refers to the difficulties that a student must overcome in the different stages of compulsory education. This failure is manifested in the difficulties to complete the learning stages of compulsory education and in the grade repetition, which leads to early school dropout and having problems in employability. Therefore, early school dropout and school failure are related, the school dropout being the final manifestation of a failure in the educational evolution of a student. In other words, the poor academic performance of students during their compulsory education stage is the main reason why they leave the education system without sufficient preparation. School failure can have many causes. Some are due to certain personal characteristics of the student, such as those derived from learning disorders (dyslexia, ADHD), developmental delays, or neurological diseases. Failure can even manifest itself in students with high abilities, for whom the educational system is not capable of providing an adequate response to their particular situation. The family or social environment of the student can also influence school failure, specifically their belonging to unstructured families or social environments where violence and marginalization hinder their educational development. Other personal events can also cause failure, such as the death of a family member, becoming pregnant, or bullying. The passage from childhood to adolescence can bring about complications in adolescence with problems such as bullying, drug addiction or new technologies, which can have a negative impact on the good academic results they had had when they were children. School failure, therefore, affects a certain group of students. But one of the most important elements that determine the quality of an educational system is its ability to react against school failure. Thus, a high level of school failure is an indicator of the low flexibility (or rigidity) of an educational system and its limited capacity to adapt and react to the poor educational performance of certain students. An educational system must have different characteristics to reduce school failure, such as individualized treatment and attention to diversity, the ability to adapt its curricula and teaching methodologies, the involvement of the educational community, or early detection of learning difficulties. The following section analyzes in detail the statistics of early school dropout and other related statistics. This analysis will serve as the

106

L. Pires

basis for an examination of the impact of the Covid crisis on the education systems of Spain and the European Union.

3 Evolution of Early Dropout Rate in Spain and the European Union The original data on early dropout from education and training is obtained from the Active Population Survey prepared quarterly by the INE. This data reflects the percentage of the total population between 18 and 24 years of age whose highest level of education is (0–2) from the ISCED 2011 and who declare that they have not received any education or training, either formal or non-formal, in the 4 weeks prior to being surveyed. The comparison of the early dropout rate of Spain with that of the rest of the countries of the European Union is considerably negative, since Spain is one of the worst situated among those 27 countries. Specifically, over the last decade, Spain has had the worst early dropout rate in the entire European Union, only being surpassed in 2021 by Romania. Table 1 shows this situation. In 2002 there were three countries with a significantly higher dropout rate than the rest of European countries, Malta at 53.2%, Portugal at 45.0% and Spain at 30.9%. Portugal and Malta then began a process leading to a very significant reduction in this dropout rate, while Spain did not begin to reduce it until 2010, when the European Union itself set the goal that, by the end of that decade in 2020, countries would reduce the dropout rate to below 10%. The global objective has been met, since the European Union has had an average dropout rate in 2021 of 9.7% (a rate that was 16.9% in 2002 and 13.8% in 2010). For 2030, the EU has set the target at 9%. The table shows how all the countries reacted and reduced their dropout rates to move closer to the target, a target that in the Spanish case stood at 15% (because it was not realistic to reach 10%), and which was almost achieved, standing at 16.0% in 2020, although in 2021 the dropout rate in Spain decreased to 13.3%, improving on Romania. However, dropout rates are worrying in Spain because it continues to lag behind European countries, and because other countries have shown that it is possible to reduce this rate quickly and significantly. Portugal, for example, which has gone from 45% to having a rate of 5.9% in 2021, is well below the European average. The analysis of the influence of Covid-19 on early leaving will be carried out by comparing the data from 2019 with those from 2020 and 2021. Thus, in Table 1, an effect of Covid-19 can already be seen in the increase in the rate of early leaving in some countries. On average, the European Union has reduced early school leaving every year since 2002 (except in 2018), and this reduction has been maintained over the last three years, so it is not observed that Covid-19 has affected this continuous reduction of educational abandonment. However, there are some countries where this effect has been noticed. Specifically, Croatia, the Slovak Republic, Finland, Sweden, Denmark, and Estonia, have experienced an increase in their early school leaving rate since 2020 or 2021, after a reduction in previous years.

14.3

6.0

14.6

13.4

15.3

Ireland

Lithuania

Netherlands

17.0

16.9

Sweden

Luxembourg

EU (27)

53.2

10.0

Finland

Malt

9.7

Austria

15.9

9.5

Slovak_Republic

Cyprus

10.4

6.7

France

9.0

16.4

13.4

Latvia

13.6

9.2

12.3

16.5

Belgium

Estonia

10.1

14.1

Czech Republic

Denmark

9.0

5.7

Portugal

49.9

17.3

13.7

5.3

12.7

18.8

14.3

6.5

41.2

7.2

45.0

Poland

11.4

13.1

4.6

15.6

5.1

16.2

Greece

7.9

Slovenia

8.0

Croatia

42.1

20.6

13.9

8.8

16.0

12.7

9.2

10.0

9.8

6.8

12.3

15.9

13.1

6.3

39.3

5.6

14.1

10.3

13.1

14.5

4.3

5.4

33.0

18.2

14.0

8.7

15.6

13.3

10.8

10.3

9.3

6.3

12.5

15.4

12.9

6.2

38.3

5.3

14.3

8.4

12.5

13.3

4.9

5.1

32.2

14.9

13.4

9.1

15.2

14.0

8.6

9.7

10.0

6.6

12.7

15.6

12.6

5.1

38.5

5.4

12.9

8.8

12.3

15.1

5.6

4.7

30.2

12.5

14.4

12.9

14.7

12.5

8.0

9.1

10.8

6.5

12.8

15.6

12.1

5.2

36.5

5.0

11.9

7.8

12.0

14.3

4.1

4.5

27.2

13.7

14.0

12.7

14.4

13.4

7.9

9.8

10.2

6.0

11.8

15.5

12.0

5.6

34.9

5.0

11.4

7.5

11.7

14.4

5.1

4.4

25.7

11.7

13.5

11.5

14.0

7.7

7.0

9.9

8.8

4.9

12.4

14.3

11.1

5.4

30.9

5.3

11.3

8.7

11.8

14.2

5.3

5.2

21.4

12.7

11.0

11.5

13.8

7.1

6.5

10.3

8.3

4.7

12.7

12.9

11.9

4.9

28.3

5.4

10.1

7.9

11.9

13.5

5.0

5.2

18.8

11.3

10.6

10.3

13.2

6.2

6.6

9.8

8.5

5.1

12.3

11.6

12.3

4.9

23.0

5.6

9.2

7.4

11.1

12.9

4.2

5.0

18.1

11.4

10.3

9.6

12.6

8.1

7.5

8.9

7.8

5.3

11.8

10.6

12.0

5.5

20.5

5.7

8.9

6.5

9.9

11.3

4.4

5.1

17.1

9.1

9.7

8.2

11.8

6.1

7.1

9.3

7.5

6.4

9.7

9.8

11.0

5.4

18.9

5.6

9.3

6.3

8.7

10.1

3.9

4.5

17.0

6.8

12.0

8.1

11.1

6.1

6.7

9.5

7.0

6.7

8.8

8.5

9.8

5.5

17.4

5.4

8.7

5.9

6.7

9.0

4.4

2.8

16.3

5.2

12.2

8.1

11.0

9.3

7.0

9.2

7.3

6.9

9.2

9.9

10.1

6.2

13.7

5.3

8.2

5.5

6.8

7.9

5.0

2.8

15.6

7.6

10.9

7.5

10.6

5.5

7.4

7.9

6.9

7.4

8.8

10.0

8.8

6.6

14.0

5.2

8.0

4.8

6.0

6.2

4.9

2.8

14.0

8.5

10.8

8.8

10.5

7.3

7.7

8.2

7.4

9.3

8.8

8.6

8.9

6.7

12.6

5.0

7.1

5.4

5.0

6.0

4.3

3.1

14.0

7.8

11.3

10.4

10.5

6.3

7.5

8.3

7.3

8.6

8.7

8.3

8.6

6.2

11.8

4.8

7.3

4.6

5.0

4.7

4.2

3.3

13.9

9.2

9.8

9.9

10.2

7.2

6.5

7.3

7.8

8.3

8.2

8.7

8.4

6.7

10.6

5.2

7.5

4.0

5.1

4.1

4.6

3.0

11.0

10.2

9.8

9.8

9.7

9.3

8.4

8.2

8.0

7.8

7.8

7.3

6.7

6.4

5.9

5.9

5.3

5.3

3.3

3.2

3.1

2.4

(continued)

12.6

11.5

7.5

9.3

9.9

8.2

7.7

8.2

8.1

7.6

8.0

7.2

8.1

7.6

8.9

5.4

7.0

5.6

5.0

3.8

4.1

2.2

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021

Table 1 Rate of early dropout rate in the European Union (27) (2002–2021)

Opportunity Costs, Covid-19, and Early Dropout Rate 107

24.2

30.9

23.0

Italy

Spain

Romania

Source MEFP database

21.9

20.7

Bulgaria

22.5

31.7

23.0

12.0

12.2

Hungary

12.8

12.5

Germany

22.4

32.2

23.1

21.4

12.6

12.1

19.6

31.0

22.1

20.4

12.5

13.5

17.9

30.3

20.4

17.3

12.5

13.7

17.3

30.8

19.5

14.9

11.4

12.5

15.9

31.7

19.6

14.8

11.7

11.8

16.6

30.9

19.1

14.7

11.5

11.1

19.3

28.2

18.6

12.6

10.8

11.8

18.1

26.3

17.8

11.8

11.4

11.6

17.8

24.7

17.3

12.5

11.8

10.5

17.3

23.6

16.8

12.5

11.9

9.8

18.1

21.9

15.0

12.9

11.4

9.5

19.1

20.0

14.7

13.4

11.6

10.1

18.5

19.0

13.8

13.8

12.4

10.3

18.1

18.3

14.0

12.7

12.5

10.1

16.4

17.9

14.5

12.7

12.5

10.3

15.3

17.3

13.5

13.9

11.8

10.3

15.6

16.0

13.1

12.8

12.1

10.1

15.3

13.3

12.7

12.2

12.0

11.8

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021

Table 1 (continued)

108 L. Pires

Opportunity Costs, Covid-19, and Early Dropout Rate

109

Fig. 1 Rate of early dropout rate in the regions of Spain, by sex. Year 2021. Source MEFP database

Focusing attention on Spain, it is observed, firstly, that there was a significant reduction in the early dropout rate in 2021, which went from 16.0% in 2020 to 13.3%. Therefore, it does not seem that Covid-19 has affected early educational abandonment in Spain. However, there are two regions that increased their early school dropout rate in 2021 when they were reducing it in 2020, Asturias and Madrid, which may indicate a possible effect of Covid-19 in these regions. The rest of the regions have continued to reduce their early school dropout rates (Fig. 1). In addition, there are notable differences within Spain between the regions, and also by sex. The following figure shows the latest data for 2021 by autonomous community and by sex. Educational dropout is significantly higher among men (16.7%) than among women (9.7%). That is a difference of seven percentage points, which equates to 52% of the total early dropout rate.1 There is great heterogeneity between the Spanish regions regarding this higher dropout rate of men compared to women, since, in some, the difference between men and women is very low (Cantabria and Rioja, with differences of 13% and 14% respectively regarding the average), and even in the Basque Country dropout is slightly higher in women (4.9%) than in men (4.7%); in the rest of the regions, the differences are greater (between 40 and 70%), with the Canary Islands standing out with a 92% difference with respect to the total number of dropouts. The dropout rate in Spain has shown high variability between the autonomous communities. Four regions already meet the EU’s objective of 10% (The Basque Country, Cantabria, Galicia, and Navarra), while the rest of the communities are between 10 and 15%, and two regions stand out below that target as they exceed 17%, Murcia and Andalusia. To analyze the heterogeneity of the Spanish regions, we are going to carry out a convergence analysis of the communities in their early dropout rate and compare it to European Union convergence rates. This convergence can be of two types, the first is the one that analyzes if the set of regions or countries 1

This percentage measures the importance of the difference between the sexes with respect to the total early dropout rate and is obtained by dividing the difference between the dropout rate of both sexes (7%) by the total dropout rate (13.3%), which gives the figure of 52%. The higher this figure, the greater the relative difference between the sexes.

110

L. Pires

is closer to each other. This convergence is known as sigma convergence (σ) and aims to assess whether there has been an approach of the regions and countries to the mean values of that variable, for which it uses the standard deviation of the natural logarithms of the variables, which reflects the existing spread. Figure 2 confirms that the dispersion is higher in Spain than in the European Union. Moreover, in Spain that dispersion has not decreased, that is, there has been no sigma convergence between the Spanish regions, and even this dispersion has increased slightly over time. The sigma convergence in the European Union shows different behavior from that of the regions of Spain. Thus, the dispersion between the countries of the European Union has clearly decreased since 2002. However, in the last year (2021) the dispersion has increased significantly, which may be caused by Covid-19. A second way to analyze convergence is to check whether the regions or countries that are lagging behind (with a higher dropout rate) move closer to those that are more advanced, as the overall dropout rate decreases. This analysis is called beta convergence (β). The central idea of this approach is to assess whether there is an inverse relationship between the growth rate (negative in this case) of the dropout rate and its initial level. For this, the graphic representation shows, on the horizontal axis, the dropout rate at the beginning of the period (2002) and on the vertical axis the average annual decrease in the dropout rate from 2002 to 2021. If there is convergence, the regions, or countries with the highest dropout rate in 2002 should be the ones that had the greatest decrease over the entire period, showing a growing trend line. 0.40

0.35

0.30

0.25

0.20

0.15

0.10 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 UE 27

Spain

Fig. 2 Dispersion of the early dropout rate in the regions of Spain and the countries of the European √ ∑i=1 (Ln D R O P OU Tit −Ln D R O P OU Tt )2 . Where, Union (27) (2002–2021). Notes Disper sion = N N LnDROPOUTit is the logarithm of the early dropout rate of each Spanish regions or each European countries i in year t; LnDROPOUTt is the logarithm of the dropout rate in Spain or European Union in year t; and N is the number of Spanish regions (17) or EU countries (27). Source MEFP database

Opportunity Costs, Covid-19, and Early Dropout Rate

111

Fig. 3 Convergence of the rate of early school dropout in the autonomous communities of Spain (2002–2021). Source MEFP database

Instead, Fig. 3 shows no such trend, as the trend is a horizontal line with an almost zero slope (0.4%) and the R2 of this trend line is very low (0.11%), meaning that this line barely explains the convergence. However, if we eliminate the three regions that are located in the upper right quadrant (Cantabria, Galicia, and the Basque Country), that is, those that, despite having a low dropout rate in 2002 (less than the average of Spain overall, which was 30.9%), are the ones that have reduced their dropout rate the most (above the average annual reduction in the entire period, which was 4.6%), there is a low convergence in the remaining regions as the line trend has a positive slope with an R2 of 32.2% (Fig. 4). In summary, the heterogeneity of the autonomous communities with respect to the educational dropout rate is maintained over time and the reduction in the dropout rate in Spain has not caused a convergence between the regions. However, it is necessary to take the data from some small Spanish communities with caution, as they have small sample sizes that are affected by strong sampling errors. The fact that the EPA is not designed to obtain data on the rate of early school leaving, but rather from labor statistics (active population, unemployment rate, etc.) explains the existence of these sampling problems. Next, we are going to carry out the same beta convergence analysis for the 27 countries of the European Union. The graphical analysis shows that in the European Union there is greater beta convergence than in the regions of Spain, with a slope of 15.8% and an R2 of 41.4% (Fig. 5). However, if we eliminate from the analysis of the three countries that in 2002 had a much higher rate of early dropout rate than the rest (Malta, Portugal, and Spain), and that have significantly reduced that rate over the following two decades (especially Portugal and Malta), the beta convergence analysis shows a slope of 15% and an R2 of 11.4% (Fig. 6). In conclusion, during

112

L. Pires

Fig. 4 Convergence of the rate of early school dropout in the autonomous communities of Spain, without Cantabria, Galicia, and the Basque Country (2002–2021). Source MEFP database

the last two decades, the countries of Europe have had a much higher convergence in their early dropout rates than the regions of Spain. The Spanish statistics on the early dropout rate include numerous elements that allow a more detailed analysis. In the first place, the statistics offer us the educational level of the mother and its influence on the educational dropout of her children. This influence is direct since the lower the educational level of the mother, the higher the dropout rate. In 2021, the children of mothers with a maximum of primary education have a dropout rate of 31.8%, which drops to 17.9% if the mothers have secondary education in their first stage, to 8.2% in the second stage, while the children of

Fig. 5 Convergence of the rate of early school dropout in the European Union (27) countries (2002–2021). Source MEFP database

Opportunity Costs, Covid-19, and Early Dropout Rate

113

Fig. 6 Convergence of the rate of early school dropout in the European Union (27) countries, without Malt, Portugal, and Spain (2002–2021). Source MEFP database

mothers with higher education only have an educational abandonment of 2.5%. The educational level of the mother has a certain direct influence on the educational dropout of her children since a higher educational level facilitates the direct help of the mother in the education of her children. However, the educational level of the mother is mainly an indicator of the social, economic, and cultural situation of the family, an indicator that numerous educational studies understand has a great influence on the academic results of the students and on early school dropout. Figure 7, shows the evolution over time of the relationship between the educational level of the mother and school dropout. In the first place, the statistics include those respondents whose mother’s educational level is not recorded, and whose dropout rate is similar to that of children whose mothers have a maximum of Primary education. If we compare the educational levels of the four groups of mothers, it can be observed that the dropout rate only falls significantly among the children of mothers with secondary education in their first stage. In the rest of the study levels, the dropout rate remains the same. Thus, children with mothers with a maximum of primary education maintain a dropout rate throughout the period of around 40% (between 37 and 45%), although in the last year, 2021, this rate dropped significantly to 31.8%, breaking the previous trend. This data may show a positive effect of Covid-19 on the relationship of many mothers with their children. Specifically, the confinement during the last half of the 2019–2020 school year allowed many mothers and fathers to establish a more intense relationship with their children’s education, which at that time was taking place online. This greater interest or approximation may have affected mothers with a low educational level more intensely, who would not have been able to obtain this information if things had remained unchanged. In the second stage of secondary and higher education, there is a slight decrease in the dropout rate. The explanation for the general reduction in the long-term early

114

L. Pires

Fig. 7 Rate of leaving education and training early by level of training of the mother (2002–2021). Source MEFP database

school leaving rate is explained, therefore, by an improvement in the educational levels of the mothers. This is verified in Fig. 8, where the percentage of students with mothers with a low educational level decreases notably, from 57.5% in 2002 to 16.6% in 2021, with the percentage of mothers increasing in the other three educational levels. The level of studies and the degree level notably influence early school dropout. In the first place, Fig. 9 divides the contribution to the total number of educational dropouts between those who obtain the Compulsory Secondary Education (ESO) certificate, but do not follow higher education or any other type of training, and those who leave the Spanish educational system without even obtaining the ESO

Fig. 8 Percentage distribution of young people (18–24) who abandon education-training early, according to the level of training of the mother (2002–2021). Source MEFP database

Opportunity Costs, Covid-19, and Early Dropout Rate

115

certificate. The obligation to receive education up to the age of 16 does not imply finishing compulsory education and obtaining the certificate, since if a student repeats an academic year or even if she/he is attending the last year of compulsory education (4th year of ESO), the day she/he turns 16, she/he can abandon studies without any legal repercussions for the student or their parents. Evidently, students who drop out without even obtaining the ESO certificate constitute the most worrying group, where dropout and school failure is more intense. The data in the following figure show how, within the long-term trend of a reduction in the school dropout rate, the contribution to the total dropout rate of students with an ESO certificate decreases from the initial 70% to around 50–60%. In other words, the percentage of students without an ESO certificate increases over the total number of students who drop out, an increase that occurred in the first decade of the twenty-first century, and then stagnated. These data show that the objective of obtaining the ESO certificate can be a first element of improvement in the reduction of school dropout and school failure. Figure 10 confirms this increase in ESO certificates, an increase that occurred after the economic crisis of 2008, since before it had been declining. Covid-19 does not seem to have affected previous trends. In many of the figures above, the economic crisis of 2008 appears as an element that decisively influences the evolution of early school dropout and the associated variables. This is logical because the situation of the labor market and the possibility that students can start working influence their decision to continue their studies or abandon them. In other words, the possibility of obtaining income in the labor market is an opportunity cost to continue studying and training. Before 2008, many students had the option of high-paying jobs in some sectors such as construction, which might have led to early school dropout. The evolution of the educational dropout rate in Spain confirms this situation since from 2002 to 2008 it did not

Fig. 9 Early school dropout by ESO qualification (2002–2021) (% of total). Source MEFP database

116

L. Pires

Fig. 10 Population aged 16–24 years with at least the ESO qualification or similar levels by age group (2002–2021). Source MEFP database

decrease, while in other countries it did (Table 1), due, among other factors, to the attraction of employment in the Spanish construction sector. With the crisis and the drastic decrease in employment opportunities for young students, early school dropout began to decrease, although the economic recovery from the crisis in 2014 did not produce an increase in the dropout rate. The reasons for the latter may lie in the (bad) experience of many youngers who left their studies in the first decade of the twenty-first century to start working in low-skilled jobs and who, after the crisis, found themselves without studies and without work. In other words, the economic and financial crisis of 2008 modified the opportunity cost of studying in Spain in the long term. Figure 11, confirms the previous statements. Until 2008, most students who had dropped out of school (70%) had an occupation, and only a third neither studied nor worked, the so-called “ninis” in Spanish (as the translation in Spanish of the conjunctions “neither” and “nor” is “ni”). This situation turned around very quickly after the 2008 crisis, with two thirds of the students who abandon being those who did not have a job and only a third who did. The economic recovery from 2014 led to a slight increase in the number of students who abandoned and had a job, although the figures for the first decade of the twenty-first century (70%) were not reached, but rather came to 50% in 2019, always due to the people who left and were unemployed being lower. The Covid crisis has again reduced the percentage of employed people in 2020 and 2021 (to around 40%), although the explanation for this reduction lies more in the economic crisis caused by the pandemic than in the pandemic itself. In fact, compared to the very serious economic crisis of 2008, since 2020 there has not been one as clear as that, since there has been a very intense but very short crisis during the months after the pandemic, followed by a significant recovery in 2021, followed again by some elements such as high inflation, which may lead to a future crisis or recession. In any case, the economic situation over the years since 2020 is quite peculiar and cannot be defined as an economic crisis such as the one that occurred in 2008 or in previous periods.

Opportunity Costs, Covid-19, and Early Dropout Rate

117

Fig. 11 Early school dropout by occupation (2002–2021) (% of total). Source MEFP database

The Ministry of Education also offers statistics on the young population that neither study nor work, (“ninis”). Figure 12, shows its evolution, which follows the same parameters that have been indicated above, specifically the influence of the economic situation. Thus, the increase in unemployment raises the number of “ninis”, which increased after the 2008 crisis and decreased after the beginning of the recovery in 2014, although the lowest level of “ninis”, in 2007, has not yet been equalled. It is also verified how a higher degree level reduces the percentage of “ninis”, since it improves the probability of obtaining a job. This figure confirms the idea that we explained in the previous paragraph, namely that the Covid crisis has been very short (in 2020) and that in 2021 the previous trend recovered again, in this case, the percentage of “ninis”. The previous figure reflects one of the most decisive elements in the evolution of educational abandonment and the employability of young people: qualifications. Until now, only the characteristics of the students who drop out have been analyzed, which in terms of qualification implies differentiating between those who have not graduated in ESO and those who do have that qualification (Fig. 9). But the reduction in dropouts is largely due to the increase in students who continue their studies after finishing compulsory education in the 4th year of ESO, that is, one of the main causes of the reduction in the rate of early school dropout in Spain is the increase in graduates in Basic Vocational Training, Middle Vocational Training or Baccalaureate. The fact that more young Spaniards complete their second-stage secondary education is positive, not only because it reduces dropout rates, but also because, with these studies, they raise their level of competencies, both for further training and for employment, skills that would be much more difficult to develop if they abandoned the school system without completing Basic Vocational Training, Middle Vocational Training

118

L. Pires

Fig. 12 Young population (15–29 years old) that neither studies nor works, by level of training (2002–2021). Source MEFP database

or Baccalaureate. At the individual level, young people who have completed upper secondary school are less likely to be unemployed and have a higher salary level, compared to those who have only completed ESO. Figure 13, shows this increase in qualification and its close relationship with the school dropout rate. While until 2008 the percentage of the population between 18 and 24 years of age with training in the second stage of Secondary or Higher Education was stable below 60%, after that date, and coinciding with the reduction in the school dropout rate, students’ qualification increased significantly until reaching over 70%. The group of students who do not obtain a higher qualification than ESO at that age but who continue studying (therefore, they do not belong to the school dropout group) has remained stable throughout the period. The effect of Covid on qualifications, if any, will be longer term. However, the data available so far shows that Covid has not reduced the previous trend of improving qualifications and the consequent improvement in early school dropout. Disaggregating studies higher than ESO, Fig. 14, shows how the Spanish population (up to 34 years of age) has improved its level of qualification, reducing the percentage of people with lower studies. The greatest increase occurs in higher education (Higher Vocational Training and University studies), mainly from 2016, when it increased from around 40% to almost 50%. In this sense, Spain amply meets one of the European Union’s 2020 education objectives, which indicates that at least 40% of the population between 30 and 34 years of age must have completed some form of higher education. This is an important achievement for Spanish education that, if used well, would be an asset to attract investment from companies that are intensive in human capital. If a country has many young people with higher education, it can take advantage of that asset by creating the conditions for that endowment of

Opportunity Costs, Covid-19, and Early Dropout Rate

119

Fig. 13 Percentage distribution of the population aged 18–24 by education qualification (2002– 2021). Source MEFP database

human capital to attract investment from multinationals that are intensive in human resources and that generate goods and services with high added value. However, the fields with the highest employment rate are those related to STEM (Science, Technology, Engineering, and Mathematics), so the enrollment of students in these studies must be increased (OECD, 2021). However, the number of university students is not the only variable of interest, since the quality of university teaching is also important. The comparison of the training quality of Spanish university students with other countries shows an unfavorable situation for Spain since people who have higher education in Spain (University studies or Higher Vocational Training) have a lower level of training than other countries. Thus, the PIAAC evaluation of the OECD, a test similar to PISA but carried out on the population between 25 and 64 years of age in 2012 (the next cycle of this test will be carried out in 2022), shows that the reading and mathematical skills of Spaniards with higher education, whether University or Higher Vocational Training, were similar to those of the population with post-compulsory secondary education (Middle Vocational Training or Baccalaureate) from several countries such as Japan or the Netherlands (MECD, 2013, p. 80). The increase in the educational training of the Spanish population has a direct effect on their job opportunities. More training means having greater employability and a higher salary. This element reduces the opportunity cost of studying. Figure 15, shows a direct relationship between the educational level and the employment rate. The employment rate is the percentage of employed workers with respect to the total population, so it is affected by unemployment levels and by the activity rate, that is, by the willingness of these people to belong to the active population. The graph shows that the employment rate is directly related to the economic situation. Thus, the

120

L. Pires

Fig. 14 Educational level of the population aged 16–34 (percentage of the total population) (2002– 2021). Source MEFP database

economic crises of 2008 and 2020 reduced the employment rate, by increasing unemployment and discouraging many people from looking for work, while in periods of economic growth (prior to 2008, between 2013 and 2020, and in 2021) employability increases in all types of training. However, throughout this period of two decades, there have been some changes between the different degrees of qualification. The most notable is the loss of employability of those who have a degree lower than the first stage of secondary education, compared to the rest of the qualifications. This situation makes school dropout more serious since the working conditions of students who abandon the school system become increasingly worse compared to students who obtain higher qualifications. On the other hand, it is also observed that the intensity of the changes softens as the level of qualification increases. In other words, an economic crisis reduces the employability of all qualification levels, but it reduces it much more at low levels than at high levels. Therefore, having a high level of qualification reduces the severity with which an economic crisis affects your employability. As with other elements analyzed above, the Covid crisis temporarily affected employability in 2020, which recovered just as quickly in 2021. The unemployment rate also improves with higher qualifications. While Spaniards aged 25–34 with less than upper secondary education had an unemployment rate of 28% in 2020, this rate decreased to 20% for those with upper secondary education and to 15% for those with higher education (OECD, 2021, p. 79). The unemployment rate of Middle Vocational Training graduates was 13.6% in 2018, while that of Baccalaureate graduates was 13.9%. On the contrary, within higher education, the

Opportunity Costs, Covid-19, and Early Dropout Rate

121

Fig. 15 Employment rates of the population aged 25–64 by level of training (2002–2021). Source MEFP database

unemployment rate is somewhat higher for graduates in Higher Vocational Training, 10.4%, compared to 8.5% of those with a university degree. In fact, the labor income of graduates in Higher Vocational Training is 20% higher than those with intermediate studies (Vocational Training or Baccalaureate), while those with university degrees have a salary that is 43% higher than intermediate studies (OECD, 2019). In short, it is highly worthwhile to study Middle Vocational Training (much better than staying only with ESO), and then continue with Superior Vocational Training. And, based on the data mentioned, it may be interesting to continue after completing Higher Vocational Training with a University Degree, especially if it is possible to validate part of Higher Vocational Training with a University Degree, an option that more and more Higher Vocational Training graduates choose. The improvement in the qualifications of the second stage of secondary education is the element that most influences the reduction of early school dropout. Figure 14 already showed a very notable increase in higher education qualifications, based on a previous qualification of second-stage secondary education studies, both in the Baccalaureate to later access university, and in the Basic and Mid-level Vocational Training, to then access Higher Vocational Training or even university. Figure 16, confirms this improvement in these studies through the number of students. The 2008 crisis is once again an element of change in the trend in the series, since the reduction in the number of university students and the stagnation in those of professional training in previous years, changes towards a growth in the number of students in these studies. Likewise, the beginning of the economic recovery from 2014 once again affects the number of students, both university students and midlevel Vocational Training students. However, Higher Vocational Training has seen

122

L. Pires

Fig. 16 Number of Baccalaureate and vocational training students (years 2002–2003 to 2020– 2021). Source MEFP database

a highly significant increase in students in recent years, reflecting the attractiveness of these studies due to their greater employability. Despite this increase, there is still room for improvement, since 69.8% of the generation of young Spaniards who in 2019 were between 25 and 34 years old have graduated, at least, in these postcompulsory studies, but this data is only two percentage points higher than that of the generation that precedes it (35–44 years old). In contrast, in the European Union, this progress between generations was 2.5 percentage points (from 82.2 to 84.7%), even though it already started from higher levels than Spain (OECD, 2019). This recent increase of Higher Vocational Training students occurs during a process of overall decrease in the number of students in general and university education. The decrease in the birth rate and the number of children is behind this evaluation. Table 2, shows the average annual growth in different periods of all qualifications. In the first place, it is verified that the growth is decreasing, from 1% in the entire period (2002–2021) to 0.2% in the last period since 2014. Moreover, nursery and primary education has experienced a decrease in students in recent years. However, the number of students in post-compulsory education is increasing, highlighting vocational training.

4 Conclusions One of the main problems of the Spanish educational system is early educational abandonment and the large number of students who do not complete, at least, Intermediate or Baccalaureate Vocational Training. These people will later have labor and social integration problems that will hamper their opportunities to enjoy the same opportunities as the rest of society throughout their lives and will need, to a greater extent, the help of the Welfare State.

Opportunity Costs, Covid-19, and Early Dropout Rate

123

Table 2 Average annual increase in the number of students in general and university education in Spain 2002–2021 (%)

2008–2021 (%)

2014–2021 (%)

Nursery education (1st stage)

5.4

0.6

− 1.8

Nursery education (2nd stage)

0.4

− 1.0

− 1.8

Primary education

0.7

0.5

− 0.3

ESO

0.4

0.9

1.5

Baccalaureate

0.0

0.7

− 0.1 9.7

Lower vocational training Middle vocational training

2.6

2.9

1.2

Higher vocational training

3.1

4.8

3.6

University

0.6

1.2

1.6

Total

1.0

0.8

0.2

Source MEFP database

The problem of early school leaving is a shared concern in all European Union countries. As mentioned earlier, the European Union set as one of its main educational goals to be achieved by 2020 that European countries would reduce the dropout rate to below 10%. Different countries have established various measures to improve their education systems and specifically reduce early school leaving. Some examples of policies that have been adopted across Europe are early warning and intervention systems to identify students at risk of dropping out and provide targeted support, seeking close collaboration between schools, teachers, counselors, and support services to monitor student progress, detect early signs of disengagement, and implement interventions to prevent dropout; flexible learning pathways to cater to diverse student needs and interests, including the provision of alternative education programs, vocational training, apprenticeships, and personalized learning approaches that offer multiple entry points and opportunities for students who may struggle in traditional academic settings; improve career guidance and counseling services to help students make informed decisions about their educational and career pathways; increase the support for disadvantaged students, including targeted measures such as extra tutoring, mentoring programs, financial assistance, and support for students with special educational needs to ensure their inclusion and success in the education system; enhance transition programs between different educational levels and stages, providing orientation, academic support, and guidance to help students adjust to new learning environments successfully; policies to foster parent engagement, including promoting parent-school partnerships, organizing workshops and information sessions, and involving parents in decision-making processes related to their children’s education; finally, many European countries have established data collection systems and evaluation mechanisms to track progress, identify areas for improvement, and ensure evidence-based decision-making. The specific policies and approaches vary among European countries, as they consider their unique educational contexts, cultural factors, and policy priorities.

124

L. Pires

Additionally, many of these policies are interconnected and implemented in a comprehensive framework to address the multifaceted nature of early school leaving and promote educational success for all students. However, Spain can adapt the most successful policies developed in some European countries. One of the most important is Germany’s dual vocational training system, since it has served as an inspiration to the Spanish educational authorities. Germany has a successful dual vocational training system that combines classroom learning with practical work experience. Spain can adopt a similar approach by strengthening vocational education and training programs, forging stronger links between schools and industries. This can enhance students’ employability skills and help address skill gaps in the job market. Spain needs more young people to continue their studies at the end of ESO. That is why it is very important to carry out educational measures focused on students who are finishing ESO studies, in the 3rd or 4th year of ESO, or the parallel studies of Basic Vocational Training (in their first and second year) which also lead in many cases to the ESO title. These actions must allow the reception of these students in the next educational level (Baccalaureate and most especially Vocational Training Media) to be done in the best conditions to prevent them from abandoning these studies without obtaining the title and the skills they provide. One of the great changes in the Spanish educational system in recent years has been the increase in the rate of graduation from high school. The contribution of Intermediate Vocational Training graduates has been key to this important achievement. That is why it is necessary to persist in encouraging these studies and correct one of their problems, the significant number of students who start these cycles and do not finish them. Thus, the substantial increase in the hiring of teachers announced by many autonomous communities derived from the Covid-19 pandemic should be used significantly to increase Middle Vocational Training cycles being offered, mainly in those cycles with greater labor insertion, such as Mechanical Manufacturing, Installation and Maintenance, Health, Transportation and Vehicle Maintenance, or Food Industries. Other policies that Spain can learn from are the following. Finland’s focus on teacher quality and its emphasis on teacher training and professional development. Spain can prioritize investing in teacher education programs and continuous professional development to improve the quality of its teaching force. Providing teachers with adequate support, resources, and autonomy can contribute to better educational outcomes and that these teachers can more successfully address the problems of early educational dropout. The Netherlands has invested heavily in early childhood education and care, recognizing the critical role it plays in a child’s development. Spain can focus on expanding access to high-quality early childhood education and ensuring affordability and inclusivity. Early interventions can lead to improved academic performance, reduced educational inequalities and early educational abandonment. Norway places great importance on achieving educational equity. Spain can adopt strategies to address educational disparities, particularly among disadvantaged students. This could involve implementing targeted interventions such as additional

Opportunity Costs, Covid-19, and Early Dropout Rate

125

resources, specialized support, and extended learning opportunities for students from disadvantaged backgrounds. Poland has undergone significant education reforms, including curriculum changes, teacher training enhancements, and increased educational autonomy. Spain can consider revising its curriculum to align with modern demands, providing more opportunities for teacher professional development, and empowering schools with greater autonomy to innovate and adapt to local needs. Sweden has implemented a school choice system that allows parents and students to choose from a range of publicly funded schools. Spain can explore the introduction of school choice mechanisms while ensuring a balance between choice and equity. Additionally, granting schools greater autonomy in decision-making can foster innovation and responsiveness to local needs. The United Kingdom has emphasized accountability in its education system through mechanisms such as school inspections and standardized testing. Spain can strengthen its accountability measures by implementing robust evaluation systems, including school inspections, teacher assessments, and student assessments. However, it is crucial to strike a balance to prevent excessive pressure on students and teachers. Covid-19 and the closure of educational centers have provided an opportunity to once again place as a political priority the problem of the large percentage of young Spaniards who drop out of school without completing Vocational Training or Baccalaureate. The exceptional situation caused by the pandemic has exposed this problem of early school leaving more intensely, especially in the fact that it shows a significant socioeconomic gap, as has been verified throughout this article. Hence the proliferation, in these months of the pandemic, of numerous educational proposals related to the improvement of the Spanish educational system. Some of these proposals would be the following: establish tutorials for small groups of students, a measure that does not have a very high cost if we compare it with the benefits obtained in those countries where it has been carried out; use online education as a complement to face-to-face education, improving its effectiveness in the most backward students; and promote some plans or programs of territorial cooperation between the Spanish Ministry of Education and the regional governments, such as the PROA Plan, aimed at centers that attended to a significant number of students in a situation of educational disadvantage, or the Program for the Reduction of Early Abandonment of the Education. Spain has begun to improve its educational system in recent years, which is reflected in a decrease in early school dropout rates. However, actions to improve the educational system in Spain have already been carried out by many other European countries. For this reason, it is necessary to deepen these measures to give the necessary impetus to the education and training of young Spaniards that allows them to integrate adequately into the Spanish, European, and world labor markets.

126

L. Pires

References Ministerio de Educación y Formación Profesional (MEFP) [Ministry of Education and Vocational Training]. Database: Explotación de las variables educativas de la Encuesta de Población Activa/Transición de la formación al trabajo. Abandono temprano [Exploitation of the educational variables of the Active Population Survey/Transition from training to work. Early dropout]. http://estadisticas.mecd.gob.es/EducaDynPx/educabase/index.htm?type= pcaxis&path=/laborales/epa/aban&file=pcaxis&l=s0 Ministerio de Educación, Cultura y Deporte (MECD) [Ministry of Education, Culture and Sports]. (2013). PIAAC: Program for the International Assessment of Competencies of the Adult Population. First cycle. Spanish report. Initial analysis. Volume I. OECD. (2019). Education at a glance 2019: OECD Indicators. OECD Publishing. https://doi.org/ 10.1787/f8d7880d-en OECD. (2021). Education at a Glance 2021: OECD Indicators. OECD Publishing. https://doi.org/ 10.1787/b35a14e5-en

Random Experiment on Relative Performance Feedback in Higher Education at URJC Cristian Macías Domínguez

1 Introduction The main objective of this research is to know the effect that feedback causes in university students at the Rey Juan Carlos University over several semesters. It is a novel study, because there are not many random experiments of this type carried out in Spain in Higher Education in the literature. The Rey Juan Carlos University, hereinafter URJC, is a public university located in the Community of Madrid that has different campuses in the region, including the Móstoles, Fuenlabrada, Vicálvaro, Alcorcón, Quintana and Aranjuez campuses. The URJC welcomes around 45,000 students each academic year in different areas of knowledge. The randomized experiment consists of dividing each class under study into two groups: a treatment group and a control group. The treatment group is chosen randomly among all the students included in the subject under study, these students are given the grade, as usual, and the percentile in which their grades are found with respect to the class during the session’s tests carried out throughout the semester. On the other hand, the typical methodology of knowing the grade without knowing what position they occupy with respect to the class is applied to the control group. The main objective of this research is to find out if there are significant differences between the means of both groups once the semester is over, assuming that knowledge of the percentile is a motivational push to improve grades for final exams. A second objective to consider in this study is the motivation that the student may suffer, for returning to the classroom, after the pandemic suffered globally by COVID-19. The students selected in the sample suffered the consequences of the pandemic in their last year of high school, which caused a hasty adaptation to work and taking exams remotely. His first year of university, that is, his first academic C. M. Domínguez (B) Universidad Rey Juan Carlos, Madrid, Spain e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_8

127

128

C. M. Domínguez

cycle, is marked by the dynamics of telecare. The URJC ruled that classes would be held on a rotating basis to avoid the overcrowding of classrooms and campuses. To do this, the students of all grades and all courses were divided into two shifts. The students of the first cycle went in person at the beginning of the course, resuming blended contact with the classrooms. The first shift of the first cycle attended the first week of the month and the second shift the second week of the month, while the higher cycle students worked from home. Once the second week was over, the cycles were reversed, the students of the first cycle worked remotely from home for the following two weeks and those of the second cycle in person and so on, sequentially throughout the course. The students under study have learned to work from home in the university environment, students who have barely maintained physical contact with teachers or with classmates. These students have shown a lack of motivation to attend class in person, as has been observed in class attendance, comparing before and after the pandemic. In order to know the sample, all students are given a questionnaire at the beginning of the semester where they are asked what the expected result is at the end of the course in relative terms. In addition, the students in the treatment group are given a survey, once the semester is over, to find out if having known the percentile during the semester can be a greater motivation to face the final exams and thus obtain better qualifications with respect to the initial idea. In this essay, there is still no evidence of the data from this survey since the completion of the complete investigation will take place in the following semester. A first approximation of the study has been carried out in the double Marketing Degree of the Fuenlabrada Campus. During the second semester of the year, the students of the treatment group of the subject Financial Economic Analysis in Marketing II, a pioneer subject in this random experiment, were aware of the relative position that their grades occupied with respect to their classmates. At the beginning of the year, a brief survey was carried out among the attending students to know the sample. During the semester, two tests were carried out, based on these tests the students of the treatment group were notified of the percentile in which they were with respect to the class. The notification was made by the internal mail of the URJC together with a brief explanation of what it meant to be in said percentile. Later, in May, the final exam of the subject was carried out in the so-called ordinary call, where once the final grades have been published, this essay is carried out.

2 Literature Review To conduct this random experiment, a review of the existing literature has been performed. The study presented in this chapter is related to the empirical literature on relative feedback performance, both in educational and workplace settings. A study conducted at a prestigious university in the United Kingdom suggests that students have limited information about the academic performance of their study efforts. Therefore, continuous feedback on grades in absolute terms reduces this

Random Experiment on Relative Performance Feedback in Higher …

129

uncertainty and, as a result, enhances the future performance of the student. This university allows each department to be free to provide feedback to its students. The research findings determine that disclosing grades to students continuously has a positive effect on their subsequent test scores: the average impact corresponds to a 13% improvement of one standard deviation in test scores. The study finds that the feedback’s impact is stronger for brighter students and for those students who have less initial information about the academic environment. Furthermore, it suggests that feedback does not have negative effects on the test scores of any student, implying that the optimal policy is always to provide feedback, assuming that a department’s goal is to maximize students’ academic performance. However, there are several explanations for why departments may optimally choose not to provide such feedback. First, departments may have incorrect beliefs about their students’ preferences or backgrounds, particularly regarding whether feedback will motivate students or lead them to slack off during the course (Bandiera et al., 2015). There are studies (Azmat & Iriberri, 2010) carried out in a secondary school in the Basque Country (Spain) that determine the importance of relative feedback. For a year, students were told their final grades, the average grade obtained and the average grade of the class, in this way the students could know if their grade was above or below the class average. The study determines the positive effect generated by the provision of feedback information on relative performance because it had a strong and motivational effect on the performance of high school students, increasing the general grades by 5%. Another study that reinforces the feedback theory in higher education (Anh & Zeckhauser, 2012) was carried out in Vietnam on university students in the subject of English. The class was divided into three groups and they were tested fortnightly: a control group made up of students who were given an absolute grade on each test taken; a treatment group that, in addition to the grade, was privately given the grade regarding the class; and another treatment group to which they were informed of the qualification and the note regarding the class in a public way, at the end of the English course they took the TOEIC exam. It was shown that the first treatment group outperformed the control group by 10 percentage points, showing that this improvement strongly supports the hypothesis that knowledge of rank with respect to other peers motivates people even when it does not bring tangible benefits. In addition, the second treatment group that was publicly ranked outperformed those that were only privately ranked. The coefficients suggest that the scores were 45% higher than the control group, which implies a higher motivation than the first treatment group. The effect of providing college students with relative performance feedback can lead to an increase or decrease in final grades. There are studies (Cabrales et al., 2019) carried out at the Carlos III University of Madrid that determine that the academic performance of the students of the treatment and control groups in the first year of the degree are similar. Once the treated students are provided with their rank information, we observe a significant decrease in their performance relative to those in the control group. In particular, during their second year, the treated students pass, on average, 0.4 points less than the students in the control group. His GPA (Grade Point Average) for that year is reduced by about 0.20 standard deviations.

130

C. M. Domínguez

Regarding satisfaction levels, there is evidence that they increase during this period in the treatment group. In the run-up to the randomized experiment, students in the treated and control groups report similar levels of satisfaction and motivation (Cabrales et al., 2019). This theory reinforces other studies that have confirmed that overconfidence causes a systematic reduction in grades (Moore & Cain, 2007). Other authors who have studied feedback in the workplace determine that they find negative effects in relative feedback. Barankay (2012) conducts a field experiment on vendors and financial incentives according to relative rank throughout the study. It is confirmed that the rankings have an impact on people’s behavior, but due to the multitasking of the analyzed sample, a novel result emerged: people can switch their attention to other tasks when they are informed that their rank is lower than what they expected by worsening their core performance. Significant gender effects were also shown to exist suggesting that rank incentives may be a phenomenon based on taste and not driven by financial incentives. There are other experimental studies in the workplace which have investigated the role of feedback under various incentive scenarios, indicating its importance (Azmat & Iriberri, 2016; Blader et al., 2015; Charness et al., 2013; Eriksson et al., 2009; Ertac, 2005; Gill et al., 2016; Hannan et al., 2008; Khunen & Tymula, 2012). These investigations do not show beliefs about increasing or decreasing expectations, except for Khunen and Tymula (2012), who determine that with the existence of incentives, those who have a lower ranking than expected increase effort and those who they rank higher than previously assumed, they reduce effort, although the overall effect is positive. The most recent literature also reflects the lack of motivation caused by COVID19. These studies determine that the new reality has forced the student to change the social and face-to-face interaction to a virtual one, and the educational environment for that of the home, this has implied an isolation from the educational climate, where teachers and classmates are a great support in their education and motivation. The student has been forced to look for new tools, classmates and teachers to guide him in some educational tasks. There are authors (Galindo & Vela, 2020) who indicate that in order to promote student self-determination, it is important to encourage the development of research that can focus on the feedback of teaching–learning strategies. Other studies carried out in the field of medicine generally confirm that remote teaching was not positive. The excessive workload carried out by teachers, the limited relationship with them and reduced motivation during the pandemic caused widespread discomfort among students, which is why this type of student welcomes the return to school attendance (Brotons et al., 2020).

Random Experiment on Relative Performance Feedback in Higher …

131

3 Objectives The main objective is focused on providing relative feedback on the academic results individually to a treatment group in the URJC students in order to know the effect caused both in the academic performance and in the individual satisfaction of the student. This practice is very common, both at work and in educational settings at earlier levels. Feedback is often provided on absolute performance or in relation to some relevant reference group (Cabrales et al., 2019). In most cases, it is believed that the more information a student has regarding their grades, the better their academic performance will be. Nevertheless, in the scientific literature it is not made clear that this hypothesis is completely true since it can be complicated by a series of characteristics inherent to the student himself. From a theoretical point of view, in addition to the incentives associated with knowing the percentile that the student occupies with respect to the class, it can influence overconfidence, causing a reduction in performance throughout the course. As explained in the review of the literature, there are articles that determine that the final results are worse for the treated group as they have greater transparency due to knowing periodically the performance of the students in the control group. In addition, we want to have proof of the student’s motivation to return to normality after the pandemic suffered in previous years through a control of face-to-face attendance.

4 Methodology The final study will be carried out in different degrees in which the researchers of said article teach classes at the URJC. To do this, students will be divided into two groups, a first treatment group and a second control group. Both groups must sign a form that guarantees the voluntariness of belonging to the random experiment and in which they will be informed concisely about the experiment carried out, guaranteeing the privacy at all times of the personal data of the student under study. The experiment will be carried out for a minimum period of two semesters, following the evolution of the students throughout the course in one of the main universities in Spain. The students of the control group will only receive information about their own performance in absolute terms in each test carried out, while the students of the treatment group, along with the grade, will be provided with the percentile range in which they are with respect to the class. In addition, once the final exams have been completed, the students will answer a survey about their satisfaction with the course and their possible academic improvement by knowing their relative position with respect to the class. Finally, an important characteristic of the study is that a questionnaire was carried out at the beginning of the course to learn more about the sample and, above all, the previous beliefs of the students, which will allow us to understand the treatment that affects the performance and satisfaction of

132

C. M. Domínguez

the students. This questionnaire is for informational purposes only and is therefore anonymous. Once the study is finished, it is intended to demonstrate two issues to understand the effects of providing feedback to students. A first question focuses on whether students who know their percentage rank with respect to the class seek to maximize their own results, or whether they are also concerned about their relative position competitively. This may be because relative performance translates into different outcomes or individuals are inherently concerned with relative position in performance. The second question considers that the prior beliefs of individuals regarding their abilities and those of their peers will help to understand if, in the absence of information, it is possible to overestimate their relative position. In a context where individuals have competitive preferences.

5 Experimental Design The study is carried out at the Rey Juan Carlos University of Madrid, specifically in the Financial Economic Analysis in Marketing II subject of the second year of the double degree in Marketing and Advertising and Public Relations at the Fuenlabrada campus. The sample includes a total of sixty-nine students, fifty-two of whom belong to the Marketing degree and seventeen to the double degree. Among all the students there were five who had already taken the subject in past years. In order to carry out the randomized experiment with URJC students, it was necessary to request permission from the university’s Research Ethics Committee. All students were informed of the random experiment and filled out a form showing their consent, always guaranteeing the privacy of their personal data. The students under study completed a questionnaire to know in detail the selected sample and the estimated quartile in their grades that they would obtain at the end of the course. This questionnaire shows that the average age of the students is 19.8 years, with 49 female students and 20 male students. The questionnaire also indicates that the average number of siblings is around 1.11, with 18.18% residing in Madrid, 61.36% residing in municipalities of Madrid and 20.46% living outside the Community of Madrid. All students have internet and a personal computer so they could follow the classes the previous year electronically without problems. Virtually all students entered the university in the 2020/2021 academic year, except for a few who entered the previous year. This confirms that all the students under study have had to adapt to teleworking during COVID-19. Regarding the quartile estimated by the students, the data show that 9.10% considered that their grade would be in the second quartile, 54.54% assured that their grade would be in the third quartile and 36.36% that his grades would be in the fourth quartile. No student indicated being in the first quartile. The treatment group was randomly selected among all the students that make up the double degree. A total of 35 students comprised this treatment group.

Random Experiment on Relative Performance Feedback in Higher …

133

Two tests were carried out during the semester, all students knew the numerical grades. The students in the treatment group, along with their grade, received a message by internal mail from the university with the percentile their grade was in with respect to the class and a brief explanation of what that meant. In the month of May, the official call exam was carried out and later a questionnaire was sent to the treatment group to determine if having known their percentile throughout the semester was an extra motivation to increase the effort for the official exam. The results of the questionnaire will be analyzed empirically when the investigation of the randomized experiment on relative performance with feedback is completed.

6 Analysis of the Results The starting hypothesis implies that students who know their percentile throughout the semester are able to get better grades at the end of it. The study determines the importance of feedback in the relative performance of URJC students, where chance is what assigns students to belong to the treatment or control group. The experiment was effective in informing the students belonging to the treatment group of their percentile compared to the control group that was not given access to this information, so they remained oblivious to their relative position with respect to the class. As mentioned above, this first trial of the randomized experiment on feedback is carried out on a very small sample of students, but it is the first step to carry out a more detailed investigation with a much larger sample of students in different subjects and grades of the URJC. To carry out the test, it is necessary to know in what relative position the students of the treatment group are in average terms with respect to the grades of all the students in the class. Table 1 indicates the most important percentiles in the distribution together with the grades corresponding to said percentile of all the students, before carrying out the experiment, in the tests carried out during the semester and the accumulated percentage. The results show that before carrying out the random experiment around 61.10% of the students were below 5, the minimum grade to pass the subject. Table 2 indicates the means, standard deviations and medians of both groups before performing the random experiment, it is observed that the students in the class had an approximate mean of 4,649 points out of 10. The analysis shows that there is a difference between the two groups in their mean scores of around 0.342 points, the mean of the control group being higher. However, the median of the treatment group is higher, this may be due to the fact that the mean includes all the extreme values of the distribution and, being such a small sample, the mean can be highly influenced by the existence of these extreme values.

134

C. M. Domínguez

Table 1 Percentile-score relationship before the experiment

Percentile

Qualification

Accumulated percentage (%)

10

1,075

9.30

20

2,025

18.50

30

2,75

31.50

40

3,65

40.70

50

4,462

53.80

60

5,01

61.10

70

5,725

70.40

80

7

83.30

90

9,15

90.70

100

10

100

Source Own elaboration from the notes obtained

Table 2 Data of the groups before performing the randomized experiment Group

Observations

Mean

Std. Dev

Percentile 50

Control group

28

4,814

2,725

4,225

Treatment group

26

4,472

2,652

4,5

Students (N)

54

4,649

2,670

4,462

Source Own elaboration from the notes obtained

Table 3 shows the number of students and the percentage that the students in the treatment group represent of the total in relation to the four quartiles and the students who did not show up. The data reflect that 40% of the students in the treatment group were below the median of the grades with respect to the class, this value being lower than the median at the beginning of the trial, which was located at 53.80%. On the other hand, only 11.44% of the students were among the highest grades in the group. Based on the data obtained in the questionnaire carried out at the beginning of the semester, 20% of students are in the second quartile while the estimate by the students was 9.10%. A total of 22.85% are in the third quartile while the estimate Table 3 Percentiles of the treatment group Students

Percentage (%)

Below or equal to the 25 percentile

7

20

Below or equal to the 50 percentile

14

40

Below or equal to the 75 percentile

22

62.85

Below or equal to the 100 percentile

26

74.29

9

25.71

Students not presented Source Own elaboration from the data collected

Random Experiment on Relative Performance Feedback in Higher …

135

made by the students was 54.54%. Finally, the presence in the fourth quartile is 11.44% and the estimate made at the beginning of the semester by the students was 36.36%. These results show the differences between reality and the estimation by students according to their beliefs and motivations at the beginning of the semester. Once the relative feedback has been carried out, the results obtained in the exam of the ordinary call for the month of May are analyzed for all the students presented to said exam. The number of students presented for this exam is greater than the number of students presented in the previous tests, as is the case with the averages of both groups. It can be noted that the mean of the control group is 0.391 higher than the mean of the treatment group and with a standard deviation lower than that of the treated group. In this case, the median agrees with the mean, since in the control group it is greater than in the treatment group. The data is reflected in Table 4. To analyze the data, an inferential study is carried out based on the contrast of hypotheses of equality of means. The contrast establishes that the population means of both groups are equal and uses the samples to determine if the observed evidence is consistent with these assumptions, that is, it consists of finding out if the data observed in the samples support the hypothesis about the populations. The test carried out in this study corresponds to the test of hypotheses for the equality of means of two normal populations, using the sample quasi-variance as an estimator. Due to the small number of the sample, a first contrast is made to verify that the variances are equal between both groups. Subsequently, the contrast of means for independent samples is carried out for the tests carried out in both groups before and after the feedback. The statistic used in this study is: t(x) = √

x¯ 1 − x¯ 2 (n1 −1)S21 +(n2 −1)S22 n1 +n2 −2

√ ∗ n11 +

1 n2

In addition, the p-value is shown, this being a direct measure of the probability that results from obtaining a sample like the current one if the null hypothesis is true. The p-value is used to indicate how much the current sample contradicts the alternative hypothesis. If the p-value is greater than the significance, we will not have sufficient evidence to reject the null hypothesis, while if it is less than or equal to, we reject the null hypothesis (Macías & Santero, 2022). In the present work we will indicate the significance values lower than 5%. Table 4 Data of the groups after performing the randomized experiment Group

Observations

Mean

Stan. Dev

Percentile 50

Control group

30

5,587

2,441

5,920

Treatment group

29

5,195

2,493

4,783

Students (N)

59

5,394

2,453

5,653

Source Own elaboration from the notes obtained

136

C. M. Domínguez

The results shown in Table 5 determine that there are no significant differences between both groups, since, as indicated by the significance results shown in Table 5, they are greater than 0.05. In this initial test, prior to the results of the final experiment where the sample will be much larger, it is observed that offering feedback on relative performance to students does not have a positive effect on their performance. The study also proves to be in agreement with the existing literature that indicates that students with feedback throughout the course may have worse grades by underestimating their position with respect to what they are unaware of, although it must be remembered that the sample has been very low, so the entire experiment has to be carried out in the following semester. In addition, based on the results, it is observed that the treatment group has always obtained lower grades than the control group, so the feedback does not show a worsening in the grades of the treatment group. This implies that the feedback has not generated a positive effect but not a negative one either. Throughout the course, class attendance was followed to find out if the students regained the motivation to return to the classroom after the COVID-19 pandemic. The average daily attendance is 38.37 students. A very low ratio considering that the enrollment of students in the subject is 69 students, which implies an attendance ratio of 55.60%, that is, just over half of the students attend class in person with an absolute minimum attendance throughout the semester of 36.23%. This trend of faceto-face attendance wants to be studied in future semesters and will be incorporated into the complete study once the randomized experiment on relative performance with feedback in Higher Education at URJC has been completed. Table 5 Significant differences between the groups before and after the experiment Control

Treatment

Total

t

Significance

Mean

4,814286

4,472115

4,649537

0,4669

0,6425

Std. Err

0,515112

0,520126

0,363432

Std. Dev

2,725721

2,652134

2,670672

Students (N)

28

26

54

Mean

5,587

5,195,632

5,394633

0,6091

0,5449

Std. Err

0,445809

0,462985

0,319456

Before

After

Std. Dev

2,441799

2,493,252

2,453795

Students (N)

30

29

59

Source Own elaboration from the data collected. Significances: < 0.05

Random Experiment on Relative Performance Feedback in Higher …

137

7 Conclusions The present randomized experiment on the relative performance feedback in students of the Financial Economic Analysis in Marketing II course at the Rey Juan Carlos University of Madrid, indicates that there are no significant differences in the grades of the students between the treatment group, a group that received relative feedback on their grades during the second semester of the year and the control group. This article is the essay of a larger study, so the sample of students is not large enough to be able to determine with complete certainty that feedback does not improve grades. During these last two semesters, the sample has been significantly expanded considering four new groups of URJC students, which implies a total sample of more than 475 students to date, although it is expected to continue expanding for the next academic year. This sample increase will show if the trend of this first essay is the general trend or if, on the contrary, the relative feedback causes an improvement in the grades of the students in Higher Education. In this study, the grades of the treatment group have always been below the grades of the control group, this implies that it cannot be assumed if the feedback causes a worsening of the grades due to greater self-confidence. At the beginning of the course, the students completed a questionnaire with the idea that the researchers could know the sample and the estimate that they considered appropriate of their grades in relative terms. It was found that the estimation of the students differs from reality. As for class attendance, it is very low compared to classroom attendance at prepandemic levels. This trend aims to be analyzed during the investigation long-term. In the complete study on the relative feedback that will be carried out in the longterm, a questionnaire will be carried out at the end of the course to the students so that they answer questions of a different nature, with the objective of knowing a posteriori the degree of implication and motivation that the students have had when knowing during the semesters analyzed their relative position with respect to the class. Because of this field experiment, robust results are expected to be obtained over several years, demonstrating that students who know the percentile of their grades throughout the course obtained higher results than their peers who do not know the percentile at the end of the course. Otherwise, as has happened in the first analysis that we have presented, it would be necessary to analyze whether excessive confidence can cause academic results to not improve or even worsen. Highlight that once the study is completed, it will add to the existing literature, a series of important considerations to consider with respect to providing students with correct feedback and will pose a series of relevant questions for those in charge of formulating policies educational.

138

C. M. Domínguez

References Anh, T., & Zeckhauser, R. (2012). Rank as an inherent incentive: Evidence from a field experiment. Journal of Public Economics, 96, 645–650. Azmat, G., & Iriberri, N. (2010). The Importance of relative performance feedback information: Evidence from a natural experiment using high school students. Journal of Public Economics, 94(7–8), 435–452. Azmat, G., & Iriberri, N. (2016). The provision of relative performance feedback: An analysis of performance and satisfaction. Journal of Economics and Management Strategy, 25(1), 77–110. Bandiera, O., Larcinese V., & Rasul, I. (2015), Blissful ignorance? A natural experiment on the effect of feedback on students’ performance. Labour Economics, 34, 13–25. Barankay, I. (2012). Rank incentives: Evidence from a randomized work place experiment. The Wharton School, Mimeo. Blader, S., Madras C., & Prat, A. (2015). The contingent effect of management practices. Columbia Business School Research Paper No. 15–48. Brotons, P., Virumbrales, M., Elorduy, M., Mezquita, M., & Balaguer, A. (2020). ¿Aprender Medicina a distancia?: Percepción de estudiantes confinados por la pandemia COVID-19. Revista Médica De Chile, 148(10), 1461–1466. Cabrales, A., Azmat, G., Bagues, M., & Iriberri, N. (2019). What you don’t know... Can’t hurt you? A natural eld experiment on relative performance feedback in higher education. Management Science, 65(8), 3449–3947. Charness, G., Masclet, D., & Villeval, M. (2013). The dark side of competition for status. Management Science, 60(1), 38–55. Eriksson, T., Poulsen, A., & Villeval, M. (2009). Feedback and incentives: Experimental evidence. Labour Economics, 16, 679–688. Ertac, S. (2005). Optimal information revelation in the presence of social comparisons: Theory and experiments. Mimeo. Galindo, N., & Vela, J. (2020). Motivación academica en tiempos de COVID-19 de estudiantes vinculados a universidades de Villavicencio: a partir de la teoría de Deci y Ryan. Universidad de Santo Tomas. Gill, D., Kissova, Z., Lee, J., & Prowse, V. (2016). First-place loving and last-place loathing: How rank in the distribution of performance affects effort provision. Mimeo. Hannan, R., Ranjani, K., & Andrew, H. (2008). The effects of disseminating relative performance feedback in tournament versus individual performance compensation plans. Accounting Review, 83(4), 893–913. Khunen, C., & Tymula, A. (2012). Feedback, self-esteem, and performance in organizations. Management Science, 58(1), 94–113. Macías, C., & Santero, R. (2022). El sistema educativo en el ámbito rural español. Un análisis comparativo con la educación en la ciudad en España. Revista Universitaria Europea, 37, 215– 248. Moore, D. A., & Cain, D. M. (2007). Overcon_dence and underconfidence: When and why people underestimate (and overestimate) the competition. Organizational Behavior and Human Decision Processes, 103, 197–213.

Gender Gap in STEM Education Rosa Belén Castro Núñez and Rosa Santero-Sánchez

1 Introduction The advances in the incorporation of women to the labour market along the past century have been one of the major social and economic achievements in recent history. It has had implication it almost all the society dimensions and it is still an ongoing process as several gender gaps remains (ILO, 2018). The 2030 UN Agenda for Sustainable Development acknowledge the challenges society needs to address for an equal participation of women in society. The agenda includes gender equality in several sustainability development goals (SDGs) from a transversal perspective, and specifically the 5th SDG is focused on gender equality and women’s empowerment. In terms of education, the 4th SDG has several targets that focus on eliminating gender disparities. Although the gaps have been reducing, last available data for OECD countries show that women earn on average 12% less than men, and that gap widens up to 23% for highly educated workers (OECD, 2022). Among the latter, one of the explanations to that gap is the different occupations men and women have, affected by an occupational segregation of women in lower paid activities. In particular, despite gender differences in education are closing, and women even outperform boys academically on average, girls are less likely than boys to choose the pathways through education that lead to the highest-paid professions, such as science, mathematics or computing (OECD, 2019a, 2019b). The low engagement of women in the so-called STEM occupations (science, technology, engineering and mathematics) has been widely studied, especially in a global context of increasingly digital and technological economy. Beyond the justice and equality argument (women and men should have equal opportunities) there is an increasing need of professionals in STEM occupations. These occupations are R. B. Castro Núñez (B) · R. Santero-Sánchez Universidad Rey Juan Carlos, Madrid, Spain e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_9

139

140

R. B. Castro Núñez and R. Santero-Sánchez

related to high value added and productivity activities, that have an impact not only in workers’ income but also in the country economic development. But on average, graduation rate in STEM programs has only increased by 1% from 2015 to 2020 and in some developed countries, including Germany, France or UK is even closer to zero (UNESCO, several years). Within this framework, the participation of women in STEM careers is still quite low and in some sub-fields, including ICT studies, it has increased from 2014 to 2020 (UNECE, several years). Finally, from a global perspective, the advances in the technology people use in their daily life push forward for an improvement in their technological and digital skills, that are acquired at schools. Addressing the gender gap in subject areas such as ICT is particularly important in the post COVID-19 pandemic scenario, which has accelerated the need for digital skills and (ILO, 2021). In this context, the gender gap in attitudes and motivations towards technological skills can contribute to additional gaps in the participation of women in society. Thus, gender-related biases in teenagers’ choices may thus have adverse consequences not only for the individual, but for society too (OECD, 2019b). Along this chapter we analyse latest data in STEM education indicators to identify patterns in the evolution of gender gaps and we review the theoretical framework that literature uses to understand the causes of those gaps. The conclusions shed some light about the experiences and lines of action in the context of the future challenges in education and labour market.

2 What Do We Mean by ‘STEM’? STEM is an acronym for Science, Technology, Engineering, and Mathematics. This term is unclear whether it refers to skills, literacy, fields of study (education), occupations, jobs or activity sector. Different institutions and researchers might use different definitions depending on analysed concept of STEM and its related database (Muñoz et al., 2021; Oleson et al., 2014; Speer, 2020). The traditional definition of STEM is based on math-intensive fields, including computer science, engineering, mathematics, physics and chemistry. This definition is considered “narrow”. Broad STEM delimitation includes traditional STEM fields and other less math-intensive, such as biological sciences, health care and business and economics sciences (Chan et al., 2021; Muñoz et al., 2021). Therefore, the intensity of the use of mathematics is a key factor to choose one or another definition. A girl or boy’s interest and performance in math and science during school years (primary and secondary) will directly affect the choices these students will make when entering tertiary education and choosing a career. Their aspirations together with other factors, like role models, parents’ occupations, or peers’ choices, are key factors in their career decision to entry and graduate in a STEM field of study. The subsequent graduation will partly influence whether labour market participation is in a STEM occupation, and/or a STEM-intensive sector (Gottfried & Bozick, 2016; Jacob et al., 2020; Muñoz et al., 2021) and its job characteristics (including

Gender Gap in STEM Education

141

Fig. 1 STEM life-cycle of an individual. Source Muñoz et al. (2021, p. 9)

earnings) (Fig. 1). In this sense, graduates in STEM programs typically earn more than non-STEM graduates (Chan et al., 2021). The analysis of STEM education differs depending on the educational stage of individuals. During the primary and secondary stages, the focus is on the results and acquisition of competencies in mathematics and science, as well as on the motivation and aspirations of students in these areas. The databases used in international comparisons are surveys such as TIMSS (Trends in International Mathematics and Science Study) and PISA (Programme for International Student Assessment). In these surveys, data are obtained at the individual and national level and there are a variety of indicators, such as average score in mathematics and science competencies, proficiency level of at least 4 over 6 (Stoet & Geary, 2018), percentage of boys and girls below the minimum level 2 and 5 or higher (Muñoz et al., 2021) and others. In relation to the STEM university programs, the literature uses as indicators both the entry to these degrees (enrolment) and graduation (graduate) in degrees and doctorates and the international organizations such as Eurostat, UNESCO and UNECE specify STEM including the educational classification levels ISCED-F 2013: 05-Natural science, mathematics and statistics, 06-Information and communication technologies, and 07-Engineering, manufacturing and construction. The educational field can be used to initially classify what constitutes a STEM occupation, job or economic sector (Oleson et al., 2014). Nevertheless, a STEM job could be less about mirroring the STEM field of study and more about the use of STEM skills in the main tasks of jobs (Muñoz et al., 2021). A pending issue is the inclusion of non-STEM workers that work in STEM fields, such as IT or scientific research (Smith & White, 2019), and STEM graduates that end up with non-STEM jobs in related fields such as teaching, businesses, and others. Teaching science and

142

R. B. Castro Núñez and R. Santero-Sánchez

mathematics from nursery school to university is one of the leading interdisciplinary education existing paradigms (Muñoz et al., 2021; UNESCO, 2017). In the labour market, each country determines how it approaches STEM occupations and sectors and there is some research comparing different measures. Still, there are some conventions at the international level that use the International Standard Classification of Occupations (ISCO), one of the main international classifications for which International Labour Organization (ILO) is responsible. The use of educational attainment is used implicitly to classify STEM workers so that most of the conceptualizations of the “STEM economy” favour professional occupations that require a bachelor’s degrees or higher (Oleson et al., 2014; Rothwell, 2013). These educational levels are associated with high qualification level, identified in the ISCO with major group 2 (professionals) and 3 (technicians and associate professionals). STEM fields in ISCO-08 correspond to 21 (science and engineering professionals) and 25 (information and communication technology professionals) in major group 2; and 31 (science and engineering associate professionals) and 35 (information and communication technicians). There exists a limitation: life science is including in 21 and 31, field is out of narrow definition of STEM. These considerations about definitions and indicators to analyse STEM concepts are not gender neutral. Therefore, the results presented in the following sections are linked to authors decision to approximate STEM education.

3 Gender Gap in STEM Education In this section, we focus on educational trajectory, from the acquisition of skills in primary and secondary education level to the pursuit of university STEM programs (degrees and doctorates). A large amount of literature from education, sociology, psychology and other disciplines has studied the gender gaps in STEM education and asked when and why those gaps appear and how they might be narrowed (Bostwick & Weinberg, 2018; Card & Payne, 2017; Delaney & Devereux, 2019; Fischer, 2017; Saltiel, 2019). This interest is not only about enrolment and graduation in STEM university programs, but also about differences in performance in math and science during the school years. As some authors have detected, gender differences in interest towards areas such as robotics are already identifiable in the early years of elementary school and are similar to differences at the secondary school stage (Sullivan & Bers, 2019). Recent research is converging toward the notion that gender differences in STEM are not due to differences in absolute cognitive ability (Ayuso et al., 2021; Valla & Ceci, 2014).

Gender Gap in STEM Education

143

3.1 Are Girls Worse Than Boys at Math? To analyse the gender differences in STEM at the primary and secondary levels, most of the literature use international surveys such as Trends in International Mathematics and Science Study (TIMSS), that monitors trends in mathematics and science achievement at the fourth and eighth grades, and Programme for International Student Assessment (PISA), that measures 15-year-olds’ ability to use their reading, mathematics and science knowledge and skills to meet real-life challenges (Muñoz et al., 2021; OECD, 2018; Stoet & Geary, 2018; UNESCO, 2017). To estimate whether a student would be capable and is motivated to study in STEM, we use mathematics score of PISA. Table 1 shows gender difference between the mean in maths score of two regions: Europe and North America (left) and Latin America (right). Values are gender gap calculated as difference between male and female students. Thus, if the value is positive, boys’ performance is higher than that for girls, and if it is negative, national average of girls is higher than that for boys. In Europe and North America, we found that girls outperformed boys in 15 (41.7%) countries in 2012 and in 21 (55.3%) in 2018. Island (2018) and Finland (2012) are the countries with better results for girls, and Italy (2018) and Luxembourg (2012) with higher gender difference. In Latin America, these results are very different. In 2012, globally boys outperformed girls, and in 2018, only Trinidad and Tobago shows a negative gender gap. Muñoz et al. (2021), using PISA 2015, show there are no significant differences by gender in performance in mathematics in Europe and Central Asia. In general, the average gender gap of the countries has decreased in recent years. In 2012, the average in Europe and North America was 0.71 and this value diminished to -0.30 in 2018, and the average for the Latin America decreased from 7.23 in 2012 to 4.50 in 2018. Focusing on Europe, countries with higher levels of gender equality, such as the Nordic countries, have gender gaps in favour of girls, who present better results in mathematics than boys. This is contrary to the results found in Stoet and Geary (2018), who using as a database from PISA 2015 and as an indicator the percentage of students having a level higher than 4 out of 6, conclude that “paradoxically, the sex differences in the magnitude of relative academic strengths and pursuit of STEM degrees rose with increases in national gender equality” (Stoet & Geary, 2018, p. 581). There does seem to be consensus that performance and interest in math during the school years affect the choices students will make when entering tertiary education and choosing a career stream. There is no consensus as to whether girls perform worse in mathematics than boys during the pre-university stages. Results vary by geographic area and by the indicators used to measure performance.

144

R. B. Castro Núñez and R. Santero-Sánchez

Table 1 Gender gap of average math score by countries. 2012 and 2018 Country

2012

2018

Country

2012

2018

Albania

− 0.71

− 3.52

Argentina

6.67

7.75

Austria

5.08

Belarus Belgium

0.73

1.15

Brazil

7.75

4.15

0.67

Chile

4.05

4.02

2.72

Colombia

12.24

9.96

Bulgaria

− 2.87

− 1.59

Costa Rica

14.34

9.14

Canada

0.86

− 0.28

Dominican Republic

0.52

0.59

Croatia

2.15

1.43

Ecuador

10.02

Czechia

3.45

− 0.85

Guatemala

1.88

Denmark

3.47

− 0.62

Honduras

6.46

0.19

Mexico

Estonia Finland

− 3.66

− 3.78

Panama

France

0.01

0.09

Peru

Germany

1.91

− 0.18

Trinidad and Tobago Uruguay

Greece

2.4

− 2.4

Hungary

1.44

3.61

Iceland

− 3.53

− 5.4

Ireland

3.48

0.03

Italy

5.48

5.43

Latvia

− 3.21

0.11

Lithuania

− 3.33

− 3.61

Luxembourg

8.63

1.87

Montenegro

0.44

3.28

Netherlands

1.93

− 1.31

North Macedonia

− 1.85

− 3.55

Norway

− 0.57

− 4.47

Poland

− 1.18

− 1.32

Portugal

1.93

− 0.12

Republic of Moldova

− 1.2

− 0.76

Romania

0.76

1.05

Russian Federation

− 1.37

− 0.25

Serbia

3.07

− 0.88

Slovakia

− 0.35

− 0.56

Slovenia

− 0.57

− 1.22

Spain

2.97

0.26

Sweden

− 2.21

− 1.38

Switzerland

0.54

1.22

7.78

5.73 3.69

6.01 − 8.47 5.71

3.54

(continued)

Gender Gap in STEM Education

145

Table 1 (continued) Country

2012

2018

United Kingdom of Great Britain and Northern Ireland

2.62

2.12

United States of America

− 1.34

1.55

Mean*

0.71

− 0.30

Country

2012

2018

7.23

4.50

Standard deviation

2.78

2.29

4.12

4.92

Min

− 3.66

− 5.4

0.52

− 8.47

Max

8.63

5.43

14.34

10.02

Nº countries with gap (−)

15

21

0

1

Nº countries with gap (+)

21

17

9

12

Source Processed by authors from UNESCO database. Note *unweighted mean

3.2 Is There Gender Difference in STEM Higher Education? Information on how girls and boys enter and stay in STEM higher education is less available at the regional level, even though UNESCO has tertiary level data on enrolment and graduation rates by gender and programs. In this section, we focus on graduation rates in STEM university programs at bachelor´s degree and doctorate level. Before presenting the gender differences in enrolment and graduation rates, it is important to review the temporal evolution of STEM graduates at the international level globally (both sexes), using ISCED (Engineering, manufacturing, construction; Information and Communication Technology—ICT; natural sciences, maths and statistics). Despite the importance of the sector in the economy and the need to have graduates, the latest data available for 2020 show that the average graduate rate is 22.7%, having grown slightly, 1.1 percentage points in 5 years (21.6 in 2015). Some countries present very high data, such as Iran (47.2% in 2015) and Turkmenistan (39.9% in 2020). Table 2 shows graduate rate in STEM programs (both sex) in 2015 and 2020, as well as the rate of change between both years, for the geographic areas of Europe and North America (left) and Latin America (right). Average graduation rates of STEM tertiary programs are higher in Europe and North America (average 25%) than in Latin America (average 18.8%). In Europe, Germany stands out especially for having a much higher percentage than average, with a graduation rate of over 36% in both years. Globally, there is a positive evolution of this indicator, with 63 countries where graduation rates are increasing (66% of the total), compared to 32 where they are decreasing. In the area of Europe and North America, 27 of the countries have a positive evolution (79%), compared to 7 that have decreased. These data are not as favourable in Latin America, where only 7 of 13 countries have grown (54% of the total for the area).

146

R. B. Castro Núñez and R. Santero-Sánchez

Table 2 Graduate rate (%) in STEM programs (both sex) Country

2015

2020

Var (%)

Country

Albania

18.1

20.1

2.0

Argentina

Austria

29.3

31.4

2.1

Brazil

Belarus

33.5

35.5

2.0

Chile

Belgium

17.4

17.6

0.2

Bulgaria

20.8

19.8

Canada

20.0

25.1

Croatia

23.9

Czechia

23.2

Denmark

2015

2020

Var (%)

15.4

15.4

15.3

18.5

3.2

20.1

20.6

0.5

Colombia

22.7

23.5

0.8

− 1.0

Costa Rica

12.9

16.2

3.4

5.0

Dominican Republic

14.3

27.3

3.4

Ecuador

16.7

25.9

2.7

Guatemala

9.8

19.6

22.5

2.9

Honduras

14.7

15.7

1.0

Estonia

26.5

27.9

1.4

Mexico

27.9

25.8

− 2.1

Finland

28.5

28.4

− 0.1

Panama

17.2

− 17.2

France

25.3

25.8

0.5

Peru

23.5

− 23.5

Germany

36.7

36.8

0.0

Trinidad and Tobago

Greece

28.2

27.3

− 0.8

Uruguay

Hungary

22.1

23.3

1.3

− 14.3 16.2

− 0.4 − 9.8

17.2

17.2

20.2

Iceland Ireland

24.9

25.3

0.4

Italy

23.3

24.5

1.1

Latvia

20.5

19.9

− 0.5

Lithuania

23.1

27.3

4.2

Luxembourg

13.8

19.0

5.1

Montenegro

20.5

Netherlands

18.6

North Macedonia

20.0

23.6

Norway

21.0

20.9

− 0.1

Poland

22.3

20.8

− 1.5

Portugal

27.9

28.0

0.1

Republic of Moldova

22.3

25.4

3.1

Romania

28.3

30.0

1.7

Russian Federation

29.0

31.4

2.3

Serbia

25.9

30.5

4.6

Slovakia

21.1

21.8

0.7

Slovenia

25.7

28.0

2.3

Spain

25.4

22.3

− 3.1

Sweden

26.0

27.3

1.3 (continued)

Gender Gap in STEM Education

147

Table 2 (continued) Country

2015

2020

Var (%)

Switzerland

24.4

25.4

1.0

United Kingdom of Great Britain and Northern Ireland

26.1

26.2

0.1

United States of America

17.4

19.2

1.9

Mean*

24.0

25.0

1.0

Standard deviation

4.64

4.64

Country

2015

2020

Var (%)

17.7

18.8

1.1

0.0

5.31

3.73

− 1.6

Min

13.8

17.6

3.7

9.77

15.42

5.7

Max

36.7

36.8

0.0

27.90

25.83

− 2.1

Nº countries decrease %

7

6

Nº countries rise %

27

7

Europe and North America and Latin America 2015 and 2020 Source Processed by authors from UNESCO database. Indicator Percentage of graduates from STEM programmes in tertiary education, both sexes (%). Note *unweighted mean

The gender gap (calculated as enrolment rate male-enrolment rate female) is presented below, differentiating between the three areas included in the field of study (Table 3). Negative values show that there is a gap in favour of women, with their enrolment rate being higher than that for men. As can be seen in Table 3, there are only female enrolment rates higher than male in Natural sciences, mathematics and statistics field in some European countries. 14 countries compared to 19/16 in 2014/ 2019, which represents between 42 and 47% of the total geographical area. In all three fields, the average enrolment rate in Europe and North America shows a positive gap, which means that more men than women are enrolled. The average gender gap is almost zero in Natural sciences, mathematics and statistics, having narrowed in recent years. However, the gasps in the fields of ICT and engineering are higher: 6.6 and 8.1 in 2014 and 2019 respectively in ICT, and 18.2 and 17.1 in 2014 and 2019 respectively in engineering. These data are aligned with previous research, such as Delaney and Devereux (2019). These authors found large gender enrolment gaps in engineering and technology, but no gaps in science. There is a perceived lack of information about engineering profession among students, and girls hold fewer positive views than boys about the areas of computer science or information and technology (Herbert & Stipek, 2008). Social dynamics in terms of gender stereotypes are at the core of the explanations for this gap, as it is analysed in the Sect. 6 of this chapter. Thus, the conclusions can be sensitive to the STEM definition and fields including in it. This should be taken into account in the design of strategies to foster the participation of women in those fields and to select the specific areas in which gender gaps are higher.

148

R. B. Castro Núñez and R. Santero-Sánchez

Table 3 Gender gap of enrolment rate (%). Tertiary students by STEM fields Natural sciences, mathematics and statistics

Information and communication technologies

Engineering, manufacturing and construction

Country

2014–15 2019–20 2014–15 2019–20 2014–15 2019–20

Albania

− 1.1

− 2.1

4.8

6.8

10.8

18.8

Austria

2.0

1.7

6.8

7.6

19.4

18.3

Belarus

− 1.1

− 0.7

6.6

8.1

35.1

32.9

Belgium

2.4

2.4

4.8

6.6

13.9

13.9

Bulgaria

− 1.0

− 1.2

3

5.4

19.6

19

Canada

1.3

0.1

4.2

7.8

16.5

15.1

Croatia

− 0.9

− 0.9

6

7

22.4

19.5

Czechia

0.3

− 0.4

8.3

10.5

17.2

14.7

Denmark

1.0

0.8

6.7

7

11.6

14.2

Estonia

− 0.1

− 0.2

10.5

12.5

21.5

17.8

Finland

0.7

0.2

13.3

13.1

25.5

24.3

France

4.8

1.7

4.4

4.7

15.1

17.5

Germany

1.0

0.7

7.1

8.1

22.9

21

Greece

2.5

2.3

1.7

2.9

18.8

17.6

Hungary

0.8

0.6

4.7

12.2

23.7

16.2

Iceland

2.3

2.0

11.2

9.2

10.8

11.1

Ireland

0.6

0.5

9.6

8.2

14.1

13.9

Italy

− 0.3

− 0.2

2.7

3.3

17.3

16.9

Latvia

− 0.3

− 0.2

8.6

10.7

20.9

21.6

Lithuania

0.2

0.0

5.3

10.9

26.3

22.8

Luxembourg

1.5

1.6

8

11.2

11.2

9.8

5.9

11.3

9.4

8.2

11.7

6.9

5.7

Montenegro Netherlands

2.2

1.9

North Macedonia

− 1.5

− 1.9

Norway

1.8

2.0

6.5

7.5

15.8

13.2

Poland

− 1.2

− 0.8

8.2

8.8

17.7

14.1

Portugal

− 0.7

− 0.2

2.5

4.1

23.8

22

− 1.6

− 1.6

6.5

6.8

19.1

17.7

Republic of Moldova Romania

0.0

9.3

16.3

Russian Federation Serbia

− 2.5

− 2.6

7.2

Slovakia

− 0.6

− 1.0

7.5

9.7

18.2

17.5

Slovenia

− 0.4

0.0

7.7

9.2

24.8

23.7

Spain

1.0

1.5

7.4

9

17.7

14.9

15 (continued)

Gender Gap in STEM Education

149

Table 3 (continued) Natural sciences, mathematics and statistics

Information and communication technologies

Engineering, manufacturing and construction

Country

2014–15 2019–20 2014–15 2019–20 2014–15 2019–20

Sweden

1.8

Switzerland

2.4

1.8

4.5

5.6

18.8

18.1

United Kingdom and Northern 2.3 Ireland

0.3

6.7

7.8

13.2

12.4

0.28

6.56

8.08

18.16

17.10

1.7

5.3

5.6

20.8

19.5

United States of America Mean*

0.58

Standard deviation

1.58

1.33

2.53

2.60

5.64

5.01

Min

− 2.5

− 2.6

1.7

2.9

6.9

5.7

Max

4.8

2.4

13.3

13.1

35.1

32.9

Nº countries gap (−)

14

14

0

0

0

0

Nº countries gap (+)

19

16

33

33

34

34

Europe and North America 2014–2019 Source Processed by authors from UNECE database. Indicator tertiary students are those enrolled in levels 5–8 of the ISCED 2011. ISCED-F 2013 is a classification of fields of education and training which accompanies ISCED 2011. Note *unweighted mean

4 Gender Gap in STEM Education in Europe: Spain Versus European Union Spain presents a positive evolution in relation to gender gaps in STEM education, although it is not one of the countries that stands out for having more favourable positions for women than men in the indicators reviewed in the previous section. For example, Spain has decreased its gender gaps in math score from 2.97 in 2012 to 0.26 in 2018 and its enrolment rate is higher for males in all STEM fields. Moreover, the STEM graduation rate for both sexes has decreased over time, from 25 to 23%. The objective of this section is to compare Spain, a country with national policies to promote gender equality with the rest of the EU, that have common programs to promote the presence of women in STEM. The Gender Equality Strategy 2020– 2025 launched by the European Union (2021) has acknowledged the need to close the gender gap in ICT studies and among STEM graduates in a context of an increasing digitalized economy that is in constant transformation. These commitments include an updated Digital Education Action Plan, the implementation of the Women in Digital Declaration and the Communication on the European Education Area (European Union, 2021). The evolution of the weight of STEM graduates over total graduation by sex is presented in Table 4. STEM has been calculated as the sum of the three fields identified above (Table 3). The gender gap has been calculated as the difference

150

R. B. Castro Núñez and R. Santero-Sánchez

between female and male STEM percentage over total. The same years have been selected (2014 and 2019) and the EU-28 is used for comparison. The average values of the weight that STEM-fields have over the total number of graduates in the EU-28 are around 25% (25.1% in 2014 and 26.1 in 2019). These data show that 1 in 4 graduates have pursued their university career in STEM-fields. The gender differences are large, since while men reach 40%, women are around 15%, generating a gender gap of 25 percentage points. Data for Spain are not very different in relation to the EU-28 average, although with slightly lower values. The weight of graduation in STEM was 24.3% in 2014 and reduced to 22% in 2019. This reduction has occurred for both men and women, affecting men more and slightly reducing the gender gap. If we compare Spain with some of the Nordic countries, which are known for the development of their equality policies, such as Sweden or Finland, we see that the gender differences are greater in these countries. This is a result of the higher rate of graduation for men, almost reaching half of the total number of graduates, than for women. Although, as we have already seen in the previous section, it is necessary to differentiate the participation in specific fields within STEM, it seems that women are still far from reaching the male values.

5 Factors Influencing Gender Differences in STEM Education The STEM life-cycle of an individual analysed in Sect. 2 (Fig. 1) shows the interaction between individual and social spheres in the decisions men and women take along their lives. Gender differences in education and the subsequent participation in labour market is deeply related to gender roles in society. These roles and stereotypes affect their academic outcomes in areas such as mathematics and science, but also general self-perception and self-efficacy. Moreover, these perceptions are influenced by their social context, from family expectations to peers’ beliefs and, ultimately, society gender biased dynamics (Olsson & Martiny, 2018; UNESCO, 2017). The phenomenon is a multidimensional and complex one, with several factors affecting women along their life that interact and overlap in different ways. Literature often organizes them around two dimensions: individual and social factors (Miner et al., 2018), that can also be approximated as internal and external factors. This division is commonly used to summarize the theoretical framework to explain gender gaps in different aspects of women participation in society. In this line, UNESCO in its report about girls’ and women’s education in STEM (UNESCO, 2017) suggests an interesting approach defined as ecological framework (Fig. 2). The idea is that factors are organized from the perspective of the student, starting by an inner circle including individual factors and a set of outer ones, that categorize those factors from closer to the individual to a more global context. Thus, for example in terms of girls’

Gender Gap in STEM Education

151

Table 4 % Graduate in STEM university programs by sex in European Union % STEM over total

Total

Gender gap in % STEM (male–female)

Female

%Var. gender gap (2019–2014)

Male

2014

2019

2014

2019

2014

2019

2014

2019

European Union—28

25.1

26.1

14.7

15.4

39.5

40.4

24.8

25.0

0.2

Austria (UE-15)

30.9

31.4

14.5

15.6

51.2

50.4

36.7

34.8

− 1.9

Belgium (UE-12)

17.9

17.6

7.9

7.8

32.9

31.6

25.0

23.8

− 1.2

Bulgaria

20.6

19.8

13.1

11.7

31.9

32.1

18.8

20.4

1.6

Croatia

23.4

27.3

14.0

16.7

37.3

43.3

23.3

26.6

3.4

Cyprus

17.0

13.6

10.3

7.7

28.4

23.9

18.0

16.2

− 1.9

Czechia

22.5

25.9

12.7

15.6

37.1

42.0

24.4

26.4

2.0

Denmark (UE-12)

20.8

22.5

12.9

13.5

31.9

34.1

19.0

20.6

1.6

Estonia

26.4

27.9

16.4

17.7

46.2

45.8

29.8

28.1

− 1.6

Finland (UE-15)

28.2

28.4

12.8

13.1

51.6

50.6

38.8

37.5

− 1.2

France (UE-12)

25.6

25.8

14.1

14.5

40.2

39.9

26.1

25.4

− 0.7

Germany (UE-12)

36.0

36.8

19.4

19.2

52.9

54.1

33.5

34.9

1.4

Greece (UE-12)

29.8

27.3

19.5

18.8

44.1

40.1

24.6

21.3

− 3.3

Hungary

20.8

23.3

10.3

11.6

38.3

40.3

28.0

28.7

0.7

Ireland (UE-12)

21.6

25.3

10.9

14.9

33.3

37.8

22.3

23.0

0.6

Italy (UE-12) 23.8

24.5

16.2

16.6

35.2

35.2

19.1

18.6

− 0.5

Latvia

21.3

19.9

11.0

9.5

40.6

39.5

29.6

30.0

0.4

Lithuania

23.7

27.3

11.5

13.5

44.5

49.3

33.1

35.9

2.8

Luxembourg (UE-12)

13.9

19.0

7.0

9.0

22.3

30.3

15.3

21.3

6.0

Malta

24.9

17.1

12.3

8.6

40.1

28.3

27.8

19.7

− 8.1

Netherlands (UE-12)

11.7

18.6

5.9

10.2

19.3

29.1

13.4

18.9

5.5

Poland

20.4

20.8

13.2

13.4

34.5

35.1

21.3

21.6

0.3

Portugal (UE-12)

27.6

28.0

18.2

17.6

41.1

42.5

22.9

24.9

2.0

Romania

28.0

30.0

19.7

21.7

39.7

42.0

20.0

20.3

0.3 (continued)

152

R. B. Castro Núñez and R. Santero-Sánchez

Table 4 (continued) % STEM over total

Total 2014

Gender gap in % STEM (male–female)

Female 2019

2014

%Var. gender gap (2019–2014)

Male 2019

2014

2019

2014

2019

Slovakia

20.8

21.8

12.3

12.2

35.6

37.1

23.3

25.0

1.7

Slovenia

26.2

28.0

13.6

15.2

45.4

47.1

31.7

31.8

0.1

Spain (UE-12)

24.3

22.0

12.6

11.0

39.3

36.0

26.7

25.0

− 1.8

Sweden (UE-15)

25.7

27.3

13.9

15.8

45.2

46.3

31.3

30.5

− 0.8

United Kingdom (UE-12)

25.6

26.2

17.0

18.2

37.1

37.5

20.1

19.3

− 0.8

Source Processed by authors from Eurostat database

participation in STEM education, the study would start with individual elements, and then analyse the influence of family and peers, school, and ultimately, society. This conceptual framework allows the identification of areas of potential action and the development of policies to reduce gender gaps in STEM.

Fig. 2 Ecological framework of factors influencing girls’ and women’s participation, achievement and progression in STEM studies. Source UNESCO (2017, p. 40)

Gender Gap in STEM Education

153

5.1 Individual Factors Traditional explanations to the lower involvement of women in technical fields relayed in biological arguments, by which men and women had differences in brain dynamics, but there is no conclusive evidence about how those differences affect skills, learning and education processes skills, whether they are relevant and if they are higher that differences among sex individuals (Eliot, 2013; Kersey et al., 2019; Wang et al., 2013). Moreover, recent advances in the comprehension of neuroplasticity highlight the relevance of environmental factors in the development of brain structure and dynamics (Spearman & Watt, 2013), including self-awareness of that influence (Eliot, 2013). Individual phycological elements are complex to isolate, as individual decisions are crucially affected by external factors. Motivation, engagement and even performance of girls in STEM studies are influenced by self-perception and self-efficacy in terms of skills, capacities and abilities and also how girls perceive the STEM professions concerning opportunities and working environment. In this regard, society stereotypes about women and the associated gender roles influence children and young people motivations, self-concept and self-efficacy in STEM studies and professions. In particular, literature suggests that two stereotypes are relevant: the idea that boys are better than girls in math and science and that science and engineer jobs are male fields (Bian et al., 2017; Hill et al., 2010). Regarding performance, last results from PISA, in 2018, show that in average terms for OECD countries, boys outperform girls by five score points in mathematics while girls outperform boys by two points in science. Indeed, in 55% of the countries and economies (35 out of 64), gender gaps at the top of the distribution of mathematics performance were not significant (OECD, 2019a). Although there is an important diversity in the gap by country as observed in previous sections, on average the gender gap in cognitive abilities related to STEM is closing. But, while the gender gap in performance is decreasing, when students are asked about their interest in occupations that gap remains significant. Among boys 8% report interest in working in ICT-related occupations, while only 1% of girls do it (OECD, 2019a) and the gender gap in interest in these fields have widened according to the results obtained in 2015 and 2018 (OECD, 2019b). Within the top-performing students in science or mathematics, only 14% of girls expect to work as professionals in science or engineering compared with 26% of the boys (OECD, 2019a). Thus, differences in performance are not enough to explain the gender gap in terms of the education and careers expectation of girls and boys. More complex psychological and external factors contribute to the explanation of that gap. Self-perception plays a major role in girls’ aspirations and is affected by the assimilation of gender stereotypes by which they perceive themselves as worst in mathematics and other STEM related skills (Ayuso et al., 2021) in a context in which they identify these fields as male domains. Indeed, a study with US children found that stereotypes that associate intellectual capacity to men appear as soon as at the age of six (Bian et al., 2017).

154

R. B. Castro Núñez and R. Santero-Sánchez

Moreover, this lower self-concept of girls in mathematics appears even when they perform similarly to boys (Petersen & Hyde, 2017). An interesting perspective focus on the perceived relative performance by boys and girls, that is at the core of the explanation Stoet and Geary develop in their work about the gender gap paradox that some countries experience. Their analysis of secondary and tertiary education results finds countries with high gender equality in global terms but with a specific gender gap for science performance and in STEM graduation (in favour of boys) larger than in countries with less gender equality (Stoet & Geary, 2018). They suggest that relative differences by gender are more important to self-perception than absolute differences. Thus, as girls are relatively better in reading, comparing with differences with boys in science and mathematics they might have the impression that they have a comparative advantage in reading (and in the same sense, boys might believe their relative strength is in mathematics). Literature also points to the relevance of attitudes about failure, competition and competitive environments in the explanation of gender differences in STEM vocations, and in particular in the context of the belief that STEM and technical professions are quite competitive environments. In this sense, results from PISA 2018 show that girls are more concern about failure than boys, especially among top-performing students, even when they show better performance (OECD, 2019b). Moreover, failure make girls doubt about their plans for future to a greater extent than to boys (two in three compared with less than one in two) in line with boy’s more positive attitudes towards competition (OCDE, 2019b). These differences towards competition form at early years and might persist latter on (Gneezy & Rustichini, 2004; Lackner, 2016; Niederle & Vesterlund, 2011). The influence of stereotypes and gender roles in STEM identities and girls’ selfperception also translates into differences in self-efficacy (Murphy et al., 2019). A survey conducted in US’ schools and universities among female students in STEM fields found that participants who experience gender bias had lower STEM selfconcept than participants that did not. Interestingly and in line with the relevance of external factors, the existence of a supportive network of peers decreased this effect (Robnett, 2016). As previously stated, these gender differences in self-efficacy are related to gender gaps in motivations, aspirations and even performance in STEM education and careers (Murphy et al., 2019). The second crucial gender stereotype is related to the low participation on of women in STEM related jobs. This remains a challenge and can reinforce a vicious circle of the male-dominated perception for these careers. The visibilización of women in STEM professions is essential to reduce gender stereotypes in that field as explained in the next section.

5.2 External Factors Individual motivations and choices cannot be understood and interpreted from an isolated perspective, as they are a result of interaction within society. In particular,

Gender Gap in STEM Education

155

these interactions affect engagement and achievements in education. Boys and girls are constantly exposed to stereotypes and bias on gender roles in the social construction of their identity, both in their closer reality, at home and school, but also in their interactions from a more global perspective, as members of a society. These interactions affect their motivation and engagement in STEM studies (Davila Dos Santos et al., 2021; Olsson & Martiny, 2018). Indeed, even when girls individually don’t approve gender stereotypes, the idea that people around them share those beliefs influences their decisions, decreasing the interest in STEM carers (Shapiro & Williams, 2012). (A) Family factors Family constitutes the first level of socialisation and the existence of gender stereotypes within families can prevent girls to engage in STEM studies (Wang & Degol, 2013). Answers by parents to specific questions included in PISA 2012 show that they expect their sons to work in a STEM field to a greater extent than their daughters, even if the children show similar performance in mathematics (OECD, 2015). Indeed, Rodríguez-Planas and Nollenberger (2018) analysed the results for several waves of PISA and conclude that parents’ gender social norms influence their children performance in mathematics. Parents transmit their expectation regarding girls’ cognitive skills, different from those expected for boys (Rodríguez-Planas & Nollenberger, 2018). The education and profession of parents also influence their children’s decisions, as they are primary role models. For instance, women scientists more frequently have parents who are scientist than their male colleagues (Tenenbaum & Leaper, 2003). Finally, even though the interactions between socio economic status and gender are beyond this analysis, literature acknowledge that the socioeconomic status of families affects differently boys and girls. For instance, results from PISA 2018 show that the gender gap in performance in mathematics and science between boys and girls of similar socio-economic status was not significant (OECD, 2019a) and thus, global values are reflecting to some extend the implications of socioeconomic contexts in girls’ engagement in STEM. (B) School and peers level factors Schools are mayor actors in motivation and career decisions of students. The type of environment their create is crucial, and the initiatives focused in reducing stereotypes regarding girls and STEM areas they develop contribute to the decrease in gender gaps in those fields (OECD, 2019b). The existence of gender-bias attitudes of teachers towards girls and STEM exercises a negative influence in their achievement and affects their decisions in enrolment in advanced STEM related courses (Lavy & Sand, 2018). Moreover, the existence of female teachers in STEM related courses influences girls’ decisions. School environments that present female role models (teachers, professionals, etc.) exercises a positive influence in the motivation of girls in enrolling in STEM careers (Breda et al., 2020; Olsson & Martiny, 2018). This is still a challenge as on average, there is a low proportion of female teachers with specialization in science and mathematics and their influence interacts with other school related factors

156

R. B. Castro Núñez and R. Santero-Sánchez

(UNESCO, 2017). Indeed, regardless the gender of teachers, teaching strategies need to incorporate methodologies and materials (textbooks and other tools) and develop learning environments that address gender differences in motivation and attitudes towards learning, avoid gender biased messages and constructing gender-responsive environments (UNESCO, 2017; UNICEF, 2020). For instance, field experiments with engineering undergraduate female students in US show that the interactions with same sex experts and professors have a positive influence. In particular, they found a positive impact on their self-conceptions, their effort and their performance in exams related to math skills and also in their interest in pursuing STEM careers (Stout et al., 2011). Also, programs that focus on early stages of education show positive results. For instance, the program “Little Adventurers” conducted in Colombia in 2019 takes a global approach to fight STEM gender stereotypes in young children (ages 3–5) in a field experiment with control group. The program combines the use of toolkit of teaching guides and material to be used at schools, video educational program by Sesame Street characters and other complementary actions. The preliminary impact results show better math skills in girls, an increase in the interest in that field and a reduction of stereotypes in instructors. But they also confirm the existence of gender stereotypes in terms of STEM toys among the children participating (Inter-American Development Bank, 2023). Both within and outside schools, peers’ influence in children decision is determinant in all socialization processes. Social and gender norms and attitudes held by peers regarding identities and abilities affect girls’ motivation and achievement in mathematics (Eble & Feng, 2022) as well as interest in STEM career choices (Robnett & Leaper, 2013). Finally, it is important to mention that most of the research on gender differences in STEM education are based on surveys and data who attend schools. Thus, general gender disparities in girls’ access to education exacerbates gender gaps in STEM field. (C) Society factors Society constitutes the ultimate global level in which women and men interact, defining their roles and constructing stereotypes and counterstereotypes. This process directly affects child’s behaviour and aspirations (Olsson & Martiny, 2018). Social and cultural norms, legislation, policies and initiatives have a multidimensional effect on individual choices, and thus, society perceptions about gender roles and stereotypes are relevant factors in explaining gender gaps in STEM. A review of different programs conducted in US identifies as productive programs those that are focused in fostering the engagement of young people, that take into account their interest, experiences and social environment and that promote the establishment of connections with schools, home and other environments (National Research Council, 2015). For instance, Techbridge is a nonprofit organization that develop after-school and summer programs that aim to foster the interest of girls in underserved communities in STEM. They conduct programs in collaboration with other institutions that include training adults that serve as role models or providing girls with opportunities to directly engage in real world application (individual projects with mentors) of

Gender Gap in STEM Education

157

STEM practices to foster their interest in them using hands-on learning in out of school environments (National Research Council, 2015).

6 Conclusions There is an increasing concern about the impact of robotization and technological change in production processes and, in particular on jobs. For instance, in developed countries, such as Europe, technology is substituting some jobs, but also creating new demand for highly qualified jobs in technological activities (Muñoz et al., 2021). Regardless the rise in the demand for these qualifications, the rate of graduates in STEM field degrees is still low, around 25% of all graduates (UNESCO, several years). The design of strategies to attract students into STEM professions needs to include specific measures focused on girls, as they show less interest and engagement than boys in these studies. Results on PISA and other surveys show that the gender gap in performance in mathematics and science is closing, although there is an important diversity of situations that reflects cultural, social and economic dynamics. Thus, academic outcomes might influence personal decisions, but the leaking STEM pipeline women face along school and labour market is a multidimensional and complex phenomenon. Personal motivations, decisions and engagement are influenced by the social context, in which schools play a key role as children socialize and make their first decisions in terms of subjects of interest within that environment. School policies, teachers’ attitudes and the material and strategies they develop in their curses might be influenced by gender bias, both conscious and unconscious. Measuring teaching practices is a first step towards improving teaching practices. For example, the World Bank recently developed Teach, a free classroom observation tool, that measures whether the teacher creates a gender bias free culture (Hammond et al., 2020). This in turn, poses barriers to the desired increase in the participation of girls in STEM studies and jeopardize the retainment within that field, especially in the transition from school to university and from university to the labour market. Schools also reflect society patterns, and the lack of female role models in STEM areas, both within professors and professionals, directly affects the construction of STEM identity, that remains male-related. Some studies on secondary and tertiary education stress out the importance of female mentors and role models to women’s participation in STEM (Hammond et al., 2020). There is a numerous literature that analyse the factors that explain the existing gender gaps in different levels of education in STEM fields, reviewed in the previous section. There is also literature focused on identifying and analysing initiatives and policies designed and implemented to reduce the mentioned gaps (a systematic review can be consulted in Prieto-Rodriguez et al., 2020, van den Hurk et al.,

158

R. B. Castro Núñez and R. Santero-Sánchez

2019). For instance, Prieto-Rodriguez et al. (2020) identify the combination of inclusive curriculum and pedagogies with exposure to female role models in education as successfully initiatives in education. Moreover, international institutions have analysed initiatives and make recommendations to reduce gender gaps in STEM, among which those to be implemented in schools play a key role. For instance, UNICEF provide a set of critical aspects that should be integrated in the design of initiatives, such as the revision of learning materials to remove gender bias and stereotypes or the development of programs to support mentorships and girls’ school-to-work transition (UNICEF, 2020). The World Bank explores the scant evidence on effective methods to close STEM gender gaps (Hammond et al., 2020). The revision of initiatives also reveals the need for a more clear and rigorous design in the measuring, reporting, assessment and evaluation of the intervention, including impact analysis methodologies. Henninger et al. (2019) performed a systematic review of 21 studies focused on the participation of women in STEM, based on a measurement of impacts or the existence of an associated impact evaluation (mainly, with experimental and quasi-experimental techniques). The analysis draws on the main areas that emerge in the wider literature, while noting the need for more rigorous evidence, key aspect to the identification of successful actions. In addition, some studies only examine immediate shifts following exposure to an intervention and the future research should seek to investigate longitudinal changes and whether they produce meaningful modifications in women’s career interests and goals (Wang & Degol, 2017). The reduction of gender gaps in education, and in particular in STEM fields, has been in the agenda along the last decades, but the covid-19 pandemic has shown that gender equality remains a crucial challenge for societies. Literature post covid19 conclude that the pandemic has worsened gender gaps in education and labour market. In the lockdown period women disproportionally assumed home-schooling, childcare and home duties compared to men, a situation that had an impact on their participation in labour market (ILO, 2021). Women remains showing a relatively higher participation in health and care services as well as a lower- and middle-skilled jobs in comparison with men. This occupational segregation is at the core of the explanations of the increase in some gender gaps in labour market along and after the covid-19 crisis, including in income, stability and even well-being. In this sense, the low participation of women in STEM related occupations is a key factor in the increase of the mentioned gaps. Moreover, from a global perspective, the differences in the level of technological content of tasks and skills of occupations have been identified as key factors in how covid-19 crisis has turned into labour losses, both in employment participation and in the quality of the remaining jobs (ILO, 2021). Thus, societies have a crucial challenge in the existing gender gaps in STEM education as their directly affect the future of current generations of students. STEMderived skills are crucial not only in terms of accessing to better jobs, but they have also become basic skills in the participation of people in society.

Gender Gap in STEM Education

159

Moreover, from a global perspective, in a world that is increasingly becoming more technological and digital, the level of STEM skills of a country labour force is source of economic development. Thus, the reduction of gender gaps in this respect should be a central objective in the political agenda.

References Ayuso, N., Murillo, A. C., & Cerezo, E. (2021). Gender gap in STEM: A cross-sectional study of primary school students’ self-perception and test anxiety in mathematics. IEEE Transaction on Education, 64(1), 40–49. https://doi.org/10.1109/TE.2020.3004075 Bian, L., Leslie, S. J., & Cimpian, A. (2017). Gender stereotypes about intellectual ability emerge early and influence children’s interests. Science, 355(6323), 389–391. https://doi.org/10.1126/ science.aah6524 Bostwick, V. K., & Weinberg, B. (2018). Nevertheless she persisted? Gender peer effects in doctoral STEM programs. NBER Working Paper 25028. Breda, T., Grenet, J., Monnet, M., & Van Effenterre, C. (2020). Can female role models reduce the gender gap in science? Evidence from classroom interventions in French high schools. IZA DP 13163. Card, D., & Payne, A. (2017). High school choices and the gender gap in STEM. NBER Working Paper 23769. Chan, P., Handler, T., & Frenette, M. (2021). Gender differences in STEM enrolment and graduation: What are the roles of academic performance and preparation? Economic and Social Report, Statistics Canada. https://doi.org/10.25318/36280001202101100004-eng Davila Dos Santos, E., Albahari, A., Díaz, S., & De Freitas, E. (2021). Science and technology as feminine: Raising awareness about and reducing the gender gap in STEM careers. Journal of Gender Studies, 31(4), 505–518. https://doi.org/10.1080/09589236.2021.1922272 Delaney, J. M., & Devereux, P. J. (2019). It’s not just for boys! Understanding gender differences in STEM. Economics of Education Review, 2019(72), 219–238. Eble, A., & Feng, H. (2022). Gendered beliefs about mathematics ability transmit across generations through children’s peers. Nature Human Behaviour, 6, 868–879. https://doi.org/10.1038/s41562022-01331-9 Eliot, L. (2013). Single-sex education and the brain. Sex Roles, 69, 363–381. https://doi.org/10. 1007/s11199-011-0037-y Fischer, S. (2017). The downside of good peers: How classroom composition differentially affects men’s and women’s STEM persistence. Labour Economics, 46, 211–226. Gneezy, U., & Rustichini, A. (2004). Gender and competition at a young age. American Economic Review, 94(2), 377–381. https://doi.org/10.1257/0002828041301821 Gottfried, M. A., & Bozick, R. (2016). Supporting the STEM pipeline: Linking applied STEM course-taking in high school to declaring a STEM major in college. Education Finance and Policy, 11(2), 177–202. Hammond, A., Matulevich, E. R., Beegle, K., & Kumaraswamy, S. K. (2020). The equality equation. Advancing the participation of women and girls in STEM. World Bank. Henninger, N., Muñoz Boudet, A., & Rodriguez-Chamussy, L. (2019). Evidence from policy interventions promoting girls’ participation in STEM. A systematic review. World Bank. Herbert, J., & Stipek, D. (2008). Are computer science and information technology still masculine fields? High school students’ perceptions and career choices. Computer and Education, 51(2), 594–608. Hill, C., Corbett, C., & St. Rose, A. (2010). Why so few women in science technology engineering and mathematics. American Association of University Women.

160

R. B. Castro Núñez and R. Santero-Sánchez

ILO. (2018). World employment and social outlook: Trends for women 2018. International Labour Office. ILO. (2021). World employment and social outlook: Trends 2021. International Labour Office. Inter-American Development Bank. (2023). Pequeñas Aventureras (Little Adventurer Girls)— Education to close the language, science, and math gaps between girls and boys in early childhood. Retrieved June 20, 2023 from https://desarrollo-infantil.iadb.org/en/innovations/lac_reg ion/pequenas-aventureras-little-adventurer-girls-education-close-language Jacob, M., Iannelli, C., Duta, A., & Smyth, E. (2020). Secondary school subjects and gendered STEM enrollment in higher education in Germany, Ireland and Scotland. International Journal of Comparative Sociology, 61(1), 59–78. https://doi.org/10.1177/0020715220913043 Kersey, A. J., Csumitta, K. D., & Cantlon, J. F. (2019). Gender similarities in the brain during mathematics development. npj Science of Learning, 4, 19. https://doi.org/10.1038/s41539-0190057-x Lackner, M. (2016). Gender differences in competitiveness. IZA World of Labor. https://doi.org/10. 15185/izawol.236 Lavy, V., & Sand, E. (2018). On the origins of gender gaps in human capital: Short- and long-term consequences of teachers’ biases. Journal of Public Economics, 167, 263–279. https://doi.org/ 10.1016/j.jpubeco.2018.09.007 Miner, K., Walker, J., Bergman, M., Jean, V., Carter-Sowell, A., January, S., & Kaunas, C. (2018). From “her” problem to “our” problem: Using an individual lens versus a social-structural lens to understand gender inequity in STEM. Industrial and Organizational Psychology, 11(2), 267– 290. https://doi.org/10.1017/iop.2018.7 Muñoz, A. M., Rodriguez, L., Chiarella, C., & Oral, I. (2021). Women and STEM in Europe and Central Asia. World Bank. Murphy, S., MacDonald, A., Wang, C. A., & Danaia, L. (2019). Towards an understanding of STEM engagement: A review of the literature on motivation and academic emotions. Canadian Journal of Science, Mathematics and Technology Education, 19, 304–320. https://doi.org/10.1007/s42 330-019-00054-w National Research Council. (2015). Identifying and supporting productive STEM programs in outof-school settings. Committee on successful out-of-school STEM learning. Board on Science Education, Division of Behavioral and Social Sciences and Education. The National Academies Press. Niederle, M., & Vesterlund, L. (2011). Gender and competition. Annual Review of Economics, 3(1), 601–630. https://doi.org/10.1146/annurev-economics-111809-125122 OECD. (2015). The ABC of gender equality in education: Aptitude, behaviour, confidence. OECD Publishing. OECD. (2018). Education at a glance 2018. OECD Publishing. OECD. (2019a). OECD programme for international student assessment (PISA) 2018 results. (Volume I) What students know and can do. OECD Publishing. OECD. (2019b). OECD programme for international student assessment (PISA) results (Volume II): Where all students can succeed. OECD Publishing. OECD. (2022). OECD employment outlook 2022 building back more inclusive labour markets. OECD Publishing. Oleson, A., Hora, M. T., & Benbow, R. J. (2014). What is a STEM job? How different interpretations of the acronym result in disparate labor market projections. Technical Report (Center on Education and Work, University of Wisconsin), Viewpoint paper 1, Sept. https://doi.org/10. 13140/RG.2.1.1454.0003 Olsson, M., & Martiny, S. (2018). Does exposure to counterstereotypical role models influence girls’ and women’s gender stereotypes and career choices? A review of social psychological research. Frontiers in Psychology, 9, https://doi.org/10.3389/fpsyg.2018.02264 Petersen, J. L., & Hyde, J. S. (2017). Trajectories of self-perceived math ability, utility value and interest across middle school as predictors of high school math performance. Educational Psychology, 37, 438–456. https://doi.org/10.1080/01443410.2015.1076765

Gender Gap in STEM Education

161

Prieto-Rodriguez, E., Sincock, K., & Blackmore, K. (2020). STEM initiatives matter: Results from a systematic review of secondary school interventions for girls. International Journal of Science Education, 42(7), 1144–1161. https://doi.org/10.1080/09500693.2020.1749909 Robnett, R., & Leaper, C. (2013). Friendship groups, personal motivation, and gender in relation to high school students’ STEM career interest. Journal of Research on Adolescence, 23(4), 652–664. https://doi.org/10.1111/jora.12013 Robnett, R. D. (2016). Gender bias in STEM fields: Variation in prevalence and links to STEM self-concept. Psychology of Women Quarterly, 40(1), 65–79. https://doi.org/10.1177/036168 4315596162 Rodríguez-Planas, N., & Nollenberger, N. (2018). Let the girls learn! It is not only about math … it’s about gender social norms. Economics of Education Review, 62, 230–253. https://doi.org/ 10.1016/j.econedurev.2017.11.006 Rothwell, J. (2013). The hidden STEM economy. Brookings Institution. Saltiel, F. (2019). What’s math got to do with it? Multidimensional ability and the gender gap in STEM. Meeting Papers 1201, Society for Economic Dynamics. Shapiro, J. R., & Williams, A. M. (2012). The role of stereotype threats in undermining girls’ and women’s performance and interest in STEM fields. Sex Roles: A Journal of Research, 66(3–4), 175–183. https://doi.org/10.1007/s11199-011-0051-0 Smith, E., & White, P. (2019). Where do all the STEM graduates go? Higher education, the labour market and career trajectories in the UK. Journal of Science Education and Technology, 28(1), 26–40. https://www.jstor.org/stable/48698379 Spearman, J., & Watt, H. (2013). Perception shapes experience: The influence of actual and perceived classroom environment dimensions on girls’ motivations for science. Learning Environment Research, 16(217), 217–238. https://doi.org/10.1007/s10984-013-9129-7 Speer, J. D. (2020). STEM occupations and the gender gap: What can we learn from job tasks? IZA DP No. 13734. Stoet, G., & Geary, D. C. (2018). The gender-equality paradox in science, technology, engineering, and mathematics education. Psychological Science, 29(4), 581–593. https://doi.org/10.1177/ 0956797617741719 Stout, J. G., Dasgupta, N., Hunsinger, M., & McManus, M. A. (2011). STEMing the tide: Using in group experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM). Journal of Personality and Social Psychology, 100(2), 255–270. https:// doi.org/10.1037/a0021385 Sullivan, A., & Bers, M. U. (2019). Investigating the use of robotics to increase girls’ interest in engineering during early elementary school. International Journal of Technology and Design Education, 29(5), 1033–1051. https://doi.org/10.1007/s10798-018-9483-y Tenenbaum, H., & Leaper, C. (2003). Parent-child conversations about science: The socialization of gender inequities? Developmental Psychology, 39(1), 34–47. https://doi.org/10.1037/00121649.39.1.34 UNESCO. (2017). Cracking the code: Girls’ and women’s education in science, technology, engineering and mathematics (STEM). Report by UNESCO. Available at https://unesdoc.unesco. org/ark:/48223/pf0000253479 UNICEF. (2020). Towards an equal future: Reimagining girls’ education through STEM. United Nations Children’s Fund. Union, E. (2021). She figures 2021. Publications Office of the European Union. Valla, J. M., & Ceci, S. J. (2014). Breadth-based models of women’s under representation in STEM fields: An integrative commentary on Schmidt (2011) and Nye et al. (2012). Perspectives on Psychological Science, 9(2), 219–224. https://doi.org/10.1177/1745691614522067 van den Hurk, A., Meelissen, M., & van Langen, A. (2019). Interventions in education to prevent STEM pipeline leakage. International Journal of Science Education, 41(2), 150–164. https:// doi.org/10.1080/09500693.2018.1540897

162

R. B. Castro Núñez and R. Santero-Sánchez

Wang, M., & Degol, J. (2013). Motivational pathways to STEM career choices: Using expectancyvalue perspective to understand individual and gender differences in STEM fields. Developmental Review, 33(4), 304–340. https://doi.org/10.1016/j.dr.2013.08.001 Wang, M., & Degol, J. (2017). Gender gap in science, technology, engineering, and mathematics (STEM): Current knowledge, implications for practice, policy, and future directions. Educational Psychology Review, 29, 119–140. https://doi.org/10.1007/s10648-015-9355-x Wang, M., Eccles, J. S., & Kenny, S. (2013). Not lack of ability but more choice: Individual and gender differences in choice of careers in science, technology, engineering, and mathematics. Psychological Science, 24(5), 770–775. https://doi.org/10.1177/0956797612458937

Digital Adoption in Times of Crisis: A Study for the European Countries Aida Garcia-Lazaro

A Churchill moment is a creative and inspiring determination to overcome a difficult challenge.

The Covid-19 pandemic has brought about significant transformations in how companies produce goods and services. With the global economy facing a lockdown in 2020, the jobs of millions of people were endangered. Particularly affected were the service sectors and other industries that lacked the option of remote work (Dingel & Neiman, 2020; Lund et al., 2020). As a result, many companies were confronted with the daunting challenge of reinventing themselves, seeking financial support, and ultimately, either surviving the pandemic or succumbing to its impact. Unsurprisingly, research suggests that the survival rate of companies was more favourable for those that were older, more productive, and had a history of pre-pandemic innovations (Abidi et al., 2022). Large companies invest up to five times more than Small and Medium Enterprises (SMEs) in technology. However, during the Covid-19 pandemic, SMEs also increased their technology adoption. Contrary to the traditional belief that innovation and technology adoption decline during economic downturns, the Covid-19 pandemic produced the opposite effect, at least for some essential digital tools. The health crisis acted as a catalyst, prompting enterprises in several European countries to accelerate their technology adoption. We contend that this process was primarily a survival mechanism in response to the lockdown rather than a mere accelerator of growth. Given the prolonged economic shutdown, many enterprises were compelled to make crucial investment choices, opting to adopt essential digital tools that allowed them to continue their operations despite the challenging circumstances. The lockdown forced companies to work remotely when the nature of the production process or the service allowed it, pushing the new technology forward. However, this rapid digital adoption has yet to be homogeneous across enterprises and industries. We aim to examine whether companies invested more in digital tools during the pandemic. We explore a sample of 22 European countries using the information in the A. Garcia-Lazaro (B) Institute for Policy Research, University of Bath, Bath, England e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_10

163

164

A. Garcia-Lazaro

portfolio of datasets available in ICT usage in enterprises from Eurostat. We concentrate our analysis of the Covid-19 pandemic on examining the remote access to (i) the e-mail system and (ii) the ICT system different from the e-mail. We also explore the association of the Covid-19 pandemic on a set of cloud computing services. Our findings suggest a significant adoption of essential digital tools across European countries and industries, particularly remote access to e-mail and basic cloud computing services. We also find a positive impact of the Covid-19 pandemic on the percentage of enterprises that increased the number of employees having remote access to the e-mail system. However, we do not find evidence that shows the Covid19 pandemic accelerated the digital adoption of more advanced and sophisticated cloud computing services. Given the data constraints we do not claim causality but to an extent a significant association between the policy measures and the percentage of companies by industry-country using these technologies. We divide this chapter into seven sections. The second section revises the digital adoption before the Covid-19 pandemic. The third section briefly introduces the model. The fourth section explains the data. The fifth section presents the estimations and results. The sixth section discusses the findings, and the final section concludes.

1 Technology Adoption Trends Technological progress has undergone a remarkable acceleration in the latter half of the century, particularly in advanced economies where advancements have been even more rapid. Over the years, we have witnessed the emergence of various groundbreaking innovations such as Information Communication Technology (ICT) from the 1980s onward, followed by Artificial Intelligence (AI), Machine Learning (ML), the Internet of Things (IoT), Big Data or Cloud Computing (CC) services as well as the advent of revolutionary materials like 3D printing. This encompassing technological revolution has been widely termed as digitalisation adoption and the rise of the digital economy. Despite the significant benefits on productivity observed in the literature (Michaels et al., 2014), the rapid technological change has placed some concerns around the effects of these new technologies on people’s lives, their jobs and the effects on society. Presumably, one of the primary concerns is the automation of some tasks and, therefore, the potential adverse effects on the labour markets. There is extensive evidence showing that the heterogeneous impact of some of the new technologies on demand for labour is mediated by skill level and education (Acemoglu & Restrepo, 2018, 2020). The empirical evidence on the newer technologies like AI, ML and the IoT is scarcer due to the lack of consistent granular datasets to estimate their impact on labour productivity and labour demand. An exception of this is Acemoglu et al., (2023) who use US job postings from Burning Glass Technologies (Lightcast) and presents evidence on the effect of AI exposure to the demand of AI jobs, and its

Digital Adoption in Times of Crisis: A Study for the European Countries

165

effects on wages. The newest studies have turned more creative in the use of nontraditional datasets to examine the effects of AI on the labour market. It has argued that the advacment of AI has not been reflected yet in the statistics of productivity and labour (Brynjolfsson et al., 2017; Furman & Seamans, 2018). It casted doubt on the deepening of these new technologies and their power to boost economic growth and productivity gains. However, the. Brynjolfsson et al. (2018) state that the nature of the changes in production from ML and AI is having a different impact than what we observed from ICT and that may be because those technologies are concentrated more across the high-skilled occupations. Moreover, some of the productivity gains may be underestimated for the current national accounts systems that until recently incorporated as intangible capital. For Brynjolfsson et al. (2017), there is a notable lag between the development of AI and ML tools and the commercialisation of the new innovative ideas building on these new tools. Thus, the complementary investment to extend these technologies will take time, as reflected in the national statistics. More research is needed to understand emerging technologies’ effects on productivity, labour demand and economic growth. In this chapter, we aim to shed some light on the effects of the health crisis in digital adoption. Particularly in what refers to an essential digital tool like remote access to e-mail and the ICT system, but also the examination of the cloud computing services, the latter fits within the emerging technologies discussed above. This study takes a step backwards and questions whether more enterprises embraced digital tools to face the Covid-19 pandemic, even before linking these technologies to productivity or workers’ displacement. Recent studies have suggested a positive trend in favour of digital adoption during the pandemic and the potential digital acceleration due to the Covid-19 pandemic (Lloyds Bank, 2020). A view suggests that the health crisis catalysed the digitalisation of business models. However, the question is whether the benefits are for most enterprises or only a few (Amankwah-Amoah et al., 2021). Although in this chapter we do not expand on the extension of the benefits for enterprises, we provide clues about the extent of the covid-19 impact on digital adoption. Cloud computing (CC) services In this section, we aim to introduce CC services, which is one of the main variables of our econometric analysis. Cloud infrastructures comprehend a vast range of online services and tools. In the last few years, they have been one of the most powerful digital tools enabling enterprises to operate. CC services are a flexible tool that the user does not have to make upfront capital-intensive investments in ICT infrastructure and services; the user only can pay for the computing resources used. From the economic viewpoint, since CC services provide the ability to access ICT services without a significant investment, that minimises the barriers for companies to adopt digital technologies. The OCED provided a definition for CC services, stating that “computing services based on a set of computing resources that can be accessed in a flexible, elastic, ondemand way with low management effort” (OECD, 2014, 2021). We emphasise the

166

A. Garcia-Lazaro

flexibility of the service and the broad spectrum of services that CC could offer. Five characteristics identify CC services: 1. On-demand self-service: It is a service where the user can provision computing capabilities, such as server time and network storage, as needed. 2. Broad network access: The system works over the network, and the services are accessed through different devices such as mobile phones, tablets, laptops, and desktop workstations. 3. Resource pooling: It indicates that the resources of a cloud computing service provider are pooled to satisfy the demand of multiple customers with a variety of virtual and physical resources. Users have no sense of the location about the services provided. 4. Accelerated elasticity: CC services can scale the services in quantity and time. 5. Measured service: CC services control and optimise resources use; they leverage a metering capability depending on the type of service, for instance, storage of files, processing, and others. Thus, resource usage is monitored and reported to the consumer and provider. This study includes a portfolio of CC services divided into three categories: the low or essential, the high and the more sophisticated CC services. We describe them in the Data section.

2 Model We examine the impact of the Covid-19 pandemic on the adoption of remote access to e-mail and ICT systems and the purchase of any CC services. Our strategy assumes that a larger share of companies using these technologies signals technology adoption. As an explanatory variable, we use two indicators: (i) an ordinal index for the workplace’s closures in each country at the time of the pandemic. This index denotes with a value of 3 the highest strictness in the government policy, indicating the closure of all the workplaces except for the essential sectors and with 0, no measures taken. The second variable of interest is the income support index, where the higher the value, the more generous the income support for households. This support is predominantly through the Job Retention Scheme implemented during the pandemic. Thus, companies with a lower burden on the operation costs due to the wage subsidy provided by the government will have more extensive scope for adopting technologies that allow them to cope with the workplace closures. Therefore, we propose the following specification: yi, j,t = γi + γ j + γ1 X j,t + εi, j,t

(1)

where yi, j,t is a vector denoting the percentage of enterprises adopting a given technology in industry i, country j at time t, which refer to 2021 for the analysis of the remote access to the e-mail and ICT systems and t = 2018, 2020 and 2021 when we

Digital Adoption in Times of Crisis: A Study for the European Countries

167

examine the purchase of CC services. X j,t is a matrix of proxies to capture the severity and support of the government policy measures during the Covid-19 pandemic. We include γi and γ j which stands for industry and country fixed effects; εi, j,t refer to the residuals.

3 Data Since we examine digital adoption during the Covid-19 pandemic in Europe, we use two datasets published by Eurostat in the section ICT usage in enterprises. We use four variables to measure the impact of the pandemic on the increase of remote access of enterprises to two primary basic services (i) the e-mail system and (ii) the ICT system different from the e-mail system. For this purpose, we use the Covid-19 Impact on ICT usage dataset, which has information at the national and industry level for 20 countries and 27 industries. Data is available for the Manufacturing sector and the services; the primary sector is excluded from the statistics,1 and the information exists for 2021. 1. Enterprises with an increase in remote access to the e-mail system entirely due to the Covid-19 pandemic. 2. Enterprises with an increase in remote access to the ICT system are different from the e-mail entirely due to the Covid-19 pandemic. 3. During 2020, enterprises have increased the percentage of persons employed having remote access to the e-mail system. 4. During 2020, enterprises that have increased the percentage of persons employed having remote access to the ICT system different from the e-mail. We also examine to what extent cloud computing (CC) services have increased due to the Covid-19 pandemic. We use Eurostat’s Cloud Computing Services dataset; the data is available for 22 countries and 27 industries. Data is available for three years 2018, 2020, and 2021. Cloud computing services are divided into (i) low CC services, (ii) high CC services, and (iii) sophisticated CC services. Thus, we include in the analysis the variables that indicate the percentage of enterprises buying the following services: Low CC services 1. 2. 3. 4.

Cloud computing services used over the internet. E-mail as a cloud computing service. Storage of files as a cloud computing service. Office software as a cloud computing service (e.g., word processors, spreadsheets, etc.).

High CC services 1

Check the table of industries in the Appendix.

168

A. Garcia-Lazaro

5. Customer Relationship Management (CRM) software as a cloud computing service. 6. Computer power to run the enterprises’ own software as a cloud computing service. 7. Finance and accounting software applications as a cloud computing service. Sophisticated CC services 8. Hosting for the enterprise database used as a cloud computing service. 9. The computing platform provided a hosted environment for application development used as a cloud computing service. We use the Oxford Covid-19 Government Response tracker to measure the restrictions during the Covid-19 pandemic, the index is available for the countries in our sample (Hale et al., 2021). This dataset contains indices about (i) containment and closure policies, (ii) economic response, (iii) health system policies, (iv) vaccination policies, and (v) miscellaneous policies. We use the indicator on workplaces closures included in containment and closure policies to proxy the strictness of the government policy on workplaces closures and the mandates of working from home. The severity of the mandates to work from home—when possible—induce enterprises to adopt digital technologies that allow them to work remotely. Thus, a more restrictive policy could lead to a higher adoption of technology depending on the teleworkablity of the sector. Workplaces closures sets 0 for no measures taken; 1 indicates recommend closing and working from home; 2 indicates require closing or working from home for some sectors or categories of workers, and 3 denotes require closing or working from home for all but essential workplaces (e.g. grocery stores, doctors, etc.). Although the strictness of the measure changed over the year, for our study, we take the maximum value in the year, as a broad measure of the tightest restrictions faced by the enterprises at that period. We also include the index of economic support from the economic responses of the Oxford Covid-19 Government Response tracker. This index records government response in providing wage subsidies or cash payments to households. Economic support holds three possible values, 0 denotes no income support, 1 indicates the government is replacing less than 50% of lost salary, and 2 indicates the government replaces 50% or more of lost salary. Since all the EU countries in our sample, implemented a type of Job Retention Scheme or wage subsidy during the pandemic (Drahokoupil & Müller, 2021); then, broadly speaking, we can argue that the economic response index captures the extent of the wage subsidy set in each country.2 The Job Retention schemes served as a lifeline for workers who could lose their job overnight otherwise. Thus, the wage subsidy maintained workers in the payroll and significantly reduced the wage cost burden for enterprises. In the UK, the Job Retention Scheme significantly mitigated the Covid-19 pandemic’s impact on employment (Martin & Okolo, 2022). We include 2

The index could overestimate the effects due to the extent of cash transfers directly to households. However, this proxy could serve as an upper-bound index.

Digital Adoption in Times of Crisis: A Study for the European Countries

169

Fig. 1 Enterprises increasing remote access to e-mail or ICT systems in 2021. Source Author’s calculations using the Covid-19 impact on ICT usage dataset, Eurostat

this variable because the reduction in the wage cost could lead to the relocation of resources to digital adoption by enterprises or to simply improve companies’ balance sheeets. Thus, the more generous the wage subsidies were, the smaller the wage cost for companies was. The less economic burden for enterprises could lead to higher adoption of digital tools to face the new working conditions. (a) Digital adoption: e-mail and ICT systems Digital adoption was heterogeneous across European countries and industries. Figure 1 shows the percentage of enterprises that reported to have increased the remote access to the e-mail (blue bars), and the enterprises that have increased the remote access to ICT systems different from the e-mail (red diamond), in both cases fully due to the Covid-19 pandemic. Malta, Austria, Germany, Netherlands, Italy and Cyprus are the countries with the highest increase in remote access to both services. Belgium, Sweden and Poland rank high in remote access to ICT systems. When we examine the percentage of enterprises that increased the share of employees with remote access to these services, we find that 60% of enterprises in Malta reported having increased the number of employees having remote access to e-mail, 50% in Netherlands and Germany and 45% in Belgium. Lithuania and Poland are among the countries with the lowest percentage of enterprises with more employees having remote access to e-mail; the results are very similar when we refer to the increase in employees with access to the ICT system different from the e-mail. Comparing Figs. 1 and 2, we observe similarities with Malta, Germany, and the Nordic countries but Poland. However, having a significant increase in the enterprises with remote access to the ICT and e-mail systems, the increase in the percentage of employees with those services is one of the smallest in Europe. Variables in Fig. 2 may be more robust indicators of the impact of digital adoption on the workforce, which could lead to higher effects on productivity as it measures an increase in the employees with access to these technologies. There is a significant heterogeneity across industries using ICT services, Fig. 3 highlights that the increase in the percent of enterprises having remote access to email, or the ICT systems are the highest within the manufacturing sector producing

170

A. Garcia-Lazaro

Fig. 2 Enterprises increasing the percentage of employees with remote access to e-mail and ICT systems. Source Author’s calculations using the Covid-19 impact on ICT usage dataset, Eurostat

Fig. 3 Enterprises increasing remote access to e-mail or ICT systems. Author’s calculations using the Covid-19 Impact on ICT usage dataset, Eurostat

(i) basic pharmaceutical products and pharmaceutical preparations, (ii) coke and refined petroleum products, and (iii) the production of computers, electric and optical products. The services with the highest increase of enterprises having remote access to both systems are (i) the legal, accounting, management, architecture, engineering, technical testing, and analysis activities, (ii) Information and communication, (iii) the ICT sector, (iv) scientific and research and development (R&D),3 and (v) other professional, scientific, and technical activities. 3

Following the Eurostat taxonomy of industries, the Scientific and R&D sector refers to the M72 sub-industry.

Digital Adoption in Times of Crisis: A Study for the European Countries

171

Industries that increased the percentage of employees having remote access to e-mail and ICT systems correspond to those reporting an increase in remote access solely (Fig. 3). However, the industries producing coke and refined petroleum products and basic pharmaceutical products and pharmaceutical preparations are those with the highest percentage of enterprises that increased the share of employees having access to e-mail (59.6% and 55% of enterprises) and the ICT systems (65.3% and 57.1%), respectively. To a lower extent, around 30–40% of enterprises reported having an increase in the number of employees with remote access to e-mail and ICT systems within industries like computers, electronics and optical products, electrical equipment, machinery and other equipment and electricity, gas, steam, and air conditioning sectors as well as services such as ICT sector, information and communication, accounting, management, architecture, engineering, technical testing, and analysis activities and scientific and R&D. The sectors with a lower adoption of remote access to e-mail and ICT systems are accommodation and food and beverages, construction, retail and trade and textile, apparel and leather products. Mainly the services are known for having a high component of low-skilled workers and, therefore, the lowest probability of teleworkability (Dingel & Neiman, 2020; Lund et al., 2020; Sostero et al, 2020) (Fig. 4). (b) Digital adoption: cloud computing services

Fig. 4 Enterprises increasing the percentage of employees having remote access to e-mail and ICT systems by industry. Source Author’s calculations using the Covid-19 impact on ICT usage dataset, Eurostat

172

A. Garcia-Lazaro

The remote access to the e-mail system and other services in the ICT systems is a basic level of technology used among enterprises to perform their activities. However, a more advanced level of digitalisation is related to cloud computing tools, which provide services such as the storage of files in the cloud, the use of e-mail as a cloud computing service, saving files in the cloud and making functional the e-mail system beyond sending and receiving e-mails. The use of office software in the cloud, like spreadsheets, text processors, and other services used in the cloud. All of these services require a cloud computing platform that enterprises can purchase. The higher education sector is an excellent example of that. Universities extensively adopted cloud computing services using Microsoft Teams and its add-ins and Google applications to expand the tools available for staff and students. Figure 5 shows the percentage of European enterprises reporting the use of cloud computing services; we have data for 2018, 2020 and 2021. The increase in the percentage of enterprises purchasing cloud computing services from 2018 to 2020 is remarkable. 10% more of the enterprises bought CC services over the internet; 8% more bought the e-mail service used as cloud computing, 6% more bought storage of files as cloud computing services and 7% services on office software. The next year, we observe a less pronounced increase but still significant in all the types of services, see Fig. 5. Cloud computing services used over the internet and e-mail as a cloud computing service are essential tools but also those that are more commonly used for enterprises. In Fig. 5, we present the low cloud computing services, reflecting the most basic types of services offered to enterprises. In Fig. 6, we display the data for the high and sophisticated cloud computing services in the first and second rows of the graph, respectively. We show two relevant facts. First, the share of enterprises buying these services is a third of those companies buying the basic cloud computing services. Second, the change in the percentage of enterprises buying high and sophisticated cloud computing services from 2018 to 2020 is significantly smaller than the increase in the basic cloud computing services (see Fig. 5). The highest increase is observed in accounting and finance software applications using CC services.

4 Estimation and Results In this section, we summarise and discuss the results of our econometric estimations using the specification in Eq. (1) and the data from the data section. Remote access to the e-mail and ICT systems Table 1 summarises the results of the estimations using as a dependent variable the ratio of enterprises who increased remote access to their e-mail or ICT systems and as a dependent variable the index on workplace closures and income support. Columns (1)–(2), include industry and country fixed effects. We find that the increase in the strictness on workplace closures led to 8.8% of the companies increasing their remote access to e-mail systems. Our estimates are consistent when we include

Digital Adoption in Times of Crisis: A Study for the European Countries

173

Fig. 5 Enterprises using low cloud computing (CC) services. Author’s calculations using the Cloud Computing Services dataset, Eurostat

income support. It led to 9.8% more enterprises having remote access to their e-mail. We also find a 4.3% increase in the ratio of enterprises having remote access to the ICT system different from the e-mail due to the strictness on workplace closures and 6.9% increase when we introduce income support; see columns (3)–(4). Columns (5)–(8) explore the effect of workplace closures and income support on the ratio of enterprises increasing the number of employees having remote access to e-mail or ICT systems. A higher strictness on the government policies about the workplaces led to a 10% increase in the companies having more employees with remote access to their e-mail. There is no evidence that the same occurred with the remote access of the ICT system. And policies on income support do not seems to be significant either. One of the explanations for our results is that the e-mail system is one of the most basic digital tools in a company; the ICT system requires a more advanced set of tools and services. Thus, our results could suggest that some enterprises may have adopted the minimal technology allowing them to continue working during the pandemic, particularly those with a lower initial level of digitalisation. Cloud computing (CC) services In this section, we estimate the impact of the government’s policy responses: (i) workplace closures and (ii) income support on the enterprises’ purchases of cloud

174

A. Garcia-Lazaro

Fig. 6 Enterprises using high and sophisticated cloud computing (CC) services. Author’s calculations using the Cloud Computing Services dataset, Eurostat Table 1 Estimations of the Covid-19 impact on technology adoption (1)

(2)

(3)

(4)

(6)

(8)

Workplace closures

0.088*** 0.088*** 0.043* 0.043*

0.106*** 0.106*** 0.019

0.019

(0.022)

(0.035)

(0.035)

Income support Constant

(0.023) (0.023)

(0.049)

(0.035)

%employed ICT system (0.035)

0.098***

0.069***

0.059

0.065

(0.024)

(0.025)

(0.042)

(0.044)

− 0.087* − 0.040 0.284***

Observations 475

%employed email system

(7)

Email system

(0.022)

ICT system

(5)

Variables

− 0.099

− 0.015

− 0.132

0.247*** 0.117

(0.089)

(0.053) (0.095)

(0.083)

(0.145)

(0.082)

(0.146)

475

471

494

494

496

496

471

R-squared

0.479

0.479

0.542

0.542

0.697

0.697

0.739

0.739

Country FE

YES

YES

YES

YES

YES

YES

YES

YES

Industry FE

YES

YES

YES

YES

YES

YES

YES

YES

Robust standard errors in parentheses; ***p < 0.01, **p < 0.05, *p < 0.1

Digital Adoption in Times of Crisis: A Study for the European Countries

175

computing services. We test a set of basic, high, and sophisticated CC services. We presume the impact of the pandemic is heterogenous across the different company sizes. Thus, Table 2 includes two control variables: the share of enterprises with 5–9 employees and the share of enterprises with more than 10 employees. Table 2 condenses the results; we include industry and country fixed effects and control variables. Column (1) shows that the increase in the strictness of the government’s workplace closures led to 3.6% more enterprises buying CC services over the internet. And more extensive income support by the government, predominantly the Job Retention Scheme, led to a 3.0% increase in the number of enterprises purchasing those CC services but the effect is not significant (Table 2). Column (2) shows the estimates on the effect of workplace closures on the purchases of e-mail as a CC service, the effect is positive and leads to 3.9% more companies buying those CC services and − 0.7% on income support, the latter not significant. Column (3) shows that stricter workplace closures is not associated to the purchase of storage files, but income support tends to increase in 3.3% the ratio of companies buying those services. Technologies in Columns (1)–(3) are classified as low or basic CC services, and they tend to be associated to the policy containment measures. Table 2 Estimations of the low and high cloud computing (CC) services Variables

Workplace closures

(1)

(2)

(3)

(4)

(5)

(6)

CC over internet

E-mail CC

Buy storage files

Customer relationship management

Computer power

Finance and accounting software

0.036***

0.039***

0.004

− 0.008

− 0.010** 0.014**

(0.012)

(0.012)

(0.007)

(0.005)

(0.005)

(0.007)

− 0.007

0.033***

0.029***

0.040***

0.017*

Income support − 0.003 (0.017)

(0.015)

(0.010)

(0.008)

(0.007)

(0.009)

− 0.074

− 0.072

− 0.074

0.013

0.003

− 0.013

(0.069)

(0.072)

(0.068)

(0.043)

(0.057)

(0.053)

Share of 0.040 enterprises 10+ (0.047) employees

0.008

− 0.008

− 0.029

− 0.021

− 0.025

(0.040)

(0.036)

(0.023)

(0.025)

(0.028)

0.091***

0.116***

0.021

0.018

− 0.001

Share of enterprises 5–9 employees

Constant

0.181*** (0.041)

(0.028)

(0.030)

(0.015)

(0.016)

(0.020)

Observations

918

921

914

935

937

940

R-squared

0.883

0.861

0.884

0.834

0.825

0.822

Country FE

YES

YES

YES

YES

YES

YES

Industry FE

YES

YES

YES

YES

YES

YES

Robust standard errors in parentheses; ***p < 0.01, **p < 0.05, *p < 0.1

176

A. Garcia-Lazaro

Columns (4)–(6) report the impact of the policy responses on the high cloud computing services that include buying computing power to run the enterprise’s own software, buying finance or accounting software applications and specific Consumer Relationship Management (CRM) software. CRM is a more elaborated system that aims to administer the interactions between the company and its customers from different channels like social media, telephone, e-mails, live chats etc. Column (4) illustrates that the ordinal increase in the workplace closures index leads to 0.1% fewer enterprises buying CRM, but it is not significant. There are also 2.9% more enterprises buying these cloud computing services when there is an increase in the index related to income support. The results suggest that a more generous income support to households, particularly with the Job Retention Scheme that all the European countries implemented to a different extent, relieves the burden of operating costs for companies. That could relocate resources from wages to investments in technologies that allow them to work under the Covid-19 restrictions. Column (5) shows that the rise of the strictness in workplace closures reduced the ratio of companies purchasing cloud computing services related to computing power while the increase in income support for workers led to 4.0% more enterprises buying computer power. A more generous policy on income support to workers also led to 3.7% more enterprises buying finance and accounting software applications used as cloud computing services. And a stricter policy on workplace closure accounts for 1.4% more enterprises buying finance and accounting software. Our results indicate that enterprises increased their basic CC services, but they seem to have reduced the purchases of more complex CC tools. We now focus on the sophisticated services, which refer to the purchase of (i) security software applications, (ii) hosting for the enterprise’s databases and the purchase of computing platforms for developing applications. These technologies are particularly concentrated on the Scientific R&D sector, other professional, scientific, and technical services and information and communication. Security and databases as cloud computing services are high in the pharmaceutical sector, motor vehicles, trailers and semi-trailers, and the legal, accounting, management, architecture and other services. Table 3 shows the summary of results using sophisticated CC services. Unfortunately, we do not have enough data to include control variables related to company size, but we still control for country and industry fixed effects. Columns (1), (3) and (4) indicate that the increase in the strictness on workplace closures led to a significant decline in the ratio of enterprises buying security software applications, hosted environment to develop applications services and the purchase of sophisticated CC services. Larger income support on the contrary increases the ratio of enterprises using these technologies. The companies saw the workplace closures taken by the government as a signal of uncertainty, and they may have decided to take precautionary measures by decreasing their investment in more sophisticated cloud computing services. However, the income support policy seems to be an effective mitigation policy for households according to the existing literature but also for companies buying cloud computing services.

Digital Adoption in Times of Crisis: A Study for the European Countries

177

Table 3 Estimations of the sophisticated cloud computing (CC) services Variables Workplace closures

(1)

(2)

(3)

(4)

Security

Databases

CP Hosted environment

Buy sophisticated CC services

− 0.397***

0.003

− 0.106***

− 0.418***

(0.036)

(0.004)

(0.015)

(0.037)

0.716***

0.028***

0.220***

0.734***

(0.041)

(0.007)

(0.020)

(0.039)

Constant

− 0.109***

− 0.010

− 0.056***

− 0.101***

(0.002)

(0.010)

(0.013)

(0.021)

Observations

598

1,568

606

598

R-squared

0.842

0.828

0.747

0.840

Income support

Country FE

YES

YES

YES

YES

Industry FE

YES

YES

YES

YES

Robust standard errors in parentheses; ***p < 0.01, **p < 0.05, *p < 0.1

5 Discussion Our findings suggest an increase in the percentage of enterprises adopting new technology during 2020 and 2021 due to the Covid-19 pandemic. The estimations show a positive and consistent association between the increase in the severity of the workplace closure measures and the increase in the generosity of income support on the percentage of companies adopting essential digital tools. The evidence is consistent for (i) having remote access to the e-mail system, (ii) having remote access to the ICT system different from the e-mail and (iii) essential or low cloud computing services such as the buy of services used over the internet, e-mail, and the storage of files. The evidence is weaker for purchasing high and sophisticated cloud computing services where the income support index results are more consistent, positive and significant. The technologies included in these results are the computer power to run the enterprise’s own software, the finance and accounting software applications and CRM software. The results also suggest an adverse effect of the increase in the severity of workplace closures and a mixed effect from larger income support on the purchase of sophisticated cloud computing services. We suspect that companies adopted the minimal set of digital tools that helped to face the pandemic. Furthermore, that explains why the essential digital tools respond positively to the more restrictive health containment measure and generous income support packages. However, the dynamics of the more advanced digital tools like the ICT system or the named sophisticated cloud computing services may respond more to long-run investment decisions. These decisions may be related to the economic recovery expectations, the economy’s re-opening, and the enterprises’ financial conditions.

178

A. Garcia-Lazaro

Considering our results, we are prudent to suggest that the Covid-19 accelerated digitalisation, or the pandemic is the digital accelerator as many people have suggested. The health crisis and the lockdown measures contributed to an increase in companies adopting the most basic digital tools such as remote access to the e-mail or cloud computing services on the e-mail, office software, storage of files and others. However, the impact does not seem significant or even negative when we refer to the more sophisticated technologies. Although, we have controlled for company size when we estimate the effect of the policy measures for the basic and high CC services, we do not have enough data for sophisticated services. We are also aware that there are other determinants that could help to make more robust our estimations, such as companies’ financial conditions and death rates. First, the medium and long-term conditions affect enterprises in decision-making. We know the literature studying enterprises’ individual conditions, such as credit access, individual financial conditions or other barriers (Foster & Rosenzweig, 2010; Parente, 1995). Second, the company’s size is a relevant factor in the investment decisions and the stage of technology adoption; large companies are ahead of small and medium enterprises. So far, our work includes fixed effects on the industries and as much as we can, we have included companies’ size, particularly for basic and high CC services, where we could gather enough data. Nevertheless, it would be desirable to have additional industry-specific information that could help to explain the heterogeneous digital adoption. With information for more years, we could explore other causal-effect methodologies. It would also allow testing the dynamics of the remote access to the ICT system and the high and sophisticated cloud computing services.

6 Conclusions We examine the digital adoption of European countries during the Covid-19 pandemic, including the years 2020 and 2021. We use the Covid-19 Impact on the ICT usage and the Cloud Computing Services datasets from the Eurostat dashboard. We tested the impact of the government policy responses on two dimensions: (i) workplace closures and (ii) income support to workers, predominantly driven by the Job Retention Scheme that works as a wage subsidy provided by the government with the condition of keeping the worker in the payroll. Our findings suggest a significant adoption of essential digital tools across European countries and industries, particularly remote access to e-mail and basic cloud computing services. We also find a positive impact of the Covid-19 pandemic on the percentage of enterprises that increased the number of employees having remote access to the e-mail system. However, we do not find evidence that the Covid-19 pandemic accelerated the digital adoption of more advanced and sophisticated cloud computing services. Thus, we are prudent in favouring the statement that Covid19 was the great digital accelerator. More research is needed to incorporate the company’s size, which, unfortunately, was unavailable at the industry level in our

Digital Adoption in Times of Crisis: A Study for the European Countries

179

dataset. That variable could modify the results, as it is known that large companies had a more significant digital intensity even before the pandemic. In those cases, if there was a positive digital adoption due to the pandemic, it may be focused on the more complex digital tools.

Appendix See Table 4. Table 4 Industries Codes

Industries

10_C10_12

Manufacture of beverages, food and tobacco products

10_C13_15

Manufacture of textiles, wearing apparel, leather and related products

10_C16_18

Manufacture of wood, paper, printing and reproduction

10_C19

Manufacture of coke and refined petroleum products

10_C20

Manufacture of chemicals and chemical products

10_C21

Manufacture of basic pharmaceutical products and pharmaceutical preparations

10_C22_23

Manufacture of rubber and plastics products, and other non-metallic mineral products

10_C24_25

Manufacture of basic metals & fabricated metal products excluding machines and equipments

10_C26

Manufacture of computer, electronic and optical products

10_C27_28

Manufacture of electrical equipment, machinery and equipment

10_C29_30

Manufacture of motor vehicles, trailers and semi-trailers, other

10_C31_33

Manufacture of furniture; jewellery, musical instruments, toys; repair and installation of machinery and equipment

10_D35_E39

Electricity, gas, steam, air conditioning and water supply

10_E36_39

Water supply, sewerage, waste management and remediation

10_F41_43

Construction

10_G45

Trade of motor vehicles and motorcycles

10_G46

Wholesale trade, except of motor vehicles and motorcycles

10_G47

Retail trade, except of motor vehicles and motorcycles

10_H49_53

Transportation and storage

10_I55_56

Accommodation and food and beverage service activities

10_ICT_T

ICT sector

10_J58_63

Information and communication

10_L68

Real estate activities (continued)

180

A. Garcia-Lazaro

Table 4 (continued) Codes

Industries

10_M69_71

Legal, accounting, management, architecture, engineering, technical testing and analysis activities

10_M72

Scientific research and development

10_M73_75

Other professional, scientific and technical activities

10_N77_82

Rental and leasing activities

References Abidi, N., El Herradi, M., & Sakha, S. (2022). Digitalisation and resilience: Firm-level evidence during the Covid-19 pandemic (pp. 1–42). IMF Working Paper WP/22/34. Acemoglu, D., & Restrepo, P. (2020). Robots and jobs: Evidence from US labour markets. Journal of Political Economy, 128(6), 2188–2244. Acemoglu, D., & Restrepo, P. (2018). Artificial intelligence, automation and work (pp. 1–43). NBER Working Papers, N. 24196. Acemoglu, D., Autor, D., Hazell, J., & Restrepo, P. (2023). Artificial intelligence and jobs: Evidence from online vacancies. Journal of Labour Economics, 40(1), S293–S340. Amankwah-Amoah, J., Khan, Z., Wood, G., & Knight, G. (2021). Covid-19 and digitalisation: The great acceleration. Journal of Business Research, 126, 602–611. Brynjolfsson, E., Mitchell, T., & Rock, D. (2018). What can machines learn and what does IT mean for occupations and the economy? MIT Initiative on the Digital Economy Research Brief, 4, 1–4. Brynjolfsson, E., Rock, D., & Syverson, C. (2017). Artificial Intelligence and the modern productivity paradox: A class of expectations and statistics (pp. 1–46). National Bureau Economic Research, NBER Working Papers, N. 24001. Dingel, J., & Neiman, B. (2020). How many jobs can be done at home? Journal of Public Economics, 189, 104235. https://doi.org/10.1016/j.jpubeco.2020.104235 Drahokoupil, J., & Müller, T. (2021). Job retention schemes in Europe (pp. 1–66). Working Paper, 2021.07, European Trade Union Institute. Foster, A., & Rosenzweig, M. R. (2010). Microeconomics of technology adoption (pp. 1–43). Center Discussion Paper, N. 984. Economic Growth Center. Furman, J., & Seamans, R. (2018). AI and the economy (pp. 1–34). National Bureau Economic Research, NBER Working Papers, N. 24689. Hale, T., Angrist, N., Goldszmidt, R., Kira, B., Petherick, A., Phillips, T., Webster, S., CameronBlake, E., Hallas, L., Saptarshi, M., & Tatlow, H. (2021). A global panel database of pandemic policies (Oxford Covid-19 government response tracker). Nature Human Behaviour, 5, 529–538. Lloyds Bank. (2020). Transformation with tech. A study into adoption of technology and digital skills for small businesses in 2020. Report of Lloyds Bank in partnership with Be the Business (pp. 1–62) Lund, S., Ellingrud, K., Hancock, B., Manyika, J., & Dua, A. (2020). COVID-19 Lives and livelihoods: Assessing the near-term impact of COVID-19 on US workers (pp. 1–10). McKinsey Global Institute. Martin, C., & Okolo, M. (2022). Modelling the differing impacts of Covid-19 in the UK labour market. Oxford Bulletin of Economics and Statistics, 84(5), 994–1017. Michaels, G., Natraj, A., & Reenen, J. V. (2014). Has ICT polarised skill demand? Evidence from eleven countries over twenty-fiver years. Review of Economics and Statistics, 96(1), 60–77. OECD. (2014). Cloud computing: The concept, impacts and the role of Government policy (pp. 1– 35). OECD Digital Economy Papers, N. 240. OECD Publishing.

Digital Adoption in Times of Crisis: A Study for the European Countries

181

OECD. (2021). Measuring cloud services use by businesses (pp. 1–56). OECD Digital Economy Papers, N. 304. OECD Publishing. Parente, S. L. (1995). A model of technology adoption and growth. Economic Theory, 6, 405–420. Sostero, M., Santo, M., Hurley, J., Fernández-Macías, E., & Bisello, M. (2020). Teleworkability and the Covid-19 crisis: A new digital divide? (pp. 1–75). JRC Working Papers Series on Labour, Education and Technology, N. 2020/05, European Commission, Joint Research Centre (JRC), Seville.

A Review of Social Conditions During the Quarantine Period-Covid-19 Taraneh Shahin

Spain was one of those countries which re-opened the science centers and schools sooner than other countries since young people suffered emotionally so much during the lockdown, so the applied decision could have been a solution for making a better social-emotional status for people. Social difficulties such as school abundance were one of those harmful sides of the Covid-19 pandemic. Shutting down the schools had prominent consequences on the student. These effects have questioned the student’s treatments and perceptions of knowledge acquisition. The lack of interest in returning to school and unjustified absences are all cons of the pandemic lockdown on student emotional wellbeing. Studying these effects from the socioeconomic aspect would be significant. We are interested in performing an empirical analysis among Spanish families and teachers to clarify the influences of the lockdown in 2020 on the student’s social-emotional status. The survey of GAD3 for Ernst and Young has been implied to understand the relationships. This survey contains 207 families and 647 schoolteachers’ answers from the 26th of April to the 21st of May. The adverse emotional effects are not just for students exclusively it also affects families and teachers. They are also struggling with them. During the quarantine, emotional feelings like frustration, abandonment, or overwhelmed surrounded tutors, made the educational-teaching situation harder for teachers and parents to maintain their children and students or the method of education. Moreover, the new learning devices and digitalization have affected teaching trajectories. Teachers are forced to maintain the student attention through these devices and continue the educating (Labrador et al., 2022). We are trying to select influential factors in the social-emotional status of students in Spain during the Covid-19 pandemic. Following the previous research and surveys,

T. Shahin (B) Universidad Rey Juan Carlos, Madrid, Spain e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6_11

183

184

T. Shahin

and quotes from Antonio Cabrales,1 the occupation position would be an influential factor. According to the latest investigation of English experts on children’s mental health, human mentality can be influenced by a difficult situation like a pandemic lockdown. Age, gender, and ethnic group were other influential elements in their research (Newlove-Delgado et al., 2021). There was a new challenge in the educational atmosphere, which is digitalization. There were some difficulties associated with the use of the Internet, cell phones, and computers. In this study, we also observed the family’s opinions about Spain’s technology capacity (Newlove-Delgado et al., 2021).

1 What is Covid-19? On 31 December 2019, a kind of disaster changed people’s lifestyles. It first reported from Wuhan, China (Xie et al., 2020). The disease called Covid-19 with an intense acute syndrome coronavirus 2, which caused deaths of millions a day. Given the unknown nature of the disease, all governments accepted a universal lockdown for at least three to five months depending on the geographical status (Pierce et al., 2020). During the pandemic, the daily life of the public entered a new era. The agreed decision stipulated serious limitations on social contact, and working conditions, so it dismissed the accessibility to citizen services. Following the multidimensional aspects of the new virus and the consequences of the lockdown, the mental health of human beings captured the attention. In this chapter, we intend to provide a more clarified comprehension of this subject for policymakers, commissioners, and service providers.

2 What Situation Had Provided After Covid-19? Researchers related to the topic reported mental and social difficulties for adults and children. As the focus of the study is on the socio-emotional statute of school children, the psychological distress of parents or carers is an essential factor. On the other side, economic problems in families during the pandemic due to the closure of workplaces, the level of their education, and their psychological background could be influential on the children’s emotions. Since the school education during the lockdown was continued online, students faced internet problems and other technological afflictions. People not only struggled with their financial and social problems but also encountered health care obstacles. The distances between health centers and the chaos in transportation had put emergency services in trouble (Newlove-Delgado et al., 2021). We intend to concentrate on the school children because of the insufficient studies in this area and the urgent need to comprehend their situation and 1

Professor at the university of Carlos lll.

A Review of Social Conditions During the Quarantine Period-Covid-19

185

requirements. Statistical evidence illustrated multiple inimical crashes of quarantine on the mind and souls of children. The significant enhancement of depression presents a necessity for a recovery period to modify the consequences of lockdown. There is an extensive disruption occurring in children and teenagers’ life we must be aware of the psychological aspect of this incidence to mitigate its negativity. Naturally, due to the lack of study about the children’s condition, we express our point of view with caution. As many surveys have been conducted to indicate the mental health status during the prevalence of Covid-19, we can refer to the NHS digital survey in England, which focus on the proportion of mental health disorder among adult and children during four months of quarantine.

3 Evidence from Other Nations The crucial factor during the life of human beings may change their habits and attitude. It would be possible to quantify the mental health anomalies in an appropriate spectrum to measure the socio-emotional situation. Performing the SDQ approach would be a proper method for assessing mental vigor (Waite et al., 2021). According to the findings of the Co-SPACE, conduct problems, hyperactivity-inattention, and emotional symptoms are presenting elements of mental health. Note standing that during the lockdown, these elements can be more annoying. The survey among English families in the first four months of lockdown, reported the changes in mental attitude and different psychological trajectories that adults and children followed. All three mental representatives experienced an increasing trend, which factors consist of depression anxiety-stress scale, child gender, special educational needs, and neurodevelopment of each child, and the low level of family income have deteriorated them. Children with a special educational needs-neurodevelopment disorders and family squabbles, suffered from a more sensitive situation. By accomplishing a linear growth method, we could confirm that certain risk factors negatively affect the poorer mental health trajectory (Raw et al., 2021). So, the family’s contextual characteristics and resilience factors obviously changed the mental health of children and adults. Conversely, growing up in a coherent family and enjoying warm family intimacy can moderate the negative effect of confining at home. Using the NHS Digital survey in the first wave of quarantine mentioned the suffering and sorrow of children and also, illustrated the socio-economic disasters of adults. The prevalence of mental disorders among children raised from 10.8 to 16.0% in 2020, and age, ethnic group, and gender were contributing invoices to this unsettled situation. Subsequently, women were travailing from psychic knots before the Covid-19 lockdown, which the current situation led the policy concern to support them. Sleep disorders among children and teenagers or the feeling excessively lonely were among the central and common turmoil. On the other side, the terrifying feeling of leaving in the house itself 18% negatively affects younger mental. As mentioned previously, any history of a mental illness complicated children’s and young adults’ conditions. Families with more conflicts and disagreements had worse conditions and suffered

186

T. Shahin

more form psychological injuries. Obviously, families with children suffered more than couples. Therefore, all these conditions faced the communities with a challenge that caused by mental and psychological problems arising from quarantine and economic troubles. Unsettled economic conditions and low levels of income were such that one-tenth of the households could not meet their food needs of themselves. Hence this harsh situation was not only about the food but also was struggling with problems such as internet connection, a proper place to study and work, etc. All in all, mentioned social disasters depicted an urgent need for the reopening of educational institutions and workplaces. Moreover, among the studies in this area, the factor of children’s age and family circumstances were again cited importantly (Bignardi et al., 2021; Newlove-Delgado et al., 2021). Regarding the complexity and dimensional nature of humankind’s brain, mental and psychological disorders could be classified differently. Performed research on the statistical information pertaining to the database of Resilience in education and development (RED), which contain a small but rich dataset, indicated synthetic information about the strengths and Difficulties of emotional problems and revised the Child Anxiety and Depression Scale among pupils. These indicators had controlled and measured before and during the lockdown of Covid-19. As indicated in the earlier lines, the history of mental illness has a crucial role in the later experiences, likewise, here students, parents, and teachers observed before and after. In the basic state of the investigation, we witnessed a drastic increase in the child depression scale of around 0.74. And, for sure other factors of mental health are also enhanced lightly. The fundamental and notable point is again related to the perusal of children and the existing depression status of this vulnerable social group. With accurate studies and explaining relationships by Bignardi and his colleagues, it would be possible to view the deepness of this threating elements. In a more detailed inquiry, specific influential control variables have merged into the model. The family education level, ownership position, and even an outstanding doer of demographical have been introduced as effective invoices. In fact, the neighborhood should be considered a leading factor. Besides, if we want to measure the effect of economic reasons by ignoring the income, it is feasible to define it as a neighborhood or moreover the SES proxy that is widely used in socioeconomic studies. All of the above explains the negative and stark relationship between the lockdown and children’s depression, which is a harmful threat for the future generation and present the necessity for more psychic and social studies (Bignardi et al., 2021). Certainly, to prevent any disproportionate changes, we tend to add more control invoices to our research. The results of psychological surveys in two provinces Huangshi, and Wuhan in China, confirm the results of other researchers. The accomplished investigations on the first victims of the disease and particularly children indicated the sensibility of the situation. The statistical information gathered online on the Crowdsourcing platform with the direct corporation of schoolchildren who were living in the above geographical areas. The advancement in depression levels is up to 22.6% among students who suffer from 18.9% anxiety, provided, and discomfort situations. Additionally, other influential factors as mostly referred to in psychological studies like gender, school grade, being optimistic during the incidences, and having worried about getting infected, wholly affect the evolution of depression symptoms

A Review of Social Conditions During the Quarantine Period-Covid-19

187

and anxiety. The substantial point in this subject which gives us a clue can be former inflectional disease situations. In 2003 Chinese students suffered from a respiratory syndrome which was associated with infectious diseases, hence, it impacted harshly the students’ mental and social health (Xie et al., 2020). So, this is not only limited to the new corona disease, but any restrictions that prevent people from being in contact with their environment and others will have destructive effects on the human condition because humans are social beings by nature.

4 What Situation is Called Quarantine? Separation from people who loved ones, confinement at home, loss of liberty, and disease problems provide an irksome atmosphere for everyone. Nonetheless, the lockdown was the only option to counteract with the Covid-19, and synthesis guidance was needed to overcome the developing situation. The emergence of a tense mental state and depression can be a normal reaction to the conditions of home confinement. Because of the plague, the nature of quarantine, and the loss of freedom, boredom, and uncertainty overseas, most countries had had a dramatic condition (Ettman et al., 2020). These global conditions that arose during the 2020 quarantine had already affected the lives of humanity in smaller ways. At the time of SARS disease, when the hospital staff went through a 9-day quarantine, after the end of the period, they all went through a period of acute stress disorder. In other research on the side-effect of SARS on society, infected and people in contact with them in lockdown reported posttraumatic stress symptoms, which hospital staff also suffered from Bai et al. (2004) and Wu et al. (2009). As stated, Spain was one of the most affected countries in 2020, with 23.822 deaths and a global case fatality rate of 11.3%. Although the number of recovered patients was convincible, the social order was so overshadowed by the event that vital measures to control the disease disrupted other medical issues. The donation and transplantation activities were unavoidably decreased, and the mortality caused by Covid-19, and on the other hand, the death by the lack of organ donation and transplantation on time provided a horrible atmosphere for Spanish society (Domínguez-Gil et al., 2020). Indeed, some results show people in some geographical areas could overcome the social and emotional effects of lockdowns. The Dutch people didn’t show any anxiety or depression disorder during the quarantine and coped with all Covid-19 related changes and threads with calm and tranquility. Statistical tests by applying different influential factors of age, sex, education level, employment status, and ethnicity stated that there were no significant differences in anxiety and depression symptoms of the Dutch population. It was interesting that even people with chronic diseases also maintained their socio-emotional conditions (Velden, et al., 2020). In all the studies conducted since the beginning of this disease, it has attracted the attention of the people of the world both physically and psychologically. And particularly the school children’s mental health who are the feature generation and society must manage them and guide them to create a sustainable generation. Of course, the emotional well-being of adults and parents

188

T. Shahin

of the children has been considered substantially, and in the above paragraphs, the caregivers’ and parents’ socioeconomic status was essential. The available evidence from the University of Chicago and the Boston University that followed the American Association for public opinion research explained that depression disorder could be measured as a categorical variable to see its effects as a ranked spectrum. Regarding determining the amount of the burden of depression during the Coronavirus disease in 2019, American societies witnessed of threefold higher level in the depression syndrome that in comparison with before had increased drastically. If we consider the psychological damage to society as a spectrum and rank it from the low level to the highest level of psychological pressure, the amount of damage to the community was directed at the age groups of 18–39 years who were adults, which Certainly, if we talk about the situation of children, the children or their dependents have been injured and are not immune from injuries. Even it would be predictable that those children would reveal some psychological disorders. It is considered that the risk factor associated with depression can be derived from two reasons, the low income as we have it in the UK and the low level of income and the low amount of savings (lower than 5000$). These phenomena expose poorer families to more stress and bring a greater burden of depression. So, it is reasonable that mental illness affects some people starkly. The damages inflicted on society in the last three years caused by the coronavirus are not only physical, additional it consisted emotional damage and psychological harm. It implies that approximately 96% of the population in 24 USA states, were forced to under-stay at home or in shelters, and struggled with mental harm (Ettman et al., 2020). Among other social damages of this deadly situation, we can mention the employment situation, for example, the number of unemployed people reached 20 million people: after the start of the quarantine in April in America. Evidence from USA provided us complete information about the case of depression prevalence and the US community, not just observed by its economic capacity but also aspects of gender, age, marital status, and household income covered in the investigation.

5 Empirical Study in Spain Studies from other countries led us to investigate the effects of the virus on school children in Spain. In the statistical investigation, we define the mental status as the socio-emotional well-being of school children and the amount of the healthiness reports as a spectrum of 6 ranges.2 Our target variable is a categorical factor, and we are interested to see the impacts of five threats to mental health on it. These threatening factors consist of conciliation, lack of resources, lack of help, insufficient classes, and the lack of spaces. To measure the effects of threatening factors during the quarantine and after the lockdown, we have used the feature input ranking. Since 2

Muy Bueno (Very Good)-Bueno (Good)-Indiferente (Indifferent)-Malo (Bad)-Muy malo (Very bad)-NS/NC (No decision).

A Review of Social Conditions During the Quarantine Period-Covid-19

189

the mental health in this study continues based on a spectrum, the pie chart implies the changes in the physical and emotional well-being of school children in Spain. We witness positive and significant improvements in mental health by considering different threatening mental problems. The enhanced number of people who have a perfect psychic condition, up to 21.2%, indicates a better emotional situation after ending the lockdown also the number of other students with good socio-emotional status has increased, and now 57.4% of them are in the good emotional condition. Consequently, as we expected the results of previous papers associated with the side effects of Covid-19 on the population, the percentages of students with a bad, and a very bad emotional status have reduced to 6.7% and 0.4% respectively. The significant decrease in the percentage of children suffering from unfavorable and dangerous mental conditions is proof of the effectiveness of reopening schools in the country. The stated values and the effect of face-to-face training similar to being among a group of friends can be proven statistically and more accurately through statistical tests to express their importance logically (Labrador et al., 2022) (Fig. 1). Information on Fig. 2 about the cumulative effects of each problem on different mental statuses implies that rough conditions like lockdown would affect all people in a society, thus the reopening of communication centers is undeniable. Table 1 presents the effects of each of five threatening factors on the socioemotional well-being of students according to the survey filled up by their parents. In the surveys conducted among the students, only the health-threatening factors were limited. Other research in the field of Covid considered various influential factors such as age, sex, employment status, family composition, technology at school, etc.

During the lock down 0.053

After the lock down

0.005

0.068

0.126

0.212

0.135

0.246

0.005

0.005

0.425

0.145

0.575

Very good

Good

Indifferent

Very good

Good

Indifferent

Bad

Very bad

NS/NC

Bad

Very bad

NS/NC

Fig. 1 Social-emotional status of school student during the lockdown and after end of the quarantine. Note The presented percentages collected from the parents or caregivers about the school children’s emotional situation first during the confinement, and the second pie chart indicates the socio-emotional well-being after the end of the confinement. Source Ismael Sanz Labrador, Luis Miguel Doncel Pedrera, Jorge Sainz González—Universidad Rey Juan Carlos

190

T. Shahin

Fig. 2 Effects of mental health-threatening factors on each mental status during the confinement

The table provided predictable results based on the prior investigation about the Covid-19 impacts on society (Labrador et al., 2022). To illustrate the changes in the socio-emotional well-being of student schools during the Covid-19 lockdown and after the end of the quarantine, we use multiple comparisons by applying multivariate regression. As the behavior of family members toward each other is a noteworthy treatment, the absence of this loving behavior can report as a practical and threatening factor for children. Considering the results from the survey; this factor can be negatively affected by the coefficient of 0.30 mental health. Another threatening factor implies negative effects, but there are insignificant however; the CI upper limits suggest that at most short increases in these symptoms occurred during lockdown (Bignardi et al., 2021). The problem of the lack of space is hazardous for children’s socio-emotional wellbeing; the negative coefficient of 0.44 with the acceptable probability value can provide an acceptable vision for families about their children requirements. The survey moves forward and checks the effects of other essential elements to show an accurate overview. The factor of sex was not related significantly to the socio-emotional well-being but by considering the age differences of parents, we can see that as the age of the guardians of the examined children increases, the mental health of the student is more threatened, which can indicate that the older people get, the less patient they are to endure difficult and complex situations. Another interesting point was the amount of influence that teachers had on students even during the quarantine period and through non-attendance classes. The cooperation and participation of teachers influenced the improvement of students’ mental and psychological conditions by up to 9%, and this relationship and proper performance in Spain did not end here, and the relationship between families and schools with a coefficient of 0.25 showed that the coordination between these two groups was very effective. And it is undeniable.

A Review of Social Conditions During the Quarantine Period-Covid-19

191

Table 1 Main effects from multivariate regression analysis Socio-emotional wellbeing during the lockdown

Conciliation

Socio-emotional wellbeing after the end of lockdown

Coefficient

Interval coefficient (CI) Coefficient

Interval coefficient (CI)

− 0.306***

− 0.619

− 0.258

0.007

− 0.021

0.216

Lack of resources

− 0.090

− 0.426

0.245

0.0286

− 0.225

0.283

Lack of help

− 0.231

− 0.578

0.114

− 0.147

− 0.409

0.114

Insufficient classes

− 0.266

− 0.588

0.1547

− 0.076

− 0.320

0.166

Lack of spaces

− 0.443***

− 0.820

0.568

0.002

− 0.066

− 0.282***

sex

− 0.179

− 0.488

0.130

− 0.230***

− 0.464

0.003

Age

− 0.021***

− 0.073

0.0008

− 0.010

− 0. 27

0.006

Employment status

0.022

− 0.0921

0.137

0.045

− 0.041

0.132

Home composition

− 0.048

− 0.273

0.176

− 0.090

− 0.260

0.079

Parent education

− 0.026

− 0.279

0.225

− 0.80

− 0.271

0.110

School enough technology

0.026

− 0.271

0.324

− 0.088

− 0.314

0.137

Teacher involvement

0.096***

0.001

0.190

0.080***

0.009

0.152

Public school

0.325

− 0.314

0.965

0.085

− 0.399

0.570

Concerted school

0.274

− 0.365

0.914

0.053

− 0.430

0.537

− 0.660

0.781

− 0.201

− 0.747

0.344

0.045

0.463

0.191***

0.033

0.350

Private school

0.060

Communication between families and school

0.254***

Family help in doing homework

0.063

− 0.077

0.203

0.003

− 0.102

0.109

Education advancement during lockdown

− 0.091

− 0.256

0.073

0.122***

− 0.002

0.246

Number

207

207

RMSE

1.0576

0.8005

R-sq

0.2386

0.2034

F-test

3.273***

2.666***

192

T. Shahin

These results show the importance of the role of the school and the teacher and in general attendance in an academic environment, which can be the foundation of the country’s decision to reopen schools. To display the strength of a relationship among variables we provide the correlation among the targets and dependents in Fig. 3. Even though age is an influential factor in our research, we must communicate about it with caution. Age is an influential factor in the conception of education and psychology, and it undoubtedly affects economic issues through the concepts of raising children and having children. These concepts can be Integrational transmission, which, in fact, the transfer of mental states, the influence of methods of raising children, educational acquisition, and welfare, from one generation (parents) to another generation (children). The mentioned theory ends with the concepts of socialization and social mobility, which of course, it had been notable in economic issues. Therefore, it is conceivable to indicate that psychic changes arising from parents or caregivers’ age transfer to children easily. And whether they experienced an awful emotional status, we can testify of a causal effect of a negative connection between the socio-emotional status of students and the age of their parents. The conception of intergenerational transmission of preferences like altruism, patience, risk aversion, and even other causal channels(schooling), can have a considerable role in the intergenerational transmission and transmitting the emotional state of the people of a family. Of course, there are shreds of evidence, that there are no noticeable effects of parents’ age on the socio-emotional estate of children (Lochner, 2008). For instance, if we focus on the study of Jay Belsky and his colleagues on the society of the west in New Zealand, they also cited that in the western world, the parents’ age is older, hence it is evident that these parents are more able to provide a better growth facility care. Basically, in their studies on the extent to which children are influenced by their parents’ upbringing, they concluded that age was not an important influencing factor. They argued that the extent of parents raising history effect on parenting. If divide parents into two groups of ages, the older the parents or caregivers, the more mature is the process of parenting. So, it would be conceivable that their childhood experiences affect their children less. But conclusively the role of age wasn’t recognized as an effective factor (Trillingsgaard & Sommer, 2018). Maternal age and parental age are always associated with the parents’ behavior which affects variously on children’s emotions. In 2018 Danish mothers reported positive and negative consequences of their manner on their children. In the conducted face-to-face interview, mothers approximately had a better physical and verbal sanctions with their 11 years old children and reported fewer social-emotional difficulties. Conversely, they had problems with children of 15 years old (controversial behavior). But generally, they introduced the factor of age as a positive element in a mother and child relationship in society, and the reason was a mother with older age and higher education was treating her children logically and provided a better atmosphere of living for their children. On the other side, they face conflict with older children associated with the generation gap. This issue to some extent is similar to our experience with the Spanish

193

Fig. 3 The correlation plot

A Review of Social Conditions During the Quarantine Period-Covid-19

194

T. Shahin

generation; well, in our surveys, every family has children of different ages, and younger parents or caregivers presented a more favorable state of mind than their children. Of course, the psychological issue of the Spanish people is also discussed here. Since Spain has happier and more sociable people and they have a warm behavior in communicating with others; so, the diversity in the results can be acceptable. The generation gap is a common phenomenon. As intergenerational solidarity and ambivalence express the inclusion of positive and negative differences between generations, we should expect any results from any ethnicity and generation. While it would be better to rely on the intellectual-social-cultural system of each country in justifying social and psychological factors. In the research of Merrill Silverstein, the generation relationship in Spain was introduced as a detached connection between parents and their children. The obtained information from OASIS, and LSOG between 2020 and 2021 which was focused on a cross national study on the intergenerational relationship; the analyzed Spanish society in their study contained approximately 61.9% of women have children. The Spanish community did not introduce as a community with intimate relationships in the family. This issue can cause by modernization, and because society is growing and renewing very fast; traditional societies unfavorably impact younger. With the steep distance-contact gradient of community in Spain, we cannot witness intimacy among people. The national political economy affects parent-children’s connections, and subsequently, it provides intensive assistance. This cross-nation study among countries expressed Spain as a developed country and performed a spectrum of intergenerational relationships consisting of amicable, detached, disharmonious, and ambivalent. This study presented Spain as a detached society emotionally, with 25% closeness difficulties, 50% conflict, and 30% arguing, which all of this could indicate that as the age of the participants in the interview increased, we saw worse mental-psychological conditions than children (Silverstein et al., 2010). Of course, having fears and disorders among family members is a natural and undeniable reaction, and parents have the right to have fears, whether it is about staying in quarantine or about ending quarantine and returning to their former social life, because of Covid-19’ effects on well-being, nervousness, and concern of parents and children, had an apparent influence which we spoke about previously. After occurring the pandemic and the lockdown, and the speed of the outbreak of the disease, scientists proposed an effective move to improve the tense situation, which is informing the population about their specific needs. Parents’ fears of Covid-19, parents’ dread about family mental health, worry about financial resources, and family difficulties; have provided anxiety and discomfort for individuals. The possibility of happening emotional problems and anxiety disorders among people in the early months of 202 increased drastically. Therefore, assisting families quickly and intervening in their parenting methods could stop the expansion of chronic disorders a lot. We explain among the evidence that the role of teachers and a practical link between school and home have a positive result on students’ mental health. Perhaps this effective communication can be seen as a reason for reducing the fear and anxiety of families from being in quarantine or even fear after the reopening of schools. These parents’ fears existed in different countries. For example, if we

A Review of Social Conditions During the Quarantine Period-Covid-19

195

look at Canadian society, we will find that four models of apprehension affected the lives of the residents of Quebec. And each of them had overshadowed changes in sleeping habits, changes in household income, and even access to health services. Factors such as changes in sleep habits and differences in the level of education had a harsh impact on parents’ fear of contracting the disease. Or on the other hand, irregular sleeping habits, when it was associated with difficulty in accessing health services, provided parents unease about family mental health. Despite the disease, families struggled with contextual problems, such as economic problems, that family income, sleep habits, and family income before Covid-19 struck it harshly. Well, it was clear that the higher the income level, the better the financial conditions during the quarantine period, or if the family members had good and sufficient sleep, as we mentioned before, they suffered from less anxiety and had better conditions during that period. That anxiety and fear were easily transferred from parents to the children, and the home confinement made it worse. This is where the role of teachers and schools is more glowing than ever. The extent to which teachers’ awareness of student education can moderate these parents’ fears and support the reopening plan was evident in the results of Table 1. The outcomes of Sabrina Suffren and her co-workers indicated that the trepidations of families incremented because of the physical and mental health of their children because of the enclosure of medical centers and cancelled appointments. The closure of schools, kindergartens, and universities in all countries was a guarantee of health, and it was only with this isolation that we could survive. So, all of these drastic changes have involved everyone in mental problems, which included both the fear of getting sick and socioeconomic problems (Suffren et al., 2021). During the quarantine, teachers encountered online teaching–learning methods with more responsibilities in the online teaching process. Moreover, the mentors’ abilities and knowledge of technology and digital log literacy turn significant and urgent (Li & Yu, 2022). In the surveys conducted, since parents presented a positive trend in the performance of teachers and their participation in the educational process and tutoring of students, then Spain has teachers and coaches who are aware of the conditions and are perfectly focused on their job. Teachers with higher education and community proper technology provided better conditions for students and teachers. As the disease harmed the worldwide economy, these two participants could help with the damage done to society (Li & Yu, 2022). After occurring the lockdown, it wasn’t clear to nations which teaching method would satisfy students’ requirements and needs for pedagogy efficiency. The evidence from Spain exhibited that an online-teaching approach can improve by more teachers’ involvement in the class contents. Quotes from the book “Visible Learning”, The role of teachers can be divided as follows: • The teaching modality—students’ perception. • Teacher expectations. • teachers’ idea of teaching, learning, assessment, and the students—the advancement of students’ Achievements based on the teacher’s participation. • Teacher perspective—whether teachers are prepared to be surprised.

196

• • • •

T. Shahin

Classroom atmosphere— desirable socio-emotional climate. Role of the teacher in the prosperity of students. The fostering of attempts. Managing the engagement of all students.

The aggregated responsibilities explain that teachers make the difference. The book focused on academic performance. The teacher’s strategies and techniques are fundamental elements of the performance. On The other hand, with the teacher’s involvement in the teaching and the instructiveness of her/his techniques, teachers can make a difference in children performance by 74%. Timely response and formative assessment are another influential factors (Hattie, 2008). The active role and high awareness of the teachers were the guides and saviors of the students even if we look for testifiers from other countries; we can see the conditions prevailing in the city of Guiyang in China. The method of teacher-student interaction was conducive and prompted the academic presentation. The researchers stated about this method that in the process of online teaching, teachers should be aware that they no longer only have the role of transmitting knowledge but also the role of leader and companion, and they should aim to improve students with sufficient time and appropriate guidance. The studies of Jijun Yao and his colleagues were related to “how to manage classes and improve students’ learning capacities”. Of course, since it was the birthplace of the Covid-19 disease, they continued the online method, but “the teaching method of each school” was different from another school. The interesting thing to note and that might be able to show us a more appropriate way to deal with situations like what happened in 2020, was their two models of teaching methods. The first module of teaching is recording video, and the second one is live broadcasting. The badger situation for parents was their disability in teaching their schoolchildren, so they suffered anxiety. And they needed caring teachers who engaged with the student’s crisis. In the video recording method, the educational content was in the form of educational clips with a self-study form. And on the contrary, in the second method of live videos, teachers used the software to follow online and live pedagogy. Therefore, communication interaction, which was the two elements needed by parents to improve both academic conditions and stressful psychological conditions, was eliminated. Instance help, learning role, and accompanying the children, improve the acquisition process of lessons. The relevant study in China showed that teachers’ performance had a direct effect on student performance, and schools with more interactive methods and live-online sessions had better outcomes in science and literal lessons. Of course, in the surveys conducted between schools with two different teaching methods, each type of school has a unified schedule. One of the reasons for the success is that the online teaching method was in the form of live videos by the teachers, and it was the effective communication between the student and the teacher, which was very close to the traditional teaching method. Undoubtedly, this companionship of the teacher with the student gave the student more self-confidence and peace until they left the student with a large number of

A Review of Social Conditions During the Quarantine Period-Covid-19

197

course concepts with greater freedom (self-study). But nothing can fill the place of a teacher’s experience, knowledge, and teaching style. It is acceptable that information technology enables students to study and provides them with more pedagogy sources. But this issue was proven during the quarantine period, which continues despite technological advances; teacher teaching feedback is the fundamental factor in transferring knowledge. In particular, teachers should also strengthen their knowledge in the role of accompaniment and mentor while teaching. Based on the evidence of Jijun Yao and his colleagues, the influential factor in the difference in students’ academic performance is the difference between the teacher and the relationship between the student and the teacher during online teaching (Yao et al., 2020). We can conclude that Spain not only in the field of teacher involvement; but also did a great job in communicating between school and home. The more academic live broadcasts, productive teaching resources, and sufficient feedback were Spanish academic teachers’ efforts, and all of them are commendable. What happened in Portugal in 2020 was associated with the pedagogical and institutional responses to the closure of schools and universities. This study elucidated that the impressive teaching method is the one that emphasizes the depth of course material accompanied by practicum. Even this closure of educational centers should be done as an opportunity for learning to modify the traditional education methods. As a proposal approach among Portugal schools, they depict a practical process with pupils alongside supervision and cooperation of teachers to overcome the difficulties of lockdown. The University of Minho in Portugal was the first place to be closed. And all activities announced suspended. The curses followed by the application of Blackboard during 2020. Throughout the quarantine period, the university emphasized maintaining the interaction with student–teacher, adjustment to the situation, and pedagogy assessment. They did this expected involvement and interactivity in the form of scientific university projects and practicum in a virtual manner, and stipulating the specific situation maintained these connections. Despite the presence of technical issues and the urgent need to adapt to the condition, students experienced a convenient condition and achieved their objectives during the second course of 2020 almost. As a matter of fact, crossing the Covid-19 crisis, needed rapid reactions as a concept of remote teaching. It is true that the research conducted in Portugal is about university students and not school students, but still, at this stage of education, the importance of a good and professional relationship between teachers, supervisors, and students is serious (Assunção Flores & Gago, 2020). The teacher’s involvement and the communication between school and home were salient as we can observe among German schools that these relationships provided highly effective trustworthiness among parents and school. Even the parents in Germany implied their reliability on school administrators with high percentages and selected the parents-teachers conferences methods as a proper way to communicate about important factors like reliability, competence, openness, and honesty (Bormann, 2022). In this section, both the active role of families and the school atmosphere and the participation of school officials can create positive perceptions between mothers and teachers regarding child-related issues (Tobin, 2022).

198

T. Shahin

On the other hand, it is undeniable that the amount of technology available in any country would make the closure of educational centers and services more bearable for the people. We agreed and even witnessed a positive relationship with the emotional state of children during the quarantine period due to the existence of quarantine this transmission, made through applications, which itself was a manifestation of technology and AI. School technology for teachers is a wild spread improvement in communication, and if combined with technological support can provide a better situation (Kraft & Bolves, 2022).

6 Last Word To provide an overview of our work, we strongly advocate for the creation of a serene environment characterized by tranquility and a sense of camaraderie, serving as a powerful counterbalance to the anxieties often associated with quarantine. Notably, parents have a crucial role to play, not only in tending to their children’s mental wellbeing but also in cultivating conducive physical conditions for their return to school. Recognizing the profound importance of personalized and suitable spaces that facilitate individual growth, our advice to parents is to establish a nurturing atmosphere that fosters concentration and supports effective completion of homework, while always considering their children’s unique mental states. Focusing specifically on Spain, our research delved into a comprehensive analysis of the experiences of a specific group of schoolchildren during a critical phase of the pandemic. Spain faced distinctive challenges and implemented targeted strategies to mitigate the virus’s impact on education and the well-being of children. Consequently, our study offers significant and enlightening insights into the outcomes of these measures within the Spanish context, shedding light on the effectiveness of regionally tailored interventions. Furthermore, this investigation of the virus’s impact on Spanish schoolchildren lays a solid foundation for conducting comparative studies across diverse contexts. By understanding the unique factors that shape the socio-emotional well-being of Spanish students, researchers in other countries can draw parallels and make informed decisions when devising strategies to support the mental health of students in their own regions. Thus, while our study focused on a specific country, its potential extends far beyond, serving as a catalyst for further research on a global scale, fostering cross-cultural insights, and driving advancements in this critical field. Moreover, as a firsthand witness to the challenges posed by the Covid-19 quarantine, I would like to express my heartfelt gratitude to the dedicated teachers and administrative staff of educational institutions. Their unwavering support, active engagement in shaping educational content, and effective communication with parents has been invaluable in providing much-needed social and mental support. Their commitment and efforts have played a pivotal role in nurturing the overall well-being of students.

A Review of Social Conditions During the Quarantine Period-Covid-19

199

References Assunção Flores, M., & Gago, M. (2020). Teacher education in times of COVID-19 pandemic in Portugal: National, institutional and pedagogical responses. Journal of Education for Teaching, 46(4), 507–516. Bai, Y., et al. (2004). Survey of stress reactions among health care workers involved with the SARS outbreak. Psychiatric Services, 55(9), 1055–1057. Bignardi, G., et al. (2021). Longitudinal increases in childhood depression symptoms during the COVID-19 lockdown. Archives of Disease in Childhood, 106(8), 791–797. Bormann, I., et al. (2022). Home–school interaction: A vignette study of parents’ views on situations relevant to trust. European Education, 1–15. Domínguez-Gil, B., et al. (2020). COVID-19 in Spain: Transplantation in the midst of the pandemic. American Journal of Transplantation, 20(9), 2593–2598. Ettman, C. K., et al. (2020). Prevalence of depression symptoms in US adults before and during the COVID-19 pandemic. JAMA Network Open, 3(9), e2019686–e2019686. Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge. Kraft, M. A., & Bolves, A. J. (2022). Can Technology transform communication between schools, teachers, and parents? Evidence from a randomized field trial. Education Finance and Policy, 17(3), 479–510. Labrador, I. S., Pedrera, L. M. D., & González, J. S. (2022). La pandemia y el bienestar emocional de los alumnos. Razón y Fe: Revista Hispanoamericana De Cultura, 285(1455), 41–54. Li, M., & Yu, Z. (2022). Teachers’ Satisfaction, role, and digital literacy during the COVID-19 pandemic. Sustainability, 14(3), 1121. Lochner, L. (2008). Intergenerational transmission. The New Palgrave Dictionary of Economics, 2. Newlove-Delgado, T., et al. (2021). Child mental health in England before and during the COVID-19 lockdown. The Lancet Psychiatry, 8(5), 353–354. Pierce, M., et al. (2020). Mental health before and during the COVID-19 pandemic: A longitudinal probability sample survey of the UK population. The Lancet Psychiatry, 7(10), 883–892. Raw, J. A., et al. (2021). Examining changes in parent-reported child and adolescent mental health throughout the UK’s first COVID-19 national lockdown. Journal of Child Psychology and Psychiatry, 62(12), 1391–1401. Silverstein, M., et al. (2010). Older parent–child relationships in six developed nations: Comparisons at the intersection of affection and conflict. Journal of Marriage and Family, 72(4), 1006–1021. Suffren, S., et al. (2021). Relations between child and parent fears and changes in family functioning related to COVID-19. International Journal of Environmental Research and Public Health, 18(4), 1786. Tobin, E., et al. (2022). Family–school connectivity during transition to primary school. Educational Research, 1–18. Trillingsgaard, T., & Sommer, D. (2018). Associations between older maternal age, use of sanctions, and children’s socio-emotional development through 7, 11, and 15 years. European Journal of Developmental Psychology, 15(2), 141–155. van der Velden, P. G., et al. (2020). Anxiety and depression symptoms, and lack of emotional support among the general population before and during the COVID-19 pandemic. A prospective national study on prevalence and risk factors. Journal of Affective Disorders, 277, 540–548. Waite, P., et al. (2021). How did the mental health symptoms of children and adolescents change over early lockdown during the COVID-19 pandemic in the UK? JCPP Advances, 1(1), e12009. Wu, P., et al. (2009). The psychological impact of the SARS epidemic on hospital employees in China: Exposure, risk perception, and altruistic acceptance of risk. The Canadian Journal of Psychiatry, 54(5), 302–311.

200

T. Shahin

Xie, X., et al. (2020). Mental health status among children in home confinement during the coronavirus disease 2019 outbreak in Hubei Province, China. JAMA Pediatrics, 174(9), 898–900. Yao, J., et al. (2020). What role should teachers play in online teaching during the COVID-19 pandemic? Evidence from China. Science Insights Education Frontiers, 5(2), 517–524.

Bridging the Gap: Addressing Inequities in Modern Education Assessment

The rapid spread of the COVID-19 pandemic worldwide not only posed challenges but also unveiled persistent inequities that have long lurked within educational systems. Education, integral to societal development, found itself at a crossroads. Amidst these transformations, assessing students’ capabilities and understanding became subjects of intense debate.

The Digital Transformation: A Double-Edged Sword The immediate thrust into online learning platforms was a necessity rather than a choice. European countries, among others, witnessed a marked increase in the utilization of digital tools for education. However, while the shift appeared seamless for some, many students grappled with accessibility. This technological disparity magnified during assessments; students unequipped with stable internet connections or the required devices were inadvertently disadvantaged. It becomes evident that the pandemic, while posing unprecedented challenges, also presents a unique opportunity. It allows for a revaluation and reformation of existing systems, pushing towards a future where assessments are both equitable and empowering. Significant changes in education should follow this volume in the future. Among them, the raise and reinforcement of Artificial Intelligence especially in Higher Education, the development of a European model of Emergency Learning, new types of assessments (not just continuous or formative assessment) due the ethical matters just to mention a few. In this sense, the European Union has the obligation to lead the fundamental changed upon which led to significant updates and develop new methodologies based on grounded data-driven policies. Especial, attention should be given to citizen-research which based on common grounds among the different countries of the European Union applies common policies to specific local contexts.

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Sainz and I. Sanz (eds.), Addressing Inequities in Modern Educational Assessment, https://doi.org/10.1007/978-3-031-45802-6

201

202

Bridging the Gap: Addressing Inequities in Modern Education Assessment

Deepening Divides: Beyond Technology While the digital gap was glaring, underlying challenges were equally pressing. Factors such as gender, socio-economic backgrounds, and geographic location began significantly influencing educational experiences. The inherent biases in educational assessments were laid bare, revealing that the deep-seated issues were not merely technological but socio-cultural and economic. Cooperative social development and interaction showed that education, except for some specific cases, requires the in-class interaction as proved by current literature. While some students see the facilities of online learning, most prefer to be in a real classroom with their classmates and colleagues. Online teachers also acknowledge the importance of in-person education to get more participation, more opportunities for debate and in-classroom participation.

The Gender Bias in STEM Education Europe’s low proportion of women STEM graduates provided a snapshot into a larger global trend. Despite girls performing commendably in subjects like mathematics and science, they often faced systemic biases that discouraged further pursuit in these fields. Educational assessments, rather than being neutral, occasionally perpetuated these biases, making it imperative to challenge and change these norms, but the leaking STEM pipeline women face along school and labour market is a multidimensional and complex phenomenon, that involves families, schools and the society as a whole. This complexity requires the reinforcement of a rigorous design in the measuring, reporting, assessment and evaluation of both specific interventions and more global policies, including impact analysis methodologies. This is key to the identification of successful strategies to reduce the gender gaps in STEM.

Spain’s Resilience: Tailored Responses in Testing Times Lockdowns and school closures induced by the COVID-19 pandemic have resulted in significant and substantial learning losses. This chapter review and discusses the estimates of the effect of school closures on student achievement in the literature. We also conducted a meta-analysis of this literature to conclude that school closures negatively impacted school learning by around 0.18 standard deviations. Alternative policies can be used to alleviate this negative impact. Spain’s response to the pandemic’s educational challenges showcased the importance of context-specific solutions. A blanket approach to education and assessment can inadvertently widen the existing gaps.

Bridging the Gap: Addressing Inequities in Modern Education Assessment

203

Reimagining the Role of Educators Teachers and instructors worldwide demonstrated remarkable adaptability, acting as frontline warriors in the educational upheaval. Beyond the knowledge they can transmit, they played pivotal roles in reinventing assessment methods. Their endeavors to create more inclusive, adaptable, and holistic evaluation methods, such as project-based assessments and open-book exams, highlighted the need for continuous evolution in pedagogical approaches.

The Rise of Continuous Assessments The pandemic catalyzed a shift from traditional, summative assessments to more formative, continuous ones. Recognizing the disparities in students’ home environments and resources, educators leaned towards evaluations that spanned the academic year, reducing the weightage of final exams and thereby lessening associated pressures. Nowadays bullying and cyberbullying in schools constitute one of the relevant public health problems, resulting in a lower degree of social interaction and integration, and increasing direct and indirect costs of the education system. In this way, it is very interesting thing to know if the government closure of schools in the Spring of 2020, due to Covid-19, could have had effects on bullying and cyberbullying in schools and how were these effects in the case of Spain. Attending these results and methods, in the current research, the analysis of bullying and cyberbullying has been conducted using the results for the topic “bullying”, and “cyberbullying” from Google Trends, with normalized search data, and extracting weekly results for the period September 2017-August 2022 for all Spanish regions. It is shown that physical presence influences clearly in the levels of bullying. The effects of the school lockdowns in reducing bullying in Spain is not fading out over a period after school lockdowns, but, eventually, the search intensity of the chosen topics returns to pre-pandemic levels, showing the reduction a temporary character.

Feedback: Moving Beyond Numbers Modern education is progressively recognizing the value of comprehensive feedback. Instead of solely relying on numerical scores, which remain important, there’s a growing emphasis on qualitative feedback. Detailed insights, highlighting strengths and areas for improvement, provide students with a roadmap for academic growth, ensuring assessments are constructive rather than just evaluative. The random experiment conducted in Spain after de pandemic with higher education students presented in this book reflects that there are no significant differences in students’ grades between the treatment group, the group that received relative feedback on their grades

204

Bridging the Gap: Addressing Inequities in Modern Education Assessment

throughout the semester, and the control group. Student attendance in class during this experiment has been reduced due to the recent COVID-19 pandemic, which could lead to inconclusive results. Therefore, it is important to note that this experiment serves as a preliminary study for a long-term research project with a larger sample size and the aim of comparing whether the outcomes in the treatment group are superior to those in the control group.

Involving Stakeholders: Crafting Inclusive Assessment Tools Creating fair and equitable assessment methods requires collaboration. By incorporating insights from diverse stakeholders—including students, parents, educators, and community leaders—assessment tools can become more reflective of collective needs and challenges, ensuring no student is inadvertently left behind. A key takeaway from our findings is the critical role of communication between families and schools. Regular communication fosters understanding, trust, and support, which are essential for students’ mental well-being, especially during times of stress like the pandemic. Moreover, the introduction of conciliation services can offer a constructive avenue for resolving conflicts between families and schools, ultimately enhancing cooperation, and ensuring equitable outcomes. The imposition of lockdown measures magnified the negative psychological consequences for students, warranting further research into the long-term repercussions. Policymakers and educational practitioners are urged to develop strategies that prioritize students’ mental health, both during the pandemic and its aftermath. By fostering an environment of serenity and collaboration, we can mitigate the anxieties associated with periods of isolation and uncertainty.

Policy Interventions: A Need for Systemic Change Governments and educational bodies play a crucial role in shaping the educational landscape. Policy interventions that prioritize equitable access to resources, teacher training for inclusive education, and research into unbiased assessment methods can form the bedrock of a more egalitarian educational system. Small group tutoring is one of the educational measures for which there is empirical evidence of effectiveness in rigorous research studies. Studies reviewed by the Education Endowment Foundation show that half an hour of small group tutoring a day for 12 weeks produces an additional four months of progress in school, compensating for the loss of three months of schooling due to the closure of schools. High dosage tutoring can particularly increase cognitive outcomes for students from disadvantaged backgrounds. The provision of online tutoring through tutors in the last years of higher education or recent graduates has already shown positive and significant effects (equivalent to what is learned in up to 5 months) in rigorous evaluations.

Bridging the Gap: Addressing Inequities in Modern Education Assessment

205

Charting the Future: From Challenges to Opportunities While the pandemic underscored the vulnerabilities of current educational systems, it also paved the way for introspection and innovation. Harnessing this momentum can guide the transformation of assessments, ensuring they are not just tests of knowledge but tools for inclusive growth. Our study emphasizes the need for a holistic approach to support students’ well-being during unprecedented challenges. By nurturing open communication, exploring conciliation options, and implementing targeted interventions, we can collectively work toward a resilient educational system that safeguards the mental health of our students.