Information Security Education - Adapting to the Fourth Industrial Revolution: 15th IFIP WG 11.8 World Conference, WISE 2022, Copenhagen, Denmark, ... and Communication Technology, 650) 3031081714, 9783031081712

Citation preview

IFIP AICT 650

Lynette Drevin Natalia Miloslavskaya Wai Sze Leung Suné von Solms (Eds.)

Information Security Education - Adapting to the Fourth Industrial Revolution 15th IFIP WG 11.8 World Conference, WISE 2022 Copenhagen, Denmark, June 13–15, 2022 Proceedings

IFIP Advances in Information and Communication Technology

650

Editor-in-Chief Kai Rannenberg, Goethe University Frankfurt, Germany

Editorial Board Members TC 1 – Foundations of Computer Science Luís Soares Barbosa , University of Minho, Braga, Portugal TC 2 – Software: Theory and Practice Michael Goedicke, University of Duisburg-Essen, Germany TC 3 – Education Arthur Tatnall , Victoria University, Melbourne, Australia TC 5 – Information Technology Applications Erich J. Neuhold, University of Vienna, Austria TC 6 – Communication Systems Burkhard Stiller, University of Zurich, Zürich, Switzerland TC 7 – System Modeling and Optimization Fredi Tröltzsch, TU Berlin, Germany TC 8 – Information Systems Jan Pries-Heje, Roskilde University, Denmark TC 9 – ICT and Society David Kreps , National University of Ireland, Galway, Ireland TC 10 – Computer Systems Technology Ricardo Reis , Federal University of Rio Grande do Sul, Porto Alegre, Brazil TC 11 – Security and Privacy Protection in Information Processing Systems Steven Furnell , Plymouth University, UK TC 12 – Artificial Intelligence Eunika Mercier-Laurent , University of Reims Champagne-Ardenne, Reims, France TC 13 – Human-Computer Interaction Marco Winckler , University of Nice Sophia Antipolis, France TC 14 – Entertainment Computing Rainer Malaka, University of Bremen, Germany

IFIP – The International Federation for Information Processing IFIP was founded in 1960 under the auspices of UNESCO, following the first World Computer Congress held in Paris the previous year. A federation for societies working in information processing, IFIP’s aim is two-fold: to support information processing in the countries of its members and to encourage technology transfer to developing nations. As its mission statement clearly states: IFIP is the global non-profit federation of societies of ICT professionals that aims at achieving a worldwide professional and socially responsible development and application of information and communication technologies. IFIP is a non-profit-making organization, run almost solely by 2500 volunteers. It operates through a number of technical committees and working groups, which organize events and publications. IFIP’s events range from large international open conferences to working conferences and local seminars. The flagship event is the IFIP World Computer Congress, at which both invited and contributed papers are presented. Contributed papers are rigorously refereed and the rejection rate is high. As with the Congress, participation in the open conferences is open to all and papers may be invited or submitted. Again, submitted papers are stringently refereed. The working conferences are structured differently. They are usually run by a working group and attendance is generally smaller and occasionally by invitation only. Their purpose is to create an atmosphere conducive to innovation and development. Refereeing is also rigorous and papers are subjected to extensive group discussion. Publications arising from IFIP events vary. The papers presented at the IFIP World Computer Congress and at open conferences are published as conference proceedings, while the results of the working conferences are often published as collections of selected and edited papers. IFIP distinguishes three types of institutional membership: Country Representative Members, Members at Large, and Associate Members. The type of organization that can apply for membership is a wide variety and includes national or international societies of individual computer scientists/ICT professionals, associations or federations of such societies, government institutions/government related organizations, national or international research institutes or consortia, universities, academies of sciences, companies, national or international associations or federations of companies. More information about this series at https://link.springer.com/bookseries/6102

Lynette Drevin Natalia Miloslavskaya Wai Sze Leung Suné von Solms (Eds.) •

•

•

Information Security Education - Adapting to the Fourth Industrial Revolution 15th IFIP WG 11.8 World Conference, WISE 2022 Copenhagen, Denmark, June 13–15, 2022 Proceedings

123

Editors Lynette Drevin North-West University Potchefstroom, South Africa

Natalia Miloslavskaya National Research Nuclear University Moscow, Russia

Wai Sze Leung University of Johannesburg Johannesburg, South Africa

Suné von Solms University of Johannesburg Johannesburg, South Africa

ISSN 1868-4238 ISSN 1868-422X (electronic) IFIP Advances in Information and Communication Technology ISBN 978-3-031-08171-2 ISBN 978-3-031-08172-9 (eBook) https://doi.org/10.1007/978-3-031-08172-9 © IFIP International Federation for Information Processing 2022 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

This volume contains the papers presented at the 15th World Conference on Information Security Education (WISE 15) held during June 13–15, 2022, which was co-located with the 37th International Conference on ICT Systems Security and Privacy Protection (SEC 2022)—the flagship event of the International Federation for Information Processing (IFIP) Technical Committee 11 (TC-11). WISE 15 was organized by the IFIP Working Group (WG) 11.8, which is an international group of people from academia, government, and private organizations who volunteer their time and effort to increase knowledge in the very broad field of information security through education. WG 11.8 has worked to increase information security education and awareness for almost two decades. This year, WG 11.8 organized the 15th conference of a successful series under the theme “Information Security Education – Adapting to the Fourth Industrial Revolution.” In response to our call for papers, we received 17 submissions from around the world. Prior to the assignment of reviewers to the submissions, members of the international Program Committee (PC) were asked to indicate their conflict of interest with submissions. This was further checked by the committee to ensure that any conflicts of interest and potential bias were adequately managed. Each paper was then subject to double-blind review by at least three members of the PC over a review period of 12 days. Where deemed necessary, additional reviewers with the necessary expertise to assess the submissions appropriately were identified to assist with assessing the quality of the work. Papers were reviewed according to a set of criteria which included relevance, novelty and originality, technical strength, discussion of related work, the quality of the design and methodology, and the paper’s plausibility. After careful consideration, the committee decided to accept eight full papers. The acceptance rate for the conference was thus 47%. This conference took place thanks to the support and commitment of many individuals. First, we would like to thank all TC-11 members for continually giving us the opportunity to serve the working group and organize the WISE conferences. Our sincere appreciation also goes to the members of the Program Committee and the external reviewers, and to the authors who trusted us with their intellectual work. We are grateful for the support of WG 11.8 Officers Erik Moore, Jacques Ophoff, and Matt Bishop. In addition, we would like to thank the SEC 2022 organizers, Christian Damsgaard Jensen, Simone Fischer-Hübner, and their team, for their efforts in helping us to host WISE 15.

vi

Preface

Finally, we wish to acknowledge the EasyChair conference management system which was used for managing the submissions and reviews of WISE 15 papers. As for the preparation of this volume, we sincerely thank our publisher Springer for their assistance. June 2022

Lynette Drevin Natalia Miloslavskaya Wai Sze Leung Suné von Solms

Organization

Conference Chair Erik Moore

Regis University, USA

Program Chairs Lynette Drevin Natalia Miloslavskaya

North-West University, South Africa National Research Nuclear University MEPhI, Russia

Conference Secretariat Matt Bishop

University of California, Davis, USA

Publication Chairs Wai Sze Leung Suné von Solms

University of Johannesburg, South Africa University of Johannesburg, South Africa

Web Chair Jacques Ophoff

Abertay University, UK

Program Committee Maria Bada Matt Bishop Nathan Clark Lynette Drevin Jun Dai Ludwig Englbrecht Steven Furnell Lynn Futcher Ram Herkanaidu Lech Janczewski Christos Kalloniatis Sokratis Katsikas Konstantin Knorr Elmarie Kritzinger Wai Sze Leung Dan Likarish

Queen Mary University of London, UK University of California, Davis, USA University of Plymouth, UK North-West University, South Africa California State University, Sacramento, USA University of Regensburg, Germany University of Nottingham, UK Nelson Mandela University, South Africa University of Plymouth, UK University of Auckland, New Zealand University of the Aegean, Greece Norwegian University of Science and Technology, Norway Trier University of Applied Sciences, Germany University of South Africa, South Africa University of Johannesburg, South Africa Regis University, USA

viii

Organization

Javier Lopez Herbert Mattord Natalia Miloslavskaya Stig Mjolsnes Erik Moore Jacques Ophoff Günther Pernul Tobias Pulls Kai Rannenberg Carlos Rieder Léo Robert Rudi Serfontein Chien-Chung Shen Alireza Shojaifar Alexander Tolstoy Susanne Wetzel

University of Malaga, Spain Kennesaw State University, USA National Research Nuclear University MEPhI, Russia Norwegian University of Science and Technology, Norway Regis University, USA Abertay University, UK University of Regensburg, Germany Karlstad University, Sweden Goethe University Frankfurt, Germany isec ag, Switzerland LIMOS, Université Clermont Auvergne, France North-West University, South Africa University of Delaware, USA University of Applied Sciences and Arts Northwestern Switzerland, Switzerland National Research Nuclear University MEPhI, Russia Stevens Institute of Technology, USA

Additional Reviewers Rasmus Dahlberg Peter Hamm Katerina Mavroeidi

Daniel Schlette Michael Schmid Manfred Vielberth

Contents

Securing the Fourth Industrial Revolution Through Programming Analyzing Error Rates in Cryptological Programming Assignments . . . . . . . . Konstantin Knorr

3

SecTutor: An Intelligent Tutoring System for Secure Programming . . . . . . . . Ida Ngambeki, Matt Bishop, Jun Dai, Phillip Nico, Shiven Mian, Ong Thao, Tran Ngoc Bao Huynh, Zed Chance, Isslam Alhasan, and Motunrola Afolabi

17

Cybersecurity in the Fourth Industrial Revolution: Charting the Way Forward in Education How to Overcome Staff Shortage in Professionals for SOCs and NSICs . . . . Natalia Miloslavskaya and Alexander Tolstoy Collaborative Cybersecurity Learning: Establishing Educator and Learner Expectations and Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Steven Furnell, Gregor Langner, Teemu Tokola, Jerry Andriessen, Gerald Quirchmayr, and Carmela Luciano A Maturity Assessment Model for Cyber Security Education in Europe . . . . . Silvia Vidor and Carlos E. Budde

31

46

60

Real-World Cybersecurity-Inspired Capacity Building ForCyRange: An Educational IoT Cyber Range for Live Digital Forensics . . . Sabrina Friedl, Magdalena Glas, Ludwig Englbrecht, Fabian Böhm, and Günther Pernul South Africa’s Taxi Industry as a Cybersecurity-Awareness Game Changer: Why and How? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Petrus Duvenage, Victor Jaquire, and Sebastian von Solms

77

92

Maritime Cyber Threats Detection Framework: Building Capabilities. . . . . . . Georgios Potamos, Savvas Theodoulou, Eliana Stavrou, and Stavros Stavrou

107

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

131

Securing the Fourth Industrial Revolution Through Programming

Analyzing Error Rates in Cryptological Programming Assignments Konstantin Knorr(B) Trier University of Applied Sciences, Trier, Germany [email protected]

Abstract. Understanding cryptological primitives like encryption, hashing, signatures, and certificates is a central skill when working as an IT security professional or software developer but it is also a major educational challenge. The paper presents a study which measures and compares error rates in cryptological programming assignments. Over a ten-week period, 20 students solved 20 cryptological Java programming assignments checked by 350 tests that were automatically verified using a grader system. The error rate in ~60.000 test results is analyzed: Students made fewer errors in substitutions than transposition ciphers, symmetric ciphers rank lower than asymmetric ones, constructor, exception and padding tests appear easier to solve than signing and its verification. Asymmetric encryption has lower error rates than signing. A discussion of the findings, limitations, and possible future improvements concludes the paper. The approach allows identifying and measuring “hard” and “easy” cryptological assignments in order to improve teaching, which is desirable from an educational perspective. Keywords: Cryptology · Error rate · Grader · Java · JUnit Tests · Programming assignment

1 Introduction Implementing cryptological software is subtle, can easily lead to attackable code, and should be done by experienced developers. Notorious problems are side channel attacks for example when implementing RSA or insufficient pseudo randomness. On the other hand, more and more information is processed and needs protection in our digital world. Recent studies underline the problems developers face when using cryptology in their software. Nadi et al. [1] identify the major obstacle as difficult API usage and missing domain and provider knowledge. Hazhirpasand et al. [2] detect a distinct lack of knowledge in understanding the fundamental cryptological concepts. Secure coding guidelines exist [3] but often developers lack a thorough understanding of cryptology. Typical problems include using ECB instead of GCM or CCM mode in symmetric encryption or confusing the different key types in asymmetric schemes. This paper addresses these shortcomings by letting students code cryptological assignments in Java. Coding cryptology helps students to better understand the underlying theory [4]. A grader automatically assesses the correctness of the code using predefined cryptological Java test cases. Examples for tests include the correct encryption of © IFIP International Federation for Information Processing 2022 Published by Springer Nature Switzerland AG 2022 L. Drevin et al. (Eds.): WISE 2022, IFIP AICT 650, pp. 3–16, 2022. https://doi.org/10.1007/978-3-031-08172-9_1

4

K. Knorr

a plaintext, the verification of a signature, or a thrown exception when using an incorrect key size. By summing up over all uploads the results can then be used to assess the error rates of the test cases, categories of test cases, assignments, and groups of assignments. This allows investigating in detail which parts of the cryptological assignments cause students most problems. Students are allowed an unlimited number of uploads of their code and are immediately confronted with the results of the test cases. They can improve their code until the deadline of the assignment. As a side effect, the final upload can also be used for grading the assignment (percentage of positive test cases = grade). The idea of using automatic graders in the classroom is not revolutionary and has raised the interest of many researchers [5–8]. The authors however do not use graders in a cryptological setting. Braga et al. [9] use test-driven software development in the construction of cryptographic software. They argue that test cases can automate acceptance tests for cryptographic software. Cryptographic test cases are considered good acceptance tests as they meet halfway between cryptologists and developers. However, they do not use Java and utilize a non-educational setting. Edwards et al. [10] investigate static analysis errors like formatting and JavaDoc style and coding flaws in half a million Java programs. They track changes between computer science students compared to other students over four semesters. Rivers et al. [11] use learning curve analysis on students’ Python program to identify problematic programming concepts. They discover that over time learning some concepts like function definitions and comparisons are easier to learn than e.g. binary operations. Sivasakthi and Rajendran [12] use questionnaires to investigate topics causing most problems when learning Java: concurrent programming, UI programming with Swing, and generic programming are hardest, object orientation and fundamental programming structure easiest. Lahtinen et al. [13] study the difficulties of novice programmers in Java, C++, and Pascal based on questionnaires for students and teachers. Finding bugs in the own code and using recursion, pointers, and references are hardest. Nadi et al. [1] evaluate Stack Overflow posts and GitHub repositories in order to identify security problems Java developers face. Most problematic are the correct use of symmetric encryption as well as signature generation and verification. Also by assessing the stack overflow web site Hazhirpasand et al. [2] detect a distinct lack of knowledge in understanding the fundamental cryptological concepts (e.g. for OpenSSL, publickey cryptography, or password hashing). Additionally, the usability of cryptological libraries undermines developers to correctly realize a cryptological scenario. Lazar et al. [14] confirm this finding: by analyzing the CVE database for cryptographic problems, they show that just 17% of the bugs are in cryptographic libraries, 83% are misuses of cryptographic libraries. Knorr describes the didactical framework used in this study in [4]. Preliminary results of the study are published in [15]. This paper extends the study presented in [15] and improves the investigation. To the best knowledge of the author, this is the first attempt to measure error rates with Java tests in cryptological programming assignment in an educational setting. The remainder of the paper has the following structure: Sect. 2 gives the necessary background information on the grader system used, on Java tests, and on programming

Analyzing Error Rates in Cryptological Programming Assignments

5

cryptology in Java. Section 3 details on the set-up of the study including a formal definition of the error rate. The results of the study are topic of Sect. 4. Finally, a discussion and future work topics conclude the paper in Sect. 5.

2 Background 2.1 Automatic Software Evaluation System Automatic assessment and grading of programming assignments have been in the focus of researchers and practitioners for many years, see [5, 6] for an introduction and overview of current systems. The main advantages of these systems are traceability of the programming progress, immediate feedback, and scalability. Especially for larger cohorts, automated grading is a vital tool in study programs. In addition, the approach is well suited for distance learning. This paper makes use of the existing and long-running ASE (Automatic Software Evaluation) system, which is described in detail in [16]. ASE has been in use since 2006 at Trier University of Applied Sciences and is mainly used for introductory classes in object-oriented programming in Java. Its architecture is a web based client-server system. Shibboleth provides authentication. TLS secures the network traffic. A detailed privacy policy explains the usage of the students’ data, see [4] for a more thorough discussion of privacy issues. ASE’s kernel comprises web sockets for managing clientserver communication and execution environments for selected programming languages like Java, C++, and Python. Each language can load plugins e.g. unit test runners or modules to check programming style. Tests can be configured using XML files. The result of the assessment of the student’s code is stored in a database and written in an XML file that is then presented in the student’s browser. ASE allows defining the maximal execution time for each test case, which is especially important for exhaustive operations like asymmetric key generation or encryption. The parameters were chosen and tested prior to the first upload of the students. The waiting time was less than 10 s per upload. Additionally, ASE uses special Java annotations to configure the tests. For example, @TestFailureMsg provides the opportunity to display a specific message in case of a failed test. See [4, 16] for more information. A typical life cycle for ASE assignments is the following: • The instructor presents the necessary learning matter in class and prepares the corresponding assignments, a sample solution, the test cases, and optionally a code skeleton for the solution. The test cases are uploaded to ASE. • Students solve the assignments and submit the solution to ASE. With ASE, they can upload an arbitrary number of solutions within the given time frame. • ASE verifies each submission by executing the test cases on the submitted code and provides immediate feedback which tests are correct or wrong with error messages defined as part of the test cases. • After the deadline, the last uploaded solution can be used for grading the assignment. The instructor reviews the assignment results, can comment on common errors in the next lecture, and optionally improves the test cases and assignments for future classes.

6

K. Knorr

2.2 Java Tests and Cryptology Test-driven software development stems from Beck [17]. The fundamental idea is to produce only code that passes predefined test cases. It encourages programmers to always have a running version of their code and to test features and code during implementation thereby avoiding integration problems typically encountered in other development models at the end of a project.

Fig. 1. JUnit test cases for Playfair (left), Elgamal (right) ciphers, and their run time in the Eclipse IDE. Test encryption3 produced a failure, as an incorrect ciphertext was calculated. Test illegalCiphertext raised an exception. All Elgamal tests passed.

For Java, JUnit [18] provides a framework for test cases. It allows for testing classes, methods, and exceptions. Major IDEs like Eclipse provide GUIs to visualize JUnit tests and their results (cf. Fig. 1). A JUnit test can have three different results: • Passed (colored green), e.g. test decrpytion1 in Fig. 1. • Error (colored red): an unexpected exception has been thrown by the code. • Failure (colored blue): an incorrect result has been calculated for that test. ASE maps the result “Passed” to true, “Error” and “Failure” to false. Test cases that expect a certain exception adequately address typical cryptological problems like incorrect key size, erroneous padding, and incorrect private or public key usage. The test case illegalCiphertext in Fig. 1 e.g. requires a specific exception to be thrown due to an invalid ciphertext (which the code does not do). A detailed discussion of the methodology can be found in [4], the source code of the tests above in [19].

Analyzing Error Rates in Cryptological Programming Assignments

7

Table 1. Assignments checked by ASE including their group and number of test cases Group

Abbr.

Assignments and number of test cases (in brackets)

Classical substitution ciphers

SC

Playfair cipher (20), FourSquare (23), Porta (24)

Classical transposition ciphers

TC

Railfence (6), Skytale (19), Bifid (24)

Classical asymmetric ciphers

CA

RSA (22), Rabin (23), Elgamal (25)

Modern asymmetric ciphers

MA

Paillier (17), NTRU (15)

Signature schemes

SIG

Rabin signatures (14), Lamport signatures (21)

Elliptic curves

EC

myEC (16), ECDH (14), ECDSA (13)

Miscellaneous

MISC

BigInteger (14), Shamir Secret Sharing (22), Java Key Store (10), Padding Oracle (5)

3 Setup of the Study 3.1 Background and Assignment Selection The study makes use of data of the elective Bachelor course “Programming Cryptology” in the winter semester 2020/21 at Trier University of Applied Sciences. Twenty students participated in the course, fifteen regularly uploaded assignments in ASE. As a prerequisite students had to visit an introductory class on object-oriented programming and a class in IT security including an introduction to cryptography in previous semesters. All assignments were in Java as Java is presently the main programming language taught to students in the programming education of the computer science department. Of the 14 weeks of the semester, programming assignments covered ten. ASE checked the assignments of seven weeks, see Table 1. In the last four weeks students had to do a project of their choice related to the topics of the class. Cryptological concepts comprise but are not limited to encryption/decryption, signature/signature verification, hashing, and pseudo random number generation. The topics covered in the course roughly follow standard cryptological textbooks like [20] for the theoretical part, [21] for the historic ciphers, and [22–24] for Java cryptology. The assignments were chosen based on the recommendations given in [3, 14] and selected for didactical or practical purposes. The assignments covered all current cryptology topics like symmetric ciphers, asymmetric schemes, key exchange (Diffie Hellman), hash functions, pseudo random functions, X.509 certificates, elliptic curves und selected post-quantum schemes (NTRU, Lamport Signatures). In total there were 34 assignments, ASE checked 20 of them, cf. Table 1. In the 20 ASE assignments students had to code cryptographic primitives from scratch (e.g. RSA encryption). The selection includes the cryptological systems covered in the previously visited IT security course and adds new systems in order to guarantee a broad spectrum of number theoretical concepts. The 20 ASE assignments all follow a similar structure: they require the definition of the Java class, constructor, and cryptological functions following the categories in Table 2. The same person wrote the assignments. A sample can be found in [4, 19].

8

K. Knorr

The remaining 14 assignments were manually checked based on the source code and videos produced by the students. They addressed the correct use of established libraries. Examples for these assignments are the development of a password safe using AES and a command line signature application using ECDSA and X.509 certificates. This split was done to balance between finding difficulties in cryptological assignments on one hand and teaching students not to code their own cryptological code on the other hand. Programmers should use existing and well-established code like OpenSSL or Bouncy Castle [24] for real life scenarios. Table 2. Categories of test cases and its numbers Category and number of test cases Explanation Encr (34)

Encrypt a plaintext with a given cipher and key

Decr (29)

Decrypt a ciphertext with a given cipher and key

EncDec (37)

Encryption and subsequent decryption using a given and a possibly randomly generated plaintext. Check if result equals the original input

Sig (8)

Sign a given plaintext with a given key and signature scheme

Ver (13)

Given a signature, public key, and plaintext: verify with the public key that the signature is valid

SigVer (11)

Given a signature scheme and possibly random plaintext: Check if the verification of the foremost-generated signature is valid

Pad (3)

Check if padding is correctly used e.g. added and removed after use

Constr (73)

Check if the constructor works as required in the assignments. Constructors are the basis for cryptological operations in the assignment. The constructor e.g. sets and checks plaintexts, ciphertexts and key(s)

Excep (52)

Any test case used for testing exceptions like incorrect key length, unpadded plaintext, or missing private key for signature generation

Misc (100)

All remaining test cases not fitting in the above categories. Examples are test cases for the assignment in the Miscellaneous group from Table 1

3.2 Formalization ASE stores the following data per assignment: student ID, name of the test case, time t of upload, and the result r of the test case. Example: r(Rabin, 6137, testEncryption2,

Analyzing Error Rates in Cryptological Programming Assignments

9

2020-10-29T13:21:35) = true. Formalized: r : A × S × T × Z → {true, false} group : A → {SC, TC, CA, MA, SIG, EC MISC} cat : T → {Encr, Decr, . . . , Misc} • A represents the assignment. The group function clusters assignments into groups, e.g. group(Rabin) =”Classical asymmetric cipher” = CA. Table 1 gives the detailed mapping. • S stands for the student ID of the participating students, a random number generated by ASE for each student guaranteeing his anonymity. • T is the set of names of the test cases, typically 10–20 per assignment, and 350 in total. The function cat and Table 2 map the test cases to categories, e.g. cat(“testEncryption2”) = ENC. • Z represents the time of the upload. Students had seven days (Wednesday to Tuesday) for solving the assignment, typically 2–3 per week. They could decide when and how often to upload. The number of uploads varied between 2 and 50. With |A| = 20, |S| = 20, |T| = 350, and the number of uploads ranging from 2–50 there are ~60,000 results in the study. The error rate e measures the percentage of negative test results per assignment, student, and test case, e.g. it divides the number of negative test results by the total number of tests. Example: The results {false, false, false, false, true} for test case testEncryption2 of the Rabin assignment by student 6137 yield an error rate e(Rabin, 6137, testEncryption2) = 0.8. Formalized: e : A × S × T → [0, 1] Note that the number of results varies per test case and that if the student did not turn in solutions for this assignment, the error rate is undefined. By averaging over the sets A, S, and T using the functions cat and group this setup allows analyzing the following aspects related to solving cryptological programming assignments: • Error rate per day and hour over all students and assignments. • Compare error rates within a single assignment in order to identify “easy” and “hard” test cases within this assignment. • Compare error rates between the assignments or groups of assignments (cf. Table 1) in order to identify “easy” or “hard” assignments or groups. • Compare error rates between different test cases and their categories, cf. Table 2. Other aspects like student specific error rates are excluded due to privacy reasons. ASE provides the data in Tab Separated Value format. These files were analyzed with Python. Matplotlib (https://matplotlib.org) generated the figures. The data files, python scripts, a sample assignment, and sample JUnit tests are available for download, s. [19].

10

K. Knorr

4 Results Figure 2 gives the total and average number of uploads per assignment. It does not take the results of the test cases into account. However, the number of uploads can be seen as a measure for the difficulty the students had with the assignments. The Shamir, Keystore, and Rabin assignment had most tries in total and in average. Porta has the smallest numbers. The differences in Fig. 2 a) and b) are due to the fact that the number of students working on each assignment varied. The average error rate over all results is 0.297. A dashed line indicates this benchmark in the following figures. Figure 3 gives the total number of true and false results per day. The two days with most correct tests in total were 2020–10–28 and 2020–10–29 due to the popular asymmetric RSA and Rabin assignments. The graph also shows days with zero uploads (gaps in time line). The corresponding assignments were checked manually, not with the help of ASE. Figure 4 shows the error rate per day. The highest peaks are in from 2020–10–23 to 2020–10–27 due the first attempts to solve the Rabin and RSA assignments. Function e can be used to assess the test cases in the different assignments. The Rabin encryption assignment serves for illustration purposes here, see Fig. 5. The encryption and constructor tests have a lower error rate than 1. decryption, 2. encryption with subsequent decryption tests, and 3. tests addressing the Chinese Remainder Theorem (CRT). This is due to the definition of the Rabin cipher: encryption is a simple modular squaring; decryption requires the more complex calculation of modular square roots with the help of the CRT, see [25].

Fig. 2. a) Total number of uploads per assignment (left) and b) average number of uploads per assignment and student (right)

Figure 6 shows e per assignment. The Padding Oracle assignment [26] has the highest error rate. It is an especially challenging bonus assignment. Solving the challenge only required solving two test cases. The classical symmetric substitution ciphers FourSquare and Porta show low error rate (e < 0.2). The transposition ciphers Railfence and Skytale have average rates (e ≈ 0.3). Playfair was the first assignment in the semester. Most students needed to become accustomed with ASE. The asymmetric encryption assignments RSA, Elgamal, and Rabin have lower error rates than (1) signature related

Analyzing Error Rates in Cryptological Programming Assignments

Fig. 3. Total number of uploads per day

Fig. 4. Error rate per day (only days with uploads > 0)

11

12

K. Knorr

Fig. 5. a) True and false test results (left), b) error rates per test case in the Rabin assignment

Fig. 6. Error rates per assignment

assignments (RabinSig, Lamport) and (2) modern asymmetric assignment like Paillier und NTRU. This could be related to the increasing number-theoretical complexity of the newer schemes. Elliptic curve assignments rank closely below average. The high error rate of the Keystore and BigInteger assignment is mentionable: Java Keystores are used to store cryptological keys and certificates. The difficulty lies in the subtle changes in keystore types and the complexity in accessing these stores. The BigInteger assignment is an introduction to working with large numbers (e.g. 4096 bit modules for RSA) in Java

Analyzing Error Rates in Cryptological Programming Assignments

13

Fig. 7. a) Error rates per test case category (left) and b) per assignment group (right)

including modular exponentiation and inversion. The test cases were all independent of each other. Most students solved them chronologically. Figure 7 a) sums up the results for test case categories. Testing constructors and exceptions has the lowest error rates. Padding, encryption, and decryption rank in midfield. The categories EncDec and SigVer, which combine two cryptological operations and are often based on a dynamically randomly generated input have the highest (e > 0.6) and second highest error rate. Encryption scores lower than decryption, signing has a lower error rate that verifying a signature. Protecting the confidentiality of data via encryption and decryption appears easier than protecting the integrity of data by signing and verifying the signature. Figure 7 b) addresses assignment groups: Classical substitution ciphers (SC) have a lower error rate than classical transposition ciphers (TC). Modern asymmetric schemes (MA) like NTRU are more error prone than classical schemes (CA) like RSA. Signature schemes (SIG) have an above average error rate, classical asymmetric encryption schemes rank below average. Elliptic curves seem accessible to students. Generally, an increase in the complexity of the cryptological schemes regarding the underlying algebraic structures and operation yields a higher error rate.

5 Discussion and Conclusion The paper presents an approach to identify easy and hard tasks when coding cryptological assignments in Java by assessing the error rate in corresponding JUnit test cases. This approach allows identifying difficult tasks in a more focused way than in many other studies like [1, 2, 12], which assess programming problems using more general approaches like questionnaires. The current study is a first step towards making problems in cryptological programming assignments measurable. This allows for improving the education in teaching programming and cryptology classes in the future. Coding cryptology is beneficial for understanding and learning cryptology. Only when the concepts are fully understood, it can be implemented correctly and test cases will pass. Sticking to the test cases also helps to split a large assignment into smaller, more conquerable parts. The test cases have been valuable in showing the students their errors. The error rates are a transparent and fair basis for grading assignments. For instructors the approach represents a major time shift from correcting students’ code ex

14

K. Knorr

post to creating sample solutions, test cases, and detailed assignments prior to classes. This approach is scalable and can therefore deal with larger cohorts e.g. in distance learning scenarios. On the downside, the following issues need to be taken into account: • Chronology of assignments: As students needed to become acquainted with ASE, this can affect the error rates in the first weeks. • Order and dependency of test cases: ASE tests each upload against all test cases of an assignment and always presents the test results in the same predefined order. Students might tend to solve the test cases in this order. There are also interdependencies between test cases: Constructor tests are often a prerequisite for other operations like encryption or signing. The EncDec category requires working implementations of encryption and decryption. The SigVer category requires that the signature can be calculated and correctly verified. Padding is often a prerequisite for encryption. The categories used are not all disjunct, e.g. encryption test cases can also raise an exception. Ambiguous test cases have been mapped to the Misc category. Labels for the test cases instead of a strict categorization could be used. • Solving time: Instead of assessing the error rate, the solving time of the first successful test can be analyzed. This has been done in [15] with no significant difference in the results. • Complexity vs. error rate: The assignments, the test cases, and their underlying cryptology have different complexities. It is e.g. much easier to understand the Skytale than NTRU. Effenberger et al. [27] differentiate between difficulty and complexity of programming assignments. For measuring the difficulty, they recommend the error rate and solving time as done in this study. Measuring the complexity is much harder. The authors recommend the number of line of codes and the type of control flow structure in the reference solution as indicators. • Number of uploads and immediate feedback: ASE allows for an arbitrary number of uploads per assignment and checks each upload against all the test cases for this assignment. Students receive their result for the test cases immediately. This is inherent to the current didactical concept of ASE: students can learn from their errors and improve their code until the deadline. Contrary, this produces a certain bias towards passed tests. • Plagiarism: Many assignments have been very challenging and time consuming for the students. They had to solve the assignments individually. Group work was not allowed. As the grade of the course comprised the final error rate per assignment, some students might have been stimulated to share their solutions and cooperate in solving the assignment. ASE does not automatically check uploads for plagiarism. This needs to be done manually based on the uploaded source code. Besides mitigating the above shortcomings of the study, ASE can be improved in several directions: An automatic feedback for instructors similar to Fig. 5 would help to identify difficult test cases faster. In addition, Python could replace Java as the underlying programming language. Several cryptological primitives like e.g. X.509 certificates are easier to handle in Python due to a more modern architecture for cryptology. The list of ASE assignments can be extended to address more topics, especially those that are

Analyzing Error Rates in Cryptological Programming Assignments

15

currently still manually checked. These are e.g. assignments addressing modern symmetric block ciphers like AES, stream ciphers like ChaCha20, key derivation functions, and hash functions. The cryptological world is currently facing tremendous changes. Quantum computers will break most current asymmetric schemes like RSA and ECDSA. So-called post quantum schemes like Rainbow, NTRU, and McEliece are currently specified and elected as future quantum-resistant schemes [28]. Most of these post quantum schemes make use of different algebraic structures like lattices, multi variate polynomials, and error correcting codes instead of modular rings or elliptic curves. These new schemes should be the content of future studies. The NTRU assignment shows first results in the current study (e ≈ 0.4) in this direction.

References 1. Nadi, S., et al.: Jumping through hoops: why do java developers struggle with cryptography APIs? In: 2016 IEEE/ACM 38th International Conf. on Software Engineering (ICSE), pp. 935–946 (2016) 2. Hazhirpasand, M., et al.: Hurdles for developers in cryptography. In: 37th International Conference on Software Maintenance and Evolution (ICSME), Luxembourg (2021) 3. Long, F., et al.: The CERT Oracle Secure Coding Standard for Java. Addison-Wesley, Boston (2011) 4. Knorr, K.: Learning and grading cryptology via automated test driven software development. In: 13th IFIP WG 11.8 World Conference on Information Security Education (WISE), Maribor, Slovenia, 21–23 September, pp. 3–17 (2022) 5. Desai, C., Janzen, D., Savage, K.: A survey of evidence for test-driven development in academia. ACM SIGCSE Bull. 40(2), 97–101 (2008) 6. Edwards, S., Pérez-Quiñones, M.: Experiences using test-driven development with an automated grader. J. Comput. Sci. Coll. 22(3), 44–50 (2007) 7. Isong, J.: Developing an automated program checker. J. Comput. Small Coll. 16(3), 218–224 (2001) 8. Krusche, S., Seitz, A.: ArTEMiS - an automatic assessment management system for interactive learning. SIGCSE 2018, 21–24 February, Baltimore, MD, USA, pp. 284–289 (2018) 9. Braga, A., Schwab, D., Vannucci, A.: The use of acceptance test-driven development in the construction of cryptographic software. In: 9th International Conference on Emerging Security Information, Systems and Technologies (2015) 10. Edwards, S., et al.: Investigating static analysis errors in student java programs. In: ACM Conference on International Computing Education Research (ICER), Tacoma, pp. 65–73 (2017) 11. Rivers, K., Hardstead, E., Koedinger, K.: Learning curve analysis for programming: which concepts do students struggle with? In: 2016 ACM Conference on International Computing Education Research (ICER), pp. 143–151 (2016) 12. Sivasakthi, M., Rajendran, R.: Learning difficulties of object-oriented programming paradigm using java: students’ perspective. Indian J. Sci. Technol. 4(8), 983–985 (2011) 13. Lahtinen, E., Ala-Mutka, K., Järvinen, H.: A study of the difficulties of novice programmers. ACM SIGCSE Bull. 37(3), 14–18 (2005) 14. Lazar, D., et al.: Why does cryptographic software fail? A case study and open problems. In: 5th Asia-Pacific Workshop on Systems, pp. 1–7 (2014) 15. Knorr, K.: Messung der Schwierigkeit von Programmieraufgaben zur Kryptologie in Java. In: Fünfter Workshop Automatische Bewertung von Programmieraufgaben, S, pp. 35–42 (2021)

16

K. Knorr

16. Herres, B., Oechsle, R., Schuster, D.: Der Grader ASB. In: Herausgeber Oliver, J., et al. (ed). Automatisierte Bewertung in der Programmierausbildung, Waxmann-Verlag, pp. 255–271 (2017) 17. Beck, K.: Test-Driven Development: By Example. Addison Wesley, Boston (2002) 18. JUnit Homepage. https://junit.org. Accessed 29 Jan 2022 19. Knorr, K.: Data files and code of the study: https://seafile.rlp.net/d/a22a20689ca1464abd79/. Accessed 27 Mar 2022 20. Stinson, D., Paterson, M.: Cryptography: Theory and Practice. CRC, Boca Raton (2018) 21. Gardner, M.: Codes, Ciphers and Secret Writing. Dover Publications, New York (1984) 22. Hook, D., Eaves, J.: Java Cryptography: Tools and Techniques, eBook (2022). https://leanpub. com/javacryptotoolsandtech 23. Weiss, J.: Java Cryptography Extensions. Morgan Kaufmann, Burlington (2004) 24. Bouncy Castle Homepage. https://www.bouncycastle.org. Accessed 12 Jan 2022 25. Rabin, M.: Digitalized Signatures and Public-Key Functions as Intractable as Factorization. MIT Laboratory for Computer Science (1979) 26. Vaudenay, S.: Security flaws induced by CBC padding — Applications to SSL, IPSEC, WTLS... In: Knudsen, L.R. (ed.) EUROCRYPT 2002. LNCS, vol. 2332, pp. 534–545. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46035-7_35 ˇ 27. Effenberger, T., Cechák, J., Pelánek, R.: Measuring difficulty of introductory programming tasks. In: 6th ACM Conference on Learning@Scale, pp. 1–4 (2019) 28. NIST Homepage for Post-Quantum Cryptography, https://csrc.nist.gov/Projects/Post-Qua ntum-Cryptography. Accessed 9 Feb 2022

SecTutor: An Intelligent Tutoring System for Secure Programming Ida Ngambeki1 , Matt Bishop2(B) , Jun Dai3 , Phillip Nico4 , Shiven Mian2 , Ong Thao3 , Tran Ngoc Bao Huynh3 , Zed Chance3 , Isslam Alhasan1 , and Motunrola Afolabi1 1

4

Purdue University, West Lafayette, IN, USA {ingambek,ialhasan,mafolabi}@purdue.edu 2 University of California, Davis, CA, USA {mabishop,smian}@ucdavis.edu 3 California State University, Sacramento, CA, USA {jun.dai,ongthao,tranngocbaohuynh,zchance}@csus.edu California Polytechnic State University, San Luis Obispo, CA, USA [email protected]

Abstract. SecTutor is a tutoring system that uses adaptive testing to select instructional modules that allow users to pursue secure programming knowledge at their own pace. This project aims to combat one of the most significant cybersecurity challenges we have today: individuals’ failure to practice defensive, secure, and robust programming. To alleviate this, we introduce SecTutor, an adaptive online tutoring system, to help developers understand the foundational concepts behind secure programming. SecTutor allows learners to pursue knowledge at their own pace and according to their own interests, based on assessments that identify and structure educational modules based on their current level of understanding.

Keywords: Secure programming

1

· Tutoring · Intelligent system

Introduction

Secure programming is one of the most fundamental elements of a software development life-cycle and it’s crucial to develop robust secure coding practices and procedures. According to a recent survey of professional developers, seventy percent of companies emphasized the importance of learning secure programming practices right from the early stages of writing code [6]. This high percentage indicates that secure programming is becoming synonymous with high quality code within the software development life cycle. According to a study conducted by IBM System Science Institute, software defects detected in later phases cost anywhere from six to fifteen times more than if the same defects were found in earlier phases [4]. c IFIP International Federation for Information Processing 2022  Published by Springer Nature Switzerland AG 2022 L. Drevin et al. (Eds.): WISE 2022, IFIP AICT 650, pp. 17–28, 2022. https://doi.org/10.1007/978-3-031-08172-9_2

18

I. Ngambeki et al.

SecTutor is a self-directed learning tool driven primarily by the learner. This learning tool focuses on the educational aspect of teaching students proper practices of improving the security and robustness of programs from the early stages. The fundamental goal of SecTutor is to instill good coding practices into learning and facilitate teaching secure programming in academic institutions. Learners can practice and develop robust programming skills and concepts to determine their current level of knowledge and understanding of secure coding and programming techniques. This paper will first examine the background of self learning, as well as the benefits of teaching secure programming practices to students in computer science. Next, the focus and need of a tool to address these practices, SecTutor, will be described, as well as a comparison of SecTutor to a previously existing security tool. The layout of SecTutor is given in diagram form, as well as an example question that a student may see is shown. Implementation details like how the tool is intelligent and the question approval system are described at a high level. Finally, the paper will conclude with how SecTutor’s data can help find common security misconceptions in education. 1.1

Background

SecTutor is based upon self-directed learning theory, known as the attainment of knowledge partially or entirely driven by the learner. Self-directed learning comprises four paradigms; self-modification, self-motivation, self-monitoring, and self-management [5]. Self-modification allows students to change their outlook on learning and take responsibility for changing their learning behaviours based on feedback. Self-motivation gives a sense of responsibility for learning and improving. Self-monitoring evaluates behaviours in learning and identifies current progress. Self-management controls learning behaviours and allows students to follow up on goals and complete assignments. The goal is to enable a self-directed learning approach to encourage students to overcome reluctance and present guidance in three different strategies; multiple entry points, gamification, and adaptive testing. Self-directed learning allows students to take control of their learning behaviors and provides flexibility in enhancing their skills through new methods of learning to meet their specific learning needs. SecTutor uses the principles of self-directed learning to allow students to learn secure programming practices using effective learning tools. According to the National Research Council (2000), students who focus on the memorization of topics, rather than taking the time to understand and make sense of the topic, often have limited opportunities to learning [2]. SecTutor guides the learning process by developing practice questions and provide feedback to identify the learner’s performance and contribute to the student knowledge model, which will inform the intelligent tutoring system the selection of content and misconceptions that the learner needs to spend more time on. The educational aspect of SecTutor aims to improve elements of proper secure programming practices and prepare learners to apply new skills and concepts. The

SecTutor: An Intelligent Tutoring System for Secure Programming

19

educational assessment deals with measuring the learner’s abilities in robust programming, assist students in learning and offer suggestions on areas of improvement. Currently, there is a gap in secure programming education that seeks to pinpoint knowledge areas to better prepare students the skills of secure coding. A recent study found that many students majoring in computer science, lack necessary fundamental knowledge in their abilities to read and write secure code and graduate without being introduced to any secure programming practices [9]. Furthermore, research has shown that basic yet important secure programming topics are not covered in the required programming courses [1]. One of the strengths of this tool is the ability to target the misconceptions students have regarding secure programming concepts. To build good coding practices, students need to have a solid understanding of how to identify and develop secure software. These primary concepts should be a required practice in all computer science courses. Researchers of the SecTutor tool previously collaborated on a project to develop a secure concept inventory to measure a student’s understanding of concepts in a specific knowledge domain was also developed by the same researchers to assess how well students were learning secure programming [10]. The goal of this project is to use the developed assessments to diagnose misconceptions and structure personalized instruction based on the learner’s current level of understanding in secure programming. This will be accomplished through constructing an adaptive test, constructing the intelligent tutorial system, integrating the learning analytic space and testing the system. The identified areas of misconceptions and foundational knowledge can be seen in Table 1. The three main categories in Table 1 are the overall flow of writing a program, looking at the way programs evolve, the principles to guide the software development, and the artifacts handled during development through execution. The topics covered in Table 1 are targeting at both undergraduate and graduate students. For any class in which there is programming, where security misconceptions may arise, SecTutor would be a great tutoring tool. 1.2

The Focus of the Tool

SecTutor focuses on the educational aspect of teaching robust coding practices from the beginning of writing programs rather than making programs robust after they are written. The key is to inculcate good coding practices into the teaching and practices of programming in institutions where it is taught. Researchers have developed a concept map that allows users to view the primary concepts of secure programming practices. The concept map is an excellent starting point to target specific concepts that will help guide a user’s progress through different learning modules. SecTutor will also provide practice questions clustered around knowledge areas calibrated by different difficulty levels. Based on a user’s selection of questions and performance, SecTutor will guide the user to appropriate content. Lastly, SecTutor uses a psychometric designed test that will assess a user’s understanding of secure programming concepts while providing individualized feedback on performance across specific domains and

20

I. Ngambeki et al.

Table 1. The identified areas of misconceptions and foundational knowledge in secure programming are broken down into three main categories: Principles, development, and execution. Principles

Assurance Complexity/Simplicity Requirements/Design Implementation Programming languages Representation

Development Threat modeling C Strings Crypto algorithms Random number generation algorithms Interdependency Error Handling Compiling Linking Testing/Debugging/Prototyping/Evaluation Tools IDE (Integrated development environments) Execution

Library/API/Third party functions Input Memory Runtime

identifying the areas the user is struggling with. The focus will be achieved in 4 stages. 1. The first stage - Establishing the content domain. During this phase, the primary research questions are: What are the concepts of secure programming and their relationship? What are the critical/foundational concepts in secure programming? 2. The second stage - Developing the item pool. In this phase, the primary research questions are: How do students understand concepts in secure programming? What are common misconceptions in secure programming? What concepts in secure programming do students find difficult? 3. Third stage - Pilot testing and refining items: The primary aim of this stage is to identify which questions from the item pool best target conceptual understanding. 4. The fourth stage - Field testing: Are the scale items valid and reliable across the target populations? The research question at this stage would be seeking

SecTutor: An Intelligent Tutoring System for Secure Programming

21

to know how effective and reliable the Sec Tutor is by testing with a large number of participants. 1.3

Why Create the Tool?

There have been several concept inventories in the past, such as: 1. The force concept inventory developed by David Hestenes [7] and his graduate student between the late 1980 s s and early 1990 s s at the Arizona State University. In the early version of the concept inventory, students were made to write out answers and were not multiple choice questions. Instead, multiple choice wrong answers were built based on common wrong answers, which Hestenes tagged as distractors. 2. Computer science concept inventory for introductory programming developed in 2016 [3]. 3. The CATS hackathon - cybersecurity inventories in 2019 [11]. This tool was created to successfully implement the development of secure programming self-efficacy amongst students in a secure programming clinic. One of the ways to successfully make self-efficacy is from constant practise and exposure, as indicated by results showing a correlation between self-efficacy and increased secure programming knowledge. The objectives of this tool are as listed below [8]. 1. Defining the content domain in secure programming and creating a concept map to describe that domain. 2. Identifying the concepts in the content domain that are foundational/critical. 3. Identifying challenging topics and common misconceptions held by students in secure programming. 4. Developing a pool of items(questions) that specifically target complex concepts and misconceptions in secure programming. 5. Testing and refining the collection of items to establish a draft secure programming concept inventory. 6. Test the scale for validity and reliability. 1.4

What Does This Project Propose?

This project is a development of a dual-purpose testing and tutoring system which will aid students in learning about secure programming at their own pace while in an extra-curricular environment. This will be done with continuous access to secure programming knowledge through an online system called SecTutor. SecTutor uses an assessment-driven approach for individuals to learn about secure programming through a personalized learning system with rigorous assessments to determine a learner’s level of knowledge and skill, used to personalize instructions for the learner. It will create a learning guide for students and give them access to an adaptive learning platform with a concept map that has been defined. The platform will also assist teachers with better analysis

22

I. Ngambeki et al.

and adaptation of teaching techniques by identifying, managing and correcting erroneous beliefs once they manifest. The primary focus of the results from concept inventories is to improve pedagogy while also achieving the below. 1. Helping instructors compare teaching over time. 2. Assisting institutions to rank instructors. 3. Helping other stakeholders make comparisons across institutions. 1.5

The Purpose of the Tool

The design of SecTutor enables it to identify students’ misconceptions through a unique test tailored to each user and designed so that the questions, administration, scoring procedures and interpretations are consistent and in adherence to laid down standards and guidelines. They do not replace examinations or grading of students’ learning; instead, they diagnose areas of programming misconceptions and help students overcome the challenges. Like textbooks, the students are motivated to use SecTutor because it will increase their knowledge about secure programming and make their performance (such as grades) and job skills better. We will promote the tool, and host workshop(s) to scale up the amount of questions. Concept inventories are designed to measure the following. The generated scores indicate how well a student understands a concept, where low scores may be indicative of a misconception. 1. Core concepts of a topic. 2. The extent to which students have achieved expert-level thinking in a domain. 3. A concept map of secure programming which will define the content domain in secure programming and identify the major and minor concepts, while portraying the relationships among these concepts. 4. Concepts ranked based on their criticality and difficulty. 5. Misconceptions in secure programming better understood. 6. Collection of multiple choice questions designed to identify misconceptions. 1.6

Related Tools

A related tool that aims to close the gap on insecure programming is the Assured Software Integrated Development Environment (ASIDE) [13]. ASIDE is a interactive static code analysis plugin built for Java in Eclipse. ASIDE attempts to provide secure programming support to developers during the actual development phase. So, ASIDE will statically analyze code during development and look for common security mistakes, and provide solutions to fix those mistakes. This differs from SecTutor in that it is only used during development, and ASIDE is geared only toward Java insecurities. SecTutor, on the other hand, is a quiz based learning site that is programming language independent, and can be used in conjunction with regular computer science curriculum (similar to using a

SecTutor: An Intelligent Tutoring System for Secure Programming

23

normal tutor, mainly outside of class time to increase areas where students are slipping) to help find and address insecure programming practices before sending students off into industry.

2

Layout

The layout of SecTutor, including how the tool is intelligent, how the users interact with SecTutor, and how SecTutor’s model is implemented follows in this section. SecTutor is implemented as a web app, built using the Python web framework Django. The account model is split into two distinct roles: teachers and students. The general account layout of SecTutor can be seen in Fig. 1. In short, the teacher accounts create questions and view results, and the student accounts take quizzes. An example of a typical question seen in SecTutor follows in Fig. 2. 2.1

How is the Tool Intelligent?

SecTutor uses item response theory [12] to recommend what subject the student should study. By using past quiz scores, an ability level θ is determined for a given interest. This ability level is used in a three-parameter model defined as: P (θ, a, b, c) = c + (1 − c)

exp (θ − b) 1 + exp (a(θ − b))

(1)

where a is item discrimination, which is how well the question can discriminate between students of low ability and students of high ability. b is item difficulty, where students with lower ability will have a harder time answering questions with high difficulty. Finally c is item guessing, which accounts for the student merely guessing the answer. The range of a, b, c is between 0 and 1. Using this value, SecTutor is able to predict a probable score for a student in a given interest. The interest that has the lowest predicted score is the next area of study that SecTutor will recommend for the student. Initial item difficulty and discrimination has been determined by testing a large and diverse population of students. Question difficulty and discrimination will change over time as the system is used. Newly added questions will determine their difficulty and discrimination when the question has been answered by enough students. Question difficulty and discrimination will change over time with more data. 2.2

The Student’s Point of View

When a student account is newly created, the student picks their interests and takes a placement quiz. Interests are the main categories that each question belongs to, see Table 1, and the student will only see questions from their interests. The placement quiz is populated by 2 random questions from each of the student’s interests. A student can always add or remove more interests.

24

I. Ngambeki et al.

Fig. 1. The layout of the SecTutor system, from the point of view of the user interface. As seen here, teacher accounts may add or approve question to the “potential questions” database. A question is added to the pool after enough approvals, to be used in the student’s quizzes. When a students gets a recommendation, item response theory (IRT) is used to determine which area should be studied.

Taking Quizzes. When a student takes a quiz, they first start off by choosing an interest. The quiz is populated with 10 random questions, starting with questions that the student hasn’t taken yet. There is no time limit, but the student may not go back to a previous question. Once the student is finished, a score is shown along with quiz results. In the example question seen in Fig. 2, the last answer is correct. This highlights the fact that if a buffer overflow occurs, both the contents of memory and the control flow may be altered unexpectedly. So, students must understand the

SecTutor: An Intelligent Tutoring System for Secure Programming

25

Fig. 2. The student’s view while taking a quiz.

attack surface of the program or system to ensure security. This question falls under the “Threat Modeling” misconception, seen in Table 1. SecTutor is designed in a way where each wrong answer can have custom feedback to further explain to the student why said answer is wrong. Quiz Subject Recommendation. Each question has both a difficulty and a discrimination value. These values are used in the quiz recommendation functionality of SecTutor. The student’s ability level (called θ in the item response theory model, see Eq. 1) is calculated based on a running average of previous question scores in a given interest. So, SecTutor is able to determine what interests the student needs to study next by recommending the interest with the lowest predicted score. If a student decides to add a new interest, and they haven’t taken any questions that fall in that interest, then SecTutor will immediately recommend that they take that quiz. More Resources. SecTutor can link to external resources for each of the student’s interests. This external resource takes the form of a concept map created during a past funded project called the secure programming concept inventory

26

I. Ngambeki et al.

(SPCI) [10]. This provides more reading material for students to study outside of taking quizzes. 2.3

The Teacher’s Point of View

The questions that the student accounts see in their quizzes are added by teacher accounts. A teacher account can add new questions, approve potential questions, and view question score performance. Viewing Student Performance. Teacher accounts have the ability to see question performance on a per question basis. A low average score can be indicative of a difficult question, or of a common misconception. This helps teachers change parts of their curriculum to address low score areas. This data can be viewed or exported. Question Approval System. Newly created teacher accounts can always add potential questions to the pool. However, until an account has permissions, this newly created account cannot view any other questions. Once the account is granted permissions, they have the ability to view “potential questions”, that being questions that have not yet been approved. If the potential question is approved by 2 separate teacher accounts, then the question will be used in the generation of quizzes for students. A question will not appear in a quiz for a student unless it has these 2 approvals. To help legitimatize a teacher approval, each teacher account has a profile page with stats about their contributions: amount of questions added and amount of questions approved. A teacher may add a short bio to their profile page as well, where teachers are encouraged to add their skills. The bulk of our questions are added by experts among the field of secure programming. Another round of question brainstorming and approvals is scheduled to happen soon, and we’ll be using the SecTutor system to gather and approve these questions.

3

Conclusion

With security being arguably the most important part of software today, SecTutor hopes to understand where students are failing to learn. SecTutor attempts to address the lack of curriculum for common security practices by identifying the weak points. Since the tool is implemented as a web app via the internet, we hope to reach an audience of thousands of students and assess their secure programming knowledge. With a higher volume of students, our tutoring system will result in a more accurate determination of what misconceptions lie in the field of secure programming. This data can be very valuable to instructors in this field, as they can tune their curriculum to match the most common misconceptions.

SecTutor: An Intelligent Tutoring System for Secure Programming

27

With our question approval system, we aren’t limited by the current inventory of questions, and can slowly expand the database of questions. This also allows the ability to test out new concepts on a large group of users. SecTutor can then employ machine learning to understand common behaviors that students have. With more data, SecTutor becomes more calibrated to identify common mistakes employed by students that lead to insecure software. Acknowledgements. This work was supported by grants DGE-1934279 and DGE2011175 from the National Science Foundation to the University of California Davis, grant DGE-1934269 from the National Science Foundation to Purdue University, and grant DGE-1934285 to the California State University Sacramento. The opinions, findings, and conclusions, or recommendations expressed are those of the author(s) and do not necessarily reflect the views of the National Science Foundation, the California State University, Purdue University, and the University of California Davis.

References 1. Almansoori, M., et al.: How secure are our computer systems courses? In: Proceedings of the 2020 ACM Conference on International Computing Education Research, pp. 271–281. ACM, New York (2020). https://doi.org/10.1145/3372782.3406266 2. Bransford, J.D., Brown, A.L., Cocking, R.R. (eds.): How People Learn: Brain, Mind, Experience, and School. National Academy Press, Washington DC, USA, expanded edn. (2000) 3. Caceffo, R., Wolfman, S., Booth, K.S., Azevedo, R.: Developing a computer science concept inventory for introductory programming. In: Proceedings of the 47th ACM Technical Symposium on Computing Science Education, pp. 364–369. ACM, New York (2016). https://doi.org/10.1145/2839509.2844559 4. Dawson, M., Burrell, D.N., Rahim, E., Brewster, S.: Integrating software assurance into the software development life cycle (sdlc). J. Inf. Syst. Technol. Plann. 3(6), 49–53 (2010). https://www.researchgate.net/publication/255965523 Integrating Software Assur-ance into the Software Development Life Cycle SDLC 5. Garrison, D.R.: Self-directed learning: Towards a comprehensive model. Adult Educ. Q. 48(1), 18–33 (1997). https://doi.org/10.1177/074171369704800103 6. Help Net Security: 70% of organizations recognize the importance of secure coding practices, March 2021. https://www.helpnetsecurity.com/2021/03/26/securecoding-practices/ 7. Hestenes, D., Wells, M., Swackhamer, G.: Force concept inventory. Phys. Teach. 30(3), 141–158 (1992). https://doi.org/10.1119/1.2343497 8. Hyder, J.: Electronics systems concept inventory. http://www.esyst.org/PDF/ Concept%20Inventory%20Presentation.pdf 9. Lam, J., Fang, E., Almansoori, M., Chatterjee, R., Soosai Raj, A.G.: Identifying gaps in the secure programming knowledge and skills of students. In: Proceedings of the 53rd ACM Technical Symposium on Computer Science Education, vol. 1, pp. 703–709. ACM, New York (2022). https://doi.org/10.1145/3478431.3499391 10. Ngambeki, I., Nico, P., Dai, J., Bishop, M.: Concept inventories in cybersecurity education: an example from secure programming. In: Proceedings of the IEEE Frontiers in Education Conference (FIE), pp. 1–5 (2018). https://doi.org/10.1109/ FIE.2018.8658474

28

I. Ngambeki et al.

11. Sherman, A.T., et al.: The cats hackathon: creating and refining test items for cybersecurity concept inventories. IEEE Secur. Priv. 17(6), 77–83 (2019). https:// doi.org/10.1109/MSEC.2019.2929812 12. Tay, L., Huang, Q., Vermunt, J.K.: Item response theory with covariates (IRT-C): assessing item recovery and differential item functioning for the three-parameter logistic model. Educ. Psychol. Meas. 76(1), 22–42 (2016). https://doi.org/10.1177/ 0013164415579488 13. Zhu, J., Xie, J., Lipford, H.R., Chu, B.: Supporting secure programming in web applications through interactive static analysis. J. Adv. Res. 5(4), 449–462 (2014). ISSN 2090–1232. https://doi.org/10.1016/j.jare.2013.11.006

Cybersecurity in the Fourth Industrial Revolution: Charting the Way Forward in Education

How to Overcome Staff Shortage in Professionals for SOCs and NSICs Natalia Miloslavskaya(B)

and Alexander Tolstoy

The National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), 31 Kashirskoye shosse, Moscow, Russia {NGMiloslavskaya,AITolstoj}@mephi.ru

Abstract. The need for specialized training of professionals, currently of increased interest to Security Operations Centers (SOCs) and their evolutionary development unit Network Security Intelligence Centers (NSICs), designed to manage network security in the intranets of various organizations, is shown. The labor functions (LFs) of NSIC staff are defined. Following the documents of the Federal Service for Technical and Export Control (FSTEC) of Russia, the roles of the personnel of the centers of the State System for the Detection and Prevention of Computer Attacks (GosSOPKA) are presented. Basic general professional competencies (GPCs) are formulated. As an example, the LFs and PCs are shown for the NSIC Head in detail. The MEPhI’s readiness to implement a Master’s degree programme for training personnel for NSICs is justified, based on the “Information Security and Business Continuity Maintenance” Master’s degree programme being implemented at the MEPhI for the past 10 years. Keywords: Professional · Network security · Network security management · Security Operations Center (SOC) · Network Security Intelligence Center (NSIC) · Academic training · Professional competencies · Knowledge · Skills · Discipline

List of Abbreviations AS CS&Ns FSES FSTEC GLFs GosSOPKA GPCs HE ICT IP IPT IPS IS

Automated System Computer Systems and Networks Federal State Educational Standard Federal Service for Technical and Export Control of Russia Generalized Labor Functions State System for the Detection and Prevention of Computer Attacks General Professional Competencies Higher Education Information and Communication Technology Information Protection Information Protection Tool Information Protection System Information Security

© IFIP International Federation for Information Processing 2022 Published by Springer Nature Switzerland AG 2022 L. Drevin et al. (Eds.): WISE 2022, IFIP AICT 650, pp. 31–45, 2022. https://doi.org/10.1007/978-3-031-08172-9_3

32

ISM ISMS ISMT LFs NSIC PCs PS SIEM SOC TI

N. Miloslavskaya and A. Tolstoy

Information Security Management Information Security Management System Information Security Maintenance Technologies Labor Functions Network Security Intelligence Center Professional Competencies Professional Standard Security Information and Event Management Security Operations Center Threat Intelligence

1 Introduction Currently, there is an annual increase in computer attacks in all spheres. Information security (IS) has become the subject not only of cybercriminals, but also of corporate games, and political and information wars. The key attack trends are overcoming a blurred network organizations’ perimeter, social engineering, attacks through the supply chain, shadow markets for selling access, in-depth research of remote banking systems that includes the identification of vulnerabilities from mobile applications, theft of customer data, bypassing the control of debiting payments, and masquerading as legitimate activities. Among the features of modern cybercriminals, there are ample opportunities to prepare for a targeted attack, high secrecy, targeting the complete seizure of infrastructure, as well as tools for all occasions (0-day vulnerabilities, attack playbooks, and bypassing virtually any information protection tools (IPTs) of any vendor). The consequences of attacks are increasingly associated with the reputation, and not only with financial losses of a hacked organization. Therefore, protection against attacks becomes an integral part of ensuring the security of the management and business processes of any organization. In every second speech at the SOC-Forum-2021, “The practice of countering cyberattacks and the construction of IS monitoring centers”, held annually at the end of the year in Moscow, it was noted that there are not enough specialists who are responsible for ensuring security at enterprises. The shortage of personnel is estimated at 500,000– 1,000,000 specialists. In particular, the Sberbank of Russia, from the point of view of its top management, needs about 20,000 specialists. As presented to the general public in 2005 [1], most of the understaffing is observed in the Security Operations Centers (SOCs) introduced by Cisco. The SOC’s experience allows us to highlight the following trend – on average its staff is completely updated over six years, which sets the task of finding new and developing existing employees [2]. Although outsourcing provides for saving on staff, the ability to attract highly qualified specialists, the advantages of your SOC with your staff and 24/7 monitoring are obvious, namely autonomy, complete control, and multi-vendor solutions used in the SOC. As an evolutionary development of CERT, within the organizations, the SOC becomes a center of competence in IS, where the staff knows exactly what to do in each specific situation and is ready to share knowledge and skills with colleagues.

How to Overcome Staff Shortage in Professionals

33

In addition, experts highlight the problem of rapid staff burnout (a condition resulting from chronic stress at work), which previously occurred at the age of 40, and now occurs at a significantly younger age – as young as 25 years. It is an especial psychological state in healthy people, resulting from intense and emotionally loaded communication during professional activities (psychiatrist H. J. Freudenberg [3]). On 28 May 2019, the World Health Organization (WHO) included occupational burnout in the International Classification of Diseases (ICD 11). The reasons lie in staff shortage and the specifics of the work in SOCs, for example, the results of IS service usage are evaluated only by errors, people experience information overload, their work is characterized by monotony and routine and their activities are carried out in an organic workspace. It is a well-known fact that hackers are more advanced than defenders. The situation is aggravated by the fact that the 20-year-old generation sees successful highly-paid TikTokers who earn more than IS specialists (an appropriate analysis is beyond the scope of this paper). Another significant problem is the lack of professors capable of equipping their students with knowledge and skills in the area of SOCs. This could be solved by the broad involvement of practitioners from the IS industry (and again, an appropriate discussion goes beyond the scope of this page-limited paper). Due to the rapid and often unpredictable development of information and communication technology (ICT) over three to five years, the need for appropriate IS specialists for ICT may likely change. Today it is clear that IS specialists require specialized knowledge and skills in big data analytics and practical usage of artificial intelligence to ensure IS. Modern ICT requires automated detection and forecasting of IS threats, and not only informed decision making and rapid response to these threats. The same applies to SOCs. In 2010, the SOC concept was improved with the introduction of a Network Intelligence Center (SIC) concept. This concept sparked the authors’ research, as a result of which the Network Security Intelligence Center (NSIC) was based on a combination of all the advantages of the SIC and Network Management Center (NOC) [4–6]. NSICs are the next step in SOCs’ evolution. However, this means that more highly qualified specialists are needed at present, and training for NSIC’s developers and users is especially relevant. The study conducted by the authors showed that there is no specialized academic training of professionals for SOCs and the even newer concept of NSICs in any university anywhere in the world. The paper’s contribution to solving this problem and the corresponding goal can be formulated as follows: to work out some initial proposals for developing a Master’s degree programme for training personnel for SOCs and NSICs as one of the key ways to overcome staff shortages for these centers. The rest of the paper is organized as follows. In Sect. 2, related work is briefly analyzed. The labor functions of NSIC staff are defined in Sect. 3. Following the documents of the Federal Service for Technical and Export Control (FSTEC) of Russia, the roles of the personnel of the centers of the State System for the Detection and Prevention of Computer Attacks (GosSOPKA) are presented in Sect. 4. Basic general professional competencies (GPCs) are formulated for the tiers of NSIC personnel in Sect. 5. The MEPhI’s readiness to implement a Master’s degree programme for training NSIC personnel is discussed in Sect. 6.

34

N. Miloslavskaya and A. Tolstoy

2 Related Work Efforts to develop a unified view on the necessary volume of knowledge, skills, and abilities of IS specialists, which would include a necessary and sufficient set of requirements for graduates of higher educational institutions and students of advanced retraining courses of various educational institutions and centers that provide long-term and shortterm training of specialists have been made for many years all over the world. The first attempts to form a unified point of view have been made at the World International Conferences on Information Security Education (WISE) between the late 1990s and early 2000s. Around the same time, certification systems for IS specialists at the initial and more advanced levels of various companies began to appear and were then expanded for new IT. However, to date, unified and globally recognized competencies of IS specialists have not been defined, although the American, Australian, and European training approaches have been formed (they are analyzed in [7] in detail). During that time, several useful standards were adopted, for example, ISO/IEC 27021:2017 formulates competencies for specialists in IS management systems (ISMS) [8] (currently it is being revised for updating). They are described from the point of view of the labor functions (LFs) of an employee with specific knowledge and skills. These competencies are divided into two categories: 12 general competencies in the context of the organization’s activities and 12 in ISM, including IS. In Russia, a complete system of IS personnel training has long been established, based on Federal State Educational Standards (FSESs) of Higher Education (HE) and Professional Standards (PSs). Thus, the FSESs of HE of the last generation from the 10.00.00 “Information Security” Enlarged Group of Specialties and Directions of Training (EGSDT) formulate the general professional competencies (GPCs) of graduates who have been trained within a specific educational programme [9]. These competencies can be linked to the provisions of the currently valid PSs related to the “Specialists in the Field of Information Security” group of occupations (professions) [10]. In 2017, through the joint efforts of a group of professors from the world’s leading universities engaged in educational activities in the field of cybersecurity, under the auspices of the Association for Computing Machinery (ACM), the IEEE Computer Society (IEEE CS), the Special Security Group of the Association of Information Systems (AIS SIGSEC) and the Working Group of the Technical Committee for Information Security Training of the International Federation for Information Processing (IFIP WG 11.8), the document titled “Cybersecurity Curricula 2017” (CSEC 2017) (Curriculum Guide in the field of Cybersecurity) was developed [11]. As the name implies, its main focus is not on competencies, but on the content and basic concepts of academic disciplines that are studied most often within the framework of the field of Cybersecurity. All of the above-mentioned can be used as a basis for the development of professional competencies (PCs) in IS, first for SOC and then for NSIC personnel.

3 Labor Functions of NSIC Personnel From an organizational point of view, a typical NSIC should include personnel consisting of operators working with current data and IS analysts who perform their in-depth analysis and carry out threat hunting, as well as NSIC management.

How to Overcome Staff Shortage in Professionals

35

The NSIC staff depends on many factors, for example, the size and complexity of the organization’s intranet, the number of previously detected IS incidents, the organizational NSIC’s model, the working hours of the staff, the level of automation of its activities, and, of course, the willingness of the organization to allocate an appropriate budget to support its NSIC operations. (The issues related to the size of the NSIC staff are outside this paper’s scope, since the main focus is on the professional training of personnel and their PCs.) The staff of a typical NSIC can be divided into several groups, called tiers, according to their LFs. As the analysis showed, there are usually three or four tiers [12–14]. Tier 1: Briefly: The Tier 1 Operator monitors SIEM system alerts, manages and configures IS monitoring tools, prioritizes alerts or issues, and performs triage to confirm that a real IS incident is taking place. Operators (Alert Investigators) continuously monitor all incoming data and its registration processes, they monitor the queue of hazard warnings generated by the IPTs and other systems, such as SIEM (Security Information and Event Management) systems, and determine the urgency of each alert, as well as perform prompt uncomplicated processing of that part of the data that for some reason cannot be processed in automated mode. These operators also monitor the operability of IS event data sources – network sensors and agents on endpoints. In the general flow of all data collected by heterogeneous sources related to IS events or presumably related to IS by established policies and rules, those data that indicate an IS incident are identified. They rely in their actions on pre-developed response scenarios (playbooks) – a sequence of operational actions when a specific type of IS incident is detected. If it takes more than a few minutes to process an event, they transmit (escalate) it to the second tier – to higher-level operators. Information about IS incidents, for which a high level of criticality is determined, is transmitted promptly - strictly within a set time (for example, 15 min). The Tier 1 operators can manage the IPTs and regularly prepare reports. Since their work requires special attention and quick reaction, they work in shifts. It is important that information about suspicious events should be transmitted not only to the Tier 2 specialists, but also between shifts, since many attacks are developing for hours, days, and sometimes years (as in the case of Advanced Persistent Threat (APT) attacks). The work of Tier 1 operators is easiest to automate. However, there are many specifics for each organization – its business processes, network topology features, etc. Tier 2: Briefly: The Tier 2 Operators remedy attacks intensified from the Tier 1 Operators, receive incidents and perform deep analysis, correlate with Threat Intelligence (TI) to identify the threat actor, nature of the attack and systems or data affected, and decide on a strategy for containment, remediation, and recovery and act on it. These operators (Incident Responders) usually have more experience. They can quickly find the cause of the detected problem, using correlation analysis of data obtained from various sources and assess which part of the organization’s intranet is under attack. They strictly follow the procedures regulating the elimination of this problem and its consequences (their versions of playbooks have been developed for them – a sequence of operational actions when responding to a specific type of IS incident). If it fails and

36

N. Miloslavskaya and A. Tolstoy

the cause of the problem is not detected, they escalate it to Tier 3 for additional investigation. The Tier 2 operators can support the introduction of new analytical methods for detecting IS threats. Tier 3: Briefly: Day-to-day, the Tier 3 Analysts conduct proactive vulnerability assessments and penetration tests, reviews alerts, industry news, TI, and data security, and actively hunt for threats that have found their way into the network, as well as for unknown vulnerabilities and security gaps. When a major incident occurs, they join the Tier 2 operators in responding and containing it. Experts (Subject Matter Experts/Threat Hunters) are top-level network security analysts (they are always highly qualified people with extensive work experience) who can understand a unique situation by applying their knowledge and experience and comparing various events and facts related to an IS incident. Using IS threat detection tools (threat hunting), they analyze in detail all the information available at that moment to investigate the IS incident escalated to them from Tier 2, and actively search for vulnerabilities in the intranet. Investigation of a single incident can take from several minutes to several weeks as it often requires the collection of additional information related to an IS incident and the involvement of technical experts. Analytical experts analyze previously unknown and unknown IS events and new vulnerabilities, relying on various sources and historical data of the organization itself, for which they must have a clear understanding of the external and internal context of the functioning of the intranet. By investigating an IS incident, they can identify its characteristic features and transmit them to the appropriate IS monitoring systems and Tier 1 operators – thus, these experts are actively involved in the development, configuration, and implementation of analytics for detecting IS threats. Their task is to reconstruct the sequence of all the events before an IS incident, as well as to prepare recommendations for eliminating its consequences and improving the overall IS level for the intranet. Their task is to reconstruct the sequence of all the events before an IS incident, as well as to prepare recommendations for eliminating its consequences and improving the overall IS level for the intranet. In addition to IS analysts, the Tier 3 specialists also include IS auditors and specialists who collect digital evidence of IS incidents for their detailed investigation and, if necessary, transfer this information to law enforcement agencies. Tier 4: Briefly: As in the case of a commander of a military unit, Tier 4 is responsible for hiring and training the NSIC staff, is in charge of defensive and offensive strategy, manages resources, priorities, and projects, and manages the team directly when responding to business-critical IS incidents. It acts as a point of contact for the business for IS incidents, compliance, and other security. This tier consists of NSIC Senior Managers and Managers with extensive experience who are responsible for compliance with all applicable requirements, policies, and regulations and for coordinating the work of the entire NSIC. This group oversees all the activities of NSIC staff and performs administrative functions, including the management of strategies and tactics of NSIC operations, recruitment, shift management, budget management, necessary personnel training, as well as assessment of the overall results of NSIC functioning and the contribution of individual employees. The special role of Tier 4 is also evident during crisis situations when it serves as a link between the

How to Overcome Staff Shortage in Professionals

37

NSIC staff and the rest of the organization. In addition, continuous communication with the relevant authorities should be carried out on this tier, if necessary. The briefly described LFs of the staff of a typical NSIC can be clarified, depending on the needs of a particular organization.

4 Staff Roles of the State System for the Detection and Prevention of Computer Attacks With the publication in August 2020 of the draft national standard GOST R, devoted to terms and definitions in the field of detection, prevention and elimination of consequences of computer attacks and response to computer incidents [15] on the website of the Federal Service for Technical and Export Control (FSTEC) of Russia, the following types of employee of the center of the State System for the Detection and Prevention of Computer Attacks (GosSOPKA) (NSIC’s analog) were identified – specialists of three tiers with an indication of the roles they perform: 1. Tier 1 – performing the following roles with the corresponding LFs: • “Specialist in interaction with staff and users” – receiving messages from staff and users of information resources and preparing information for submission to the National Computer Incident Coordination Center (NCICC); • “Specialist in detecting computer attacks and incidents” – analysis of IS events, registration of computer attacks and IS incidents; and • “GosSOPKA Center Maintenance Specialist” – ensuring the functioning of funds placed in the GosSOPKA center, as well as additional IT infrastructure protection tools. 2. Tier 2 – performing the following roles with the corresponding LFs: • “Security assessment specialist “ – inventory of information resources, identification of vulnerabilities, analysis of identified vulnerabilities and IS threats, establishing compliance of measures taken with the IS requirements; • “Specialist in the elimination of the consequences of computer incidents” - coordination of actions to respond to computer incidents and bring IT infrastructure to normal operation; and • “Specialist in determining the causes of computer incidents” - analysis of the causes of computer incidents, analysis of the consequences of incidents, and preparation of a list of computer incidents for submission to the NCICC. 3. Tier 3 – performing the following roles with the corresponding LFs: • “Analyst” – analysis of information provided by specialists of Tier 1 and Tier 2; identification and analysis of IS threats, forecasting their development, development of proposals for the revision of regulatory and methodological documents on IS;

38

N. Miloslavskaya and A. Tolstoy

• “Technical expert” – expert support under the specialization (malware, IPTs configuration, application of specialized technical means, security assessment, etc.), formation of proposals to increase the level of security; development of proposals for the revision of regulatory and methodological documents on IS issues; • “Specialist” – regulatory legal and methodological support of the activities of the GosSOPKA center; and • “Head” – management of all the activities of the GosSOPKA center, interaction with the NCICC, amendments to the relevant regulatory and methodological documents. It should be noted that Tier 4 is not allocated in [15]. From our viewpoint, two roles from Tier 3, namely the “Specialist” and “Head” roles, can be attributed directly to Tier 4 in the case where the 4-tiers model suits an organization. This set of roles with their LFs is taken further as a basis for defining PCs for them.

5 NSIC Personnel Competencies The qualification characteristics of the NSIC personnel as IS specialists can be presented in the form of a set of competencies. A competency refers to a combination of observable and measurable knowledge, skills, and abilities, as well as individual attributes and work experience that contribute to enhanced NSIC personnel performance and ultimately result in organizational success [16]. Knowledge is the cognizance of facts, truths, and principles gained from formal training and/or experience. A skill is a developed proficiency or dexterity in mental operations or physical processes that is often acquired through specialized training, using the skill results in successful performance. An ability is a power or aptitude to perform physical or mental activities that are often affiliated with a particular profession. The ability to apply knowledge and skills in a productive manner, which can be characterized by such behavioral attributes as aptitude, initiative, willingness, communication skills, team participation, leadership, and others, shows the professional’s effectiveness. When training the NSIC personnel of all tiers, it is necessary to proceed from the fact that they must have modern PCs required to solve specific professional tasks and implement specific LFs. The PCs formation refers to the tasks solved by the existing education system, the regulatory framework of which in Russia is the complex of the FSESs of HE. The analysis of the above-mentioned FSESs and PSs made it possible to establish the connection of NSIC personnel LFs objects and subjects with specific educational and professional standards: • FSES of HE in the 10.04.01 “Information Security” training direction (Master’s degree level); • PS 06.032 “Specialist in security of computer systems and networks”; type of professional activity: protection of information in computer systems and networks (CS&Ns); • PS 06.033 “Information security specialist in automated systems”; type of professional activity: ensuring the security of information in the automated systems (ASs).

How to Overcome Staff Shortage in Professionals

39

A specific Master’s degree programme that trains NSIC personnel within the framework of an educational standard should establish the following GPC (the graduate must be able to): • GPC-1 justify the requirements for the ISMS and develop a draft technical specification for its creation; • GPC-2 develop a technical design of a system (subsystem or component of a system) to ensure IS. At the stage of developing a specific Master’s degree programme curriculum and specific academic disciplines’ syllabus to train specialists for NSICs, a competencebased approach should be applied. Based on this approach, we will formulate PCs for employees of all NSIC tiers: 1) Tier 1: required knowledge: fundamentals of information and network security, Windows, UNIX, and Linux operating systems, network protocols and TCP/IP, computer networking, routing and switching, familiarity with SQL, C, C++, C#, Java, or PHP programming languages; required technical skills: alert monitoring, security alert triages, monitoring of security sensors’ and endpoints’ “health”, data collection, intrusion detection, work with packet analysis tools, IDS/IPS, penetration and vulnerability testing, firewalls, anti-virus, and anti-malware) and SIEM systems, etc.; non-technical skills: critical thinking, problem-solving abilities, capability to communicate, initiation of the Tier 2 work, and so on. 2) Tier 2 – competencies of Tier 1 and additionally: required knowledge: forensics fundamentals, in-depth intrusion detection, hackers’ tools and techniques, exploits, TI basics; required technical skills: log files’ analysis, correlation of data from various sources, the basic assessment of malware and exploits, detection of affected systems and data, procedures for responding to incidents and formulating recommendations for their elimination, support for new analytic methods for detecting threats, collection of digital evidence of incidents on the network and hosts, etc.; non-technical skills: critical thinking, problem-solving abilities, capability to communicate, initiation of Tier 3 work and so on. 3) Tier 3 – competencies of Tier 2 and additionally: required knowledge: in-depth knowledge of networks, endpoints, and TI, advanced network forensics, malware reverse engineering, specific applications or underlying IT infrastructure, general IS management issues; required technical skills: “Threat Hunting”, involved in developing, tuning, and implementing threat detection analytics, including anomaly detection, aggregation, and advanced data analysis, Pen Testing, project management, IS management, IS incident management training, etc.; non-technical skills: critical thinking, problem-solving abilities, capability to communicate and to be a contact point for business-critical incidents, personnel management and shift scheduling, resource management, strategic and tactic planning, and so on.

40

N. Miloslavskaya and A. Tolstoy

The competencies listed in ISO/IEC 27021:2017 are fully suitable for Tier 3 personnel [8]. Of course, the training of NSIC personnel should be carried out constantly, for which the advanced retraining courses conducted by various training centers are intended. It is necessary because information, communication, and network technologies are evolving constantly, and new technologies are also emerging, which is reflected in the dynamically changing environment of the intranet and NSIC functioning as an integral part of it. LFs and PCs have been developed for all the roles listed in Sect. 4, based on [17]. As an example, LFs and PCs are given only for the NSIC Head, representing Tier 4. LFs: • Manages the performance of work on information protection (IP) in organizations; • Organizes the categorization of informatization objects, the identification of IS threats and technical channels of information leakage, work on special inspections, special studies, certification of informatization objects, licensing of organizations, the use of certified IPTs; • Organizes the development of organizational and administrative documents in the field of IP; • Participates in the consideration of technical assignments for research and development work on the development of products subject to protection; • Develops proposals for inclusion in the plans and programs of work on IP. Participates in the work on the introduction of new IPTs; • Promotes the dissemination of best practices in the organization and the introduction of modern organizational and technical measures, means, and methods of IP; • Organizes measures to prevent leakage of restricted access information by officials of organizations performing work related to information constituting a state secret and/or containing other restricted access information when using open communication channels; • Monitors compliance with the requirements of regulatory legal acts and other documents on IP; • Organizes work to determine the need for IPTs, their order, receipt, and distribution; • Participates in the selection and placement of specialists; • Organizes work on certification, training, professional retraining, and advanced training of specialists in the field of IS, etc. PCs – knowledge: to know: • The laws of the Russian Federation regulating relations related to the protection of state secrets and other restricted access information; • Regulatory and methodological documents on issues related to IP, planning of activities and control of work on IP; • Methods and means of monitoring the IP effectiveness; • Organizational structure of objects to be protected; • Organization of activities of these objects and functioning of control, communication and automation systems on them;

How to Overcome Staff Shortage in Professionals

41

• Rules for the organization of certification, licensing and certification in the field of IP; • Measures for the protection and control of information, including when using open communication channels; • Methods of identifying IS threats and methods of IP against them; • The procedure for concluding contracts for conducting special studies and special inspections; • Technical capabilities of special equipment of IP units, the basics of its operation, organization of maintenance and repair; • Methods for assessing the capabilities of technical exploration; • The procedure for categorizing objects; • Requirements for registration of inspection certificates, test reports, prescriptions for the right to operate basic and auxiliary equipment and systems; • Methods for assessing the professional level of IS specialists, certification of specialists; • Fundamentals of economics, production organization and personnel management; • Fundamentals of labor legislation; rules on labor protection and fire safety, etc. PCs – skills: • To form and establish the directions of the NSIC’s activities in the field of IP; • To provide recommendations, to define goals and stimulate the successful solution of tasks related to IP at various levels of their execution; • To distribute responsibilities and powers on all NSIC’s tiers; • To analyze the security of ICT systems and develop recommendations for the operation of the IP systems (IPSs); • To formulate information security policies for ICT systems; • To summarize scientific and technical literature, normative and methodological materials in the field of IP; • To form models of the IS threats and the IS intruders for ICT systems; • To identify the most appropriate approaches to the ICT systems; • To develop private IS for ICT systems, including access control and information flow policies; • To apply the current legislative framework in the field of IS; • To develop and to apply methods for assessing the security of software and hardware IPSs; • To be able to evaluate the IP effectiveness; • To be able to analyze the system in order to determine the security and trust levels; • To apply the procedure for categorizing objects; • To apply methods for assessing the professional level of IS specialists and their certification; • To analyze IS policy for the adequacy of established IS requirements and measures; • To compile and to issue an analytical report on the results of the analysis; • To conduct research in order to find the most appropriate practical solutions for IP; • To select and summarize scientific and technical literature, methodological materials on software and hardware and methods of IP, etc.

42

N. Miloslavskaya and A. Tolstoy

6 Mephi’s Readiness to Implement a Master’s Degree Programme for Training Personnel for NSICS The Master’s degree programmes being implemented at the MEPhI have for many years been designed to train IS professionals in the 10.04.01 “Information Security” direction. One of the four Master’s degree programmes of the MEPhI is the “Information Security and Business Continuity Maintenance” programme, which has been successfully implemented for 10 years [18]. It ensures the implementation of the following disciplines designated in the 10.04.01 FSES of HE as the basic disciplines of the professional module: “Information Security Management” and “Information Security Maintenance Technologies” (ISMT), as well as the “Information Security Incident Management”, “Information Security Risk Management”, “Business Continuity Management”, “IT Security Assessment”, “Secure Information Systems”, “Computer Network Security”, “Modern Information Technologies in the Banking Sector” and “Confidential Information Loss Prevention” disciplines. Since 2020, a new “Network Security Management Centers” discipline has been introduced into the Master’s degree programme. The availability of all of the above disciplines proves the full readiness of our programme to meet the increasing need for training highly qualified specialists for SOCs. The following disciplines, such as “Analytical Methods for Big Data Processing” and “Methods for Visualizing of Network Security-Related Information” should be added. Among those listed, we can single out the “ISMT” discipline, the content of which most fully corresponds to the LFs of the personnel of all NSIC tiers. It is an original discipline developed at the MEPhI 20 years ago and is currently taught in all educational institutions of the Russian Federation, where Masters in IS are trained. The educational programme of the Master’s degree programme in accordance with the 10.04.01 FSES establishes a link between the “ISMT” discipline and GPC-1 and GPC-2 by formulating GPC Achievement Indicators (AI) [19, 20]. For the “ISMT” discipline, these indicators will be as follows: • for GPC-1: GPC-1.1.1 to know the basics of domestic and foreign standards in the field of IS; GPC-1.2.1 to be able to design the terms of reference for the ISMS creation; GPC-1.3.1 to have the skills to participate in the ISMS development; • for GPC-2: GPC-2.1.1 to know the methods of conceptual design of ISM technologies; GPC-2.2.1 to be able to choose and justify the use of methods for solving IS problems; GPC-2.3.1 to have the skills to perform work in the development and operation of ISM systems and tools. With reference to the “ISMT” discipline, the above-mentioned GPC can be compared with the generalized LFs (GLFs) and related LFs given in PS 06.032 and PS 06.033. At the same time, only those GLFs and LFs that are directly related to NSIC are highlighted: PS 06.032 “Specialist in security of CS&Ns”: GLF-S: Assessment of the security level of CS&Ns: LF-S/01.7: Conducting control checks of the operability and effectiveness of the applied software and hardware IPTs.

How to Overcome Staff Shortage in Professionals

43

LF-S/02.7: Development of IS requirements and IS policies for CS&Ns. GLF-D: Software and hardware IPTs development for CS&Ns: LF-D/01.8: Development of requirements for software and hardware IPTs for CS&Ns. LF-D/02.8: Design of software and hardware IPTs for CS&Ns. LF-D/03.8: Development and testing of IPTs for CS&Ns. LF-D/04.8: Support for the development of IPTs for CS&Ns. PS 06.033 “Information security specialist in ASs”: LTF-D: Development of AS IS systems: LF-D/01.7: Testing of AS IS systems. LF-D/02.7: Development of design solutions for AS IP. LF-D/03.7: Development of operational documentation for AS IS systems. LF-D/04.7: Development of software and hardware tools for AS IS systems. GLF-E: Formation of requirements for the AS IP: LF-E/01.8: Justification of the need in AS IP. LF-E/02.8: Identification of threats to the security of information processed by the AS. LF-E/03.8: Development of the AS IS system architecture. LTF-E/04.8: Modeling of secure AS in order to analyze their vulnerabilities and the effectiveness of IP tools and methods. For the practical training of NSIC personnel, it is necessary to use the Cyber Polygon being created in many countries in general and universities in particular, the deployment of which at the MEPhI has been delayed due to the transition to distance learning because of the pandemic [21].

7 Conclusion At present and as the next step of their evolution, more and more professionals for SOCs and NSICs are required all over the world. The contribution of this study in overcoming staff shortage was to formulate some important initial proposals for the development of a Master’s degree programme for training highly qualified personnel for SOCs and NSICs with different roles and LFs as the first obvious solution. Another step could be the development of a professional standard for training specialists for SOCs and NSICs. Thus, the obvious conclusion is that the issue of training personnel for intelligent network security management is extremely relevant and requires an immediate solution. The paper presents some considerations that can serve as a starting point for the beginning of this process. It should be noted that the results obtained are more valuable at the international rather than the national level, since SOCs and NSICs are created in companies in many countries, not only in Russia. They are based on the generalization of all the international sources that have already been analyzed in [7]. Acknowledgement. This work was supported by the MEPhI Academic Excellence Project (agreement with the Ministry of Education and Science of the Russian Federation of August 27, 2013, project no. 02.a03.21.0005).

44

N. Miloslavskaya and A. Tolstoy

References 1. Zimmerman, C.: Ten Strategies of a World-Class Cybersecurity Operations Center. MITRE 2014. https://www.mitre.org/sites/default/files/publications/pr-13-1028-mitre-10-strategiescyber-ops-center.pdf. Accessed 4 Apr 2022 2. Yankin, A.: SOC: the staff decides everything. Information Security. ITSEC. 2016. Iss. 1. http://lib.itsec.ru/articles2/Oborandteh/soc-kadry-reshayut-vse. Accessed 4 Apr 2022. (In Russian) 3. Freudenberg, H.J.: Staff burn-out. J. Soc. Issues 30(1), 159–165 (1974). https://doi.org/10. 1111/j.1540-4560.1974.tb00706.x 4. Miloslavskaya, N.: Network security intelligence center as a combination of SIC and NOC. In: Postproceedings of the 9th Annual International Conference on Biologically Inspired Cognitive Architectures, BICA 2018 (Ninth Annual Meeting of the BICA Society). Procedia Computer Science. 2018, vol. 145, pp. 354–358. https://doi.org/10.1016/j.procs.2018.11.084 5. Miloslavskaya, N.: Developing a network security intelligence center. In: Postproceedings of the 9th Annual International Conference on Biologically Inspired Cognitive Architectures, BICA 2018 (Ninth Annual Meeting of the BICA Society). Procedia Computer Science, vol. 145, pp. 359–364 (2018). https://doi.org/10.1016/j.procs.2018.11.085 6. Miloslavskaya, N.: Security zone infrastructure for network security intelligence centers. In: Postproceedings of the 10th Annual International Conference on Biologically Inspired Cognitive Architectures, BICA 2019 (Tenth Annual Meeting of the BICA Society), Samsonovich, A.V., Klimov, V.V. (ed.) Procedia Computer Science, vol. 169, pp. 51–56, 15 April 2020. ISSN 1877–0509. https://doi.org/10.1016/j.procs.2020.02.113 7. Miloslavskaya, N., Tolstoy, A.: State-level views on professional competencies in the field of IoT and cloud information security. In: Proceedings of 2016 4th International Conference on Future Internet of Things and Cloud Workshops. The 3rd International Symposium on Intercloud and IoT (ICI 2016). Vienna (Austria), 22–24 August 2016, pp. 83–90. https://doi. org/10.1109/W-FiCloud.2016.31 8. ISO/IEC 27021:2017 Information technology — Security techniques — Competence requirements for information security management systems professionals. 21 p. 9. Methodological Materials of the XXII Plenum of FUMO VO IB. Drafts of FGOS VO 3++ UGSNP 10.00.00 «Information Security». Krasnodar: Publishing House – South, 2018. 140 p. (In Russian) 10. Scientific, Methodological and Normative Materials of the XX Plenum of FUMO VO IB. Collection of Professional Standards for the Group of Occupations (Professions) “Information security specialists”. Moscow, 2016. 198 p. (In Russian) 11. Cybersecurity Curicula 2017. Curriculum Guidelines for Post-Secondary Degree Programs in Cybersecurity. A Report in the Computing Curricula Series Joint Task Force on Cybersecurity Education. https://www.acm.org/binaries/content/assets/education/curricula-recommend ations/csec2017.pdf. Accessed 4 Apr 2022 12. Orion, C.: Security Operations Center Roles and Responsibilities. 2019. https://www.exa beam.com/security-operations-center/security-operations-center-roles-and-responsibilities/. Accessed 4 Apr 2022 13. Role and Responsibilities of a SOC Analyst. 2021. https://www.infosectrain.com/blog/roleand-responsibilities-of-a-soc-analyst/. Accessed 4 Apr 2022 14. Balaji, N.: How to Build and Run a Security Operations Center 2021. https://gbhackers.com/ how-to-build-and-run-a-security-operations-center/. Accessed 4 Apr 2022 15. GOST R «Data Protection. Detection, Prevention and Elimination of the Consequences of Computer Attacks and Response to Computer Incidents. Terms and Definitions». Draft. https://fstec.ru/en/component/attachments/download/2770. Accessed 4 Apr 2022. (In Russian)

How to Overcome Staff Shortage in Professionals

45

16. Budzko, V., Miloslavskaya, N., Tolstoy, A.: Forming the abilities of designing information security maintenance systems in the implementation of educational programmes in information security. In: Drevin, L., Theocharidou, M. (eds.) Information Security Education – Towards a Cybersecure Society. WISE 2018. IFIP Advances in Information and Communication Technology, vol. 531, pp. 108–120. Springer, Cham (2018). https://doi.org/10.1007/ 978-3-319-99734-6_9 17. Unified Qualification Reference Book of Positions of Managers, Specialists and Employees. The “Qualification Characteristics of the Positions of Managers and Specialists in Ensuring Information Security in Key Information Infrastructure Systems, Countering Technical Intelligence and Technical Protection of Information” Section. https://docs.cntd.ru/document/902 156801#6500IL. Accessed 4 Apr 2022. (In Russian) 18. Miloslavskaya, N., Senatorov, M., Tolstoy, A., Zapechnikov, S.: “Business continuity and information security maintenance” masters’ training program. IFIP Advances in Information and Communication Technology. Dodge, R.C., Futcher, L. (eds.): Information Assurance and Security Education and Training - 8th IFIP WG 11.8 World Conference on Information Security Education, WISE 8, Auckland, New Zealand, 8–10 July 2013, Proceedings, WISE 7, Lucerne Switzerland, 9–10 June 2011, and WISE 6, Bento Gonçalves, RS, Brazil, 27–31 July 2009, Revised Selected Papers. vol. 406, pp. 95–102. Springer, Berlin (2013). https:// doi.org/10.1007/978-3-642-39377-8_10 19. Methodological Materials of the XXIII Plenum of FUMO VO IB. Part 1. Drafts of Exemplary Basic Educational Programmes for Bachelor’s and Master’s Degrees (according to the FGOS VO 3 ++) USSNP 10.00.00 “Information Security”. Stavropol, 2019. 52 p. (In Russian) 20. Training of Masters in the 10.04.01 Direction: Working Curricula and Competence Models. http://www.mephi.ru. Accessed 4 Apr 2022. (In Russian) 21. Miloslavskaya, N., Tolstoy, A.: Cyber polygon site project in the framework of the MEPhI network security intelligence center. In: Proceedings of the Annual International Conference on Brain-Inspired Cognitive Architectures for Artificial Intelligence (BICA*AI 2020), Natal, Brazil, 15 November, vol. 1310, pp. 295–308. Springer, Cham (2021). https://doi.org/10. 1007/978-3-030-65596-9_36

Collaborative Cybersecurity Learning: Establishing Educator and Learner Expectations and Requirements Steven Furnell1(B) , Gregor Langner2 , Teemu Tokola3 , Jerry Andriessen4 Gerald Quirchmayr5 , and Carmela Luciano6

,

1 University of Nottingham, Nottingham, UK

[email protected]

2 Austrian Institute of Technology, Vienna, Austria 3 University of Oulu, Oulu, Finland 4 Wise & Munro, The Hague, The Netherlands 5 University, of Vienna, Vienna, Austria 6 University of Salerno, Fisciano, Italy

Abstract. Effective provision of cybersecurity requires practitioners to work collaboratively to solve practical real-world problems. However, the extent to which these skills are supported by current higher education programmes is potentially limited. This paper presents an investigation into the needs of related learners and the educators who support them, examining the provisions within current cybersecurity education at degree level, and the extent to which they go beyond traditional knowledge transmission approaches. The findings illustrate a broad appreciation of the interdisciplinary nature of cybersecurity and recognition of the value of using collaborative learning approaches. At the same time, however, these aspects are not represented strongly enough within the current provision. Keywords: Cybersecurity education · Cybersecurity Awareness · Collaborative learning · Educators · Learners

1 Introduction Traditional forms of cybersecurity education are often focused on knowledge transmission [1]. This means that knowledge is taken as something tangible that can be transferred from an expert (i.e., the teacher) to a beginner. When practiced well, the learner may acquire such knowledge, but not the resilience to apply it in various contexts [2]. This is especially troubling in cybersecurity, given the dynamic nature of the environment and the increasing need for collaboration, sometimes even required by legislation (cf. collaboration obligations laid out in the European NIS Directive [3]). We consequently need forms of education that understand the interdisciplinary nature of the field, as well as at the development of joint action in context: being able to act in a variety of situations and knowing how to do this together. © IFIP International Federation for Information Processing 2022 Published by Springer Nature Switzerland AG 2022 L. Drevin et al. (Eds.): WISE 2022, IFIP AICT 650, pp. 46–59, 2022. https://doi.org/10.1007/978-3-031-08172-9_4

Collaborative Cybersecurity Learning

47

This paper is based on work from the Collaborative Cybersecurity Awareness Learning (COLTRANE) project funded under the European Union ERASMUS + programme [4]. The premise of the project is that cybersecurity is recognised as needing more practically relevant and applicable learning outcomes than merely the acquisition of related knowledge. This is well illustrated by findings from the profession, with a recent study from the Chartered Institute of Information Security indicating that analytical and problem-solving skills are considered to be the most important skills for people entering the profession (selected by 53% of respondents, ahead of technical subject matter skills, communication skills, and management skills, which were considered to be the primary requirement by 24%, 22% and 1% of respondents respectively) [5]. The profession, therefore, needs support from more innovative forms of education that focus on the development of joint actions. Cyber practitioners must be able to act in a variety of situations and be able do so in collaboration with others. The COLTRANE consortium consists of six partners across five countries (Austria, Finland, Italy, The Netherlands, and the United Kingdom). The project aims to contribute by using cyber-ranges and collaborative learning platforms to create hands-on activities, as well as opportunities for collaborative reflection. It will ultimately support the co-design of learning activities, with a toolkit for teachers and evaluation of learning activities by students. However, the starting point for the work was to determine the views of related educators and learners, in order to establish where COLTRANE’s intended approaches could offer opportunities for improvement. To this end, the paper presents the findings of related data collection from sample participants in the partner countries, leading to the analysis and resulting recommendations that will inform the COLTRANE work as it moves forward. The findings presented build directly on data collection activities that included a session at a previous WISE event [6].

2 Approach While education relating to cybersecurity can be identified at several levels, the target groups of interest for COLTRANE are educators and learners working and studying at higher education institutions within Europe. A necessary foundation for later work, it was appropriate to investigate how these groups currently understand and experience cyber security education. For the purposes of this study, the target groups are more specifically defined as follows: • Educators: Representatives from selected higher education institutions in Europe. They are involved in teaching delivery to students, with a focus on cybersecurity in the field of computer science. They contribute actively to the education at their institution through lectures, exercises, and combinations of both. • Learners: Both graduate and undergraduate students, coming from the field of computer science as well as related disciplines. Additionally, the choice of target institutions was focused on those with specific cybersecurity degree programmes, or those where cybersecurity was significantly represented within the teaching in wider degrees (e.g., related units/modules in a more general computing or computer science or information systems programme). This paper presents

48

S. Furnell et al.

the details of the resulting investigation and is based upon two rounds of data collection, which were undertaken across the COLTRANE partner countries. The first was a questionnaire-based survey activity, which was intended to establish a broad understanding of how cybersecurity education is currently delivered and experienced by the participants involved, as well as to explore the perceived relevance and appetite for the collaborative learning approaches that COLTRANE seeks to pursue. The results from this phase were then supplemented by a series of discussion-based workshops, which sought to enable further exploration, drawing upon themes emerging from the survey findings, as well as allowing the stakeholders to introduce further issues from their own experiences.

3 Surveying Cybersecurity Educators and Learners The work began by identifying several broad issues that it would be relevant to investigate with the educator and learner communities, including: • Topics encompassed within, and related to, cybersecurity (i.e. what is cybersecurity perceived to be and how is it related to different disciplines?) • Topic coverage and delivery techniques in current deliveries (i.e. what do institutions currently cover and how do they do so?). • Perceived sufficiency of use of / exposure to different learning methods (i.e. are the current approaches to delivering cybersecurity education based upon appropriate techniques and meeting expectations?) Two questionnaires relating to the above themes were prepared, with questions framed to address educators and learners as distinct audiences (i.e., questions to educators were presented in terms of how they provide cybersecurity education, while those addressing learners were framed in terms of how they had experienced it in their studies to date). In responding to the survey, educators were expected to answer from the perspective of their institution’s overall provision rather than only their individual contribution. Learners were asked to reflect upon their expectations and experiences to date, recognising that students early in their studies would be responding more from the former perspective, whereas graduates would be able to reflect on a wider experience. The questionnaires were promoted directly via the COLTRANE partners’ research networks and contacts. For example, in the UK promotion was supported by the Chartered Institute of Information Security (CIISec) and the National Cyber Security Centre (NCSC), with both sent out via their academic partner networks. In total this opportunistic sampling yielded responses from 62 educators and 322 learners (65% undergraduates, 32% Masters, and 3% at doctoral level). While these numbers were judged to be sufficient to get a broad sense of overall opinions and experiences, they did not enable any meaningful analysis of country-based sub-samples, or indeed to cut the data according to other parameters (e.g., level of study). In both groups, the sample populations proved to be 75% male. While this is not a balanced sample, it can be argued to be a broadly representative one, insofar as it reflects the recognised problem of gender-imbalance in the digital sector in general and the cyber sector specifically. For example, looking at wider

Collaborative Cybersecurity Learning

49

findings from the UK, it is reported that females account for 28% of the workforce in the digital sector and 16% in the cyber sector [7]. Also, in Europe it is known that the vast majority of persons employed as ICT specialists are men [8] and the situation is similar for the cybersecurity sector where women represent fewer than 20% of cybersecurity professionals in Europe [9]. Looking at the experience of the respondent groups in the surveys, the significant majority (74%) of educators had over five years of experience in delivering cybersecurity, and only 11% had less than two years. Meanwhile, the learners were at varying stages of study, but more than three quarters were beyond their first year (the stage of study is likely to have some bearing on their ability to offer fully informed opinions about the extent and style of cybersecurity coverage in their programmes, as some will have had only partial exposure in practice). While the length constraints of the current paper preclude a full discussion of all findings, some key points are relevant to highlight. The main body of the questionnaires for both audiences began by asking them to rate a series of topics in terms of their perceived relevance to cybersecurity. Figure 1 depicts the result of this activity from the perspective of the learners, and these suggest that while people (unsurprisingly) make a primary association between cybersecurity and computer science, clear links are rightly perceived with a range of wider topics (and even the topics that would be less obviously associated, such as drama and history, can be related if considered from a suitable perspective – e.g. the use of drama to support the delivery of education and awareness, or the historical context for the evolution of security incidents and safeguards). The findings from the educator audience were broadly similar, and the only area of significant difference was that a much higher overall proportion (80%) considered Psychology to be highly or often relevant. Having established the interdisciplinary influences that may feed into the topic, another significant finding was the appreciation of sub-topics that fall directly within the cybersecurity domain. The nature of this coverage was assessed using the categories and Knowledge Areas (KAs) from the Cyber Security Body of Knowledge (CyBOK) [10] as a frame of reference, with educators being asked which areas their courses covered and learners being asked to indicate which topics were important to them1 . The results in Fig. 2 clearly show that some topics are more likely to be receiving coverage than others. Meanwhile, their perceived importance to learners appears broadly similar, with no topics standing out as significantly more critical than others and only Formal Methods falling below a quarter of learners viewing it as important. Having established a sense of what is delivered, the survey was also able to gather some insights into perceptions of how the delivery is achieved, with respondents being asked about the perspectives covered and the methods used to do so. The related findings are summarized in Fig. 3, and it is interesting to note that the use of traditional lectures is the only aspect where both audiences generally believe that provision is appropriate. In various other cases, we can see that significant proportions of learners were inclined to feel that they were receiving too little exposure to the aspects in question. Indeed, there are some aspects (e.g., collaborative learning and interdisciplinary activities) for which 1 Note that CyBOK’s Applied Cryptography KA is not listed as it had not been released at the

time of the study.

50

S. Furnell et al. Business and management Computer science Drama Economics Education Electronic Engineering History Law and criminology Mathematics Physics Psychology Sociology 0%

10%

Highly relevant

20%

30%

40%

Often relevant

50%

60%

Rarely relevant

70%

80%

90% 100%

Not relevant

Human, Organisational and Regulatory Aspects

Fig. 1. The extent to which learners considered different discipline areas to be relevant in contributing to the topic of cybersecurity. 33%

Risk Management & Governance

82%

28%

Law & Regulation

71%

30%

Human Factors

66% 34%

Privacy & Online Rights

Attacks and Defences

63% 38%

Security Operations & Incident Management

Systems Security

73%

35%

69%

39%

Cryptography

79%

27%

Operating Systems and Virtualisations

66% 34%

Distributed Systems Security 20%

Formal Methods

53%

34% 38%

Authentication, Authorisation & Accountability Software and Platform Security

82%

30%

Adversarial Behaviours

Forensics

Infrastructure Security

74%

39%

Malware & Attack Technologies

81%

40%

Software Security

76%

42%

Web & Mobile Security 30%

Secure Software Lifecycle Network Security

37%

Hardware Security

35%39% 30%

Cyber-Physical Systems

76% 53% 87%

40%

25% 29%

Physical Layer and Telecommunications 0%

10%

20%

Important

30%

40%

50%

60%

70%

80%

90%

100%

Covered

Fig. 2. Comparing the extent to which educators indicated that CyBOK knowledge areas were covered in their degrees to and the extent to which learners felt the topic was important to them.

only around half the audience believes the current provision is appropriate. Meanwhile, learners and educators notably diverge in their perceptions about whether the coverage of technical topics is appropriate but have a more similar view in terms of the (in)sufficient

Collaborative Cybersecurity Learning

51

Real-world Interdisciplinary Non-technical perspectives / cyber security Technical cyber examples/case Problem-based learning activities topics security topics studies

Collaborative activities

Practical/handson labs

Lectures

coverage of non-technical content. Given that there are many instances where the educators and learner views agree about shortcomings, the observations collectively suggest that there is some potential to reconsider and rebalance the provision.

Learners Educators

17%

71%

12%

11%

82%

6%

Learners Educators

16%

56%

29%

Educators

2%

61%

36%

Educators

2%

67%

31%

Learners

2%

68%

31%

Learners

7%

75%

18%

Learners

10%

20%

30%

Too Little

2%

39%

59% 0%

7%

50%

43%

Educators

2%

59%

39%

Learners

12%

46%

42%

Educators

4%

46%

49%

Educators

4%

46%

50%

Educators

2%

51%

48%

Learners

2%

60%

39%

Learners

5%

51%

43%

40%

About Right

50%

60%

70%

80%

90%

100%

Too Much

Fig. 3. Comparison of educator and learner perceptions about the extent of use of different delivery methods and the coverage of different topic perspectives.

Recognizing the interest of educators in collaborative activities, we also investigated how collaboration is reflected in their practices. This revealed 31% claiming to make extensive use, 50% making some use, 16% not making use but able to see the potential, and only 3% indicating that they did not feel it worked for their topic. Thus, the overall picture clearly suggests an openness to the use of collaborative learning approaches, even though only a minority are already making extensive use of it.

4 Workshop-Based Exploration Having used the survey to gain a broad appreciation, the investigation then explored some of the key issues in more detail via a series of related workshop sessions. As with the

52

S. Furnell et al.

questionnaires, these targeted both educators and learners, seeking further insights into the perceived effectiveness of the current provision and the potential for enhancement via collaborative awareness. It was expected that participants would come from different backgrounds (e.g., learners could be studying at different levels, and some would be from dedicated cybersecurity degrees, whereas others would be taking related modules within wider programmes). A series of resulting seven workshops (each averaging around 90 min and involving 6–7 participants, plus convenors) was staged between May and August 2021. Of these, three were specifically with educators (in Austria, Finland and the UK), two with learners (Finland and UK), and two were mixed sessions (held at the 14th World Conference on Information Security Education – WISE, and 16th International Conference on Availability, Reliability and Security - ARES). As with the surveys, this gave a broad range of backgrounds in terms of their specific experiences but offered a suitably comparable group for the level of discussion being sought. Specifically, following an outline of the COLTRANE project the sessions addressed four main themes of discussion, as follows: 1. Interpreting cybersecurity - exploring participants’ interpretation of what cybersecurity is in order to establish their starting point in the later discussion. 2. Delivering cybersecurity education - developing an understanding of what the participants currently do in their cybersecurity education delivery. 3. Reflecting on existing approaches - reflective discussion, comparing the described delivery activities to employer needs and identifying the breadth of approaches. 4. Looking ahead – considering what could be changed or done better, and identifying opportunities for COLTRANE to contribute. The workshop convenors guided the discussion, with the focus tending to be placed on the discussions and reflections relating to current approaches. The sub-sections below summarise the key points arising. The discussion does not seek to distinctly call out findings from educators or learners, as in many cases, the general nature of their comments was in broad alignment. However, where specific views or perspectives came from a particular audience, this is denoted in the text. 4.1 Interpreting Cybersecurity The aim here was to establish a foundation for the later discussions by exploring the participants’ interpretation of what cybersecurity is and what it includes. The aim was to get a sense of whether it was perceived to be a technical, computing-led topic or if there was already an appreciation of it having a broader basis and a relationship with other disciplines. This aspect was allocated a relatively small proportion of the discussion, as it was mainly intended to get the participants ‘warmed up’ and comfortable to start interacting. Nonetheless, there were some relevant learning points from how the discussions tended to unfold and the perceptions that were revealed. For example, the discussions generally tended to begin with people attempting to offer a short definition of the term rather than get under the surface of what it encompasses. Consideration of technical topics seemed to emerge more naturally, and the following UK response reflects the sort of points that were being made: “Strong foundations in computer science, deeply

Collaborative Cybersecurity Learning

53

technical discipline overall”. At the same time, there was recognition of the wider, nontechnical perspective and reference to the protection of people and the notion of risk were also forthcoming without convenors needing to steer the participants in those directions. The interdisciplinary aspect emerged as an essential consideration and was particularly recognised by those learners coming into the topic from other subject areas (and therefore did not have computer science as their primary background). Overall, while the breadth of cybersecurity did seem to be generally recognised, the way in which the acknowledgement of non-technical/non-technology aspects emerged in the discussions perhaps suggests that they are regarded as ‘secondary’ elements (this broadly echoes a survey of cybersecurity programmes reported in [11], which found that topics lacking sufficient coverage were typically in organisational, human, social, operating and maintenance subjects). It is worth considering how related awareness and educational materials are constructed because the more desirable position is for the breadth of topics to be considered equally, and to recognise that non-technical aspects can be just as crucial in practice. 4.2 Delivering Cybersecurity Education Having discussed what the participants considered cybersecurity to be, the focus then moved to develop an understanding of what they currently do in terms of their cybersecurity education delivery (concerning the educators) or how it is currently experienced (from the learners’ perspective). To some degree, the comments here continued to reflect the technical bias that had been observed in the initial discussion. For example, feedback from one of the Finnish educators commented upon the “lack of cyber content outside the technical domain”, while many comments from Finnish learners tended to focus on the technical aspects explicitly: – “Many technical aspects covered with hands-on focus” – “Technical concepts well covered and requires deep knowledge” – “Possibility to go deeper in focused topics, e.g., forensics or individual tool usage” Looking at the current practicalities of delivery, educators suggested several challenges to further extending the provision: • There is a lack of academics confident in cybersecurity education, limiting the ability to deliver it. Additionally, while teachers might appreciate the importance of collaboration, they may not know how to conveniently incorporate it in their delivery of courses in their specialty area. This aligns with the challenge-related findings from the educators in the survey, and the workshop participants suggested the need for some further ‘train the trainer’ provision for academics. • There is sometimes a lack of teaching resources, particularly in relation to hands-on activities. • Although it is recognised that further integration is needed (both in terms of integrating additional topics into cybersecurity degrees and integrating cybersecurity coverage

54

S. Furnell et al.

into other, non-specialist programmes), a practical barrier is often that the curriculum/programme is already full and does not have space for more content. As such, new content can only come at the expense of something else. • When looking at provisions outside of computer science, it was felt that academics running the degree programmes may be resistant to cybersecurity being added into their courses, and there would be concerns about the pitching of the delivery (e.g. technical level) not matching the needs of the non-computing audience. Another challenge is the lack of practice-oriented teaching resources, especially for hands-on activities, and while the value of providing exercise-focused activities is recognised, the time required to develop them was observed as a significant obstacle (e.g., “Creating scenarios sounds good, but the maintenance of such clusters is too time-consuming” – Austrian educator). The challenge of time limitation was also a factor raised in the ARES workshop, but from the perspective of what it is feasible to fit within the formal, educator-led delivery (“Challenging to offer sufficient exposure to topics in the limits of the formal hours available”). This should not be overlooked as an underlying factor, and ultimately if the content of a programme needs to be changed to incorporate something new, it is often going to be at the expense of something else. The learner experience tended to vary quite significantly depending upon the degree they were studying. This included a distinction between undergraduates and postgraduates, as well as between programmes where cybersecurity was the primary focus versus those in which it was a topic of study amongst a wider set (e.g., for students encountering it within general computer science degrees). The learners tended to refer to practical activities and hands-on labs, further illustrating the value that is clearly placed on such aspects. This implicitly acknowledges a need to ensure that programmes are fostering learners’ cybersecurity skills rather than increasing their related knowledge. The educators were almost exclusively from computing-related backgrounds and departments, and so their experience of delivering the topics primarily related to having computing students as their audience. While some had the occasional experience of delivering to students outside the topic area, this tended to prompt observations around the difficulty of delivery to those who lack the ‘necessary’ background (noting that the necessity in this case refers to the educators’ interpretation, and the knowledge prerequisites therefore assumed in their materials). There was special recognition from learners that awareness and education around cybersecurity should extend further than currently occurs. To quote from the Finnish learner workshop: “Technical institutions could organise different content for a wider audience, but should be separate from technical content”; “The wider audience should know of the existing risks”. 4.3 Reflecting on Existing Approaches Having collected the views of the participants, the next stage was a reflective discussion, considering the breadth of delivery approaches and the likely alignment to employer needs. This linked into considering whether existing delivery supported collaborative cybersecurity awareness, in the manner envisaged by COLTRANE.

Collaborative Cybersecurity Learning

55

With technical skills having been prominently discussed in the discussion of existing delivery, it was notable that this follow-on discussion saw some explicit acknowledgment of the need for broader aspects: – “Theoretical knowledge also needed to keep technical content learning effective” (Finnish learner) – “Reporting, communication and documentation not covered” (Finnish learner) – “Soft skills are taught at work, not at university, which is later than it would be necessary to teach them” (ARES participant) While soft skills were acknowledged as necessary, it was not considered appropriate to try to ‘teach’ them in isolation (to quote one UK educator, “Teach and practice soft skills within assessments”). Additionally, an ARES participant observed that specific delivery structures do not lend themselves to the delivery of broader skills (“where there are short, focused courses on particular topics (e.g., OT security) they don’t cover the wider angle”). Perhaps unsurprisingly, it was recognised that a variety of delivery and learning approaches help students engage more. Although it was not directly stated, a point that was implied by some of the comments (“Students like the skills that help them to find a job in the future”) is that it is often necessary to strike a balance between what learners want and expect, versus what educators believe they need. A relevant comment from an educator in the ARES session was that they had given specific attention to career development (e.g., career paths, salaries, etc.) which was of interest to the students and would perhaps not normally have been considered as part of their expected delivery. The attitude of learners was also significant in informing what educators felt it relevant and valuable to provide for them. The following comment illustrates the point, while also highlighting a positive approach to tackling the issue: “Our students often see modules as being just about ‘getting credit’ - using ‘real world examples/case studies’, as mentioned, helps - but have also found that utilising cybersecurity industry professionals - in particular alumni - to chat to the students about what they do and the importance of their job (as related to the module), shows the students the relevance of what they’re learning”. 4.4 Looking Ahead The final segment of each workshop session focused on what could be changed or done better in view of the earlier discussion. While the not strictly limited to the needs of the project, this was intended as a means of identifying opportunities for COLTRANE to contribute and the context of collaborative awareness-raising. Prompted by the experience of moving to online, remote delivery during the COVID19 pandemic, the flexibility of delivery was an important aspect moving forward. There was also a general recognition of the need to keep content relevant and updated – e.g. “We need to review security in new technologies and add new modules to our curriculums” (UK educator). One notable comment, offered by an educator in the WISE session, suggested “Having a lot of people do a small bit perfectly instead of having everyone doing a

56

S. Furnell et al.

lot (and the same thing)”. This aligns strongly with the notion of collaborative provision and COLTRANE’s plans for providing a repository. For example, it makes perfect sense for case studies that one teacher researches and prepares the materials, and then makes the resulting slides, assignments etc. available to others (in exchange of other cases provided by them). Indeed, this would also go some way to addressing the resourcing challenges associated with creating activities that were highlighted in Sect. 4.2. Sharing work and national collaboration in providing cyber education were both factors that received support. Educators in the WISE session mentioned using materials from external sources, such as OWASP and from GitHub. Meanwhile, the benefit of collaboration was also recognised in the learner community, with one of the participants in the Finnish session suggesting the “Possibility for inter-institutional co-operation”. At the same time, other comments were clear that this is not always straightforward. For example, when considering reusing teaching materials, one of the Austrian educators observed, “It is more difficult to use other people’s slides than your own”. Interdisciplinarity was an issue that garnered support, with the following quotes, all taken from the Austrian session, illustrating aspects of this recognition: – “Interdisciplinary [is] very important” – “No business process is possible without cybersecurity” – “Awareness must also be created among educators!” Having said this, enhancing the provision of interdisciplinary perspectives would require attention to the design of appropriate case studies in which different disciplines could contribute, and where related learners could participate: “Advice and case studies on delivery across multidisciplinary programmes” (UK educator). The issue of learner-centred provision was also emphasized in the discussion by Austrian educators, with the term ‘edYOUcation’ being used to reflect the need to address learners from their own backgrounds and standpoints. Other workshops also reflected aspects of this, and in particular the need for extending provision to learners outside the computing area was also flagged, with examples such as the following being offered as thoughts for future directions: – “Cyber Hygiene courses for varying audiences outside the technical domain” (Finnish educator); – “a more introductory level course with medium level technical skill expectations would be useful” (Finnish learner). At the same time, broader delivery already happens in some cases. As an example, an educator from the ARES workshop described having had the opportunity to present cybersecurity content to students in the law department. Meanwhile, educators in the Austrian workshop commented on the potential challenge (“how to bring cybersecurity to people without much knowledge of cybersecurity?”), while also highlighting where collaborative approaches could help (“shared repository of cases for educators and learners to learn basics and lower entry threshold”).

Collaborative Cybersecurity Learning

57

5 Analysis and Discussion While varying in the specific levels of participation in the different activities across the partner countries, the overall data collection activity represents a significant body of work when considering both the survey and workshop findings. Moreover, there was a notable level of consistency in some of the findings and messages emerging from the educator and learner communities involved, which provide a useful basis to feed-forward and inform further COLTRANE activities. The overall findings determined that while cybersecurity education is now a recognised and established topic within both undergraduate and postgraduate studies, there is an apparent skew towards technical areas in the delivery of the topic. Additionally, while both learners and educators recognise interdisciplinary overlap with various other topic areas, the delivery of cybersecurity is currently dominated by the computing perspective. While this is unsurprising insofar as it is relating to the protection of computing-related assets in computing systems and devices, there is potential for other topic areas (e.g. business, economics, law, psychology) – which are often highly relevant to addressing security in practice - to be better represented than is currently the case. The findings suggest a potential disconnect between the theory and practice of cybersecurity delivery. While the survey and workshop findings from both the educator and learner communities appeared to recognise the importance of cybersecurity as an interdisciplinary issue, the reality of delivery appears to be more grounded in considering it a computing-focused subject and skewed towards technical aspects of the topic. Given the number and breadth of knowledge areas represented within CyBOK, it is not realistic to expect any cybersecurity degree – at undergraduate or postgraduate level – to fully cover all of them. This was clearly reflected in the survey results from both the educators and learners, and it was also apparent that some knowledge areas are more likely to receive coverage than others. Programmes need to be clear on what they are aiming to deliver and for learners to have a similarly clear idea of what programme will best meet their needs (if they already have a particular aspect of cybersecurity in mind as a target at that stage). At the same time, cybersecurity education must not result in insular and siloed approaches. The survey and the workshop highlighted recognition of interdisciplinary perspectives that are not necessarily reflected in current provision. Even if someone has elected to focus their study on a particular technical subset of the discipline, there would still be value in their education exposing them to a broader context and showing where this topic ‘fits’ into the broader picture – of both cybersecurity and the business and societal needs that it serves. Facilitating this through collaboration with other learners representing these wider elements would arguably be the most effective means of introducing a parallel to the real-world situations that will be encountered in practice.

6 Conclusion and Recommendations The findings from the task as a whole lead to a number of recommendations that should be considered in the subsequent COLTRANE activities, as well as for the provision of cybersecurity education more widely.

58

S. Furnell et al.

• Ensure that approaches and materials are holistic and represent the breadth of topics relevant to cybersecurity. This should specifically include non-technical aspects of cybersecurity, as well as appropriate emphasis toward fostering and supporting soft skills. • Technical and non-technical aspects should each receive clear recognition in terms of their contribution to cybersecurity, and one should not be presented (implicitly or explicitly) as being secondary to the other. While specific provisions may focus attention upon some topic areas and not others, this should ideally still be done in a manner that shows where these topics fit within the wider context. • Mapping the topic coverage to an appropriate reference framework (e.g. CyBOK) will be advantageous in enabling learners to understand which aspects of cybersecurity will receive attention in particular courses/programmes or modules/units. • Delivery methods should incorporate an appropriate level of practical activities (including problem/scenario-based approaches) in order to ensure that learners have the opportunity to go beyond knowledge acquisition and develop at least a baseline level of accompanying skills. • While it is important to ensure that soft skills receive specific attention, they should not be approached as a distinct and separate topic in their own right. They should be seen as being implicit within cybersecurity more generally, and the coverage and assessment of them should be integrated alongside the more discipline-specific material. • Incorporate opportunities for educators from outside the computing topic area to provide input into delivery, to evidence both the relevance of cybersecurity within their discipline and how approaches from their disciplines can be relevant in addressing it. Correspondingly, look for opportunities to the computing-based educators to offer contribution into the delivery for other disciplines that need to understand cybersecurity in the technology context. • Incorporate opportunities for cybersecurity learners to interact with learners from other related disciplines. This may be achieved within programmes, where there are learners from multiple backgrounds, or between programmes through specifically constructing opportunities for collaborative learning. • Consider means by which learners with backgrounds in other disciplines can be offered opportunities to contribute their wider perspective. These provide a basis to inform both the methodology and approach for the COLTRANE educational framework: a conceptual account of principles and guidelines for designing educational building blocks, including the use of technology. The framework will therefore be implemented as educational scenarios through which educators in higher education can exploit the platform technology to design their materials. Acknowledgements. The authors would like to thank the ERASMUS+ project COLTRANE (grant number KA2-Projekt 2020–1-AT01-KA203–078070) for supporting the research presented in this paper. The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

Collaborative Cybersecurity Learning

59

References 1. Cybint. Why Hands-On Skills Are Critical in Cybersecurity Education. 5 August 2021 (2021). https://www.cybintsolutions.com/hands-on-skills-in-cyber-security-education/. 2. Bereiter, C.: Liberal education in a knowledge society. In: Liberal education in a knowledge society (2002) 3. European Union. Directive (EU) 2016/1148 of the European Parliament and of the Council of 6 July 2016 concerning measures for a high common level of security of network and information systems across the Union. Official Journal of the European Union, 19 July 2016 (2016). http://data.europa.eu/eli/dir/2016/1148/oj 4. COLTRANE Homepage. https://coltrane.ait.ac.at. Accessed 11 Dec 2021 5. Wilson, P.: The Security Profession 2020–2021. Chartered Institute of Information Security (2021). https://www.ciisec.org/White_Papers 6. Andriessen, J., Furnell, S., Langner, G., Quirchmayr, G., Scarano, V., Tokola, T.: Foundations for collaborative cyber security learning: exploring educator and learner requirements. In: Drevin, L., Miloslavskaya, N., Leung, W.S., von Solms, S. (eds.) Information Security Education for Cyber Resilience. WISE 2021. IFIP Advances in Information and Communication Technology, vol. 615. Springer, Cham, pp. 143–145 (2021). https://doi.org/10.1007/978-3030-80865-5 7. DCMS. Cyber security skills in the UK labour market 2021 - Findings report. Department for Digital, Culture, Media & Sport, 23 March 2021 (2021). https://www.gov.uk/government/ publications/cyber-security-skills-in-the-uk-labour-market-2021 8. Eurostat. ICT specialist by sex’ in ICT specialists in employment. 13 July 2021 (2021). https:// ec.europa.eu/eurostat/statistics-explained/index.php?title=ICT_specialists_in_employment 9. (ISC)2 . Women in Cybersecurity: Young, Educated and Ready to Take Charge (2019). https:// www.isc2.org/-/media/ISC2/Research/ISC2-Women-in-Cybersecurity-Report.ashx 10. Rashid, A., Chivers, H., Lupu, E., Martin, A., Schneider, S.: The Cyber Security Body of Knowledge. Version 1.1.0, 31 July 2021 (2021). https://www.cybok.org/media/downloads/ CyBOK_v1.1.0.pdf 11. Blažiˇc, B.J.: Changing the landscape of cybersecurity education in the EU: will the new approach produce the required cybersecurity skills?, Educ. Inf. Tech. (2021). https://doi.org/ 10.1007/s10639-021-10704-y

A Maturity Assessment Model for Cyber Security Education in Europe Silvia Vidor(B)

and Carlos E. Budde

University of Trento, Trento, Italy [email protected]

Abstract. Maturity assessment models have repeatedly been applied to education and to cyber security with the aim of improving the performance and management of private and public institutions. However, no attempts have been made so far to create a framework for the evaluation of cyber security education, which is an increasingly pressing matter due to the demand for cyber security professionals in the European Union. This paper contributes with a proposal for a maturity assessment model of cyber security education, including a discussion of one of the main issues in the field: the definition of knowledge units for the standardization of cyber security education in Europe. Keywords: Cyber security · Security education assessment · Maturity models

1

· Maturity

Introduction

The growing threat of cyber attacks for both public and private organizations has stimulated, in recent years, the slow but steady growth of educational and training programs aimed at providing graduates and workers with the necessary instruments to implement cyber security measures, as well as creating dedicated professional figures [1]. Beyond national security, this is of particular relevance for the industrial sector, where e.g. SMEs can seldom afford exclusive resources for security-specialised personnel, and moreover whose level of information security is generally low [2,3]. Nevertheless, in the European context, overarching educational efforts in the area of cyber security training remain sparse. On top of this, the lack of standardized educational frameworks and curricula makes it difficult to investigate the factors that contribute to the development and diffusion of quality cyber security education. While methods for evaluation in the wider fields of education and cyber security have been previously defined, to the best of our knowledge that has not happened for cyber security education [4–6]. Due to its relatively recent emergence and lack of standardized curricula, we argue that cyber security education presents specific characteristics and unattended issues that cannot be fully addressed through more general analyses. c IFIP International Federation for Information Processing 2022  Published by Springer Nature Switzerland AG 2022 L. Drevin et al. (Eds.): WISE 2022, IFIP AICT 650, pp. 60–74, 2022. https://doi.org/10.1007/978-3-031-08172-9_5

A Maturity Assessment Model for Cyber Security Education in Europe

61

For example, it has been argued that cyber security should be seen as a metadiscipline and thus includes a variety of disciplinary variants, leading to the necessity for any cyber security education-related framework to include a series of degrees potentially very different in structure [7]. Cyber security education would then need a dedicated analysis, starting from a framework for the definition of key domains impacting the overall quality of cyber security education in Europe and the evaluation of the different levels of capability of educational organizations [1,8]. Objective. The aim of this paper is, then, to propose a maturity assessment model for the use of organizations involved in the field of cyber security education, particularly universities and institutions involved in cyber security training. Content and Outline of This Work. In Sect. 2 we provide a brief review of existent maturity assessment models, describing their definition, history, main elements and two examples of models used in the education and cyber security fields. Then, in Sect. 3 we describe our proposal for domains, parameters and levels constituting a maturity assessment model for cyber security education in Europe. Our decisions are oriented to the expected (main) end beneficiary of such education, namely industry in general and SMEs in particular. In Sect. 4, starting from the surveys on formal cyber security education in Europe [1] and professional needs in cyber security [8], we discuss one of the main issues in the evaluation of cyber security educational programs: the identification of standardized knowledge units. We conclude the paper with some considerations over the next steps to be taken, specifically concerning the evaluation of the model we have developed.

2

Review of Existing Maturity Assessment Models

This section summarizes the main elements in relation to maturity assessment models, specifically concerning their definition, their history and development from the field of software development to business, education and cyber security, and their essential components. Two examples of maturity assessment models used in education and cyber security are briefly described in Sects. 2.4 and 2.5 to provide a specific insight into the structure of popular frameworks in the areas of relevance to our work. 2.1

Maturity Assessment Models: A Definition

Maturity assessment models (also known as maturity assessment frameworks or maturity evaluations) can be described as collections of elements, or attributes, representing capability and progression in a particular discipline or sector. The content of a model typically includes best practices and standards of practice of the field [9]. As a consequence, maturity models provide a benchmark against which an organization can evaluate the current level of capability of its practices, processes, and methods and set goals and priorities for improvement.

62

2.2

S. Vidor and C. E. Budde

History and Development of Maturity Assessment Models

The origins of maturity evaluations can be traced back to the Capability Maturity Model (CMM), which was originally developed with the aim of evaluating software contractors’ capabilities when working with the US Department of Defense. The CMM was formalized by Watts Humphrey of the Software Engineering Institute in 1988 [10], and later defined in detail with five maturity levels, each characterized by specific process areas and practices, by Mark C. Paulk, Charles V. Weber, Bill Curtis, and Mary Beth Chrissis [11]. Though its original use case was that of evaluating government contractors specifically operating in the software development field, the CMM - and its derivation, the Capability Maturity Model Integration (CMMI) - have been regarded more generally as methods to evaluate the maturity of processes and have thus gained traction in other sectors, such as business. Today, the CMM and its derivations constitute the basis upon which the great majority of maturity models are built. The progressively increasing popularity of CMM has resulted in a widespread diffusion of maturity models to areas not necessarily related to IT or business but considered as being in need of improved management, such as education. The education and training community began issuing its own versions of maturity models (particularly related to the introduction of digital technologies in education) in the early 2000s s [12]. The aim was to identify and measure performance in a series of key indicators in the education and training fields - e.g. “learning effectiveness”, “change readiness” or “time to competency” - to evaluate the ability of the concerned educational institution to reach its goals, and eventually improve. More recently, maturity has also become central in the evaluation of the cyber security capabilities of organizations, with the consequent development of models specifically targeted to this field by both private and public institutions. In particular, maturity assessment models in the field of cyber security have been mainly employed as a means to manage, measure and monitor the efficacy of both cyber security implementation methods and their governance [13]. As of today, some of the most widely used cyber security maturity models are incorporated into international cyber security standards such as ISO/IEC 27001 and NIST, which define requirements for the maintenance and improvement of information security in organizations [4]. 2.3

Characteristics of Maturity Assessment Models

The essential components of a maturity model include [4]: – Levels (see Fig. 1) - constitute the measurement element of the model, and are generally organized in a scale from 1 (least mature) to 5 (most mature); – Attributes - constitute the content of the model, at the intersection of domains and maturity levels; – Appraisal and scoring methods - constitute the standards for the measurement of the levels;

A Maturity Assessment Model for Cyber Security Education in Europe

63

– Model domains - constitute the areas of importance for the analysed topic and can be specified in attributes.

Disciplined process

Standard, consistent process

Predictable process

Continuously improving process

1

2

3

4

5

Initial

Repeatable

Defined

Managed

Optimizing

Fig. 1. The five levels of CMM [11]. The denominations of the levels (and sometimes its amount) vary between different authors and sectors of application.

According to [14], maturity models can also be divided into three different types: progression, capability and hybrid models. Progression models focus on the model attributes and represent the scaling of one or more attributes, where the progression from one maturity level to the other indicates some progression of attribute maturity. Their purpose is to provide a roadmap of improvement as expressed by increasingly better versions of an attribute as the scale progresses; for this reason, they are the preferred kind of model for companies such as consultancies. An example of progression model is represented by the Smart Grid Maturity Model (SGMM), which is used to plan the development of smart grid facilities for electric power utilities [15]. Capability models, which are represented by the CMM and its derivations, focus instead on the broader organizational capability of the analysed institution, in an effort to evaluate the maturity of the “culture” instead of that of the single attribute. As a consequence, the levels of capability models represent states of organizational maturity specifically focused on so-called “process maturity” leading to capability models being also known as process models. Finally, hybrid models combine elements of both the progression and capability models. The transitions between one level and the other describe capability maturity, as in capability models, but the structure of the model reflects that of a progression model, for example by starting from an existing sectoral code of practice. 2.4

The e-Learning Maturity Model

As mentioned in Sect. 2.2, maturity assessment models in the field of education tend to focus on the introduction of digital technologies in the teaching and learning experiences, as well as on the ways to best integrate them for maximum results in terms of learning. Consequently, one particularly popular subject of maturity assessment has been e-Learning, with several higher education institutions developing and applying their own version of the model. Among the most used models for the evaluation of e-Learning capability and maturity in educational institutions is the e-Learning Maturity Model (eMM),

64

S. Vidor and C. E. Budde

developed by Stephen Marshall of the Victoria University of Wellington in 2006 (an earlier version dates back to 2004). The eMM, in the words of its creator, is “aimed at [. . . ] changing organisational conditions so that e-learning is delivered in a sustainable and high quality fashion to as many students as possible” [16]. Based on the CMM and on the Software Process Improvement and Capability dEtermination (SPICE) model, the eMM builds upon the idea that the ability of an institution to be effective in sustaining and delivering e-Learning depends on its capability to operate optimally in five “process categories” (known as domains in the CMM) as defined in Table 1. Each of these categories is defined by a series of “dimensions of capability” (ranging in number from 3 for the Evaluation category to 10 for the Learning category), which are evaluated on a maturity scale from 1 to 5. Each dimension can be further broken down into essential or useful practices that are necessary to achieve the objective of the process from the perspective of the considered dimension. Table 1. The five process categories of the eMM as described in [16]. Category

Description

Learning Development

Processes that directly impact on pedagogical aspects of e-learning. Processes surrounding the creation and maintenance of e-learning resources. Processes surrounding the oversight and management of e-learning. Processes surrounding the evaluation and quality control of elearning through its entire lifecycle. Processes associated with institutional planning and management.

Support Evaluation Organisation

An example of application of the eMM to the context of e-Learning in Finnish universities has been described in [17]. 2.5

The Cybersecurity Capability Maturity Model

In 2012, the U.S. energy sector and the Department of Energy (DoE) developed a dedicated maturity model, known as Cybersecurity Capability Maturity Model (C2M2), which is aimed at public or private organizations wishing to improve their cyber resiliency through the implementation and management of cyber security practices. The most recent version of the model, dated July 2021, includes input from internationally recognized cyber security bodies, such as the National Institute of Standards and Technology (NIST) [9]. Similarly to the eMM, the C2M2 includes 342 practices divided into 10 domains: – – – – –

Asset, Change, and Configuration Management; Threat and Vulnerability Management; Risk Management; Identity and Access Management; Situational Awareness;

A Maturity Assessment Model for Cyber Security Education in Europe

– – – – –

65

Event and Incident Response, Continuity of Operations; Third-Party Risk Management; Workforce Management; Cybersecurity Architecture; Cybersecurity Program Management.

Practices are the actions that the concerned organization can take to improve its capability into the considered domain. Each practice is also organized into a series of objectives, representing achievements supporting the domain at hand. In applying the model, each domain is evaluated according to three maturity levels (1 to 3), differently from the original CMM. A simplified representation of the C2M2’s structure is shown in Fig. 2.

Fig. 2. Practices and objectives in each domain according to the C2M2. Notice that objective can be of “approach” or “management” nature [9].

The C2M2 is highly complex, partly due to the extensive amount of aspects included in the evaluation of cyber security maturity within an organization and partly due to the inclusion of additional standards, frameworks and requirements compared to the previous versions.

3

Maturity Evaluation of Cyber Security Education

In order to develop a maturity assessment model for cyber security education in Europe, we took inspiration from the 6P assessment implemented by Manufacturing Industry Digital Innovation Hubs (MIDIH), a EU H2020 program aimed at implementing technological, business and skills building services to European stakeholders in the field of digital innovation [18].

66

S. Vidor and C. E. Budde

Interest in cyber security education in Europe has been increasing steadily in the past years, particularly among industry actors, who are in need of highquality education on the subject for their current and future employees. Maturity assessment models can provide a tool for industry in general, and SMEs in particular, to understand where to turn to for high-quality cyber security-educated hires [2]. We thus chose the MIDIH over other relevant models in the fields of security or education due to its closer alignment with the needs of industry, which is not present in models such as the eMM. The original 6P model, which is a derivation of the CMM, includes six domains (Product, Process, Platform/Technology, People, Partnership and Performance) and five levels of maturity; we propose an adaptation to the field of cyber security education as described in the following sections. 3.1

Proposed Domains and Parameters

In line with the original 6P model, we have chosen to define six domains for the assessment of maturity of organizations involved in cyber security education. However, differently from 6P, we have made the decision to change two of them to fit specifically the cyber security education field as opposed to a wider focus on innovation. In particular, the “Product” domain has been changed into the “Students” domain to better reflect the reference to the subjects of education, while the “People” domain has been changed into the “Educators” domain, given its exclusive attention to teachers and trainers compared to the wider perspective of the original domain as defined in the 6P model. For each domain, our model provides a description of the parameters that are subject to the evaluation of maturity, as follows: 1. Students – concerns the students’ interest in cyber security-related subjects, their background and experience in the field, and their enrollment in different types of educational offerings (e.g. seminars, full degrees) on the topic. For instance, the percentage of students that choose a clear-cut cyber securityrelated career or subject, such as cryptography, falls within this domain. Another example are surveys that reveal the intention of pre-university students to follow studies in the field of cyber security. 2. Educators – concerns the level of experience and expertise on the topic of cyber security (and its subtopics) on the side of professors, teachers and trainers. For instance, cyber security-related qualifications and certifications obtained by educators are relevant to this domain. 3. Process – concerns the coverage of different knowledge units, depending on which ones are considered as a priority by the concerned country, or by supranational authorities. For example, measures related to the adherence of degree courses (or syllabi) to curricula requirements for cyber security education fall within this domain. 4. Platform – concerns the use and integration of platforms for cyber security education, such as cyber arenas, into the teaching program. The modality and frequency of the platforms’ use would also be relevant to achieve full maturity in this domain.

A Maturity Assessment Model for Cyber Security Education in Europe

67

5. Partnership – concerns the presence of partnerships between stakeholders (e.g. academia, industry, public institutions) on the topic of cyber security education and their contributions to the improvement of education quality. An example would be that of a partnership between an university offering cyber security degree courses and a company offering internships to selected students. 6. Output – concerns the overall impact on society of the process of cyber security education, in the form of outgoing student quality. For example, the percentage of former students employed in the cyber security or closely related sectors is included in this domain, as well as the percentage of former students that choose to start an academic career in cyber security. The specifications of the six domains throughout the five levels of maturity are listed in Table 2.

Table 2. Proposed maturity assessment model for cyber security education. The model can be read horizontally as a progression between different levels of maturity w.r.t. a specific domain, or vertically as a transversal maturity scenario w.r.t. a specific level. Domain

Consolidating

Defined

Managed

Optimizing

Students Self-initiative; students enroll in generic courses for the wide public

Initial

Seminar; students attend mini-courses on basic topics

Course(s); students attend full courses on fundamentals

Degree; students pursue an educational path dedicated to cyber security

Career; students follow a comprehensive and coherent educational path (with practical experiences) in cyber security

Educators

Inexperienced; superficial knowledge

Competent; some years of training or experience

Formally educated; specific knowledge

Limited practical experience; multiple-area knowledge

Experienced and formally educated; implementation of theoretical and practical teaching

Process

Cyber security as extra topic

Few fundamentals covered

Fundamentals mostly covered

Fundamentals and few optional topics covered

Fundamental and optional KUs required by national (or other) institutions are fully covered

Few tools available for educators

One platform for students to train basic skills

Some platforms for basic skills and technologyoriented needs

Integrated use of various platforms for theoretical and practical competences

Multi-topic between selected stakeholders

Adaptive, ad hoc with actors from different environments

Wide network with balanced representation of different-background stakeholders

Internship-level knowledge

Career-ready, wide-ranging theoretical and practical competences

Platform No use of tools or platforms

Partnership

No partnerships Single, limited-topic with already known contacts

Output

No quality measurement available

Selective, Knowledge of general concepts topic-specific competences (e.g. phishing)

68

3.2

S. Vidor and C. E. Budde

Proposed Levels of Maturity

In line with the approach of the CMM, which the 6P model derives from, we have chosen to maintain the original five levels of maturity in our cyber security education maturity assessment model, so as to define in a detailed manner the evolution from one level to the other. We have, however, changed the name of the second level coherently with the characteristics of the level, which do not fit the original “Repeatable” label. Even though it is probable that organizations involved in cyber security education may fare differently in the six domains, registering different levels of maturity in different areas, the model that we propose can also be read vertically, depicting five scenarios of maturity that give a full picture of the state of cyber security education in an organization: 1. The Initial scenario describes the starting point of cyber security education, where the topic is still unaddressed or only of minor interest, students lack knowledge in the field, and the availability of experienced teaching personnel is low. 2. The Consolidating scenario describes a situation of growth of interest and knowledge on the topic of cyber security on the side of students and institutions, with short-term courses on basic issues (e.g. fundamentals for cyber hygiene for office jobs) being taught by educators with sufficient foundations on the matter at hand. Partnerships and platforms still play a minor role, but might be present. 3. The Defined scenario describes a case in which cyber security fundamentals are covered in full-length courses (with the use of at least one training platform, for example through Massive Online Open Courses [20]) taught by personnel with a formal education in cyber security. Partnerships on multiple topics are possible, but still restricted to stakeholders from the same environment (e.g. academia-academia). 4. The Managed scenario describes an advanced (but not yet optimal) level of maturity of cyber security education, where students (who may have previous experience or knowledge in the field) can follow dedicated degree courses and may benefit from the existence of ad hoc partnerships between the educational institution and other stakeholders for the improvement and practical application of their education, e.g. through internships. 5. The Optimizing scenario describes the final level of maturity for cyber security education, with the possibility to pursue full educational paths on the topic (extensively covering both fundamental and optional knowledge units) taught by experienced personnel and allowing the development of theoretical and practical competences (with the help of platforms such as cyber ranges), to be later spent in the job market. Scenarios may also represent different cycles or generations of progress of cyber security education. The passage from one level to the other, then, would be enabled by the completion of the elements in the previous one. For example, the improvement in terms of educators’ preparedness and competence on the

A Maturity Assessment Model for Cyber Security Education in Europe

69

topic of cyber security is made possible by the quality of their education in the previous scenario, when they held the position of students. The model we propose for the evaluation of the maturity of cyber security education may be used, where needed, in combination with other maturity assessment models for a more well-rounded assessment of the quality level of an educational or training institution. This is particularly relevant in the case of other models evaluating specific aspects of education, such as the e-Learning Maturity Model mentioned in Sect. 2.4. While the eMM analyzes a different aspect of education, that is, the impact of e-Learning on the wider learning experience of students and on the institutions offering such service, given the recent popularity of online learning (also in the field of cyber security education), the parallel assessment of these two aspects can help in determining the overall quality of cyber security teaching or training in the examined organization. Still, there are some elements that remain of difficult definition within our proposed model: among them is the identification (and prioritization) of the cyber skills that are necessary for a complete cyber security education, that is, the identification of which knowledge units educational institutions should be required to cover to stimulate the development of those skills—both in terms of fundamental and optional topics. This issue, which relates to the Process domain of our model, is discussed in the following section. 3.3

Validation and Evaluation

Albeit based on MIDIH and other consolidated works, our model remains at theoretical level. Subsequent refinements and modifications must be preceded by a validation phase, where the model is evaluated by educational practitioners and also intended end users, e.g. SMEs which require cyber security personnel. In this respect, a questionnaire is being designed that describes the maturity assessment framework of Table 2. Survey participants are asked whether the selected domains are relevant as indicators for the maturity degree of the cyber security education of their respective institutions. The survey includes open answers, where respondents can propose additional domains, thus signaling areas possibly not being covered by the six domains of our model. When deemed necessary, interviews will be carried out with selected respondents. The survey will be primarily disseminated among educational and training organizations involved in cyber security education in Europe. This includes universities, other cyber-security relevant centres of public studies, and also possibly private education institutions. Industrial sectors not necessarily involved in education per se, but whose IT assets demands personnel expected to have undergone education at the Managed or Optimizing maturity level—e.g. telecommunications and consultants—will also be targeted as survey respondents.

4

Knowledge Units

Given the extraordinary growth in cyber threats to private and public organizations alike in Europe, as well as the increasing emphasis on security- and

70

S. Vidor and C. E. Budde

privacy-by-design, it is by now beyond doubt that there is the need for capable specialists in all areas of cyber security [1,19]. These specialists need to possess a variety of skills and competences that are generally acquired through education and, in part, through practical experience. At the moment, however, institutions teaching cyber security around the European Union are not adopting a common approach to cyber security education, covering a variety of topics and on occasions not distinguishing adequately between fundamental and optional subjects. The lack of a clear definition of the cyber security curricula across the continent, thus, makes it difficult to evaluate the level of maturity of cyber security education, and represents a critical issue for addressing Europe’s need for a unified approach to the subject. 4.1

A CyberSecurity Education Framework

To try and address this issue, we build upon the framework developed in [1] to perform a review of existing European MSc programs in cyber security. This framework has been extended to professional education in [8]. Our aim is to create a comprehensive, credible structure containing easily recognizable and common terminology in order to provide a point of reference for the organization of cyber security curricula. The framework is based upon a comparison of a series of existing cyber security curricula and taxonomies, such as: – the ACM Cybersecurity Curricula framework, developed by the Association for Computing Machinery in collaboration with the IEEE Computer Society, the Special Interest Group on Information Security and Privacy of the Association for Information Systems, and the Committee on Information Security Education of the International Federation for Information Processing; – the National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework, developed by the US National Institute of Standards and Technology. While more bodies of knowledge were originally considered, such as the Proposal for a European Cybersecurity Taxonomy developed by the EU Joint Research Centre and the Cybersecurity Body of Knowledge (CyBOK) developed by the UK National Cyber Security Programme and the University of Bristol, the ACM and NICE frameworks were preferred due to their reputation. The final version of the framework identifies nine knowledge areas, each including a series of more specific knowledge units. Knowledge units are sets of topics connected by a common theme, while knowledge areas are aggregates of related knowledge units. The framework is shown in detail in Fig. 3. 4.2

Known Issues and Pivotal Decisions

While the definition of a common framework can help to determine which subjects need to be covered in cyber security education, there are some persistent problems that remain to be solved e.g. with extensions or modifications to the framework illustrated in Fig. 3.

A Maturity Assessment Model for Cyber Security Education in Europe

71

Fig. 3. The relevant knowledge areas identified in [1, 8] for cyber security (left) and the relative knowledge units (right). The concepts were taken from the ACM and NICE frameworks, although further sources were considered.

72

S. Vidor and C. E. Budde

The primary issue concerns the hierarchy of knowledge units: there is currently no concretely specified distinction between “fundamental” or “core” units, and “optional” ones. This is shown for instance by the survey performed in [1], which included questions verifying the coverage of the cyber security-related knowledge units defined in the framework throughout higher education institutions in 25 EU countries. This survey put in evidence that current cyber security courses across Europe cover different topics in quantitatively different ways. For example, even though all knowledge units identified by the framework were covered with mandatory courses, this only applies to the entire set of respondents. More specifically, not all countries covered all knowledge units: Spanish, French and German institutions covered more than 80% of the knowledge units with dedicated mandatory courses; in contrast, Slovakia, Romania and Ireland covered less than 10%. The focus of these three last countries was on Connection Security, Data Security and Organizational Security (Slovakia); Connection Security (Romania) and Data Security (Ireland), while other countries offered much more variety [1]. Such variations, not only from country to country but also across different higher education institutions in the same nation, can be interpreted by saying that at the moment, the definition of one knowledge unit as fundamental over another is a quite complex task. The differences from country to country also raise another question, that is, whether it is preferable for cyber security education across the European Union to adopt a homogeneous approach in terms of which knowledge units include in the cyber security curriculum, and in which measure, or whether it is better to allow each country to specialize in one or more specific sub-field(s)—e.g. cryptography in Spain, access control in Belgium. The second option would entail, of course, that maturity assessments for the Process domain would need to be adapted to the curriculum developed by the country in which the analysed institution is situated.

5

Conclusion

In this paper, a proposal for a maturity assessment model was presented with the aim of defining a method for the evaluation of the maturity of institutions involved in cyber security education in the European Union. While some issues pertaining to the definition and standardization of knowledge units remain, our proposal tries to build upon the existing models in the areas of education and cyber security to lay the foundations of a common approach to cyber security education in Europe. Future Work. As a next step, the model will be tested across educational and training institutions in the cyber security field to verify its applicability and to try to provide ad hoc solutions for the issues we have identified and described in Sect. 4.2. Depending on the next steps taken at the European level concerning cyber security education, however, it remains possible that the model will need to be adapted to every country’s specific necessities or desires for specialization,

A Maturity Assessment Model for Cyber Security Education in Europe

73

requiring a dedicated evolution of the Process domain to better account for these peculiarities. Acknowledgments. This work has been partially supported by the EU Horizon 2020 Programme under grant 830929 (CyberSec4Europe), and by EIT Digital under task 20097 (Blended Cybersecurity Master).

References 1. Dragoni, N., Lafuente, A.L., Massacci, F., Schlichtkrull, A.: Are we preparing students to build security in? A survey of European cybersecurity in higher education programs. IEEE Secur. Priv. 19, 81–88 (2021). https://doi.org/10.1109/MSEC. 2020.3037446 2. Manso, C.G., Rekleitis, E., Papazafeiropoulos, F., Maritsas, V.: Information security and privacy standards for SMEs. ENISA report (2016). https://www.enisa. europa.eu/publications/standardisation-for-smes 3. Ruiz, J.F., et al.: Security characteristics description, security and market analysis report. SMESEC deliverable D2.1 (2017). https://www.smesec.eu/deliverables. html 4. Aliyu, A., et al.: A holistic cybersecurity maturity assessment framework for higher education institutions in the United Kingdom. Appl. Sci. 10(10), 3660 (2020). https://doi.org/10.3390/app10103660 5. Marks, A., AL-Ali, M., Atassi, R., Abualkishik, A.Z., Rezgui, Y.: Digital transformation in higher education: a framework for maturity assessment. Int. J. Adv. Comput. Sci. Appl. 11(12), 504–513 (2020). https://doi.org/10.14569/IJACSA. 2020.0111261 6. Ozkan, B.Y., van Lingen, S., Spruit, M.: The cybersecurity focus area maturity (CYSFAM) model. J. Cybersecur. Priv. 1(1), 119–139 (2021). https://doi.org/10. 3390/jcp1010007 7. Parrish, A., et al.: Global perspectives on cybersecurity education for 2030: a case for a meta-discipline. In: ITiCSE 2018 Companion. Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education, pp. 36–54 (2018). https://doi.org/10.1145/3293881.3295778 8. Karinsalo, A., et al.: D6.3 - design of education and professional framework. Cyber Security for Europe (2021). https://cybersec4europe.eu/wp-content/uploads/ 2021/06/D6 3 Design-of-Education-and-Professional-Frame-work Final.pdf 9. Muneer, F., et al.: Cybersecurity capability maturity model, version 2.0. U.S. department of energy, office of cybersecurity, energy security and emergency response, Washington, D.C. (2021). https://c2m2.doe.gov/C2M2%20Version%202. 0%20July%202021.pdf 10. Humphrey, W.: Characterizing the software process: a maturity framework. IEEE Softw. 5(2), 73–79 (1988). https://doi.org/10.1109/52.2014 11. Paulk, M.C., Curtis, B., Chrissis, M.B., Weber, C.V.: Capability maturity model for software, Version 1.1. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania (1996). https://resources.sei.cmu.edu/asset files/ TechnicalReport/1993 005 001 16211.pdf 12. Wagenstein, H.N.: A capability maturity model for training & education. Chapter one: background and rationale. PMI Global Congress 2006. North America, Seattle, WA. Newtown Square, PA: Project Management Institute (2006)

74

S. Vidor and C. E. Budde

13. De Bruin, R., von Solms, S.H.: Cybersecurity governance: how can we measure it?. In: 2016 IST-Africa Week Conference, pp. 1–9 (2016). https://doi.org/10.1109/ ISTAFRICA.2016.7530578 14. Caralli, R., Knight, M., Montgomery, A.: Maturity models 101: a primer for applying maturity models to smart grid security, resilience, and interoperability. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennysilviania (2012). https://resources.sei.cmu.edu/asset files/WhitePaper/ 2012 019 001 58920.pdf 15. Software Engineering Institute: Smart Grid Maturity Model, Version 1.2. Model Definition. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania (2018). https://doi.org/10.1184/R1/6583835.v1 16. Marshall, S.: eMM Version 2.3 Process Descriptions. Victoria University of Wellington, New Zealand (2007). http://e-learning.geek.nz/emm/documents/ versiontwothree/20070620ProcessDescriptions.pdf 17. Haukij¨ arvi, I.: E-learning maturity model – process-oriented assessment and improvement of e-Learning in a Finnish University of Applied Sciences. In: Passey, D., Tatnall, A. (eds.) ITEM 2014. IAICT, vol. 444, pp. 76–93. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-45770-2 9 18. European Commission: Manufacturing Industry Digital Innovation Hubs FactSheet (2021). https://cordis.europa.eu/project/id/767498 19. Crumpler, W., Lewis, J.A.: The Cybersecurity Workforce Gap. Center for Strategic & International Studies (2019). https://csis-website-prod.s3.amazonaws.com/s3fspublic/publication/190129 Crumpler Cybersecurity FINAL.pdf 20. Fischer-H¨ ubner, S., et al.: Quality criteria for cyber security MOOCs. In: Drevin, L., Von Solms, S., Theocharidou, M. (eds.) WISE 2020. IAICT, vol. 579, pp. 46–60. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59291-2 4

Real-World Cybersecurity-Inspired Capacity Building

ForCyRange: An Educational IoT Cyber Range for Live Digital Forensics Sabrina Friedl(B) , Magdalena Glas , Ludwig Englbrecht , Fabian B¨ ohm , and G¨ unther Pernul University of Regensburg, Universit¨ atsstraße 31, 93053 Regensburg, Germany {Sabrina.Friedl,Magdalena.Glas,Ludwig.Englbrecht,Fabian.Boehm, Guenther.Pernul}@ur.de https://go.ur.de/ifs

Abstract. The Internet of Things (IoT) is finding increasing application in different areas, whether for private users or in industrial plants. The IoT increases the attack surface for Advanced Persistent Threats (APTs) due to insufficiently secured IoT devices and networks. The heterogeneous structure of the IoT poses several new challenges for the application of IoT forensics (IoTF). Due to limited resources and storage capacity on the devices, the application of traditional forensics is not possible. Therefore, the nature of these IoT devices urges forensic experts to extract and analyze possibly relevant data in a real-time manner from running devices by applying Live Digital Forensics (LDF). Although LDF investigations are not commonly applied in the IoT context yet, IoTF could benefit largely from a combined arms approach with LDF. Thus, security experts with sufficient skills and knowledge will be required to perform such procedures. Addressing the challenge to equip future forensic experts with these skills and knowledge, we propose a concept for an educational IoT Cyber Range for LDF for postgraduate cybersecurity learners. For a realistic learning experience, we outline the simulation of a simplistic, underlying IoT system. In order to create an environment that is as realistic as possible, we describe an illustrative scenario that serves as a motivational story. Following the scenario, learners carry out several tasks of an IoTF investigation for solving the scenario. Keywords: IoT forensics · Live Digital Forensics (LDF) · Cybersecurity · Cyber range · Digital forensics · Internet of Things (IoT)

1

Introduction

The IoT connects billions of devices, enabling them to collect or transfer data and communicate with each other [19]. As of 2019, the total number of IoT devices is estimated at around 10 billion and is projected to jump to an amount of 30.9 billion connected IoT devices worldwide by 2025 [10]. This development, c IFIP International Federation for Information Processing 2022  Published by Springer Nature Switzerland AG 2022 L. Drevin et al. (Eds.): WISE 2022, IFIP AICT 650, pp. 77–91, 2022. https://doi.org/10.1007/978-3-031-08172-9_6

78

S. Friedl et al.

in combination with the nature of the applied IoT environments, poses several challenges to performing digital forensic investigation within an IoT network [21]. Examples of these challenges are vastly heterogeneous devices and highly fuzzy network delineations. However, the need to carry out forensic investigations is demonstrated by the ever-increasing amount of cyberattacks and especially of targeted and professional APTs. As these are often combined with some types of fileless malware, the investigations of an incident within an IoT environment have to be performed in real-time. Especially due to the specific features of IoT devices, such as limited resources and the fact that running devices cannot be switched off, the application of LDF is particularly useful. Thus, methods and techniques from the context of LDF are applied to understand what is happening on a running system and extract valuable data that would have been lost otherwise [22]. Generally, conventional devices are more secure than IoT devices due to traditional security practices. In addition, IoT devices need to be smaller and more compact for their intended uses, leaving less space for security defenses. These are reasons for the drastic increase in the IoT attack surface [15]. The involvement of IoT devices in crimes confirms the necessity for IoTF. To carry out investigation in the IoT, experts need a specific set of skills and knowledge. As stated in the ISACA report from 2021 [11], organizations worldwide face problems recruiting qualified cybersecurity experts. DF experts represent a portion of the available cybersecurity experts and are thus available in even smaller numbers. There are various ways to introduce new content to cybersecurity learners in a comprehensible way, e.g., a lecture, a workshop [9], or a cyber range [23]. While traditional lectures rely on theoretical knowledge transfer, workshops and cyber ranges present new ways of experiential learning, directly integrating the learners. Cyber ranges are particularly useful for understanding and learning how to deal with cybersecurity incidents by visualizing the learning environment and guiding the learner through a use case scenario [16]. As outlined above, the specific characteristics of IoT devices make it difficult to apply conventional forensic analysis. Thus, an IoT environment is a particularly illustrative use case to show learners, who are new to digital forensics, what LDF is and how to apply it. In this paper, we strive to investigate the potential of cyber range training for this matter, addressing the following research question: RQ. How can learning and training for Live Digital Forensic investigations be supported by an educational IoT cyber range? We propose a concept for a cyber range for IoTF training which is based on a simulation of a simple IoT system. For the LDF investigation, we include a visual decision-support tool into the cyber range to allow a visual perception of the system’s behavior under attack as well as a set of forensic tools to perform the actual investigation. The remainder of this work is structured as follows. Section 2 provides a selection of definitions that are relevant for understanding the present publication. Further, an overview of related work regarding similar concepts is offered.

ForCyRange: An Educational IoT Cyber Range for Live Digital Forensics

79

In Sect. 3, the developed concept is laid out and explained in detail. Following, Sect. 4 provides an illustrative scenario with three incidents that reflect the three fundamental objectives of information security (confidentiality, availability, integrity). Finally, Sect. 5 concludes the concept and gives an outlook on future work.

2

Background and Related Work

In this section, we explain terms that serve as a basis for understanding the paper and within applied approaches. In addition, a brief overview of related work in the area of learning environments for forensic investigations in conjunction with cyber ranges is provided. 2.1

Internet of Things (IoT)

The term IoT was coined in 1999 by K. Ashton [12]. Since then, it has been applied to various connected devices in different settings, like consumer, domestic, business, and industry [20]. Dorsemaine et al. [7] define the IoT as a “group of infrastructures, interconnecting connected objects and allowing their management, data mining and the access to data they generate” including objects/things that are “sensor(s) and/or actuator(s) carrying out a specific function that are able to communicate with other equipment” [7]. IoT technologies can be utilized in various application areas, for example, smart homes, smart cities, or smart manufacturing and agriculture [4]. In general, an IoT environment is divided into three zones, (1) Cloud (e.g., public, hybrid, private), (2) Network (e.g., cellular, industrial, mesh), and (3) Devices (e.g., sensors, wearables). At the basis of the IoT architecture, the devices produce or collect data, send it over a network into the cloud, where the data is aggregated, sorted, and processed. The processed data is then made available for users (e.g., private persons) and provides insights into communication between IoT devices [26]. 2.2

Digital Forensics, IoT Forensics and Live Digital Forensics

Digital Forensics (DF) is a sub-area of classic forensics [8]. In DF, processes and events on IT systems are investigated in relation to criminal offenses in order to obtain digital evidence that is legally permissible [14]. IoTF is based on DF and is first defined by Oriwoh et al. in 2013 [18] with three zones (internal network, middle, outside/external network). This concept is then further specified by Zawoad et al. (2015) [26], who define IoTF according to the structure of an IoT environment as a composite of three forensic types, (1) Cloud Forensics, (2) Network Forensics, (3) Device Level Forensics. Stoyanova et al. (2020) [21] then provide a fourth forensic type, valuable in IoTF, (4) Live Digital Forensics (LDF). LDF, also known as dynamic analysis, describes a Real-Time System (RTS) and is subordinate to IoTF and DF. In LDF, data is collected while the system is still running. This practice provides additional contextual information that is otherwise lost when collecting data after a system shut-down [1].

80

2.3

S. Friedl et al.

Cyber Ranges

In the field of cybersecurity training, the concept of experiential learning in cyber ranges has gained attention in recent years [25]. The term cyber range refers to an environment that replicates parts of an organization’s digital infrastructure (e.g., networks, tools, or applications) to perform realistic cybersecurity training and testing [16]. The concept emerged from the military sector, where it was derived from the term shooting range [6], providing the possibility to conduct target practice in a safe and isolated space. Thus, a cyber range enables learners to experience cyber attacks and practice cybersecurity defense strategies in a realistic way without harming the operating system they are trained for. This replication of virtual environments is not limited to Information Technology (IT) but can also include Operational Technology (OT) devices, either simulated or even in the form of actual hardware [25]. Yamin et al. [25] present a taxonomy for cyber ranges that allows to comprehensively classify cyber range concepts. The six domains of the taxonomy are briefly described hereafter. While the term cyber range refers to the virtual environment itself, a cyber range exercise is a specific training or testing activity conducted on the cyber range. The scenario describes the context and storyline for such an exercise. A cyber range may hold multiple scenarios that can be repeatedly performed on the cyber range in the course of exercises. Learning comprises tutoring and scoring elements of the cyber range, commonly included in a Learning Management System (LMS), presenting the tasks learners need to solve within the training and complementary learning material. The environment of a cyber range is the technical topology in which a scenario is executed. The teaming aspect of a cyber range refers to assigning different roles to the people involved in a cyber range exercise. As commonly used in cybersecurity exercises, a Red Team is assigned the task to exploit vulnerabilities of a system while a Blue Team is responsible for its defense. Further roles, yet less commonly used, are a White Team managing the overall exercise and a Green Team maintaining the cyber range infrastructure. Management of a cyber range exercise incorporates the organization of both the cyber range infrastructure and the learner interacting with it. Monitoring describes which mechanisms are applied to monitor the state of the technical infrastructure and the learners’ actions during the exercise. 2.4

Related Work

Related work focuses on the one hand on the application of cyber ranges in or for digital forensics. On the other hand, a workshop concept with a serious-game approach to learn forensic procedures is closely related to this topic. Vykopal et al. [24] propose the KYPO cyber range, which provides complex cyber systems and networks in a virtualized, fully controlled, and monitored environment. Due to the use of cloud resources, the deployment is time- and costefficient. KYPO is accessed via a web browser at every stage of the virtualized network lifecycle. Starting from the preparation and configuration of artifacts

ForCyRange: An Educational IoT Cyber Range for Live Digital Forensics

81

to the deployment, instantiation and operation. It allows users to focus on the desired task and provides a hands-on platform for cybersecurity courses and training. A few of the important features are the training scenarios for malware forensics, network security, and penetration testing. Leitner et al. [13] present the AIT Cyber Range that is designed on the principles of scalability, flexibility, and utilization of open source technologies. Therefore, they implemented an architecture comprised of a computing platform, infrastructure- and software-provisioning and a scenario engine. They describe various sectors in which the training content is presented. In this way, the authors aim to contribute to the general understanding and exchange of information on the design and use of cyber ranges. Blauw and Leung [2] describe an adventure workshop called ForenCity, which takes the form of a web-based scavenger hunt based on the serious-game approach. In this game, users have to apply their knowledge in digital forensics and process digital evidence in both the virtual and physical worlds. Thereby, the authors created an environment with augmented physical items and spaces at their institution’s campus. Virtual characters guide the users through a police investigation that develops further while they engage with physical and virtual properties around them. The previously discussed work addresses a lack of skilled cybersecurity analysts and cyber ranges as well as the challenges of doing LDF investigations in an IoT scenario. Vykopal et al. [24] and Leitner et al. [13] describe cyber ranges with a possible forensic application but without a detailed implementation based on a use case. Contrary, Blauw and Leung [2] describe the use case in-depth but they do not use a cyber range as an underlying basis. To the best of our knowledge, the aforementioned work does not focus on the application of LDF investigation in an IoT environment and the training of forensic analysts and learners with a cyber range. To address this research gap, we propose ForCyRange, a conceptual approach of a cyber range encompassing a virtual learning environment guiding through an IoTF scenario.

3

ForCyRange Concept

To ensure the adequacy of an educational cyber range, it is crucial to precisely define the target group it addresses, as well as the skills and knowledge it aims to impart [17]. For this reason, the target group and learning objectives of ForCyRange will be outlined in the following before proposing the design of the cyber range. 3.1

Target Group and Learning Objectives

ForCyRange aims to provide a practical introduction to LDF. In doing so, it targets individuals who are not yet working as forensic analysts but aspire to do so in the future. Since LDF is a complex process, learners are required to have some prior knowledge in cybersecurity and related concepts. Thus, the

82

S. Friedl et al.

target group of the concept are postgraduate learners with a background in cybersecurity. Training for this target group can be part of both organizational training for employees or a curriculum for cybersecurity at a higher education institution. With this target group in mind, we define the following learning objectives. These can be seen as overarching goals to be achieved by the concept and shall serve as a guideline for the design of training scenarios. – LO1: Understand the purpose and goal of LDF investigations. – LO2: Learn typical components and procedures of LDF investigations. – LO3: Develop and enhance soft skills required by an analyst during an LDF investigation. – LO4: Identify anomalies and secure the associated digital traces. LO1 and LO2 describe the transfer of knowledge about LDF. LO3 targets LDF-relevant soft skills that can be trained in the cyber range. This ranges from decision-making under time pressure to effective collaboration in a team of forensic analysts. These objectives shall be pursued by any scenario executed on the cyber range. LO4, on the other hand, refers to the hard skills one should acquire in the course of the training. As these skills highly rely on the tools and technical environment used within a scenario, LO3 can be further specified for each scenario. Depending on the design of a scenario, these learning objectives can be supplemented by others. For example, it is conceivable to design a scenario in a way that it also imparts knowledge about the IoT system in scope. 3.2

Proposed Design

ForCyRange is characterized by a flexible design that allows creating diverse scenarios within the cyber range. The cyber range concept contains different roles and components. We define the two roles Trainer and Trainee. The former oversees and manages the training. The latter participates in the training in the role of a forensic analyst. The four technical components are (1) an IoT System Simulation, (2) a Middleware for data processing, storage, and transmission, (3) a Forensic Workstation, and (4) a Learning Management System. The concept is illustrated in Fig. 1 and is detailed in the following. Forensic Analyst (Trainee): The Trainee interacts with the cyber range over a web-based user interface containing the Forensic Workstation and the Learning Management System. Each Trainee is provided with their distinct cyber range instance consisting of the four technical components described above. The role of the Trainee does not necessarily have to be taken on by an individual but by a small group of participants jointly solving the tasks. Trainer: One or more Trainers are responsible for the design and implementation of the scenario. Thereby, scenarios can be automated so that the Trainer does not need to interfere with the environment during the training but only monitors the Trainees and the technical environment to intervene if problems

ForCyRange: An Educational IoT Cyber Range for Live Digital Forensics

83

Fig. 1. Design of ForCyRange.

occur, e.g., if the Trainee faces technical issues or has trouble understanding a particular task. IoT System Simulation: The network to be forensically examined in the training is a system of connected IoT devices. For the needs of the cyber range, an IoT system is simulated realistically and complemented with one or more cyber attacks whose traces are to be detected by the Trainee. This simulation is referred to as IoT system hereafter. The IoT system is equipped with a set of tools for data acquisition and monitoring of, e.g., network traffic, active processes, and file editing. Use cases represented with the IoT system can be manifold, ranging from a smart home or office setup like described in the illustrative scenario in Sect. 4 to the simulation of IoT system as part of a smart factory. The attacked IoT system produces real-time data during the training. This provides a realistic environment for the Trainee to explore and perform various forensic investigations. Middleware/Database: A Middleware retrieves the raw data from the IoT devices. This allows to retrieve relevant data from the IoT system during an investigation without having to install and run the client of a data analysis tool on the IoT system. Several pre-processing steps required to analyze the data in the Forensic Workstation are performed on the middleware. The processed data is then stored in a database. Forensic Workstation: The LDF investigation is performed on a Forensic Workstation consisting of a visual decision-support tool, hereafter referred to as Visual Decision-Support Tool, and a set of common forensic tools (Forensic Tool Set). LDF, performed on a running system, is characterized by the strong time pressure under which analysts need to make decisions. B¨ohm et al. [3] propose the usage of a tool for visualizing security-relevant data during a LDF investigation. This intends to support decision-making during the investigation and to assist the analyst, for example, in selecting the right forensic tool at the right time.

84

S. Friedl et al.

We follow this approach by integrating a Visual Decision-Support Tool into the investigation process performed by the Trainee in the cyber range training. As a central component of the Forensic Workstation, the Visual Decision-Support Tool visualizes the data produced by the IoT system during the attack. It supports the Trainee in identifying which aspects of the IoT system need to be analyzed further and which forensic tool is required for the particular issue. Learning Management System (LMS): Throughout the investigation, the Trainees interact with a LMS, guiding them through the scenario. The LMS contains textual explanations, images, and videos to transfer the theoretical knowledge required for the investigation. On the one hand, this includes knowledge about goals (LO1) and established processes (LO2) in LDF, and on the other hand, information about the training scenario. The latter mainly includes a description of the IoT system and the impact the attack has on the devices. The skill learning aspect of the cyber range (LO4) considers the well-known theory of “flow” by Cs´ıkszentmih´alyi et al. [5], which examines how to create a balance between the person’s skills and the difficulty of an activity. This balance aims to provide a state of “flow” in which a person fully engages with an activity without being overwhelmed or frustrated. The authors propose that this state requires not only the appropriate skill set for a given task but also clear goals and immediate feedback for one’s actions. For this reason, the LDF investigation is broken down into a sequence of tasks that the Trainee needs to solve using the Visual Decision-Support Tool and the Forensic Tool Set. Therefore, the individual tasks are explained in detail to the Trainee within the LMS. The trainees are closely guided which tool out of the given toolset is suitable for which of the analysis steps. As the training is aimed at trainees who are new to the field of digital forensics, we believe that this simplification compared to an actual forensic investigation is necessary in order not to overwhelm the trainees. The Trainee submits a solution in the form of a flag to the LMS, e.g., the number of a port over which an attack is executed. This way, the Trainee is set a sequence of clear and manageable goals receiving immediate feedback for every flag they submit. Setting a time limit for each task challenges the Trainees to make decisions under time pressure (LO3). 3.3

Classification of the Concept

The proposed ForCyRange concept is finally classified in Table 1 using the previously outlined taxonomy by Yamin et al. [25] to provide comparability to other cyber range concepts. In summary, ForCyRange is a flexible concept for training LDF in a realistic IoT environment, enabling a Trainee to get a first practical insight into LDF. Thereby the Trainees learn to understand the challenges of LDF and how to tackle them. The IoT use case and the composition of the Forensic Workstation can be customized to the purpose of a specific training respectively target group. This allows designing scenarios that include specific systems, networks, and tools. Especially when the concept is used for organizational cybersecurity training,

ForCyRange: An Educational IoT Cyber Range for Live Digital Forensics

85

Table 1. Concept description based on the taxonomy by Yamin et al. [25] Domain

Description

Scenario

LDF investigation of an attacked IoT system using a Visual Decision-Support Tool and a Forensic Tool Set.

Environment Simulated IoT system and a Forensic Workstation, connected by a Middleware Learning

Trainee interacts with a LMS consisting of textual explanations, images, videos, and tasks, possibly enriched with gamification elements (e.g., scoring mechanism for flag submission)

Teaming

Blue Team: Trainee; Red Team: Automated; White Team/Green Team: Trainer

Management Trainer implements the scenario, creates learning material for the LMS, and manages the technical infrastructure. Monitoring

Trainer monitors infrastructure and the Trainee during the training, possibly partly automated

this offers the possibility of creating a training environment that is close to the type of systems and tools the Trainee will face in an actual forensic analysis in the future.

4

Illustrative Scenario: OutSmart-The-Burglar

A realistic scenario will demonstrate the application and capabilities of the ForCyRange concept. The scenario is an LDF investigation of a combined cyberphysical attack on a medical practice, inspired by a recent burglary in Austria1 . The attack consists of three incidents, each incident covering one central aspect of cybersecurity, namely confidentiality, availability, and integrity (rf. Fig. 2). We first describe the general scenario and its technical environment before outlining the Trainee’s learning process. 4.1

Storyline of the Scenario

A series of burglaries in a medical practice were reported, during which medical equipment worth e 3 million was stolen. The burglaries happened during the day, through a window facing the practice’s backyard. When the theft was detected, police was immediately informed. The attackers were able to enter the building undetected (Incident 1 ). Thereby, no physical traces of burglary, e.g., broken windows, could be determined. The CCTV camera monitoring one of the videos was not working during the burglary, so it does not provide any evidence (Incident 2 ). It stands to reason that the burglars gained access via the smart 1

https://newsbeezer.com/austriaeng/endoscopy-equipment-stolen-from-viennahospital/.

86

S. Friedl et al.

control of the windows and doors (Incident 3 ), which are part of a smart home solution the practice uses to control doors, windows, ventilation, and lighting. Before resetting the devices, the cybersecurity experts involved consult a forensic expert (the Trainee) to preserve traces of the burglary. As the investigation happens while medical interventions are performed in the practice, the IoT network cannot simply be switched off as it also controls ventilation and lighting in the surgery- and treatment rooms. For this reason, an LDF investigation of the running system is necessary. The storyline of the LDF investigation is closely linked to the learning process of the scenario. Thus, it is described in Subsect. 4.3 along with the actions respectively tasks the Trainees need to perform in the course of the investigation.

Fig. 2. OutSmart-The-Burglar : sequence of incidents.

4.2

Environment

The environment of the scenario is illustrated in Fig. 3. The IoT System Simulation consists of a Smart Door, a Smart Window, and a CCTV camera. The smart devices send status updates to a central IoT-Hub running the software OpenHAB2 for monitoring the status of the devices. A simulated malware is running on the IoT-Hub, enabling the virtual attackers to perform the three aforementioned incidents. For each of the three incidents, the Visual DecisionSupport Tool assists the Trainee in the initial identification of the incident. For the Visual Decision-Support Tool running on the Forensic Workstation that the Trainee is using for their investigations, we propose to use the LDF visualization tool3 presented by B¨ohm et al. [3]. They state the need for conducting LDF investigations due to newly proliferating fileless malware and the challenging timely manner in which decisions during investigations have to be made. Therefore they developed a prototype that visually supports the decision-making process during LDF. The tool comes with several visualizations that enable the user to identify abnormal network traffic and file system activities. The Forensic Tool Set the Trainee is provided with consists of the tools Tcpdump4 and Wireshark5 for network traffic analysis and LogDNA6 for log analysis. 2 3 4 5 6

https://www.openhab.org/. https://github.com/bof64665/LDF ReactFrontend. https://www.tcpdump.org/. https://www.wireshark.org/. https://www.logdna.com/.

ForCyRange: An Educational IoT Cyber Range for Live Digital Forensics

87

Fig. 3. OutSmart-The-Burglar : scenario design.

4.3

Learning

This subsection covers the sequence of tasks to be completed by the Trainee. As described in Sect. 3, the tasks are complemented by several learning units which impart theoretical knowledge. The tasks are not designed as a simple set of instructions to be followed by the Trainee. On the contrary, the training requires the Trainees to apply the theoretical knowledge they gain throughout the training to the actual use case and analytically determine possible courses of action - either individually or in discussions with other Trainees. If a Trainee is stuck on a particular task, they can request hints for the respective tasks within the LMS, which guide them closer to the solution. This is to avoid frustrating the Trainees and thus jeopardizing the aforementioned “flow” of the learning process. Incident 1 - Data Breach of Smart Door (Confidentiality): The back door of the practice is equipped with a smart lock. It produces log data whenever the lock is opened or locked with a key card by one of the employees. The log file synchronizes with the IoT-Hub for central monitoring. Through a phishing attack, the attackers managed to install malware on the hub and thus control the IoT system. The malware produces a ZIP file of the door’s log data every 20 min. The ZIP file is uploaded to the attackers’ server via a web-based API. The analysis of this log data enables the attackers to obtain information about when no employees are present at the practice during the day. This allowed the attackers to plan at what time the burglaries could be performed without being detected. At first, the Trainee needs to identify the process ( Flag 1) which creates the ZIP file and which external port ( Flag 2) it is uploaded to. This is achieved by detecting the process as an anomalous data flow in the visualization of the

88

S. Friedl et al.

Visual Decision-Support Tool. To investigate the incident in-depth, the Trainee uses Tcpdump to analyze the respective network traffic, which tells them the IP address of the attackers’ server ( Flag 3). To retrieve more information about the attackers, the Trainee uses an IP location finder to determine the geographic location ( Flag 4) of the IP address. The trainees are informed that IP location finders sometimes provide incorrect information and that this information must therefore be carefully checked. Incident 2 - DoS Attack on CCTV Camera (Availability): The attackers want to enter the medical practice unnoticed and attack the CCTV cameras installed on-site to do so. The burglars started a Denial of Service (DoS) attack on the cameras which are connected to the IoT-Hub. Thereby, the availability of the video material is threatened. On a normal day, the camera sends the video material ten times a day via a specific IP address to the linked cloud storage. However, if the attackers send numerous requests to the camera, the requests will no longer be processed, and the camera eventually freezes or stops working. The Trainee now needs to investigate the cause of the CCTV cameras malfunction. Therefore, they use the Visual Decision-Support Tool to determine in which time period the connection load between the camera and external IP addresses rises in comparison to the traffic load before ( Flag 5). This way, the Visual Decision-Support Tool assists the decision to look further into the traffic load spike. The Trainee is provided with a LogDNA output to analyze log files in real-time. The logs are already filtered for the specific time period of a traffic spike. In the log dataset, the Trainee searches for keywords like “ERROR”/ “503” to find out or verify the root cause of the DoS attack ( Flag 6). Incident 3 - Manipulation of IoT-Hub Monitoring (Integrity): In the next step of the burglary, the attackers follow the same goal, to enter the medical practice unnoticed. Through the monitoring feature of the IoT-Hub, the status of the smart windows (open/closed) is displayed. This enables observation and control from a distance on the IoT-Hub. To enter the practice unnoticed, the burglars use the malware on the IoT-Hub to bypass the alarm system on open windows. This means, the data displayed on the IoT-Hub does not represent the actual physical status of the window, whereby the break-in via the window is not evident from the data. Thereby, the integrity of the IoT-Hub is violated. Finally, the Trainee needs to detect the file manipulation caused by the malware with the help of the Visual Decision-Support Tool and extract the manipulated file ( Flag 7). The analyzed output of the Visual Decision-Support Tool supports the decision to have a closer look at the likely manipulated file. This way, the Trainee receives a pcap-file, which then should be analyzed with the tool Wireshark. Now, the Trainee should examine the given pcap-file and search for a possibly malicious file to extract and analyze it more deeply to understand the file change or manipulation ( Flag 8).

ForCyRange: An Educational IoT Cyber Range for Live Digital Forensics

5

89

Conclusion and Future Work

To address the research question of this work, we propose ForCyRange, a conceptual design for an educational IoT cyber range focusing on LDF investigations. The concept enables the training of postgraduate cybersecurity learners in both academic and organizational contexts. We define two roles within the concept, the forensic analyst or Trainee and the Trainer. The Trainee performs an LDS investigation of an IoT system under attack by interacting with the cyber range over a web-based user interface. The Trainer is responsible for the design and implementation of the scenario and the complementing learning material. The underlying design we propose for ForCyRange consists of the four components (1) IoT System Simulation, (2) Middleware/Database, (3) Forensic Workstation, and (4) LMS, which build the technical foundation of the cyber range. These components can be flexibly configured, which allows the creation of diverse scenarios within the cyber range. To demonstrate the application of the concept and illustrate its capability, we present the scenario OutSmart-The-Burglar. In this scenario, Trainees perform an LDF investigation on a Smart Home IoT system using a set of open-source tools for LDF. The concept’s suitability regarding the transfer of LDF skills and knowledge is yet to be empirically verified. Thus, for future work, we pursue a prototypical implementation of ForCyRange with the illustrative scenario mentioned above. This prototype will be evaluated within a user study among postgraduate students and professionals. The evaluation should help determine the Trainees’ learning progress and is envisioned as a knowledge survey of the topic before and after working on the scenario in ForCyRange. Furthermore, we want to extend the concept to include a more complex teaming structure., e.g., forensic analyst teams with different tasks and tool sets who need to collaborate to perform an LDF investigation successfully. This approach could make scenarios more diverse and reinforce soft skill development within the training. Acknowledgement. This work is partly performed under the INSIST project, which is supported under contract by the Bavarian Ministry of Economic Affairs, Regional Development and Energy (DIK0338/01).

References 1. Adelstein, F.: Live forensics: diagnosing your system without killing it first. Commun. ACM 49(2), 63–66 (2006) 2. Blauw, F.F., Leung, W.S.: ForenCity: a playground for self-motivated learning in computer forensics. In: Drevin, L., Theocharidou, M. (eds.) WISE 2018. IAICT, vol. 531, pp. 15–27. Springer, Cham (2018). https://doi.org/10.1007/978-3-31999734-6 2 3. B¨ ohm, F., Englbrecht, L., Friedl, S., Pernul, G.: Visual decision-support for live digital forensics. In: IEEE Symposium on Visualization for Cyber Security, VizSec 2021, New Orleans, LA, USA, 27 October 2021, pp. 58–67. IEEE (2021) 4. Boyes, H., Hallaq, B., Cunningham, J., Watson, T.: The industrial internet of things (IIoT): an analysis framework. Comput. Ind. 101, 1–12 (2018)

90

S. Friedl et al.

5. Csikszentmihalyi, M., Csikzentmihaly, M.: Flow: The Psychology of Optimal Experience, vol. 1990. Harper & Row, New York (1990) 6. Davis, J., Magrath, S.: A survey of cyber ranges and testbeds. Technical report, Defence Science and Technology Organisation Edinburg (Australia) Cyber and Electronic Warfare DIV (2013) 7. Dorsemaine, B., Gaulier, J., Wary, J., Kheir, N., Urien, P.: Internet of things: a definition & taxonomy. In: Al-Begain, K., AlBeiruti, N. (eds.) 9th International Conference on Next Generation Mobile Applications, Services and Technologies, NGMAST 2015, Cambridge, United Kingdom, 9–11 September 2015, pp. 72–77. IEEE (2015) 8. Eckert, W.G.: Introduction to Forensic Sciences. CRC Press (1996) 9. Englbrecht, L., Pernul, G.: A serious game-based peer-instruction digital forensics workshop. In: Drevin, L., Von Solms, S., Theocharidou, M. (eds.) WISE 2020. IAICT, vol. 579, pp. 127–141. Springer, Cham (2020). https://doi.org/10.1007/ 978-3-030-59291-2 9 10. IoT Analytics: Cellular IoT & LPWA Connectivity Market Tracker (2020). https:// iot-analytics.com/state-of-the-iot-2020-12-billion-iot-connections-surpassingnon-iot-for-the-first-time/. Accessed 4 Apr 2022 11. ISACA: ISACA Report - State of Cybersecurity 2021, Part 1 (2021) 12. Kramp, T., Van Kranenburg, R., Lange, S.: Introduction to the Internet of Things. In: Enabling Things to Talk, pp. 1–10. Springer, Heidelberg (2013). https://doi. org/10.1007/978-3-642-40403-0 1 13. Leitner, M., et al.: AIT cyber range: flexible cyber security environment for exercises, training and research. In: Proceedings of the European Interdisciplinary Cybersecurity Conference. EICC 2020. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3424954.3424959 14. McKemmish, R.: What is forensic computing? Australian Institute of Criminology Canberra (1999) 15. Mishra, N., Pandya, S.: Internet of things applications, security challenges, attacks, intrusion detection, and future visions: a systematic review. IEEE Access 9, 59353– 59377 (2021) 16. National Initiative for Cybersecurity Education (NICE): The cyber range: A guide. Technical report (2020) 17. Newhouse, W., Keith, S., Scribner, B., Witte, G.: National initiative for cybersecurity education (NICE) cybersecurity workforce framework. NIST Spec. Publ. 800(2017), 181 (2017) 18. Oriwoh, E., Jazani, D., Epiphaniou, G., Sant, P.: Internet of things forensics: challenges and approaches. In: Bertino, E., Georgakopoulos, D., Srivatsa, M., Nepal, S., Vinciarelli, A. (eds.) 9th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing, Austin, TX, USA, 20–23 October 2013, pp. 608–615. ICST/IEEE (2013) 19. Rahman, M.S., Kabir, M.H.: A survey analysis and model development for Internet of Things (IoT) system for city buildings: Dhaka city, Bangladesh perspective. In: TENCON 2018–2018 IEEE Region 10 Conference, pp. 1229–1234. IEEE (2018) 20. Rose, K., Eldridge, S., Chapin, L.: The internet of things: an overview. The Internet Soc. (ISOC) 80, 1–50 (2015) 21. Stoyanova, M., Nikoloudakis, Y., Panagiotakis, S., Pallis, E., Markakis, E.K.: A survey on the Internet of Things (IoT) forensics: challenges, approaches, and open issues. IEEE Commun. Surv. Tutorials 22(2), 1191–1221 (2020) 22. Sudhakar, K.S.: An emerging threat Fileless malware: a survey and research challenges. Cybersecurity 3(1), 1 (2020)

ForCyRange: An Educational IoT Cyber Range for Live Digital Forensics

91

23. Vielberth, M., Glas, M., Dietz, M., Karagiannis, S., Magkos, E., Pernul, G.: A digital twin-based cyber range for SOC analysts. In: Barker, K., Ghazinour, K. (eds.) DBSec 2021. LNCS, vol. 12840, pp. 293–311. Springer, Cham (2021). https:// doi.org/10.1007/978-3-030-81242-3 17 24. Vykopal, J., Oslejsek, R., Celeda, P., Vizv´ ary, M., Tovarn´ ak, D.: KYPO cyber range: design and use cases. In: Cardoso, J., Maciaszek, L.A., van Sinderen, M., Cabello, E. (eds.) Proceedings of the 12th International Conference on Software Technologies, ICSOFT 2017, Madrid, Spain, 24–26 July 2017, pp. 310–321. SciTePress (2017) 25. Yamin, M.M., Katt, B., Gkioulos, V.: Cyber ranges and security testbeds: scenarios, functions, tools and architecture. Comput. Secur. 88 (2020) 26. Zawoad, S., Hasan, R.: FAIoT: towards building a forensics aware eco system for the internet of things. In: 2015 IEEE International Conference on Services Computing, SCC 2015, New York City, NY, USA, 27 June – 2 July 2015, pp. 279–284. IEEE Computer Society (2015)

South Africa’s Taxi Industry as a Cybersecurity-Awareness Game Changer: Why and How? Petrus Duvenage(B)

, Victor Jaquire , and Sebastian von Solms

University of Johannesburg, Johannesburg, South Africa {pcduvenage,vjaquire,basievs}@uj.ac.za

Abstract. The South African economy is not only the second largest in Africa, but it is also the most diversified, industrialized and technologically advanced on the continent. This technological advance is evidenced by its citizenry’s comparatively high-level of digital connectivity and interconnectedness. On the reverse side, South Africa (SA) has the third highest number of cybercrime victims internationally. Maintaining and expanding its competitive technological advantage requires of South Africa to have a robust national cybersecurity endeavour. This needs to include an innovative, high-impact cybersecurity awareness campaign that effectively reaches a diverse population. This paper’s primary aim is to propose such a high-impact drive, namely a broad-based national cybersecurity awareness campaign that levers the South African minibus taxi industry. The paper’s three objectives pertain to the ‘why’ and ‘how’ of such a campaign. The paper’s first objective is to substantiate the need for a broad-based cybersecurity awareness campaign (in short: why is it needed?). The second objective is to substantiate why the taxi industry constitutes an optimal platform for a game changing campaign. Thirdly, the paper advances a proposition on a Taxi Industry Cybersecurity Awareness Campaign (TICAC) (i.e. how can the taxi industry be a game changer?). We qualify the TICAC as a tentative, high-level conceptual proposition subject to much further research on the theoretical/academic and practical levels. It is hoped that the paper would be of value to also other countries – developed and developing – in utilising private and/or public transport industries as platforms for cybersecurity awareness initiatives. Keywords: Cyber security awareness · Public transport · Minibus taxi industry · Developing countries · Wi-Fi · South Africa

1 Introduction With COVID-19 dominating the headlines throughout 2020 and 2021, “the consistent development and deployment of information and communication technology (ICT) infrastructure and its concomitant services has meant a continued trend towards digital transformation for societies, businesses and governments alike” [1]. Individuals and © IFIP International Federation for Information Processing 2022 Published by Springer Nature Switzerland AG 2022 L. Drevin et al. (Eds.): WISE 2022, IFIP AICT 650, pp. 92–106, 2022. https://doi.org/10.1007/978-3-031-08172-9_7

South Africa’s Taxi Industry as a Cybersecurity-Awareness Game Changer

93

institutions are moving more and more of their existential activities online and at a faster rate. The pandemic has unquestionably deepened digital reliance. The pandemic has not only accelerated digital reliance, but also the concurrent exploitation of the cyber sphere by cyber criminals and other malicious actors. Trend Micro, for example, estimates that 2021 witnessed a 47% year-on-year increase in cyber threats internationally [2]. Research cited by Interpol [3] found that in 2021 cybercrime reduced Gross Domestic Product (GDP) of Africa countries by more than 10% and at projected cost of an estimated 4.12 billion USD. Consequently, cybersecurity has further expanded its presence as a priority item on the agendas of governments and enterprises [4]. In terms of technology and governance, ‘zero trust’ models and practices are now on the forefront [5]. Technology and governance alone are of course not the sole solution. Humans as part of the ecosystems remain the weakest link in cybersecurity. Insecure cyber practices by individuals not only pose a risk to themselves but also the institution(s) they work for or otherwise digitally interact with. This has been amplified by the COVID-era’s ‘new normal’. Against this backdrop, the already vibrant interest in interdisciplinary cybersecurity awareness research has been boosted further. Academic research into innovative, fit-for-purpose cybersecurity awareness mechanisms is particularly pertinent. In South Africa, like the rest of Africa, there is a particular need for game changing initiatives that can increase broad-based cybersecurity awareness [6]. Public transport, this paper contends, could offer a useful platform for cybersecurity awareness. However, academic research on leveraging public transport for cybersecurity awareness in general remains limited. Therefore, this paper’s primary aim is to substantiate and propose a broad-based national cybersecurity awareness campaign that leverages the South African minibus taxi industry. For ease of reference, this initiative is referred to in the paper as TICAC (Taxi Industry Cybersecurity Awareness Campaign). As suggested by its title, the paper’s three objectives pertain to the ‘why’ and ‘how’ of such a campaign. These objectives are to: • Substantiate the need for a broad-based cybersecurity awareness campaign (in other words: why is such a campaign needed?). • Motivate the assertion of the taxi industry being an optimal platform for the campaign (i.e. why the taxi industry can be a game changer?). • Advance a proposition on Taxi Industry Cybersecurity Awareness Campaign (TICAC) - i.e. how the taxi industry can be a game changer? We qualify the TICAC as a tentative, conceptual proposition subject to further research and not in any a roadmap for implementation. While the paper utilises South Africa as a case study for such an initiative, it would also be of interest to practitioners and academics of other countries. In tabulated format, the structure of the paper - linked to the objectives - is depicted in Table 1 (on the next page).

94

P. Duvenage et al. Table 1. Paper objectives and structure

Objective 1: Substantiate why there is a need for a broad-based cybersecurity awareness campaign Section 2

The critical need for innovative, broad-based cybersecurity awareness initiatives in South Africa

Objective 2: Motivate why can the taxi industry can be a game changer Section 3

Requirements to which a broad-based cybersecurity awareness campaign in South Africa should comply to be effective

Section 4

Reasons why the South African minibus taxi industry with WiFi offering could be a game changer

Objective 3: Outline how the taxi industry can be a game changer Section 5

International research on the use of public transport for cybersecurity awareness

Section 6

Proposal on a Taxi Industry Cybersecurity Awareness Campaign (TICAC)

Section 7

Security considerations that should be considered in the rolling out of TICAC

This section contextualised the paper and presented its aims, objectives and structure. In accordance with the structured outlined above, the next section seeks to respond to the question: Why is there a critical need for a broad-based cybersecurity awareness campaign in South Africa?

2 The Need for Innovative, Broad-Based Cybersecurity Awareness Initiatives in South Africa As was noted in the preceding section, the acceleration of certain trends in the cyber threat landscape has led to an even higher prioritisation of cybersecurity awareness internationally. While globally important, there is a particular acute need for an innovative broad-based cybersecurity awareness in South Africa. To contextualise, substantiate and unpack this assertion, this section is structured as follows: • Sect. 2.1 shows South Africa as a continental leader in terms of technological advancement and digital connectivity. Although these are positives, these factors also increase the attack surface vulnerable to exploitation. • In overviewing the South African cyber treat landscape, Sect. 2.2 contends that these vulnerabilities are exploited by malicious actors and that South Africa is indeed highly cyber insecure and targeted. • Sect. 2.3 shows that South Africa’s cyber insecurity to a substantial degree relates to certain deficiencies in its national cybersecurity maturity (of which cybersecurity awareness is an important pillar). • Sect. 2.4 specifically appraises and identifies some shortcomings in cybersecurity awareness in South Africa.

South Africa’s Taxi Industry as a Cybersecurity-Awareness Game Changer

95

Flowing from these four Subsects. (2.1–2.4), we conclude Sect. 2 with asserting the need for fit-for-purpose, broad-based awareness cybersecurity initiatives in South Africa. 2.1 South Africa as a Technologically Advanced and Connected Continental Leader South Africa’s economy is not only the second largest in Africa, but it is the most diversified, industrialized and technologically advanced on the continent [7]. Overall it is also the most technologically innovative and leads the continent in fields such as AI, and IoT [1, 8]. This accelerating technological advance is further evidenced by the sophistication of South Africa information and communication sector as well as its population’s comparatively high-level of digital connectivity, interconnectedness and uptake of digital devices. The following attest to this digital connectivity and interconnectedness: • South Africa’s active mobile broadband subscriptions (102 per 100 inhabitants) not only tops that of African countries but is far in excess of the world average of 75 per 100 inhabitants [1]. • South Africa is one of only four African countries where the proportion of individual Internet users in 2020 was above the world average of 51.4% [9]. • It is estimated that in 2021, 36.45 million South Africans accessed the internet through mobile devices [9]. • South Africa (with Mauritius and the Seychelles) are leading the e-government ranking in Africa [1]. 2.2 Cyber Insecurity and the South African Threat Landscape Although interconnectivity and technological advancement generally are positives, such factors could (for reasons discussed in Sect. 2.3), also have a negative reverse side. As is clear from the following statistic, this is the case in that South Africa: • Has the third highest number of cybercrime victims internationally [10]. • In 2021 was found to be the 24 most cyber insecure of 75 countries surveyed by Comparitech [11]. While South Africa’s risk score incrementally improved from in 2019, South Africa is sliding further down the list, “meaning it is not keeping up” [11]. • In 2020 had a 100% annual increase in mobile banking application fraud and is estimated to have experienced 577 malware attacks an hour [3]. • Was recorded by Trend Micro to have had 230 million threat detections from January 2020 to February 2021 - nearly 200% more than the next worst affected African country of Kenya [3]. • Is one of the top ten countries “globally where threat actors received the highest volume of cryptocurrency from illicit addresses” [3].

96

P. Duvenage et al.

2.3 Assessing South Africa’s National Cybersecurity Maturity Maintaining and expanding its competitive advantage (outlined in Sect. 2.1) would require of South Africa to have a robust national cybersecurity endeavor. However, the concise overview of the South Africa threat landscape (Sect. 2.2) suggests that this is not the case. This is also evidenced by South Africa’s ranking in the ITU’s 2020 Global Cybersecurity Index (GCI) report [1]. The ITU [4] assessed national cybersecurity across five pillars, namely (i) legal measures, (ii) technical measures, (iii) organisational measures, (iv) capacity development measures and (v) cooperation measures. The ITU found South Africa to be substantially lagging its peers in its national cybersecurity effort [4]. In the 2020, the ITU [4] ranked South Africa 59th globally and only 8th in Africa. The ITU [1] graphically depicts its assessment of South Africa national cybersecurity maturity as follows:

Fig. 1. South Africa’s national cybersecurity maturity [1]

Moving from the wider context provided in the preceding subsections (Sect. 2.1 to 2.3), the next section more specifically examines the status of cybersecurity awareness in South Africa. This is done in line with Sect. 2’s aim to substantiate and explicate the need for initiative broad-based cybersecurity initiatives in South Africa. 2.4 Defining and Appraising South Africa’s Public Cybersecurity Awareness (as Part of ‘National Cybersecurity’ Maturity) “Effective cybersecurity awareness raising is essential in keeping citizens, businesses, governments, youth, and organizations alert. With the current shift to digital services, governments need to ensure that all users are aware of the risks they face while carrying out digital activities” [4]. Cybersecurity awareness is of course a crucial element of national cybersecurity and accordingly features so in the cybersecurity maturity models of inter alia the ITU and Oxford University. The ITU [4] positions “cyber security awareness” as a subset of

South Africa’s Taxi Industry as a Cybersecurity-Awareness Game Changer

97

“capacity development”. Phrased differently, “cyber security awareness” is one of the main factors used in assessing “capacity development”. As indicated in in Fig. 1 above, South Africa performs poorly in “capacity development”. As far as could be ascertained from consulted academic literature, there is no recent survey or comprehensive analysis of specifically the status of public cybersecurity awareness in South Africa. From available public sources, the following can be surmised: • There is no broad-based (public) cybersecurity awareness programme centrally coordinated or driven by government. • Several entities in government and the private sector are engaged in cyber security awareness initiatives but these are directed at specific target groups and are not centrally coordinated. • Corporate entities in general are committed to cybersecurity awareness in respect of their workforces. A Mimecast’s State of Email Security 2020 report, for example, found that 99% of South African respondents offer security awareness training [12]. • Existing cybersecurity awareness initiatives in South Africa, however, are not effective and/or do not have a sufficiently broad impact. According to Mimecast [12], to illustrate, “half of South African employees surveyed” admitted to opening emails they considered suspicious.” • South African Internet users “are [comparatively] inexperienced and less technically alert than users in other nations’ [10]. When the effectiveness of South Africa cybersecurity public awareness campaigns is contrasted against Internet penetration, the country falls out in an intersection of high concern, namely “high Internet penetration” but deficient and insufficient “public cybersecurity awareness campaigns” [4].

3 Requirements for Effective Broad-Based Cybersecurity Awareness Campaign (in the South Africa Context) The preceding section indicated the need in South Africa for an effective public (broadbased) cybersecurity awareness campaign. To be a game changer, this cybersecurity awareness campaign should duly consider the requirements to which this campaign should comply. Therefore, this section aims to derive such requirements. Dlamini & Modise [13] rightly state that there is no single fit-for-all cybersecurity awareness campaign. Instead, a broad-based public cybersecurity awareness campaign should be specifically designed by means of a process that has the following six elements [14]: “Cybersecurity awareness goals and objectives, Identify intended audience, Define topics to be covered, Define delivery methods to be used, Develop a strategy for rollout, and Develop evaluation methods.” Although all of the above cited elements are informative, this paper was qualified as not being a roadmap for implementation. Instead the paper limits itself to deriving the requirements to which the envisaged campaign should comply. No postulations could be found in consulted literature on requirements to which a broad-based cybersecurity awareness campaign, comparable to the one envisaged in the paper, should comply.

98

P. Duvenage et al.

Therefore, the six requirements discussed in the next paragraph were identified through inductive reasoning that is specific to South Africa’s political and socio-economic context and the nature of the envisaged campaign. In addition to considering the above-mentioned six elements of a cybersecurity awareness campaign [13], we also consider aspects such as the South African political and socio-economic realities and its national cybersecurity posture to derive requirements. Based on our assessment of the noted elements and aspects, the inter-related requirements to which the cybersecurity awareness campaign should comply are as follows: 1) The broadest feasible target group should be reached and aimed at the spectrum of the population which stands to benefit the most. 2) Because the target audience should be broad, the campaign should allow for some segmentation (i.e. be flexible in that it allows for some variation in topics and pitch-level to different audience sub-groups (such as scholars, students, blue collar workers, office workers and manual labourers). 3) The delineation of the target group should consider socio-economic realties such as South Africa’s highly unequal income distribution and digital divide (See Sect. 4). 4) An innovative delivery method(s) and platform(s) capable of reaching a broad audience should be used. 5) The delivery method and strategy should incentivise the target audience’s engagement. 6) The campaign should be cost effective and not be solely rely on government for financing. Building on the requirements to which a broad-based cybersecurity awareness campaign should comply, the next section examines why the South African minibus taxi industry could be a game changer.

4 Why the South African Minibus Taxi Industry with a WiFi Offering Could Be a Game Changer This section appraises why the South African minibus taxi industry as platform for a broad-based cybersecurity awareness campaign. Phrased differently, this section substantiates our contention that there are substantial grounds for our assertion that the taxi industry (with a WiFi offering to commuters) could change the cybersecurity awareness landscape. To this end we firstly demarcate the preferred (cyber security awareness campaign) target group and, secondly, describe the taxi industry as a potential platform for delivering the campaign. As was noted in Sect. 3, the envisaged cyber security awareness campaign has, as first requirement reaching the broadest population group feasible. In addition, requirement 3 pertains to South African techno and socio-economic realities. Based on these requirements we contend that a broad-based campaign should favour lower-middle and low income groups as well as those with no direct income. (e.g. scholars, the unemployed, jobseekers and the poor). These groups are henceforth collectively referred to in the paper as the ‘target group’.

South Africa’s Taxi Industry as a Cybersecurity-Awareness Game Changer

99

According to a 2019 estimate, this target group comprises 76% of the South African population [15]. The overwhelming part of the target group does not have fixed affordable Internet. It is estimated that only 10% of South African homes have fixed affordable Internet and that low income earners could pay up to 80 times more for Internet access than better earning citizens [16]. However, by virtue of smart phone ownership, most of the target group has a device to access the Internet. A recent Deloitte’s [17] survey shows that smart phone possession in all income groups within South African exceeds 90%. Smart phone ownership in the lowest income group stands at 94% [17]. The relatively high cost of data, however, is a barrier to the target group accessing the Internet. While some cities, towns, communities and institutions host free WiFi services these offerings are predominantly localised and by no means service the whole of the target group. Consequently, there is a dire need in South Africa for low cost or free WiFi accessible to the majority of the population. The target group, as described above, is very reliant on public transport in the form of minibus taxis (hereafter refer to as the ‘taxi industry’). This industry’s estimated 200 000 taxis accounts for “75% of all daily transport – about 15 million commuter trips daily to work, schools and universities, to access healthcare or for leisure” [18]. In addition to local services, longer-distance services are also provided. The majority of commuters favours taxis, over less expensive buses and trains, because they deliver a more effective and efficient service [18]. For the most part, taxi operators belong to taxi associations [19]. Those associations, which are granted certain routes by South African government authorities, have formal leadership structures [19]. Such associations exist on the national, provincial, regional and local/primary taxi level. While the taxi industry and some associations have been marred by (sometimes violent and deadly) competition, these associations nonetheless provide nodal points of engagement with the industry on TICAC. To the best knowledge of the authors, there currently are no coordinated free Wifi offerings that use the taxi industry as platform. Consulted literature made no mention of such offerings. With the qualification that it is a supposition to be tested with further research, we assume current WiFi offerings of any kind by the minibus taxi industry in South Africa to be at best very rare. As far we could ascertain there is a near void in this regard. Linking a Free WiFi offering (with imbedded cybersecurity awareness campaign elements) would thus constitute an innovative proposition incentivising both industry participation and target audience’s engagement. Once again with the qualification that it is a contention requiring further research, the combination of the tax industry’s expected buy-in and the envisaged high-level of uptake by a broad target audience would present attractive advertising prospects to profit-driven business entities. Practically, such advertising (by for example cellular service providers) could take the form of at least the partial sponsoring of data and devices. While private sector sponsoring alone would not necessarily fully fund the envisaged cybersecurity awareness campaign, it could contribute significantly to its cost-effectiveness and affordability. With government’s financial support a free WiFi offering would be realistically achievable. Based on the foregoing analysis of some potential role-players (taxi industry and business entities) and the target group, it can be concluded that the taxi industry with

100

P. Duvenage et al.

a sponsored, free WiFi offering could potentially be a cybersecurity awareness game changer. Measured against the criteria stated in Sect. 3, the proposal on the taxi industry’s utilisation thus satisfies requirements 1, 3, 4 and 6. The proposition we advance in Sect. 6, will show the proposal to also meet requirements 2 and 5 (as stated in Sect. 3). Since it could be of value in the design of our proposition on the taxi industry as a game changer, the next section reviews international research of possible relevance in the use of public transport for cybersecurity awareness.

5 International Experience: The Use of Public Transport for Cybersecurity Awareness The previously stated aim of the literature review was to identify published academic research with regard to the utilisation of public transport for cybersecurity awareness. For this purpose, a search with the string “cybersecurity awareness through mobile platforms on public transport” was done on the following platforms: Scopus, EBSCO, Institute of Electrical and Electronics Engineers (IEEE) Explore, Springer Link, and Google Scholar. The addition of further search terms such as “taxi(s)” and “minibus taxi/s”, and the utilisation thereof in various combinations - with the terms cited in the section’s introductory sentence – also rendered very limited existing research directly relevant to TICAC’s configuration. In fact, at the time of the literature review, no published works could be found that expressly note the implementation and/or drive of cybersecurity awareness initiatives through mobile platforms utilising the public transport/taxi industry as a focus area for awareness content delivery. However, broadening the search parameters by adding the words “digital” and “advertising” to the search string, yielded numerous publications with regard to the successful utilisation of mobile advertising and other digital displays. These publications focus on past and current research on advertising methods, ideas and toolsets utilising mobile platforms within public transport. In respect of digital advertising, Evans, Moore & Thomas [20] found that the “rapid expansion of wireless technologies has provided a platform to support intelligent systems in the domain of mobile marketing”. Qin, Zhu, Lu, Xue & Li [21], as a further example, researched the optimisation of mobile advertising in “vehicular networks” such as taxis and busses. Using “extensive simulations” based on “three real data sets of taxi and bus traces” Qin et al. [21] assert that certain mobile methods/tools can “greatly improve the coverage and the intensity of advertising”. It stands to reason that a mobile cybersecurity awareness campaign shares similarities with a mobile platform advertising campaign. Therefore, it is highly plausible that some mobile advertising’s methods and toolsets can be adapted to a taxi-based cybersecurity awareness campaign in South Africa. This will require much further research and development - which falls outside the paper’s aim to advance a high-level conceptual proposal. To summarise, while the literature review did not find existing research on the use of public transport for cyber security awareness per se inferring from mobile advertising research, it does however support the feasibility of a based cybersecurity awareness campaign using a mobile platform.

South Africa’s Taxi Industry as a Cybersecurity-Awareness Game Changer

101

6 How Can the South African Minibus Taxi Industry Be a Game Changer? Building on the preceding sections, we now proceed to advancing a high-level, conceptual proposal on how the South African minibus taxi industry could be utilised as a game changing broad-based, mobile cybersecurity awareness campaign. The aim is to provide contours that can guide further research and the engagement of role players. Given its nature (high-level concept) and aim, the proposal is generic and does not mention specific government institutions or private sectors entities. Our proposal is premised on the necessity for functional public private partnerships. As with national cybersecurity in general, the Taxi Industry Cybersecurity Awareness Campaign (TICAC) needs to a joint venture between government and the private sector (taxi industry, content/application developers and other business entities - notably cellular services providers). TICA’s success will depend on synergy between these roleplayers in taking up the opportunity to drive, implement and mature this cybersecurity awareness initiative. In short, the South African minibus taxi industry can only be game changer through a collective effort involving all four categories of role players . The four categories of role-players and some tentative ideas on the respective functions, is graphically depicted in Fig. 2 on the next page below.

Fig. 2. High-level outline of a Taxi Industry Cybersecurity Awareness Campaign (TICAC)

The core concept, as per Fig. 2, can be expanded with a combination of the following aspects:

102

P. Duvenage et al.

• Sponsored secure WiFi offerings to commuters: • Providing an extra enticement (in the form of free data), as an incentive to utilising their taxi service, • Ensuring a platform through which a large contingent of the (online) population can be engaged to receive awareness content w.r.t. cybersecurity. • Sponsored secure content development (such as films, animation or games): • Primarily for the utilisation on mobile platforms within the taxi industry for cybersecurity awareness purposes, • Once the initial infrastructure is established and implemented, the bouquet of content can be expanded and/or matured to free education and training content for students on the move. • Sponsored secure delivery mechanism development, such as: • Secure web based login before data or internet services can be utilised by a commuter, • Mobile device vulnerability assessment and free security implementation options to commuters, and • Continuous threat monitoring and mitigation. • Government support services, such as: • Policy development that could include encouragements for participating companies, such as tax and/or other incentives, • Coordination, • Assistance with, and or funding of content development, • Impact/success rate assessment, and • Client feedback. Our research found several existing solutions (in a fast growing list of applications and services) that are currently available and utilised for mobile (and other) cybersecurity education and awareness purposes. A study conducted by Zang-Kennedy & Chiasson [22] identifies 119 tools, 54 digital games as well as 34 films and animation for “educating users about cybersecurity and privacy”. The same study found digital games to be the most widely used type of tools and identified five web-based games that are also available as mobile applications. Which of these solutions would be most suitable to the South African context would of course require further research. This section outlined a proposition on TICAC and observed on applications and services available and utilised for mobile (and other) cybersecurity education and awareness purposes. The next section draws on existing research to identify some initial security considerations that should be noted in configuring TICA.

South Africa’s Taxi Industry as a Cybersecurity-Awareness Game Changer

103

7 Initial Security Considerations Security in the design, implementation, operation and maturing of TICAC is of course of paramount importance. Implementing such a campaign in an unsecure manner will in a very real sense defeat the purpose of cybersecurity awareness. The broadening of our literature review to public WiFi security and the maturing of cybersecurity awareness campaigns more generally, did render aspects of value. This being a high-level proposal, only brief reference can be made to the following three inter related security aspects for consideration in configuring TICAC, namely (i) the socio-technical nature of cybersecurity, (ii) WiFi networks’ vulnerability to exploitation and the cybersecurity paradox, and (iii) the need for TICAC’s progressive maturing. 7.1 Socio-Technical Nature of Cybersecurity Countless studies and incidents have showed humans to be the weakest link in the cybersecurity chain [23, 24]. This underlines the concern that it cannot be left up to TICA’s target audience to securely connect to the convenience of free Wifi/data typically trump security considerations. Breitinger [24], for example, found that within a specific area 78.7% of people not only access public unsecured WiFi networks, but did so means of automatic connection. As aptly phrased by Leuprecht, Skillicorn, & Tait [23] “The way humans and technology interact, blurs and dissolves the concepts of being ‘inside’ or ‘outside’ a cybersecurity space”. Even more so because it is intended for a developing country with deficient cybersecurity awareness (see Subsect. 2.4), TICAC’s secure design and implantation should therefore duly consider the human factor and the status of South Africa’s cybersecurity awareness. 7.2 WiFi Networks’ Vulnerability for Exploitation and the Cybersecurity Paradox Public WiFi is not only particularly vulnerable to exploitation by adversaries (such as criminals), but can also be used by governments for surveillance purposes [25, 26]. In this regard, Leuprecht, Skillicorn & Tait [24] observe on the “paradox” in that “governments want to ensure cybersecurity, but at the same they want access to the data of individuals and organizations for surveillance purposes”. In expanding this paradox “governments want companies and citizens to protect themselves, but … do not want them to use encryption and other cybersecurity measures, as this might allow terrorists and criminals to hide their traces” [24]. Consequently, “governments thus often attempt to balance good and evil by allowing encryption but requiring backdoors” to remotely access the encrypted devices. Such backdoors can also be exploited by others and merely shift cybersecurity threats from the front door elsewhere” [24]. Since the TICAC is essentially a joint venture between government and the private sector - with a free public WiFi offering at its core - addressing the security challenges noted in this paragraph is imperative but complex.

104

P. Duvenage et al.

7.3 The Need for TICAC’s Progressive Maturing TICAC’s secure and success implementation will depend on the following of a wellformulated maturity process involving all role-players. Oxford CMMs’ [1] five stages of maturity for building cybersecurity awareness serve as an example. These stages range from “Start-up” scenario where the need for cybersecurity awareness “is not recognised or is only at initial stages of discussion”; to the “Dynamic” Scenario in which a country “creates new regional/international cybersecurity awareness-raising programmes that contribute to …. international awareness-raising.” By highlighting initial security considerations, this section completed our high-level outline of TICAC. The next section concludes the paper by summarising our findings and observing on areas for further research.

8 Conclusion This paper examined the South Africa’s taxi industry as a potential cybersecurityawareness game changer. The research we conducted resulted in three primary findings. Firstly, there is a dire need for a broad-based cybersecurity awareness campaign in South Africa. Secondly, the taxi industry was found to be indeed a potential game changing platform (for the rollout of a broad-based cybersecurity awareness campaign). Thirdly, we found that a TICAC will have to have, at its core, a joint venture between government and the private sector (taxi industry, cellular services providers, other business entities and content/application developers). The high-level proposition on TICAC we then advanced was qualified as a tentative, conceptual outline requiring much further research. As an immediate priority, such further research will be aimed at determining the views of identified role-players on the feasibility and configuration of a TICAC (See Fig. 2). A related immediate priority research priority relates to the difficulties and challenges pertaining to the government-private sector partnership so vital to TICAC’s success. It is imperative that these challenges and viable solutions be identified and described. In addition, further research will focus on identifying cyber security awareness tools and applications most suitable to a TICAC. Taking a broader view, the pertinence and viability of a TICAC in developing countries, other than South Africa, presents another promising research area.

References 1. International Telecommunication Union (ITU): Global Cybersecurity Index 2020. https:// www.itu.int/epublications/publication/global-cybersecurity-index-2020/en/. Accessed 11 Nov 2021 2. BusinessTech: South Africa is a playground for cyber criminals – here are the scams to be aware of (2021). https://businesstech.co.za/news/technology/531640/south-africa-is-a-playgr ound-for-cyber-criminalsf/. Accessed 29 Nov 2021 3. Interpol: The African Cyberthreat Assessment Report 2021 (2021). https://www.interpol. int/content/download/16759/file/AfricanCyberthreatAssessment_ENGLISH.pdf. Accessed 9 Dec 2021

South Africa’s Taxi Industry as a Cybersecurity-Awareness Game Changer

105

4. ITU: Digital trends in Africa 2021: Information and communication technology trends and developments in the Africa region, 2017–2020 (2021). https://www.itu.int/pub/D-IND-DIG_ TRENDS_AFR.01-2021. Accessed 11 Nov 2020 5. Microsoft: New insights on cybersecurity in the age of hybrid work (2021). https://www.mic rosoft.com/security/blog/2021/10/27/new-insights-on-cybersecurity-in-the-age-of-hybridwork/. Accessed 2 Dec 2021 6. Bada, M., Von Solms, B., Agrafiotis, I.: Reviewing national cybersecurity awareness in Africa: an empirical study. In: 3rd International Conference on Cyber-Technologies and Cyber-Systems, Athens, Greece (2018) 7. Olalelec, D.: Top 10 leading African countries in Global Innovation Index 2020. Techeconomy (2020). https://techeconomy.ng/2020/09/top-10-leading-african-countries-in-global-inn ovation-index-2020/. Accessed 13 Oct 2021 8. Chigozi, E.: The Top Ten Most Technological Advanced Countries in Africa. AnswersAfrica (2020). https://answersafrica.com/top-10-most-technologically-advanced-countriesin-africa.html. Accessed 1 Dec 2021 9. Statista: Number of mobile internet users in South Africa from 2016 to 2026 (2021). https://www.statista.com/statistics/558867/number-of-mobile-internet-user-in-south-africa/. Accessed 11 Dec 2021 10. Accenture: Insight into the cyber threat landscape in South Africa (2020). https://www.acc enture.com/_acnmedia/PDF-125/Accenture-Insight-Into-The-Threat-Landscape-Of-SouthAfrica-V5.pdf. Accessed 9 Oct 2021 11. Bischoff, P.: Which countries have the worst (and best) cybersecurity? CompaSystems (2021). https://www.comparitech.com/blog/vpn-privacy/cybersecurity-by-cou ntry/. Accessed 29 Nov 2021 12. Mimecast: State of Email Security Report – 2021 (2021). https://www.mimecast.com/global assets/documents/ebook/state-of-email-security-report-2021.pdf. Accessed 12 Dec 2021 13. Dlamini, Z., Modise, M.: Cyber security awareness initiatives in South Africa: a synergy approach. In: 7th International Conference on Information Warfare and Security, University of Washington, Seattle, USA (2012) 14. Mashiane, C.T., Dlamini, I.Z., Mahlangu, T.: A rollout strategy for cybersecurity awareness. In: Proceedings of the 14th International Conference on Cyber Warfare and Security, Stellenbosch, Cape Town, South Africa (2019) 15. Mamacos, E.: Is-your-family-poor-middle-class-or-rich. News 24 (2019). https://www. news24.com/parent/family/relationships/is-your-family-poor-middle-class-or-rich-take-thistest-and-find-out-20190821. Accessed 12 Dce 2021 16. Malaba, K.: How is South Africa’s digital divide making inequality worse in the country? Global Citizen (2021). https://www.globalcitizen.org/en/content/south-africa-digital-dividemakes-inequality-worse/. Accessed 2 Jan 2022 17. Deloitte: Global Mobile Consumer Survey 2019- South Africa Cut. https://www2.del oitte.com/content/dam/Deloitte/za/Documents/technology-media-telecommunications/zaGMCS-2019-final-report.pdf. Accessed 30 Nov 2021 18. Fobosi, S.: South Africa’s minibus taxi industry has been marginalised for too long. https:// theconversation.com/south-africas-minibus-taxi-industry-has-been-marginalised-for-toolong-this-must-change-142060. Accessed 12 Dec 2021 19. Daily Maverick: Cocked and 100% loaded: Taxi industry calls the shots (2020). https:// www.dailymaverick.co.za/article/2020-07-14-cocked-and-100-loaded-taxi-industry-callsthe-shots/. Accessed 11 Nov 2021 20. Evans, C., Moore, P., Thomas A.: An intelligent mobile advertising system (iMAS): locationbased advertising to individuals and business. In: 6th International Conference on Complex, Intelligent, and Software Intensive Systems (2012). https://doi.org/10.1109/CISIS.2012.24

106

P. Duvenage et al.

21. Qin, J., Zhu, Y., Zhu, H., Lu, L., Xue, G., Li, M.: POST: exploiting dynamic sociality for mobile advertising in vehicular networks. IEEE Trans. Parallel Distrib. Syst. 27(6), 1770–1782 (2016). https://doi.org/10.1109/TPDS.2015.2467392 22. Zhang-Kennedy, L., Chiasson, S.: A systematic review of multimedia tools for cybersecurity awareness and education. ACM Comput. Surv. 54, 1 (2020). https://doi.org/10.1145/3427920 23. Leuprecht, C., Skillicorn, D., Tait, V.: Beyond the castle model of cyber- risk and cybersecurity. Gov. Inf. Q. 33(2), 250–257 (2016). https://doi.org/10.1016/j.giq.2016.01.012 24. Breitinger, F., Tully-Doyle, R., Hassenfeldt, C.: A survey on smartphone user’s security choices, awareness and education. Comput. Secur. 88, 101647 (2020) 25. Dunning, J.P.: Taming the blue beast: a survey of Bluetooth based threats. IEEE Secur. Priv. 8, 20–27 (2010). https://doi.org/10.1109/MSP.2010.3 26. Fuller, J.: How bluetooth surveillance works. How Stuff Works (2008). https://electronics. howstuffworks.com/bluetooth-surveillance.htm. Accessed 5 Jan 2022 27. Oxford Global Cybersecurity Centre: Cybersecurity Capacity Maturity Model for Nations (CMM) (2021). https://gcscc.ox.ac.uk/the-cmm. Accessed 13 Jan 2022

Maritime Cyber Threats Detection Framework: Building Capabilities Georgios Potamos1(B)

, Savvas Theodoulou1 , Eliana Stavrou2 and Stavros Stavrou1

,

1 Open University of Cyprus, Nicosia, Cyprus {georgios.potamos,savvas.theodoulou}@st.ouc.ac.cy, [email protected] 2 Applied Cyber Security Research Lab, University of Central Lancashire Cyprus, Larnaca, Cyprus [email protected]

Abstract. In recent years, attackers have shifted their attention in the Maritime domain, exploiting vulnerabilities of Maritime IT/OT systems and human resources, and impacting the situational picture onboard the ships and/or at shore infrastructures. Therefore, developing human skills and systems’ detection capabilities in the Maritime domain are a critical element to effectively manage the cyber risks related to different types of ships and/or Maritime infrastructures. Such capabilities can greatly contribute to developing a Maritime Cyber Situational Awareness, which can promote the Maritime domain key mission objectives, such as maintaining preparedness, safety, and security. The complexity of the Maritime environment poses a great challenge in detecting cyber threats and/or anomalies in the behavior of systems, due to the different ship-to-shore systems that form this environment. Not being able to detect cyber threats or detect them early enough can affect the effectiveness of decision-making and impact the mission objectives of the Maritime domain. This work investigates detection aspects in the Maritime domain and contributes towards a novel Maritime cyber threat detection framework, to guide the development of relevant human skills and systems’ cyber threat detection capabilities. A highlight of this work is the development of a Maritime attack matrix based on MITRE ATT&CK matrices, to clearly specify the cyber threats related to the Maritime surveillance and navigation systems. Keywords: Cyber threats detection · Maritime cybersecurity · Cybersecurity training · Maritime cyber situational awareness

1 Introduction Over the last few years, the Maritime cyber threat landscape has greatly expanded, giving the opportunity to attackers to modify the situational picture onboard ships and/or at shore infrastructures [1], by changing the maneuvering behavior of ships, preventing the detection of malicious actions and causing data leakage. The attack surface in © IFIP International Federation for Information Processing 2022 Published by Springer Nature Switzerland AG 2022 L. Drevin et al. (Eds.): WISE 2022, IFIP AICT 650, pp. 107–129, 2022. https://doi.org/10.1007/978-3-031-08172-9_8

108

G. Potamos et al.

the Maritime environment contains navigational, surveillance Informational and Operational Technology (IT/OT) and Industrial Control Systems (ICS) [2]. These systems have expanded the attack surface in the Maritime domain, due to the digitalization, the automation, and the integration of systems between ship-to shore. The ship-to-shore data exchange is feasible through the satellite and/or radio data links [3], which are able to provide the required bandwidth to support the exchange of real time data [4]. As a result of the improved communications between ships and shore, and the expanded cyberattack surface, attackers may perform several types of attacks (e.g., malware, Manin-the-Middle, spoofing, Denial of Service) exploiting vulnerabilities on both IT/OT systems/networks of the ship and/or the Maritime infrastructures (e.g., ports, energy transportation infrastructures, operation centers). Detecting cyber threats and the relative anomalies of systems in the Maritime environment (onboard/ashore) is a complicated and challenging task due to the different shipto-shore systems that form this environment. To achieve effective detection of incidents, the security operator shall combine the existing information received by the network and information computing systems, and the information from the interaction of the operational technology systems with the metadata collected from the surveillance systems that are used to produce the Maritime Situational Awareness (MSA). In addition to the need to develop mechanisms and tools to automate the detection of Maritime incidents, it is equally important to focus on the human aspect and train the personnel across all levels of the Maritime workforce hierarchy. Currently, existing work covering the real time detection of cyberattacks in the Maritime context [5, 6] is limited. This poses a great challenge in upskilling the personnel to be able to detect Maritime-related security incidents as they need to have a clear view of many aspects, such as the Maritime cyber threat landscape, the systems involved, be aware of detection technologies and practices, etc. This research focuses on the human and technological aspects for the detection of Maritime cyber threats, proposing a Maritime cyber threat detection framework to build relevant capabilities. The proposed framework can be perceived as a roadmap to build a strong human capacity that is able to effectively detect and handle cybersecurity incidents. It is envisioned that the framework will be utilized to build relevant curricula and run training exercises over appropriate training infrastructures. Section 2 briefly discusses existing work. Section 3 presents the proposed framework, profiling the Maritime domain, presenting the cyber threat landscape, and specifying core capabilities that need to be developed across the Maritime workforce. Section 4 presents a training activity that was designed taking into consideration the proposed framework. Finally, Sect. 5 provides conclusions.

2 Literature Review Detection of cyber threats should be a high priority for every Maritime organization [6]. In this context, human aspects are of great concern. To this end, cybersecurity training of all stakeholders in the shipping industry [7] should be pursued, due to the necessity

Maritime Cyber Threats Detection Framework: Building Capabilities

109

to increase the cyber risk awareness and preparedness, by developing material and practical exercises to build adequate skills [8]. For this purpose, several studies describe the training methods and aspects, covering most of the cybersecurity functional areas [9]. However, there are limited directives with respect to human resources and how to build competencies to detect cyber threats against the Maritime IT/OT systems. Cybersecurity competency frameworks have been proposed in the context of several EU pilot projects (FORESIGHT, SPARTA, Cyber4Europe, CONCORDIA and ECHO) launched to provide valuable insights as to the roles and the cybersecurity competencies that need to be developed in different sectors. However, more investigations and solutions are needed towards creating an organizational culture able to timely detect and mitigate cyber threats against Maritime OT systems. A recent work [10], highlights the relationship between Maritime safety, security, and training and presents a framework to develop a risk aware culture in the Maritime domain. Moreover, applying early warning and deep learning tools [11] can assist in building such capabilities and support the domain’s mission objectives [12]. The delivery and continued development of effective training and exercises is critical to the execution and sustainment of detection activities. For the practical exercises, cyber ranges are commonly used as the proper environment to perform offensive and defensive training drills [13, 14].

3 Maritime Cyber Threats Detection Framework 3.1 Description of the Framework The proposed framework serves as a roadmap to build the necessary human skills and systems’ cyber threats detection capabilities in the Maritime domain. The framework is intended to guide the development of training curricula which can be utilized to train all responsible actors in the Maritime environment and develop their knowledge and skills to be able to detect cyber threats and support effective decision-making. Moreover, the proposed framework is intended to act as a starting point for those that are interested in implementing an infrastructure to detect cyber threats in the Maritime domain. Such an infrastructure can also serve as a training environment and support training curricula. It is envisioned that the framework can be used by personnel that is responsible to train the Maritime workforce and by personnel that manages cybersecurity incidents. The intended audience for the framework includes the organization’s Managing director and the IT managers, the training managers, and the ship IT managers [15]. Senior Management should employ the framework as a guiding tool for protecting a ship’s management systems. Trainers and IT managers should consider the framework to implement training curricula and develop a cyber threats detection infrastructure in the Maritime domain. To support the aims of the framework, this work focuses on identifying the knowledge and the skills related to detection aspects in the Maritime domain that should be acquired by all actors and/or build into systems.

110

G. Potamos et al.

Fig. 1. Proposed maritime cyber threats detection framework

Figure 1 presents the high-level structure of the framework and its core components. NIST Cybersecurity Framework is taken into consideration to guide the design of the proposed framework and support its objectives. The NIST Cybersecurity Framework aims to assist organizations to manage their cybersecurity risks through five functional areas: Identify, Protect, Detect, Respond and Recover. Although there is a dedicated functional area for detection purposes, other functions can support detection tasks. To be able to effectively develop human skills and systems’ cyber threat detection capabilities, one should develop knowledge and skills relevant to all tasks that can support this objective. Identifying all relevant tasks is challenging as they are listed across the NIST Cybersecurity Framework. Given that these tasks are listed in different functional areas, beyond the Detection function, often is not evident which of these are strongly interlinked and can impact the effectiveness of dedicated detection tasks. This research work aims to identify all tasks related to detection aspects and provide insights, increasing understanding of the tasks that need to be undertaken to build human skills and systems’ cyber threats detection capabilities. In the scope of this work, the functional areas that were identified to specify core tasks from a detection perspective include: Identify, Protect, Detect, Response, Recover. For each functional area, the tasks that can support detection aspects have been identified and discussed in Sect. 3.2, taking into consideration key Maritime elements, e.g., the Maritime OT systems/sensors appropriate for the navigation and surveillance. Table 1 should be referenced to obtain information about the tasks that should be performed to build detection capabilities. From the analysis in Sect. 3.2, 3 key areas have been identified where is imperative for personnel to build knowledge, at varying degrees depending on their responsibilities. These areas cover domain-related, human, and technical aspects. All these aspects are essential to build strong cyber threats detection capabilities in the Maritime domain and support the establishment of Maritime Cyber Situational Awareness (MCSA). The criticality of the Maritime domain requires the continuous determination of the situation to support the domain’s missions. Given

Maritime Cyber Threats Detection Framework: Building Capabilities

111

that IT/OT operations support Maritime tasks, the cybersecurity domain is interlinked with the Maritime domain and contributes to the MCSA. To start building cyber threats detection capabilities, one first needs to obtain a clear view of what the Maritime domain entails, identify all the elements that need to be protected and the relevant cyber threats that can jeopardize Maritime operations. The key characteristics of the Maritime domain are presented in Sect. 3.3. The relevant cyber threat landscape is covered in Sect. 3.4. The aim of these sections is to highlight the key aspects and point the stakeholders of the proposed framework, e.g., curricula designers, to the right direction of the knowledge that needs to be developed. The next area that needs to be taken into consideration concerns human aspects. To build the workforce’s capabilities and contribute to an effective MCSA, it is essential to acknowledge all stakeholders that need to be educated and trained. To achieve effective learning, a focused approach is required that will drive the specification of the cyber threats detection capabilities that should be pursued through a training. This research work specifies a set of 10 key capabilities (Sect. 3.5) that can be considered in training curricula. The aim is to provide a starting point to curricula designers, indicating which capabilities should be considered as a baseline to detect effectively and efficiently cyber threats in the Maritime domain. As indicated in Sect. 3.5, the Maritime workforce has different backgrounds and (technical or non-technical) expertise. It is envisioned that the suggested capabilities can be further elaborated, depending on the audience and the level of knowledge and skills that need to be developed. Developing an appropriate cyber threats detection infrastructure in the Maritime domain is a challenging task, due to the complexity of this environment. This work aims to highlight key implementation aspects that are essential to establish an effective MCSA. With regards to capabilities building, this work considers the development of a Maritime Cyber Range that simulates the operation of an onboard and/or ashore Maritime Security Operations Centre (Maritime SOC). A SOC can be developed to continuously monitor and detect cybersecurity incidents in the Maritime domain. The challenge is to identify the positions that sensors need to be located so the SOC can collect and analyze appropriate information from Maritime assets. Appropriate training curricula can be delivered over the Maritime Cyber Range, taking benefit of the SOC’s operation to build appropriate cyber threats detection skills and knowledge. To demonstrate the usage of the framework, a training activity is presented in Sect. 4. An attack scenario against Maritime OT Systems is developed as part of the training activity. Moreover, the infrastructure to deliver such training activities will be briefly discussed. 3.2 Detection Aspects Across the NIST Cybersecurity Framework The complexity of the Maritime domain challenges the detection of cyber threats as a clear view of all relevant tasks that should be performed is needed. The NIST Cybersecurity Framework lists tasks across 5 functions as illustrated in Table 1. There are tasks across all functional areas that contribute towards cyber threats detection objectives, however, this knowledge is not always evident and can impact the effectiveness and efficiency of threats’ detection. This research work focuses on the identification of all the tasks across the CSF that are related to cyber threat detection aspects in the Maritime

112

G. Potamos et al.

domain, even though they may belong to a different functional area, e.g., identify, protect, etc. The purpose is to highlight to curricula designers the areas where knowledge and skills need to be developed, promoting the development of a strong cyber threats’ detection capacity. Table 1 identifies the categories across the different NIST Cybersecurity Framework functions that include tasks relevant to detection aspects and provides insights as to the knowledge and skills that should be considered in training curricula to build cyber threats detection capabilities. With regards to identity function, the business environment, asset management and governance are driving factors to identify the cyber threats detection requirements relevant to Maritime operations. Risk management entails an assessment of the cyber threat landscape, which is expected to drive the selection and configuration of detection mechanisms. Moreover, the tasks under the protection function contribute to the formation of the cyber threats detection strategy that concerns the utilization of sensors and technologies to monitor Maritime operations and the identification of malicious activities. The detection function involves dedicated tasks related to continuously monitor events and anomalies to identify cyber threats in a timely manner. The effectiveness of the listed tasks is strongly interlinked with tasks from the previous mentioned functions, which specify the detection needs and requirements and drive the specification of the deployed detection mechanisms. Once an event is detected, it needs to be communicated to all appropriate stakeholders and decision-makers, so that actions are taken to respond, limit the impact and recover from a potential compromisation. Table 1. Detection aspects in the maritime domain Function

Category

Detection aspects

Identity

Asset Management

Identify the assests (data, personnel, devices and systems, and facilities) that should be monitored to detect potential cyber threats

Business Environment

Understand the organization’s mission objectives and the workforce’s responsibilities to formulate the detection requirements in the Maritime domain. Identify the roles of the workforce with Maritime Situational Awareness responsibilities that may be impacted by cyber threats

Governance

Legal and regulatory cybersecurity obligations are identified to drive the specification of detection requirements. Policies, procedures and processes to monitor communications, detect and manage cybersecurity incidents are specified. Cyber threats detection responsibilities are aligned with internal roles

Risk Assessment The cyber threat landscape in the Maritime domain is identified and will drive the configuration of the detection mechanisms Risk Management Strategy

The criticality of operations that might be affected by a cyber threat should be established and drive the specification of the monitoring, detection and response and recovery strategy (continued)

Maritime Cyber Threats Detection Framework: Building Capabilities

113

Table 1. (continued) Function

Category

Detection aspects

Protect

Awareness and Training

Understand the training needs related to detection aspects. Educate personnel, according to their role and their information and security related duties in the Maritime domain. Specify proficiency levels mapped to the workforce’s roles to indicate the level of cyber threats detection capabilities that need to be developed per role

Protect

Data Security

Data security requirements (confidentiality, integrity, and availability) should drive the specification of the detection strategy. Decide where to use sensors and detection mechanisms/algorithms to monitor data and identify potential malicious activities, e.g., data exfiltration, data spoofing, etc.

Information Protection Processes & Procedures

Security policies, processes and procedures address roles and responsibilities for the collection and evaluation of all the available information from the Maritime IT/OT/ICS sensors and actuators

Maintenance

Maintain and repair sensors, services, and components that may be impacted by a cyber threat, to maintain the detection capability

Protective Technology

Technical security solutions are managed to ensure the security and resilience of the critical Maritime management systems, like ECDIS, Radars, ICS management system (Machinery Monitoring and Control System). Build onboard/ashore SOCs, install sensors needed for the detection of cyber threats

Detection Anomalies and Events

Detection of Maritime anomalies caused by exploitation of the Navigational aid’s/Surveillance mean’s vulnerabilities. Estimation of the potential impact to Maritime operations using of HIDS – NIDS (signature/behavioral based) – XDR (NDR EDR). Visualization of alerts and development of customized dashboards to communicate results to different stakeholders (e.g., Captain, Navigation Officers, ashore operators) at a strategic, operational and technical level Perform analysis of log files from a variety of sources (e.g. individual host logs, network traffic logs, firewall logs, and intrusion detection system logs) to identify possible threats to network/host security Use data fusion and algorithms embedded in the Maritime Cyber Situational Awareness infratsructure to detect alomalies and malicious activities realated to the exploitation of ECDIS, Radar, and AIS vulnerabilities (continued)

114

G. Potamos et al. Table 1. (continued)

Function

Category

Detection aspects

Security Continuous Monitoring

SOC utilizes tools for behavioural based analysis, including data from data fused sensors, to identify cybersecurity events and verify the effectiveness of the protective measures. Information is monitored from: a) Surveillance and Navigational Sensors (RADAR – AIS – RF receivers - GNSS), b) Industrial Sensors (PLCs) c) Network sensors deployed in relevant positions onboard a ship and ashore, if there is any integration with relative shore systems Utilize host-based sensors (HIDS) and network sensors (NIDS) to monitor: a) the Personnel activity for internal threats, b) the IT/OT/ICS systems, c) unauthorised access of the onboard/ashore systems, d) external service providers activity, e) execution of malicious code f) decoy artifacts and systems, g) exfiltration

Detection Processes

Well defined roles and responsibilities are established for the detection of cyber threats onboard a ship and/or ashore against a Maritime infrastructure. For this purpose, detection activities comply with all applicable requirements Detection processes are tested and evaluated with the implementation of relative scenarios Event detection information is communicated to all appropriate parties Detection processes are continuously improved

Respond

Response Planning

Response plan is executed during or after a cyberattack, according to the involved role/responsibility, or the attached compromised or to be compromised system. The secure and safe operational continuity of the Maritime asset is considered as a core requirement

Communications Personnel (crew members) know their roles and procedures during the detection operations Analysis

Generate reports with accurate and comprehensive information about the detected event If the attack is executed onboard a ship, share information about the incident/event with the shore to support decision-making and coordinate further actions

Respond

Mitigation

Perform actions to mitigate the impact of the attack, ensuring safety and security, collision avoidance, proper manouver of ships

Improvements

Lessons learned related to the detection activities. Improve detection algorithms, signature and behavior-based rules, to ensure the effectiveness of the detection tools/systems during future attacks (continued)

Maritime Cyber Threats Detection Framework: Building Capabilities

115

Table 1. (continued) Function

Category

Detection aspects

Recover

Recovery Planning

Recovery plan is executed during the attack to re-establish functionality of the Network sensors and the Management systems utilized for the Maritime Cyber Situational Awareness

Improvements

Lessons learned drive improvements of the recovery measures to ensure the timely restoraration of affected operatons from future attacks

Communications Repair security communications with shore systems. Recovery activities are communicated to internal stakeholders, including executive and management teams

3.3 Maritime Domain Identification To be able to protect a domain and to detect cyber incidents, one must have a clear view of what the domain entails. Therefore, curricula designers should consider including learning content with regards to the Maritime characteristics. This section highlights the key aspects that should be taken into consideration. The Maritime domain contains onboard and ashore systems with different operational and behavioral characteristics, considered as the Maritime assets. As presented in Fig. 2, the environment contains ships, which are described as complex mobile platforms, functioning stand alone or interconnected, able to exchange data/information between them and with ashore assets/infrastructures. Ships can be of different types, e.g., cargo, tanker, passenger, fishing etc., including Unmanned Ship Vehicles (USV) or other automated ships. On the other hand, ashore infrastructures can include Maritime operational centers (governmental or private sector’s), observation/surveillance posts, and critical infrastructures such as ports or energy transportation facilities (onboard/ashore). To be able to identify and manage cybersecurity risks, it is essential to identify potential vulnerabilities and the systems involved. The Maritime domain is considered a complex environment, including a range of vulnerable systems, which combine several technologies (IT/OT/ICS) and protocols. As shown in Fig. 2, many hacking positions are available to deploy attacks [13]. For example, a Man-in-The-Middle or a Denial-ofService (DoS) attack launched against critical OT systems may impact navigation safety; moreover, malwares can cause data manipulation and leakage, impacting severely the Maritime domain’s mission objectives [11]. The exposure of the Maritime domain to a range of cyber threats, forced the Maritime industry to develop procedures and capabilities to protect its mission objectives [13], effectively maintaining preparedness, personnel and environmental safety, and operational security. One should also understand the guidelines that create a comprehensive regulative and legislative framework that promotes the identification and management of cyber risks in the Maritime domain: a) BIMCO, in cooperation with many shipowners [15], published guidelines to manage risks onboard ships, considering the NIST Cybersecurity Framework, b) The International Maritime Organization (IMO) adopted resolution MSC 428 (98), to address the cyber risks in existing management systems.

116

G. Potamos et al.

Fig. 2. Vulnerable systems onboard ship. Hacking positions onboard/ashore.

Additionally, IMO recommended in a circular MSC-FAL.1/Circ.3, high-level measures to safeguard ships from current cyber threats and vulnerabilities, c) ENISA published guidelines for cyber risk management for ports [16], d) The Advance Bureau of Shipping published guidelines in four key cyber areas: cybersecurity, automated systems safety, data management and software assurance, e) The Digital Container Shipping Association analyzed the Maritime Cyber Risks, based on the NIST Framework, f) The International Association for Classification Societies has issued a “Recommendation on Cyber Resilience (No. 166)” [17], f) The Maritime Bulk Liquids Transfer Cybersecurity Framework Profile [12] which assists in cybersecurity risk assessments for all involved entities. The aforementioned guidelines should be included in prospective trainings to educate participants of the driving factors that formulate cybersecurity requirements, especially relevant to cyber threats detection aspects. 3.4 Maritime Cyber Threat Landscape This section highlights key characteristics of the Maritime cyber threat landscape that need to be taken into consideration when designing a training curriculum. Curricula designers should have a clear view of the knowledge that should be developed to promote a good understanding of the cyber threat landscape and its complexity in the Maritime domain. The Maritime domain is a complex environment with a range of onboard and onshore assets that need to be protected from cyber threats. The attack surface contains the Maritime IT/OT systems, which operate onboard the ships and/or ashore at the relevant Maritime infrastructures. Part of these systems are integrated with ashore management systems or other systems onboard which are interconnected with other ships, creating an interlinked and complex operational environment. In addition, a range of systems/sensors are connected with Command and Control (C2) management systems. The interactions of the C2 management systems and the sensors/actuators are illustrated in Fig. 3. The

Maritime Cyber Threats Detection Framework: Building Capabilities

117

discussion that follows, highlights key operations that need to be protected, and the cyber threats that can jeopardize them. These aspects should be included in prospectve training curricula with the aim to educate participants of the assets tha can be impacted by cyber threats, understand how the operations and mission objectives can be compromised, and recognize the role that the workforce plays to protect the assets they are working with.

Fig. 3. Sensors and management systems (C2) in the maritime domain

The aforementioned C2 management system supports the Maritime domain’s mission objectives, and thus it is imperative to identify the cyber threats that are relevant to the systems’ operation. Specifically, on a ship, the main management system used to receive fuse and analyse the collected information from the surveillance and navigation sensors is called Electronic Chart and Display Information System (ECDIS). It operates as a plotter providing functionalities for a secure and safe sail. Additionally, the ECDIS collects data from many potentially vulnerable sensors and composes the Maritime picture (or Maritime Situational Awareness/MSA) around the ship. Similarly, a MSA C2 management system, often with extended capabilities in terms of Command, Control, Communications and Computers (C4), operates to ashore centers, capable to collect, process and visualize the data received from the surveillance sensors, mostly from AIS and Radar NMEA. Reported cyberattacks on the sensors are: (1) AIS: Spoofing [18], hijacking, data manipulation [19] and DoS [3], executed via software or wirelessly (RF) [20], (2) Radar: Jamming, DoS, and obfuscation [8], and (3) GNSS: Spoofing [21], DoS [22]. In addition, ECDIS is vulnerable to malware attacks, causing corruption or loss of chart data [23] and positioning spoofing [24]. The other main management system capable to collect and analyse data from the main and auxiliary machines, serving as a Human Machine Interface (HMI), is mostly known as the Machine Monitoring and Control System (MMCS), sometimes also refered as the Ships Management System. In modern ships or ashore infrastractures, the information from sensors/actuations of the industial systems (e.g., SCADA) is collected through PLCs, which are connected to the MMCS, through industrial switches. The cyber threats related

118

G. Potamos et al.

to the industrial systems of a ship are well covered by the MITRE ATT&CK matrix for ICS. ICS sensors in the Maritime domain include monitoring devices, fuel/oil/water temperature and presure sensors able to provide indications for the operational status and the performance of the the main and auxiliary machines. Additionally, actuators are responsible to modify the operational status of the industrial equipment, according to the sensor’s indications. Both of sensors/actuators enlarge the attack surface of the Maritime mean, especially in the case that are interconnected with HMI for remote monitoring and when they have connectivity for automatic operational actions (e.g., on/off mechanisms of the main, power and auxiliaty machines including heater pumps and cooling systems, alerting, automatic indication of capasity, etc.). For the above reasons, these sensors/actuators are referred as Industrial Internet of Things [15]. Moreover, cargo control management systems utilize digital systems to support the loading procedures. Such systems may be connected with various systems onboard, ports, Marine terminals and stevedores. Typical ICS threats that should be considered include, among other, industrial protocols’ (e.g., modbus) tree attacks, including gaining unauthorised SCADA system access, identification of modbus devices and general master-slave attacks, and also Denial of Service attacks against the industrial systems and sensors. Other ICS sensors that can increase the attack surface include safety sensors such as fire pumps (eg. CO2 , foam, water pumps, tighness indicators). Safety sensors are also part of the Ships Management System, alongside access control sensors that can protect the personnel and the critical parts of the ship or the environment in general. For example, such devices could include CCTV equipment, access cards, password protected panels, interconnected buoys or lifebuoys, doors status indicators (e.g., preasure), etc. Passengers’ access systems also include IoT devices such as in the case of entertaining systems. Many of these sensors are connected to the onboard and/or ashore IT infrastructure for remote monitoring purposes, troubleshooting, etc., and are often considered as Internet of Things (IoT) devices. As more ICS/IoT devices are introduced onboard/asshore, the attack surface expands further and increases the risk of compromising Maritime operations. It is well known that IoT devices suffer from well-known vulnerabilities, such as insecure passwords, outdated components and unencrypted communications. Therefore, introducing IoT devices should not be taken lightly, and should follow best practices to secure them. To further analyse the cyber threat landscape in the Maritime domain, the MITRE ATT&CK framework is utilized to develop a blueprint for detecting cyber threats. The purpose of the proposed blueprint is two fold: a) to provide insights to the behavior and techniques that hackers use to exploit the vulnerabilities of the Maritime sensors and actuators, and b) to guide security teams where to focus their detection efforts across the cyberattack kill chain and enhance systems’ detection capabilities. Such knowledge is valuable when designing training scenarios as it can provide a clear view to the curricula designers of the malicious activities that can take place across the cyberattack kill chain and assist in selecting topics to cover in the training and build relevant skills and knowledge. For the development of the blueprint, the MITRE ATT&CK Enterprise, ICS and mobile matrices are taken into consideration, as they are are widely accepted by the cybersecurity community as a comprehensive resource of adversarial Tactics, Techniques and Procedures (TTP). Therefore, the matrices are a valuable resource to

Maritime Cyber Threats Detection Framework: Building Capabilities

119

provide guidance for the identification of relevant attacks against Maritime-related IT and OT systems. Taking this into consideration, this work contributes a Maritime attack matrix (Fig. 4) that combines and customizes information of the MITRE ATT&CK framework attacks, across all matrices, that are applicable in the Maritime domain and clearly identifies the cyber threats related to the Maritime surveillance and navigation systems. The aim is to create a matrix that indicates the cyber threats that can impact the Maritime Navigational and Surveillance Sensors/Systems and the respective Management Console as they are core elements that support the safety and security of Maritime operations. As indicated from the proposed Maritime matrix in Fig. 4, during the: • Reconnaisance stage, the attacker could gather all the available host and network information, including information about Navigational and Surveillance Systems. Moreover, active scanning can be performed to identify the flow of the NMEA messages, recognizing several Maritime components at the bridge and identifying their connectivity. For example, through active scanning, the ECDIS workstation can be identified, including its connectivity with AIS, Radars, GNSS and possibly with other navigation sensors. • Resource development, all the necessary infrastracture is prepared to facilitate the execution of attacks using RF links/wired connections, using malware or exploits. • Initial access, techniques can be implemented to compromise the network devices and peripherals (including the satellite communications equipment), RF ship–to-ship or ship-to-shore connections and vulnerable RF surveillance sensors. Especially for the ship, VSATand RF links are the main access points. • Execution, several techniques can be deployed to manipulate Maritime data (NMEA Radar/GNSS/AIS) and change the operational status of the Maritime OT devices/systems. • Percistence stage, the attacker could modify configuration files or the settings to the receivers of the Navigation and Surveillance systems in order to maintain access. • Priviledge escalation, techniques similar to the other MITRE ATT&CK matrices can be implemented, which are applicable for Maritime OT systems. • Defence escalation, the attacker can take advantage of Maritime/navigational messages that are transmitted in plain text and implement spoofing techniques. • Discovery, the target is to identify the navigation and surveillance devices and the data flow among them. • Lateral movement, remote access techniques and direct connectivity (e.g., via USB stick) are feasible, taking advantage of the enhanced connectivity between sensors and management systems. • Collection, the target is to capture the Maritime traffic which is exchanged between the bridge components. • Command and control, the attacker is capable to mimic the ship’s bridge network using techniques to connect the ship’s surveillance sensors with the attacker’s C2 system, e.g., attacker’s ECDIS. • Exfiltration, the attacker can apply techniques to exfiltrate NMEA (Radar/AIS/GNSS) data. • Impact, the results of an attack against Maritime OT systems are described, causing manipulation of the Maritime Picture, Denial of Sevice and/or loss of availability and control.

120

G. Potamos et al.

3.5 Cybersecurity-Aware Maritime Roles and Cyber Threats Detection Capabilities Development In the Maritime environment, human roles are separated in three main categories: managers/commanding officers, officers/directors and employees/operators with different roles and responsibilities. The detection of cyber threats in the Maritime domain is an aspect that concerns all stakeholders, including those that have IT/cybersecurity responsibilities and those that have a managerial role onboard a ship and/or ashore. For example, a weaponized cyberattack might affect the movement/behavior of the ship and result into a collision. Therefore, a ship’s IT Manager (as a key person) should have clear understanding of both Maritime and cyber domains to ensure the safety of a ship. To be able to support the mission objectives of the organization [12], all involved stakeholders need to develop capabilities to detect potential cybersecurity incidents that can impact decisionmaking. The only difference is the level of knowledge and skills they need to develop to support their role. The key Maritime roles [15] to consider in potential trainings include the Managing director, the Fleet manager, the Captain/Commanding officer, the organizations’ and the ships’ IT Managers, the Chief engineer, the Navigation/Operation officer, the Operators, the Safety manager and the Training manager. To design effective training programs, a microlearning approach is required to build human capacity in detecting cyber threats in the Maritime domain. To achieve this, an initial set of baseline cyber threats detection capabilities should be utilized to guide the development of essential capabilities. It is envisioned that once the workforce develops these essential capabilities, then further training can be pursued to build more advanced skills and knowledge. This work specifies a set of 10 essential capabilities that can be considered in training curricula. Table 2 lists key capabilities that need to be developed across strategic/operational/technical levels to support detection objectives [15]. The proposed capabilities take into consideration the guidelines in [25] and the key aspects of NIST Cybersecurity Framework that have taken a center stage in this work and served as drivers to specify the baseline capabilities listed in Table 2. Specifically, the workforce first needs to understand the Maritime domain, identify its key assets and the relevant cyber threats that can impact their operation. Special attention should be given to educating the workforce about cyber threats that can jeopardize the operation of onboard assets and can lead to severe impact, e.g., ship collision. • Another essential skill that needs to be cultivated concerns the workforce’s communication skills. During an incident, the workforce should be able to communicate information effectively and efficiently to the appropriate stakeholders. This is critical for timely resolving an incident with the minimum impact. This aspect should be driven by an appropriate detection, response, and recovery policy, which can be considered a fundamental element to achieve governance. Therefore, appropriate capabilities should be developed with regards to the design and development of such policy. To be better prepared and be able to detect an incident effectively and efficiently, personnel should reference multiple intelligence sources across the cybersecurity and Maritime domains and retrieve information about relevant threat actors, their capabilities, attack indicators, etc. Such knowledge can further be utilized to configure security systems, e.g., SIEM. Building technical capabilities are also part of the proposed baseline

Compromise VPN connection

Exploit via SDR RF Interfaces

Develop Capabilities using Malware and Exploits

Obtain Capabilities using Malware, Tools, Exploits, vulnerabilities, Certificates

Deliver GNSS/AIS NMEA Spoofed Messages to Surveillance Receivers/Systems

Masquerade Legitimate Application/ Service

Exploitation of Remote Services

Compromise Infrastructure

ΝΜΕΑ related Attacks

Search for integration between Ship to Ship or Ship to shore

Gather Victim Ship/Maritime Infrastructure Information for the existing Relationships, Roles)

Compromise Accounts relative to Management System’s

Gather Victim Network and Host Information relative Navigational and Surveillance equipment

Modify Services status

Scripting to manipulate NMEA traffic

Command-Line Interface

Broadcast receivers NMEA Traffic/ Falsified AIS messages

Drive-by Compromise the Navigational and Surveillance System’s

Acquire Infrastructure to receive and maliciously retransmit RF signals

Active Scanning for NMEA Mgs

Deliver Malware on ECDIS, Map User Interface app, via other Means or Removable Devices

Execution

Initial Access

Resource Development

Reconnaissance

Exploitation for Privilege Escalation

Privilege Escalation

Valid Accounts

Spoof Reporting Message

Obfuscate Files or Information

Modify Trusted Navigation and Surveillance Data

Masquerade Legitimate Application/ Service

Defense Evasion

Discovery

Remote System Discovery

Ships/Maritime Infrastructures Network Configuration Discovery

Network Service Scanning

Network Sniffing

Tracking of the Ship

Navigational and Surveillance Management Account Discovery

Fig. 4. Maritime attack matrix

Port Knocking

Modify configuration of the Navigational Process Injection aid, or the Surveillance System

Broadcast receivers

Persistence

Collection

Connection Proxy

Commonly Used Port

Command and Control

Ghost ship attack

AIS/GNSS/ Radar NMEA Replay Attack

Exfiltration

Port Knocking

Traffic Signaling

Non-Standard Port

Data Obfuscation

GNSS Spoofing

Create Maritime Anomaly

Denial of Service

Block Reporting Message

Input Capture Standard Block Serial GUI Input Application COM Capture, Layer Protocol Keylogging, Credential API

Man in the Attack via the Middle ECDIS/ Plotter via USB Between Sensors connection and C2 systems

Exploitation Capture NMEA of Remote traffic Services

Lateral Movement

Manipulation of MSA

Manipulation of Control

Loss of Safety

Loss of Availability

Denial of View

Loss of Control

Denial of Service

Impact

Maritime Cyber Threats Detection Framework: Building Capabilities 121

122

G. Potamos et al.

set. An essential capability concerns analyzing logs from different sensors to identify potential malicious activities. Considering that cyber threats increase in intensity and complexity, searching effectively through logs becomes challenging. Therefore, taking into consideration the onboard/onshore assets that need protection, alongside threat intelligence information and the various logs collected from sensors, the capability to construct basic to advanced log queries should be developed to address the dynamics of the cyber threat landscape. Having a clear view of the attack stage, whether the attack is at an initial stage, or it is already elaborated and has impacted Maritime operations, can assist decision-makers to prioritize actions. Therefore, it is imperative for the appropriate personnel to have the knowledge and the skills to map the security incident to the cyberattack kill chain. All the aforementioned information should be synthesized and communicated through appropriate dashboards to different stakeholders working at a strategic, operational, or technical level. The aim is to present relevant information related to security events and incidents to the appropriate decision-makers, supporting them in their tasks to detect malicious activities and recover from compromization. Depending on the workforce’s role, dashboards can be designed to present strategic, to operational, to technical level details. The goal is to build a cyber resilience capacity, where the workforce recognizes its working environment, realizes and anticipates the threat, identifies security incidents, and also plans and applies a response when required. Curricula designers can select the proposed baseline set as a whole or they can select a subset of the proposed capabilities and build a training curriculum. The capabilities list is not meant to be exhaustive, but rather indicative, aiming to serve as a starting point that can be further elaborated depending on the training needs, the personnel’s role, and the proficiency level that detection capabilities need to be demonstrated by personnel.

4 Cyber Threats Detection Training Activity This section presents a training activity that was developed over a Maritime Cyber Range [13] with the target to develop cyber threats detection capabilities. First, the highlevel architecture of the training platform, which is appropriate to deliver the activity is presented, and then the design aspects of the training activity are discussed. The discussion also indicates how the proposed framework was utilized to drive the design and development of the training activity. 4.1 Training Platform – High-Level Architecture To be able to deliver training activities and build cyber threats detection capabilities, an appropriate learning environment needs to be developed. In the context of authors’ research activities, a Maritime Cyber Range [13] was developed to simulate realistic cyber threat scenarios and to deliver offensive and defensive training activities in the Maritime domain. As part of the training infrastructure, a Security Operations Centre (SOC) was developed to support continuous security monitoring of Maritime operations. Table 1 was consulted to identify the aspects that need to be taken into consideration

Maritime Cyber Threats Detection Framework: Building Capabilities

123

Table 2. Cyber threats detection capabilities in the Maritime domain ID

CAPABILITIES

LEVEL

C1

Identify onboard/onshore assets that can be impacted by cyber threats

S/O/T

C2

Discuss the Maritime attack surface and the impact to the operations and mission objectives of the organization

S/O/T

C3

Explain how specific cybersecurity threats may compromise the operation of S/O/T onboard assets

C4

Communicate information about an onboard security incident to the appropriate stakeholders

S/O/T

C5

Design and implement a detection, response and recovery policy taking into consideration the organizations’ mission objectives, regulations, and best practices

S/O/T

C6

Utilize multiple intelligence sources across the cybersecurity and Maritime domains to detect a security incident

O/T

C7

Construct search queries to tailor log analysis considering onboard/onshore assets

T

C8

Map Maritime security incidents to the cyberattack kill chain to identify the stage of the attack and inform decision-making

T

C9

Detect a security incident and take actions to eliminate the impact to the onboard/onshore assets

O/T

C10

Development of customized dashboards to communicate results to different stakeholders at a strategic, operational, and technical level

T

Level: S – Strategic, O – Operational, T – Technical.

when developing a SOC within the Maritime Cyber Range. Specifically, the tasks listed under the Detection function in Table 1 have been taken into consideration, driving the decisions taken related to the technologies utilized and the implementation aspects. This section outlines the high-level architecture (Fig. 5) of the developed training platform to enable curriculum designers to gain an understanding of potential learning environments they need to develop to support learning objectives related to detection aspects. As depicted in Fig. 5, the developed training platform leverages the abilities of already deployed technologies to detect cyber threats, including End Point Detection and Response (EDR), Host and Network Intrusion Detection and Prevention Systems (H/NID) and Security Information Event Management Systems (SIEM). HIDS provide the capability to detect host-based malicious activity, where NIDS provide detection of malicious, spoofed, or manipulated traffic. Expanded capabilities of the SIEM to visualize a cyber incident impacting the Maritime traffic are developed as part of the training platform. Also, it is recommended that the detection capabilities and alerting should remain onboard the ship, since the connection with the shore is not ensured. As indicated in Table 1, to develop continuous and effective security monitoring, sensors should be deployed at appropriate network positions to collect the IT and OT network traffic. Endpoints and network sensors are deployed in each network segment.

124

G. Potamos et al.

Fig. 5. High-level description of a maritime cyber threat detection training platform

As illustrated in Fig. 5, the deployment of network sensors provides the capability to collect, monitor, and analyze the traffic exchanged between sensors/actuators and the respective Maritime management systems. The traffic monitoring process is also useful in case a system is integrated through a ship to shore connection. By default, the architecture of the training platform permits the deployment of training scenarios considering onboard/ashore infrastructures. With regards to the implemented detection mechanisms, signature and behavior anomaly detection capabilities can be effective when sophisticated attacks are executed. In particular, the training activity described in Sect. 4.3 considers an AIS “Replay Attack” on ECDIS, in combination with a Man-in-the-Middle (MiTM) technique and malware deployment. The detection of the attack is performed using a SIEM, configured with an appropriate behavior-based rule/query to detect the event when the port is killed. Figure 6 indicates Zeek’s rule for the detection of the AIS replay attack. Moreover, this attack can be detected by a command-and-control system, which is able to apply fusion algorithms to correlate the received AIS messages with the detected RF signals. As a result, the system can realize the spoofed tracks. Taking into consideration the proposed Maritime Cyber Threat Detection Framework to design a new training activity, Sect. 4.2 presents the activity, alongside the learning strategy to engage participants and achieve an effective training. Details of the attack scenario are presented in Sect. 4.3. 4.2 Description of Training Activity The motivation behind the development of the training activity was to start building structured capabilities related to cyber threats detection to create human capacity in the Maritime domain. Initially, Sect. 3.5 was consulted to decide on the capabilities to develop through the training activity. Table 2 was referenced to facilitate the selection. Capabilities C1–C3 and C7–C8 have been selected as the initial aim was for all

Maritime Cyber Threats Detection Framework: Building Capabilities

125

Fig. 6. Zeek’s rule for the detection of AIS replay attack

stakeholders to develop a good understanding of the cyber threat landscape and realize the significance of the problem and how it can affect Maritime operations. Such knowledge should be developed across the workforce with strategic, operational, and/or technical roles. Moreover, it was considered essential for participants to be able to identify the stage of the attack and realize the importance of communicating information to decision-makers to coordinate further actions before an attack can elaborate further. Taking into consideration the cyber threat landscape presented in Sect. 3.4 and the increased interest of cyber criminals to attack onboard systems, it was decided to focus the training activity on a relevant attack impacting critical onboard systems. The training activity presents a case study, demonstrating an attack scenario, and critically analyzing its execution. The attack scenario covers the exploitation of a vulnerability related to AIS, impacting the Maritime OT system called ECDIS. Through the demonstration of the scenario and the critical analysis, the participants are expected to identify critical bridge systems (C1), such as AIS, radar, GNSS and ECDIS, that can be affected by a potential cyber threat, and discuss their connectivity needs/dependencies. Moreover, participants are expected to analyze their operational use and how their operation may be affected (C2) from specific cybersecurity threats (C3). Special emphasis is given to ECDIS operation as it is the primary system to construct MSA. The training activity also aims to build capabilities related to identifying the stage of the attack (C8) and to communicating information to the relevant decision-makers before the attack is elaborated down the cyberattack kill chain and cause greater impact. The attack mechanisms will be demonstrated and critically discussed to facilitate understanding and learning. Beyond gaining knowledge and developing critical thinking related to the cyber threat landscape and its impact to Maritime operations, it is considered equally important to demonstrate aspects of the network monitoring capabilities that need to be developed

126

G. Potamos et al.

(C7). The focus will be to develop understand around the appropriate use of sensors in the Maritime domain to detect a cyber threat and to critically analyze the difference between a cyber incident and a Maritime anomaly, and how they may be related and detected. 4.3 Cyber Threats Attack Scenario The attack scenario to be demonstrated as part of the training activity, covers the exploitation of a vulnerability related to AIS, impacting ECDIS. The specific vulnerability, which permits the manipulation of AIS messages as they are exchanged in plain text, provide the capability to demonstrate operational scenarios impacting the Maritime Situational Awareness (MSA) both in a ship and/or in an ashore Maritime infrastructure. The proposed Maritime matrix in Sect. 3.4 is referenced to select the techniques to implement in the context of the training scenario. Figure 7 indicates the techniques utilized from the Maritime matrix and maps them to the cybersecurity kill chain.

Fig. 7. Attack scenario mapped to the maritime matrix

The steps below describe the attack evolution as implemented in the training platform.

Maritime Cyber Threats Detection Framework: Building Capabilities

127

• Step 1: The attacker successfully infects the target machine (ECDIS workstation) through a malicious Electronic Chart’s (ENC) update and setups a MiTM attack controlling the traffic between AIS and ECDIS. • Step 2: Attacker compromises the ECDIS through the existing TCP/IP port utilized to receive the AIS Traffic. • Step 3: The attacker who previously captured the preferable AIS situation (AIVDM/AIVDO messages) regarding the area of interest and used dispatcher to decode these messages, spoofs part of the messages by replaying the AIS tracks (the attacker also manipulates the timestamp of the messages and stores them in a file). • Step 4: Through the compromised ECDIS, the falsified/spoofed messages are retransmitted to ECDIS when the ship enters a specific circular area (Latitude - Longitude). As a result, the attacker modifies the MSA of the ship with the insertion of spoofed AIS messages to ECDIS. Depending on the objectives of training activities, the proposed Maritime attack matrix can be utilized in a similar approach to guide the development of new cyberattack scenarios supporting offensive and/or defensive tasks.

5 Conclusion Successful detection of cyber threats in the Maritime domain is critical for uninterrupted Maritime operations. Developing detection capabilities can mitigate the impact of cyber threats, support decision making and maintain the security and safety of the ships and/or the shore Maritime infrastructures. This work contributed towards a new framework to assist the development of (a) relevant Maritime cybersecurity skills and (b) cyber threats detection capabilities for the Maritime domain, including a novel Maritime attacks matrix based on MITRE ATT&CK matrices. Future work will include the delivery of the training activity and the design of new curricula using the proposed framework. Acknowledgement. The authors would like to acknowledge the FORESIGHT project funded by the European Union’s Horizon 2020 research and innovation program (grant agreement: 833673), and the partners on the project.

References 1. Meland, P.H., Bernsmed, K., Wille, E., Rødseth, Ø., Nesheim, D.A.: A retrospective analysis of maritime cyber security incidents. TransNav Int. J. Mar. Navig. Saf. Sea Transp. 15(3) (2021) 2. Hyra, B.: Analyzing the attack surface of ships, p. 166 (2019) 3. Caprolu, M., Pietro, R.D., Raponi, S., Sciancalepore, S., Tedeschi, P.: Vessels cybersecurity: issues, challenges, and the road ahead. IEEE Commun. Mag. 58(6), 90–96 (2020). https:// doi.org/10.1109/MCOM.001.1900632 4. Satellite Internet Market: Global Industry Analysis and Forecast 2021–2027: by Frequency Band, Bandwidth, Orbit, End User and Region. Maximize Market Research. https://www. maximizemarketresearch.com/market-report/satellite-internet-market/125907/. Accessed 30 Jan 2022

128

G. Potamos et al.

5. Jacq, O., Brosset, D., Kermarrec, Y., Simonin, J.: Cyber attacks real time detection: towards a cyber situational awareness for naval systems. In: 2019 International Conference on Cyber Situational Awareness, Data Analytics and Assessment (Cyber SA), pp. 1–2, June 2019. https://doi.org/10.1109/CyberSA.2019.8899351 6. Jacq, O., Boudvin, X., Brosset, D., Kermarrec, Y., Simonin, J.: Detecting and hunting cyberthreats in a maritime environment: specification and experimentation of a maritime cybersecurity operations centre. In: 2018 2nd Cyber Security in Networking Conference (CSNet), pp. 1–8, October 2018. https://doi.org/10.1109/CSNET.2018.8602669 7. Canepa, M., Ballini, F., Dalaklis, D., Vakili, S.: Assessing the effectiveness of cybersecurity training and raising awareness within the maritime domain. In: Online Conference, pp. 3489– 3499, March 2021. https://doi.org/10.21125/inted.2021.0726 8. Tam, K., Jones, K.: MaCRA: a model-based framework for maritime cyber-risk assessment. WMU J. Marit. Aff. 18(1), 129–163 (2019). https://doi.org/10.1007/s13437-019-00162-2 9. Hatzivasilis, G.: Modern aspects of cyber-security training and continuous adaptation of programmes to trainees. Appl. Sci. 10(16) (2020). https://doi.org/10.3390/app10165702 10. Hopcraft, R.: Developing maritime digital competencies. IEEE Commun. Stan. Mag. 5(3), 12–18 (2021). https://doi.org/10.1109/MCOMSTD.101.2000073 11. Mrakovi´c, I., Vojinovi´c, R.: Maritime cyber security analysis – how to reduce threats? Trans. Marit. Sci. 08(01), 132–139 (2019). https://doi.org/10.7225/toms.v08.n01.013 12. Maritime Bulk Liquids Transfer Cybersecurity Framework Profile. United States. Coast Guard. https://www.hsdl.org/?abstract&did=797741. Accessed 12 Feb 2022 13. Potamos, G., Peratikou, A., Stavrou, S.: Towards a maritime cyber range training environment. In: 2021 IEEE International Conference on Cyber Security and Resilience (CSR), pp. 180– 185, July 2021. https://doi.org/10.1109/CSR51186.2021.9527904 14. Tam, K., Moara-Nkwe, K., Jones, K.: The use of cyber ranges in the maritime context: assessing maritime-cyber risks, raising awareness, and providing training. Marit. Technol. Res. 3(1) (2020). https://doi.org/10.33175/mtr.2021.241410 15. The Guidelines on Cyber Security Onboard Ships. https://www.bimco.org/about-us-andour-members/publications/the-guidelines-on-cyber-security-onboard-ships. Accessed 23 Oct 2021 16. European Union Agency for Cybersecurity. Cyber risk management for ports: guidelines for cyber security in the maritime sector. LU: Publications Office (2020). https://data.europa.eu/ doi/10.2824/671060. Accessed 11 Feb 2022 17. Armando, A., Henauer, M., Rigoni, A.: Next Generation CERTs. IOS Press (2019) 18. Balduzzi, M., Pasta, A., Wilhoit, K.: A security evaluation of AIS automated identification system. In: Proceedings of the 30th Annual Computer Security Applications Conference, New Orleans Louisiana USA, pp. 436–445, December 2014. https://doi.org/10.1145/2664243.266 4257 19. Ray, C., Gallen, R., Iphar, C., Napoli, A., Bouju, A.: DeAIS project: detection of AIS spoofing and resulting risks. In: OCEANS 2015 - Genova, pp. 1–6, May 2015. https://doi.org/10.1109/ OCEANS-Genova.2015.7271729 20. Faragher, R., MacDoran, P.F., Mathews, M.B.: Spoofing mitigation, robust collision avoidance, and opportunistic receiver localisation using a new signal processing scheme for ADS-B or AIS, p. 11 (2014) 21. Hareide, O.S., Jøsok, Ø., Lund, M.S., Ostnes, R., Helkala, K.: Enhancing navigator competence by demonstrating maritime cyber security. J. Navig. 71(5), 1025–1039 (2018). https:// doi.org/10.1017/S0373463318000164 22. DiRenzo, J., Goward, D.A., Roberts, F.S.: The little-known challenge of maritime cyber security. In: 2015 6th International Conference on Information, Intelligence, Systems and Applications (IISA), Corfu, Greece, pp. 1–5, July 2015. https://doi.org/10.1109/IISA.2015. 7388071

Maritime Cyber Threats Detection Framework: Building Capabilities

129

23. Preparing for Cyber Battleships – Electronic Chart Display and Information System Security 24. Svilicic, B., Brˇci´c, D., Žuškin, S., Kalebi´c, D.: Raising awareness on cyber security of ECDIS. TransNav Int. J. Mar. Navig. Saf. Sea Transp. 13(1) (2019). https://doi.org/10.12716/1001. 13.01.24 25. Parrish, A.S., et al.: Global perspectives on cybersecurity education for 2030: a case for a meta-discipline. ITiCSE (2018). https://doi.org/10.1145/3293881.3295778

Author Index

Afolabi, Motunrola 17 Alhasan, Isslam 17 Andriessen, Jerry 46

Langner, Gregor 46 Luciano, Carmela 46 Mian, Shiven 17 Miloslavskaya, Natalia

Bishop, Matt 17 Böhm, Fabian 77 Budde, Carlos E. 60

31

Ngambeki, Ida 17 Nico, Phillip 17

Chance, Zed 17 Dai, Jun 17 Duvenage, Petrus

92

Englbrecht, Ludwig

77

77

Huynh, Tran Ngoc Bao 17 Jaquire, Victor

Quirchmayr, Gerald

46

Stavrou, Eliana 107 Stavrou, Stavros 107

Friedl, Sabrina 77 Furnell, Steven 46 Glas, Magdalena

Pernul, Günther 77 Potamos, Georgios 107

Thao, Ong 17 Theodoulou, Savvas 107 Tokola, Teemu 46 Tolstoy, Alexander 31

92

Knorr, Konstantin

3

Vidor, Silvia 60 von Solms, Sebastian

92