Security, Privacy and User Interaction [1st ed.] 9783030437534, 9783030437541

Citation preview

Markus Jakobsson   Editor

Security, Privacy and User Interaction

Security, Privacy and User Interaction

Markus Jakobsson Editor

Security, Privacy and User Interaction

Editor Markus Jakobsson ZapFraud Inc. Portola Valley, CA, USA

ISBN 978-3-030-43753-4 ISBN 978-3-030-43754-1 (eBook) https://doi.org/10.1007/978-3-030-43754-1 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2020 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

For A and Art. Thank you for putting up with me.

Foreword

Online, most people know me by my nom de guerre, Sinon Reborn, and the fact that I hookwinked, over a few busy months, an array of people in positions of power or fame, including celebrities and key people at financial institutions at Wall Street, Bank of England, and the White House. However, what made my deception unusual was that my goal was never to damage or steal, but always to prank my marks. The approach I used did not involve malicious code or hacking, but it was about cleverly selected account names, cunning pitches, and an understanding of what motivated my marks. At the same time, I never asked for the combination to the safe or any form of sensitive data; far from it. My modus operandi was more often than not to invite them to a party, an unusual party; perhaps with a strange theme, but a party nonetheless. If I were a criminal, I would have been able to use the same techniques to become a successful cybercriminal. There is no doubt: Given the right angle and the right pitch, you can make almost anybody do almost anything. This is what social engineering is about. Social engineering, in a way, is like martial arts: It is about using your opponent’s force against them. In the context of social engineering, that means understanding the psychology and context of the target and to play on their vulnerabilities and insecurities. And this is what criminals increasingly do to plunder everybody from little old ladies to major corporations. It is a crime that is not based on criminal technology (unlike, for example, traditional viruses and associated attacks). It is also a crime for which the development of technological countermeasures has lagged severely behind. This begs the question: If everybody were to leave their doors unlocked, would it be reasonable to resent a rise in burglaries? Burglary would still be wrong, but the blame would not just be with the criminals, but with the people leaving their doors open, too. In the same way, today’s Internet services are very much like a city of unlocked doors, and it is because we, as a society, have not bothered understanding the problem. The unlocked doors for us are embodied by a failure to understand protocols, user interfaces, and how these can be used . . . and abused. This book shines a light on this vast problem and explains both shortcomings and fixes—but not only in the context of social engineering, but about other failures of vii

viii

Foreword

proper communication, too. It asks and answers the question of how people think and how we can measure this. It also asks and answers how systems should be designed with this in mind—whether we consider how to ask permissions from users to use their data or how we can make sure that existing technologies (such as second-factor techniques) are not abused by criminals. Returning to the analogy of martial arts, this book considers how to avoid having criminals use the force of little old ladies and of corporations to commit crimes. But doing so requires that we understand the limitations of these potential victims (as well as their strengths) and that we consider what limits the criminals. The book is based on a series of case studies that, together, build the case that it is time for us, as technologists and decision-makers, to change how we approach this problem. Manchester, UK February 2020

James Linton, a.k.a. Sinon Reborn

Preface

In 1997, I graduated from University of California, San Diego, with a PhD in Cryptography. Entering my first real job as a Member of Technical Staff at Bell Labs, I was convinced that the answer to most problems—or at least, to most problems worth solving—was cryptography. That, admittedly, was rather naïve. I spent a few blissful years believing cryptography was the answer, doing research on privacy, randomness generation, and electronic payments—until it gradually started to dawn on me that the problems I was solving were strangely disconnected from the real security problems society seemed to suffer. The problem, as it turned out, was that I (like almost all my peers at the time) did not take the end user into consideration. I made the mistake of believing either that the end user will follow the instructions to the dot or sometimes will never do anything right. In either case, I decided to ignore the user, since I had no control over what he or she might do. That, as it turns out, is often a massive mistake and one that can severely cripple otherwise very well designed solutions. I wrote this book in the hope that by explaining why the end user matters, others could avoid making the mistake so many others have made. It is my strong belief that considering security and privacy only, while ignoring the end user and the user interfaces, is not a winning strategy. Let me say it again: The user does matter. While we can neither count on him or her always doing the right thing nor always doing the wrong thing, we still need to understand what motivates the typical user and how the typical user interprets information. Most users, whether of a new phone, a new IoT system, or a new piece of software, want to plug it in, flip the switch, and then go on with their lives. They do not want to carefully understand the implications of various possible configurations; they do not want to have to understand what can go wrong; most of them will not even want to read the instructions. This is unfortunate because the end user does matter—and probably more than anything else when it comes to applied security. But that does not mean that we must make the end user understand the delicate details. We will never succeed with that. Take phishing prevention as an example. Early on, the most prominent countermeasure was that of user awareness. Security experts told people to watch out for ix

x

Preface

Fig. 1 The figure illustrates when awareness campaigns pay off: On the x-axis is the difficulty to teach a concept, and on the y-axis is the difficulty for a user to apply their knowledge. For example, “Only install software from trusted sources” is both easy to teach and apply and therefore makes a meaningful awareness campaign. In contrast, anything with lots of different cases, like how to identify whether an email comes from a trusted source or not, is difficult to teach. Similarly, by going against the user habits, like the advice “do not click on links” does, makes knowledge hard to apply. This book addresses the space in gray

poorly spelled emails and emails with flawed grammar. Users were told not to click on hyperlinks, and they were told that they must be careful. But honest people, at times, misspell words and use incorrect grammar. And you can bet that companies will continue sending out emails with hyperlinks—even the very same companies that warned their users never to click on links. At the same time, criminals, when sufficiently motivated, can spell correctly (or hire somebody who can), and they can use correct grammar. They can even use the emails sent by honest service providers, simply modifying the hyperlinks, and use these modified emails to phish people. Telling people to be careful simply is not enough. End-user security should not rely on the end user, in other words, and user awareness efforts should be seen as a last resort, when everything else fails. Therefore, there is one thing that will be glaringly absent from this book, in spite of it having a focus on the end user. This book will not be about user awareness campaigns. Mind you, I do not believe awareness never has value— but it is not the right approach for most security problems, as illustrated in Fig. 1. Generally speaking, I believe that considering user awareness the first approach of protecting a system is wrong. First of all, it suggests that security is the enduser’s responsibility . . . which I think is no longer a reasonable assumption. Second, emphasizing awareness may cause organizations to allocate budgets in ways that hurt them, rather than help them. Instead, I believe organizations must identify how to best solve user-related problems algorithmically and then determine what awareness campaigns to invest in. I believe that the first line of defense should be technical, whenever possible. Moreover, it should be a technical solution that takes the end user into consideration.

Preface

xi

This book makes this case and does that by considering the user from three different perspectives, corresponding to the three parts of the book. The first part of this book considers the typical user, and what he or she will do—or sometimes fail to do. The second part considers what a cheater might do. The cheater is somebody who is not complying with the preferred usage, whether this constitutes a criminal act, breach of terms of service, or simply, undesirable behavior. It makes sense to consider the actions and abilities of the cheater both as this cheater may attempt to dupe honest users or more generally affect the marketplace by performing undesirable actions. If we wish to protect the typical user, it is necessary to understand what the cheating user will attempt to do. The third and final part shows how to use an understanding of both the typical user and cheating user to design a practical and secure system. Throughout the book, these three different perspectives are described using a series of examples related to security problems that result in losses and risks on a daily basis, whether these are financial losses, political losses, or losses of privacy. We even consider environmental losses, demonstrating that secure design also impacts disciplines that normally are not considered to even be associated with the realm of computer security. However, this book is not about these particular security problems or these particular solutions; instead, it is about the principles that these are used to illustrate. Simply speaking, the main tenet is (as already stated) that the user matters. But how can we turn that belief into actionable goals and tasks? That is what this book sets out to do. It is my hope that you, the reader of this book, will make sure that you are not trapped in the dangerous and still all-too-common belief that one can build systems without taking the end user into consideration or to first build the system and then address user-related issues as they arise. Instead, I hope that you will be inspired to improve how we, as a society, design and build systems that provide beneficial services that increase our common well-being, as opposed to flawed systems that introduce risks and vulnerabilities. That all starts with recognizing that people make mistakes, and how these mistakes can become opportunities to criminals. Last but not least, many of the specific solutions used to illustrate the principles of this book correspond to patents or other intellectual property rights that belong to a variety of companies working to improve security, privacy, and user interfaces. Therefore, before you commercially deploy any of these technologies, I encourage you to either perform a patent search or to contact me (or one of the contributors of the book) to ask for guidance to make sure that you do not infringe upon these rights. Portola Valley, CA, USA June 2020

Markus Jakobsson

Acknowledgements

This book would never have been possible without its many contributors. The contributors are listed for each chapter and are, in alphabetical order: Enrico Bertini (Chap. 4), Cristian Felix (Chap. 4), Payas Gupta (Chap. 1), Arthur Jakobsson (Chap. 3), Jay Koven (Chap. 4), David Maimon (Chap. 4), Nasir Memon (Chaps. 1 and 4), Toan Nguyen (Chap. 1), and Hossein Siadati (Chaps. 1 and 4). In addition, the research underlying the individual chapters has also benefitted from the guidance of and feedback from a large number of colleagues. They are, in alphabetical order: Michael Barrett (Chap. 2), Keir Finlow-Bates (Chap. 5), Kevin Gallagher (Chap. 1), Steve Gerber (Chaps. 2, 3, and 8), Seda Gurses (Chap. 1), Sima Jafarikah (Chap. 5), Trishank Karthik (Chap. 1), James Linton (Chap. 4), Toan Nguyen (Chap. 6), Sameer Patil (Chap. 1), Paul Sherer (Chap. 6), Hossein Siadati (Chap. 5), Guy Stewart (Chap. 5), Yifan Tien (Chap. 4), John Wilson (Chaps. 4), and Nico Zapiain (Chap. 2). The work underlying Chap. 1 was partially supported by NSF grant 1359601. Last but not least, the creation of this book was made possible by the help of Maja Jakobsson, who helped convert contributions from peer-reviewed papers to chapters.

xiii

Contents

Part I Considering the Typical User 1

Mind Your SMSes: Mitigating Social Engineering in Second Factor Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hossein Siadati, Toan Nguyen, Payas Gupta, Markus Jakobsson, and Nasir Memon

5

2

Permissions and Privacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Markus Jakobsson

23

3

Privacy and Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Markus Jakobsson and Arthur Jakobsson

41

Part II Considering the Malicious User 4

A Framework for Analysis Attackers’ Accounts . . . . . . . . . . . . . . . . . . . . . . . . . Hossein Siadati, Jay Koven, Christian Felix da Silva, Markus Jakobsson, Enrico Bertini, David Maimon, and Nasir Memon

63

5

Environmentally and Politically Conscious Crypto . . . . . . . . . . . . . . . . . . . . . . Markus Jakobsson

91

Part III Designing Solutions Based on Typical and Malicious Users 6

Social Engineering Resistant 2FA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Markus Jakobsson

7

The Rising Threat of Launchpad Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Markus Jakobsson

8

Discouraging Counterfeiting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Markus Jakobsson

9

Seeing the Future . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Markus Jakobsson xv

Part I

Considering the Typical User

Many security professionals make the mistake of designing systems for themselves. Early computer users, for example, can think back to their experience with MS-DOS as an example of this general design approach. I do not mean here that MS-DOS was not secure, but rather, that it was only accessible to a very select few. It was not until the introduction of the graphical user interface that computing became accessible to the masses. These days, any company with a user interface as part of their product needs to carefully consider the user experience (UX) in order to be successful, and large companies commonly employ large teams of highly competent UX specialists. However, many of today’s security and privacy systems still suffer the “MS-DOS problem.” The problem is not whether the layout is good or the text is of the proper size and color. The problem, rather, is that the technical systems—what is under the hood—are not designed with an understanding of the end user. That is beyond the scope of traditional UX design. An illustration of the problem is shown in Fig. I.1. Starting with the maybe most obvious example of an interaction that does not take the typical user into consideration, we see that the “Default” permission corresponds to the startingly incomprehensible Com.sec.android.provider.badge.permission.READ—among other things. This is not likely to mean much to the typical user. As a slightly less extreme example, the app also needs access to the user’s text messages—“SMS or MMS”— but the typical user is not likely to see why. It is therefore likely to be seen as abusive by a user who attempts to understand the permissions request, given that there is no evident reason why Facebook (the app maker) should need that information. A technically knowledgeable observer may suspect that the permission relates to automation of second factor authentication (2FA), but if that were so, it is still overkill, as it would have sufficed for the app to be given access to text messages sent by the app maker to the user to perform that task. However, this more narrow capability is not supported by Android, and an app that wants access to some text messages needs to ask permission to access all text messages. The other permission requests in the figure are also, to various degrees, unclear or worrisome (and often both) to typical users.

2

I Considering the Typical User

Fig. I.1 The figure shows an example of a permissions request that is likely to be discouraging to many users. The typical user will not be able to evaluate the permission request, or may feel abused

This is not an isolated example of something that could have been improved. While the example I use is, perhaps, slightly more extreme than many other instances, the problem is pervasive. Moreover, the problem is not in any way limited to permissions requests. As another example of a problem that is the result of unfortunate user interaction is the deceptive use of display names. The display name of an email is a text associated with the email address of the sender, and chosen by this party. For example, an email from Wells Fargo’s bill paying service will come from Wells Fargo Online . Here, Wells Fargo Online is the display name and the email address. It should, of course, come as no surprise that typical users may get deceived by an email from a hypothetical sender Wells Fargo Online or even from a hypothetical sender Wells Fargo Online , neither of which is associated with Wells Fargo. The problem here is that identity information is conveyed by the display name—Wells Fargo Online—as much (or indeed more than) the extent to which the email address is used to convey identity . . . and there are no controls over who can use what display name. If you wish, you can change your display name today, making it Wells Fargo Online, or anything else you wish. The criminals can, too. In other words, the way the conveyance of identity is designed is vulnerable to abuse and deception. It is not hard to see why criminals use this form of identity deception to pose as trusted parties, whether trusted companies or colleagues, and

I Considering the Typical User

3

Fig. I.2 The figure shows a Google-issued warning shown to a user receiving an email from a “new” sender with the same display name as a (to the recipient) known sender. By drawing the attention of the recipient to the potential risk, abuses such as Business Email Compromise are countered

request funds transfers and sensitive data from their intended victims. This problem is commonly referred to as Business Email Compromise (BEC). The BEC problem is not made any better by Apple’s “smart addresses” which is a viewing preference that causes only the display name to be shown—not the email address. The same aggravation of the problem is common for smart phone mail readers, which due to the very limited screen real estate commonly do not display the email address of the sender, but only the display name. This makes it even harder for people to detect deception attempts. Many email security companies have introduced automated detection methods that identify high-risk emails by determining when emails from non-trusted parties use display names of trusted parties. To detect deceptive display names, the security system identifies the use of a display name that is associated with a trusted party, and which is used with an email address that is not normally associated with this display name. If that happens, the security system can remove or replace the display name. For example, this may result in an email from Wells Fargo Online being shown1 as coming from Warning! Fraud Risk . An offending email can also be blocked, quarantined, or marked up with a warning—see, e.g., Fig. I.2 for an example of the latter. We have offered two examples of security/privacy problems above, and argued that these relate to how typical users interpret information—or misinterpret it. In order to solve these two security/privacy problems—as well as countless others— one has to begin by understanding what the source of the problem is. • In the first example, related to privacy permissions, the problem is that typical users do not know what they are being asked to share, and cannot determine whether it is reasonable. Sometimes, in addition, the requester is forced to ask

1 The

decision of whether to change the display name to a warning or to block the email altogether may depend on a risk assessment or a policy of the organization receiving the potentially deceptive email.

4

I Considering the Typical User

for too broad permissions, as they simply do not have any meaningful option. This may collectively lead to reduced install rates, to less engaged users, to undesirable sharing of data, and to fears of undesirable sharing of data. • In the second example, related to deceptive emails, the problem is that users cannot determine when something is safe, and when it is not. Since most emails people open are safe, and sent from the people they appear to be sent from, it is natural for people to let down their guards and treat the display names as a meaningful indicator of whom the emails are from. However, the above examples summarizes the problems, not the source of the problems. This is much deeper. In my view, the source of these two problems— and many like these—is that the designers of the systems did not think about what could go wrong, and did not recognize how the end users would understand (or misunderstand) the information they were given. This first part of this book consists of three chapters that illustrate different aspects of this issue: • In Chap. 1, we describe a problem related to second factor authentication (2FA), and how the problem could easily be dramatically reduced by understanding the typical user. • This is followed, in Chap. 2, with an example related to how to convey permissions requests that are more accessible to typical users; the principle there is to model the system, and the permissions, around what is comprehensible instead of what is commonly done today (which was illustrated in Fig. I.1). • In Chap. 3, a case study is described in which, among other things, typical users are given the option to choose how tracking is performed and associated information is used to provide services. The common aspect for these three chapters is that they focus on the needs and capabilities of the typical user—as opposed to those of a highly capable specialist. This is an important ingredient in any system that takes the end user into consideration.

Chapter 1

Mind Your SMSes: Mitigating Social Engineering in Second Factor Authentication Hossein Siadati, Toan Nguyen, Payas Gupta, Markus Jakobsson, and Nasir Memon

Abstract SMS-based second factor authentication is a cornerstone for many service providers, ranging from email service providers and social networks to financial institutions and online marketplaces. Attackers have not been slow to capitalize on the vulnerabilities of this mechanism by using social engineering techniques to coerce users to forward authentication codes. We demonstrate one social engineering attack for which we experimentally obtained a 50% success rate against Google’s SMS-based authentication. At the heart of the problem is the messaging associated with the authentication code, and how this must not have been developed with security against social engineering in mind. Pursuing a top-down methodology, we generate alternative messages and experimentally test these against an array of social engineering attempts. Our most robust messaging approach reduces the success of the most effective social engineering attack to 8%, or a sixth of its success against Google’s standard second factor verification code messages.

H. Siadati Google LLC, Infrastructure and Cloud, New York, NY, USA T. Nguyen Department of Security R&D, Salesforce.com Inc., San Francisco, CA, USA P. Gupta Pindrop, Atlanta, GA, USA M. Jakobsson () ZapFraud Inc., Portola Valley, CA, USA N. Memon New York University, Computer Science and Engineering, Brooklyn, NY, USA © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2020 M. Jakobsson (ed.), Security, Privacy and User Interaction, https://doi.org/10.1007/978-3-030-43754-1_1

5

6

H. Siadati et al.

1.1 Problem Statement It is increasingly recognized that human factors constitute the weakest link in the security of individuals and organizations. In the last 10 years, social engineering has become a staple in the arsenal of criminals, whether petty thieves or nation state backed actors. Social engineering is used to commit fraud; to trick users to install malware; to willingly share sensitive corporate information; and, to perform transfers of large sums of money [23, 25]. A common use of social engineering, in the form of phishing attacks, is to steal user credentials. During the last few years, it has also increasingly been used to defeat out-of-band authentication methods. To authenticate a user initiating a critical action—e.g., a password reset, user account setup or a fund transfer—an out-ofband authentication system generates a one time random secret (also known as “Verification Code” or “Security Code”) and sends this to the user by a separate channel (for example, on a mobile phone using a number on file for the user). On receiving the code, the user relays it back to the service provider using the original channel (for example, the web channel which was used for password reset). If the user-entered code matches what was sent, the user is authenticated and the requested action is completed. The assumption is that the second channel will be hard for an attacker to access even if the attacker has some knowledge of user credentials. The main objective of out-of-band authentication is to build a second layer of authentication that can be used even when a user’s credentials are partially or fully known to an attacker. (Knowledge of user credentials is a realistic threat given the frequency of large-scale data breaches [22], the success rates of targeted phishing attacks and the prevalence of malware [27, 46, 48].) Social engineering attacks against out-of-band authentication essentially trick the user to relay the verification code to the attacker. Symantec has reported an increase in the number of such attacks [47] and previous work shows that a social engineering attack on out-of-band authentication methods can have success rates of 25% [44]. Balduzzi et al. found 22 instances of such attacks using a mobile honeypot in China [5]. These attacks are based on tricking an account owner to forward a verification code to the attacker. An attack of this kind typically begins with the attacker triggering a verification code to be sent to a user from a service provider. Then by sending a deceptive message to the user, the attacker dupes him into sharing the code. This is typically done by posing as the service provider. Once the attacker receives the code, he can complete the action on the user’s account. Figure 1.1 shows the steps of the attack. It is easy to see that different attackers may try different social engineering approaches in an attempt to trick users to forward verification codes. These different attack messages will have different success rates. The attacker’s success rate does not only depend on the attack message, though, but also on the formulation of the message sent by the service provide along with the verification code. We identify social engineering pitches capable of coercing 50% of users, in the context of Google’s reset code. Google’s SMS-based second factor authentication

1 Mind Your SMSes: Mitigating Social Engineering in Second Factor Authentication

7

Fig. 1.1 The figure illustrates the typical flow of a social engineering attack on 2FA, in which an attacker uses deception to trick a victim to forward a security code that enables the attacker to compromise the victim’s account. A related attack can be used against token-based 2FA solutions

(2FA) epitomizes simplicity: “Your Google verification code is 654722.” This is an approach taken by many service providers, including Microsoft (“Microsoft account verification code: 1505”), Wells Fargo (“Use Wells Fargo verification code 817882”) and many others. Other verification messages contain a warning—such as that used by AT&T (“AT&T Free Msg: Your Temporary password is 987876. If you did not request a temporary password, call 1-800-ATT-2020. We’ll never contact you to ask for this password.”) These are slightly more robust against attack, with a 34% success rate for our most effective experimental attack. However, such messages are abusable by an attacker who uses a similar message to get a victim to call a rogue number under the attacker’s control. In this chapter we investigate how social engineering attacks, specifically on SMS-based out-of-band authentication, could be mitigated by improving the messages used during the authentication process. This chapter therefore mainly focuses on designing better verification messages. Different verification messages are tested against a variety of systematically generated and tested attack messages. In order to perform a systematic study of the effectiveness of verification messages in mitigating social engineering attacks, we first identify what attack messages seem most likely to coerce users. Then, we turn to developing and testing a collection of verification messaging alternatives, and to enunciate the principles underlying the most robust approaches. We find that when a warning “Please ignore this message if you did not request a code” precedes the authentication code, it stands up to social engineering attempts1 better than any other formulation, resulting in a susceptibility to attack of just 8%. The main contribution of this chapter lies in 1 Whereas

we only tested this method in the context of SMS-based 2FA, we have no reason to believe that the identified principles would not apply to other security-related messaging contexts.

8

H. Siadati et al.

the notable reduction of the expected yield as seen by a cunning attacker employing social engineering to steal verification codes. However, of potential independent interest are the lightweight experimental techniques we developed and used in the underlying study. These include our techniques to identify likely contenders— whether for most successful attacker approach or most robust verification message. Another such technique is for a realistic testing of the exact yields of various attacks, when applied to various verification messages. These lightweight tools allow us to concurrently argue about hypothetical approaches in two dimensions, corresponding to what the good guys should do in light of what the attackers might try, and to select the verification message with the greatest robustness against attack. In Chap. 6, another approach to address the problem is described.

1.2 Designing Attack Messages As can be seen from the previous section, the attack does not require much knowledge about the target. The attacker only needs to know the email address and phone number associated with the account, and both can be easy to acquire. Data breaches are one source of such information. For example, the Anthem breach [33] involved the loss of email addresses and phone numbers, as well as other personal information, of 80 million customers. Malware is another means of collecting such information. Specifically, smartphone malware such as Asacub [27], Ackposts [46], and Godwon [48] are known for harvesting contact lists, along with other valuable information. Moreover, vulnerabilities in messaging apps have been shown to enable attackers to harvest large amounts of contact information in a short interval of time [21, 28, 32]. Given the ease with which an attacker can gain access to information to launch an attack, the main challenge lies in tricking users to send their verification codes. However, it turns out that users are quite susceptible and easy to deceive. For example, previous work has shown that if the following message is sent by the attacker soon after a victim receives a verification code due to action triggered by the attacker, it achieves a success rate for the attacker of 25% [44]:

Please verify that your phone is still associated with your Gmail account by replying to this message with the code we have just sent to you.

One reason for the ease of deceiving users is due to the shortcomings of the messages used by service providers. This begs the question—can better messages be designed that may be more robust against attacks? However, any such verification message must resist different types of attack messages.

1 Mind Your SMSes: Mitigating Social Engineering in Second Factor Authentication

9

To be successful, social engineering attack messages must justify to the user why she has received an unsolicited verification code, and the reason that she should send the code back. To study the extent to which the attack can be successful, we systematically generated a number of attack messages using three categories of storylines: 1. Compliance: Companies contact customers to update them about new policies and terms of service. Users expect to be contacted by service providers if they misuse an online service. 2. Service: Users are persuaded by promotions from service providers (e.g., larger email inbox), or by communication messages from their friends offered via service provider (e.g., voicemail). 3. Security: Companies usually communicate with users concerning the security of their account. Users are asked to change their passwords, notified of changes in the security settings of their accounts, and contacted if an abnormal activity, such as login from a new computer, is observed. Therefore, users expect to receive such messages from service providers. In addition, previous work has studied persuasion tools for phishing [1, 10, 36]. Abraham et al. [1] have classified phishing persuasion techniques to “Curiosity, empathy, and excitement”, “Fear”, and “Greed”. We have used these classes of persuasion techniques for the tone of phishing messages in the previously listed classes. Combining storylines with different classes of persuasion, we generated 16 different attack messages (See Table 1.1) to test attacks on SMS-based out-of-band authentication systems. We first tested them against the Google SMS-2FA system (using a field experiment explained in Sect. 1.3) and found significant variation in success rates for the attacker. For example, we found that one attack yields a success rate of 50% against Google’s SMS-2FA system. The attack messages also formed the basis of our study as we examined the efficacy of designed verification messages to resist deceptive attack messages. The next sections explain the principles and the procedures that helped us to identify effective countermeasure messages.

1.3 Methodology In this section, we describe the methodology used to reach the goal of finding an effective verification message that reduces the susceptibility of users to code forwarding attacks.

10

H. Siadati et al.

Table 1.1 Generated attack messages. We approximated the phishing ratio of each message using an online experiment. Attack messages with a phishing rate above 20% (highlighted in the table) were selected for further field experiments # x1 x2

x3 x4 x5

x6 x7 x8 x9

x10 x11 x12 x13

x14 x15 x16

Message We have received a report of harassment related to your Gmail account. Please reply with the verification code we just sent to you to receive the details privately We have received a complaint of abuse of your Gmail account. Please reply with the verification code we just sent to you to receive the details privately You have a voicemail on Google Voice. To listen, please reply with the message code we just sent to you You have a private message on Google Voice. To listen, please reply with the message code we just sent to you Your Gmail storage is expiring soon. To continue receiving new emails, please reply with the confirmation code we just sent to you We are verifying your Gmail account. To continue your Gmail service without any interruption, please reply with the verification code we just sent to you To continue using SMS-based verification for your Gmail account, please reply with the verification code we just sent to you You have received a voice mail with delivery confirmation. Respond with the code to have it delivered We are processing your request. To cancel the requested action on your Gmail account, you may reply with the verification code we just sent to you within 30 min Your Gmail account has been accessed from Nigeria. If you do NOT want to authorize further access, please reply with the confirmation code we just sent to you We received a request from you to delete all your Gmail messages from last month. To cancel, please reply with the verification code we just sent to you Did you request a password reset for your Gmail account? To cancel, please reply with the verification code we just sent to you Your account has been limited due to a login attempt from an unknown location. To block the access, please reply with the verification code we just sent to you Your password has been reset. Use the one-time code sent in the previous SMS. Log in now using the one-time code in the previous SMS and select a new password There was a password reset request for your Gmail account. To cancel, please forward the verification code we just sent to you Did you request a password reset for your Gmail account? Delete this message if you did. Otherwise, send “Cancel” + the verification code we just sent to you

Category Compliance

Compliance

Service Service Service

Service

Service

Service Security

Security

Security

Security

Security

Security

Security Security

1 Mind Your SMSes: Mitigating Social Engineering in Second Factor Authentication

11

• Phase 1: Identify Strong Attack Message Candidates. Using Amazon Mechanical Turk, an online service for requesting end-users to perform small tasks, we asked a large number of users how they would have acted in hypothetical situations, where each such hypothetical situation corresponds to an attack. This way, we tested the likely response for each candidate attack message, where each was tested on a large number of users. This type of security role playing is illustrated in Fig. 1.2a, b. The outcome of phase 1 was an approximate ranking, in terms of strength, of the candidate attack messages. • Phase 2: Test the Strongest Attack Messages Using a Naturalistic Phishing Experiment. We enrolled a large number of users in an experiment where they would be sent a verification code (all of the format Google uses) and then be exposed to an attack using exactly one of the attack messages. The strength of the attack messages would be determined by assessing which one caused the highest percentage of users to forward the verification code. While the subjects were never at risk, since the verification codes that were “stolen” did not actually come from an actual service provider of theirs. (A naturalistic phishing experiment is one that cannot, practically speaking, be distinguished from a real attack by the “victim”, but which is designed not to inflict any harm.) The outcome of phase 2 was a realistic assessment of the risks associated with the different attack messages selected in phase 1. • Phase 3: For the Strongest Attack Message, Test What Countermeasure Worked Best. The countermeasures to the attacks are the formulations in the messages sent along with the verification codes, and correspond to the various warnings described before. Using another group of users from those in phases 1 and 2, different candidate verification code messages were tested in combination with the strongest attack message from phase 2. The result of this phase was the verification message that made it the least likely that a user would fall for the strongest attack among those we tested.

In the first phase, the subjects were asked to select among four options: (1) I will delete both of the messages, (2) I will reply quickly to the second message, (3) I will reply to the second message whenever I am free, and (4) Other. We counted the subjects that selected options 2 or 3 as phished users as these options signify intention to send the verification code to the attackers. We surveyed 800 subjects from MTurk to test 16 variations of attack messages (subjects were randomly and almost equally assigned to the variations). Using the results of this experiment, we eliminated the attack messages with a yield below 20% from further evaluation in

12

H. Siadati et al.

Fig. 1.2 Two snapshots of phone screens presented for an online survey. The participants were asked what they would do if they receive these two messages one after the other. (a) Google’s verification message. (b) Content of verification code forwarding attack message

the next experiment. The phishing messages that we tested, as well as the selected messages for the field experiment (highlighted), are shown in Table 1.1.

1.3.1 Designing the Experiment We followed a phishing experiment design that closely resembles the verification code forwarding attack [44]. For this purpose, we used two phone numbers, phone A and phone B, with a Mountain View, CA area code (the area code of Google’s headquarters). During the experiment, phone A was used to send verification codes that were supposedly coming from a legitimate service provider (Google in our experiment), and phone B was used to send the phishing messages from attackers and receive users’ replies. We detail the recruitment process and the experiment’s procedure in this section. 1. Subject Recruitment: We recruited participants for our experiment by posting an advertisement for a user study on Craigslist [11] (New York/ Job, etc., and New York/ Community/Volunteers) and via Amazon Mechanical Turk (US Only) [37]. Due to the nature of the study, we only recruited participants who were between the age of 18 and 65. We did not disclose the use of phishing or any other details of the user study and only described it as a “study concerning the usage of communication devices (e.g., phone, computers, etc.)”. The subjects gave us consent to use the information that they provided, including email addresses and phone numbers, to contact them for the purpose of the study. As the first step of the study, we asked the subjects to fill out a survey over the phone by calling and replying to a number of questions to an interactive voice response (IVR) system. The IVR system asked a total of eight questions

1 Mind Your SMSes: Mitigating Social Engineering in Second Factor Authentication

13

about their age, gender, number of devices, and online activities. Along with recording the subjects’ responses, the IVR system stored the caller ID of the participants. We used these phone numbers as the contact point of subjects for the experiment. We identified those who called from landlines or VOIP phones, using Twilio’s phone lookup service, and eliminated them from the rest of the study because they could not receive SMSes. However, all participants who took the IVR survey were given a chance to win a raffle for five $20 Starbucks gift card and one $200 gift card. After finishing the phone survey, we sent an email to the participants and told them that we will send them an email containing a link to the final survey within the next few days. This was done to prime them to believe that we will not communicate with them via any other channel, including SMS. 2. Sending a Verification Message: In this step, we sent a verification message from phone A to the subjects’ phone number. The content of the verification message was varied for different group of subjects depending on the treatment under study. However, in order to avoid likely bias from the verification code in the message, we used the same verification code (i.e., “312985”) for all treatments. We sent these messages on Tuesdays and Wednesdays between 3– 5 p.m. 3. Sending a Phishing Message: 30 seconds after the subjects received the verification code, we sent a phishing message to each participant using phone B (attacker’s phone). The content of phishing message was varied based on the treatment under study. The message aimed to deceive the users to send the verification code. We measure the success of different attack messages based on the percentage of subjects who sent the verification code to phone B. 4. Online Exit Survey: We debriefed the participants a day after we sent the phishing messages. This delay was applied mainly because we did not want to tip off potential “victims” ahead of time (some participants sent the code after 4 h). We also invited them to fill out an online survey mostly concerned with the SMS-based 2FA and their experience with SMS messages they received during the experiment.

1.3.2 Phase 2 Experiment: Identifying an Effective Attack Message We designed this experiment to identify the most effective attack message among the messages that we selected in the first phase. We estimated their phishing rate using the phishing experiment described in Sect. 1.3.1. In the phishing experiment, we experimented with the selected attack messages against Google’s verification message: mg = Your Google verification code is 312985. We kept the verification code constant for all the subjects to avoid potential bias.

14

H. Siadati et al.

1.3.3 Phase 3 Experiment: Identifying the Success of Warning Components Against the Most Effective Attack Message We designed the phase 3 experiment to evaluate the effectiveness of the new verification messages against the most effective attack message from phase 2. To create the new verification messages, we combined the warning components that were previously selected as part iof the experiment with the Google’s verification message (mg ). We used the new verification messages as the first message in the experiment. We used the most effective message found in phase 2 as the attacker’s message in this experiment. It was not clear upfront whether the warning should be placed in the front of the message or at the end of the message. While some researchers find it more effective to place the warning before instructions [51], others disagree [35]. To evaluate both of the ordering methods, we created two variations of verification message for every warning; (a) Original message preceding Warnings: by appending warning components to mg e.g. mg .w3 forming “Your Google verification code is 312985. Please ignore this message if you did not request a code.” (b) Warnings preceding original message by appending mg to warning components e.g. w3 .mg forming “Please ignore this message if you did not request a code. Your Google verification code is 312985.” In total, we tested nine variations, four messages where warnings are preceded by mg message and four messages where mg is preceded by warnings. Moreover, we also tested the original mg message against the most effective attack message found in phase 2.

1.3.4 Ethical Considerations We followed recommendations from the literature for designing ethical and accurate phishing experiments [19, 26]. First, we conducted experiments on real and diverse populations rather than only students, in field studies instead of “closed-lab” environments which might alert subjects about the real study and likely affect the outcome of the experiments. Our recruited subjects were not aware of the phishing experiments for which they were being tested. Instead, they expected to receive an online survey about their computer usage (the exit survey). Thus, the result of our experiment could reflect the real success rate of code forwarding attacks in the wild. Second, we did not tamper with subjects’ accounts and did not trigger real verification messages from the service provider (in this case, Google). We created and sent verification messages from our own phone number. No harm or service disruption was made to real subjects’ accounts. Finally, our study was approved by the New York University Institutional Review Board.

1 Mind Your SMSes: Mitigating Social Engineering in Second Factor Authentication

15

1.4 Evaluation In this section we present the results obtained from our experiments. We also discuss the effects of the content and order of the two components of the verification messages sent in the phase-3 experiment, and study the confounding factors that may influence the results. Our approach for evaluation is not only statistical, but findings are also supported by the recurrent trends in the results of the experiments. In addition, we report some other interesting findings about user attitudes towards the SMS-2FA and differences among participants who were successfully phished and those who were not. The experiment involved a total of 334 subjects using the procedure described in Sect. 1.3.1. Out of the 334 participants, 100 were chosen randomly for the phase-2 experiment, and 234 for the phase-3 experiment. The success of a phishing attack in our experiments is computed based on the phishing ratio, i.e., the number of subjects who sent the verification code they received to the attacker.

1.4.1 Success of Attacks in Phase 2 We tested nine different attack messages against Google’s standard verification message on a total of 100 subjects. The attack messages demonstrate different success rates as shown in Table 1.2. The attack message x16 , “Did you request a password reset for your Gmail account? Delete this message if you did. Otherwise, send “Cancel” + the verification code we just sent to you.” was the most effective attack message with a phishing success rate of 60%. We used this message for evaluating our designed verification messages.

1.4.2 Success of Verification Messages in Phase 3 In phase 3, we used x16 , the most effective attack message from phase 2, to evaluate the performance of new verification messages we designed. Each new verification message is composed of a warning component (wi ) and Google’s standard verification message (mg ). To study the effect of the order of warning and verification code components, we tested both: (a) a warning preceding the code (wi .mg ), and (b) the code preceding a warning (mg .wi ). Google’s standard verification message (mg ) was also included in the experiment separately as the control group. Table 1.2 shows the result of the experiment for each verification message. We found that the verification message w8 .mg , “Please ignore this message if you did not request a code. Your Google verification code is 312985” had the lowest phishing rate, and can reduce susceptibility of the users to code forwarding

16

H. Siadati et al.

Table 1.2 Phishing rates for the experiments. In the second phase, a number of generated attack messages were tested against Google’s standard verification message mg = “Your Google verification code is 312985” to find the most effective attack message. In the third phase the performance of our new verification messages were evaluated against best attack message (i.e., x16 ) from phase 2 (a) Phase 2 Verification message mg mg mg mg mg mg mg mg mg (b) Phase 3 Verification message mg .w3 w3 .mg mg .w8 w8 .mg mg .w13 w13 .mg mg .w16 w16 .mg mg

Attack message x2 x4 x5 x8 x11 x13 x14 x15 x16 ∗ Attack message x16 x16 x16 x16 x16 x16 x16 x16 x16

# subjects 15 10 10 10 10 15 10 10 10

# subjects 26 26 26 26 26 26 26 26 26

% phished 34 38 31 8* 50 23 23 11 50

% phished 33 10 40 30 30 13 10 10 60

% avg. phished 36 20 36.5 17 50

attacks dramatically. Out of 26 participants, only 2 (8%) fell for the attack when using this message. This is six times less than the phishing rate of 50% for the mg . Using Bonferroni corrected pair-wise comparisons, we compared different verification messages using Fisher’s Exact test. Specifically, Fisher’s Exact test with Bonferroni correction between mg and w8 .mg shows that there is a high statistically significant difference between the phishing ratio of two verification messages (pvalue