Critical Statistics: Seeing Beyond the Headlines [1 ed.] 1137609796, 9781137609793

Citation preview

‘Never be lied to by statistics again: this book will teach you everything you need to know to combat dodgy data, spot shoddy stats, and to start constructing your own robust and reliable statistics.’ – Jude Towers, Lancaster University, UK ‘This is precisely the type of book needed to empower the public to disentangle the valid from the invalid in the information age.’ – Jennie E. Brand, UCLA, USA ‘This book is a must for those taking introductory statistics courses. It provides real-world every day examples of the use of statistics in everyday life.’ – John Jerrim, UCL, UK ‘70% of married women have cheated on their partners’ ‘32,000 people in the US die from gun violence every year’ The number is the story – but what if the story isn’t true? Statistics can inform and enlighten – but they can also mislead and deceive. This accessible and entertaining new textbook will provide you with the skills you need to understand the barrage of numbers we encounter in our everyday lives and studies. Almost all the statistics in the news, on social media or in scientific reports are based on just a few core concepts, including measurement (ensuring we count the right thing), causation (determining whether one thing causes another) and sampling (using just a few people to understand a whole population). By explaining these concepts in plain language, without complex mathematics, this book prepares you to meet the statistical world head on and to begin your own quantitative research projects. Key features include: • Abundant real-world examples demonstrating the ubiquity of numbers, where they come from and how to understand them. • ‘Seeing Beyond the Headlines’: A step-by-step guide to applying each chapter’s concepts to real statistics. • A wealth of online resources including more examples and exercises, links to key datasets and a portal to share your own examples of bad statistics which can be found on www. macmillanihe.com/devries-critical-statistics. Ideal for students facing statistical research for the first time, or for anyone interested in understanding more about the numbers in the news, Critical Statistics helps you see beyond the headlines and behind the numbers. Robert de Vries is a lecturer in Quantative Sociology at the University of Kent, UK, where his innovative Critical Thinking course introduces first-year social scientists to the mysteries of statistics. His research focuses on social and economic inequality.

Cover image by Andrew Davis

‘Never be lied to by statistics again: this book will teach you everything you need to know to combat dodgy data, spot shoddy stats, and to start c­ onstructing your own robust and reliable statistics.’ —Jude ­Towers, Lancaster University, UK

‘This book provides a straightforward and timely tutorial in how to make informed decisions regarding which statistics are to be trusted. This is precisely the type of book needed to empower the public to disentangle the valid from the invalid in the information age.’ —Jennie E. Brand, UCLA, USA

‘This book is a must for those taking introductory statistics courses. Rather than being a dry, technical textbook, it provides real-world every day examples of the use of statistics in everyday life.’ —John Jerrim, U ­ niversity College London, UK

‘This book provides a welcome complement to the wide range of “how to do ­statistics” books that are available. It takes students in measured steps, ­providing useful exercises along the way.’ —Gillian Whitehouse, University of Queensland, Australia

‘Critical Statistics provides an accessible and entertaining tour through the ways that statistics can be used to mislead us. It’s a thorough introduction for people who shudder at the thought of data, but people who see themselves as experts will learn something from this too.’ —Mark Taylor, University of Sheffield, UK

‘This is a highly readable introduction to the ways numbers are manufactured and misrepresented in today’s society, teaching the importance of thinking ­critically about statistics and showing how to do better in our own learning and research. In the age of fake news, this is essential reading for all students of the social sciences.’ —Richard Harris FAcSS, University of Bristol, UK

‘This is the perfect statistics book in an era in which it is so difficult to navigate the numbers and data we are exposed to in our everyday life. It helps the reader – anyone, from students to more expert readers – understand how difficult it is to interpret and utilize statistics in the news, and it teaches how to make better use of the incredible amount of data available today.’ —Maria Sironi, University College, London, UK

‘This is a most impressive teaching resource. De Vries adopts an e­mbedded approach to introduce students to statistical reasoning, guaranteed to increase student engagement with quantitative methods and to encourage a ­much-needed critical eye to quantitative evidence.’ —Stella Chatzitheochari, University of Warwick, UK

‘Darrell Huff’s How to Lie with Statistics is rightly a classic but after over 60 years is inevitably showing its age. Robert de Vries “seeing beyond the ­headlines” is a much needed Huff for our times. It takes the reader skilfully through the ­essentials of statistics by taking apart the way in which numbers are routinely misused by the media. The result is not only an engaging ­introduction to ­statistics but an essential guide in the “post truth” era of “alternative facts” and “fake news”.’ —John MacInnes, University of Edinburgh, UK

Critical Statistics: Seeing Beyond the Headlines Robert de Vries

© Robert de Vries, under exclusive licence to Springer Nature Limited 2019 All rights reserved. No reproduction, copy or transmission of this publication may be made without written permission. No portion of this publication may be reproduced, copied or transmitted save with written permission or in accordance with the provisions of the Copyright, Designs and Patents Act 1988, or under the terms of any licence permitting limited copying issued by the Copyright Licensing Agency, Saffron House, 6–10 Kirby Street, London EC1N 8TS. Any person who does any unauthorized act in relation to this publication may be liable to criminal prosecution and civil claims for damages. The author has asserted his right to be identified as the author of this work in accordance with the Copyright, Designs and Patents Act 1988. First published 2019 by RED GLOBE PRESS Red Globe Press in the UK is an imprint of Springer Nature Limited, registered in ­England, company number 785998, of 4 Crinan Street, London N1 9XW. Red Globe Press® is a registered trademark in the United States, the United Kingdom, Europe and other countries. ISBN 978–1–137–60980–9 hardback ISBN 978–1–137–60979–3 paperback This book is printed on paper suitable for recycling and made from fully managed and sustained forest sources. Logging, pulping and manufacturing processes are expected to conform to the environmental regulations of the country of origin. A catalogue record for this book is available from the British Library. A catalog record for this book is available from the Library of Congress.

Don’t take anyone’s word for it. — Motto of the Royal Society (roughly translated)

Contents

List of Figures and Tables

ix

Abbreviations xi Preface xii Tour of the Book

xiv

Guide to the Website

xv

Acknowledgements xvii

1.

2.

99% of statistics are made up

1

The world runs on numbers Statistics in the fake news era Don’t be part of the problem

6 9 12

Where do numbers come from?

17

Making the news 20 #nofilter 23 Where’s the harm? 27 A lie can run around the world before the truth can get its boots on 32

3.

4.

Samples, samples everywhere …

38

It’s samples all the way down … Size matters Bad samples Spotting biased samples The magic of sampling

39 44 54 57 62

Measure for measure

68

The dark side of immigration in open, generous Sweden Define your terms How to lie with definitions Counting is hard Asking the right questions There’s no such thing as a perfect statistic

68 71 74 77 80 86

vii

Contents

5.

6.

7.

8.

What does it mean to be average?

91

Average man The mean doesn’t always mean what you think it means Why doesn’t everyone know this already? The median: The mean’s under-appreciated brother Averages are not real

91 94 97 99 105

Fraction of a man

111

There are two kinds of data in the world What’s the point of percentages? Percentages – backwards and forwards Risky business Statistics aren’t real

111 113 118 120 124

Cause and effect

131

Kill or cure CSI: Causation The drugs don’t work ‘Have smartphones destroyed a generation?’ Establishing causation is not impossible The rush to infer causation

133 135 139 143 148 150

Bad graphics

156

Electioneering 156 Charts as a collection of ‘visual metaphors’ 158 Bad charts: A spotter’s guide 161 Nonsense graphs 182

9.

Context is everything

192

‘Is that a big number?’ Four questions Camera tricks

194 194 206

10. Do it yourself The gender pay gap Writing up the results How to use statistics to tell the truth

210 211 224 230

Notes237 Index246

viii

LIST OF FIGURES AND TABLES

Figures 3.1 Bar chart visualising the results of the IFOP infidelity survey 52 3.2 IFOP infidelity bar chart, with error bars added 53 4.1 Screenshot captured from @SenatorBaldwin Twitter feed 75 5.1 Histogram showing the distribution of Instagram followers in your group 95 5.2 Histogram showing the distribution of Instagram followers in your group, plus Anthony 97 5.3 Histogram of a hypothetical distribution of weights 103 5.4 Histograms of possible distributions of 100m times 104 5.5 Histogram of hypothetical IQ distribution among Bingons and Cromulans 106 8.1 Photograph of James Barber’s election leaflet 156 8.2 My corrected visualisation of the figures provided in the Liberal Democrat leaflet 158 8.3 William Playfair’s bar chart of imports and exports in Scotland 159 8.4 Florence Nightingale’s polar area diagram of the causes of death of British soldiers in the Crimean war 160 8.5 Box office receipts of Transformers movies and best picture winners 162 8.6 Screen capture of Fox Business bar chart 163 8.7 Screen capture of Gizmodo’s comparison of iPad battery sizes 164 8.8 Pie chart showing the gender breakdown of movie protagonists 166 8.9 Pie chart showing the gender breakdown of movie protagonists, now three-dimensionalised 167 8.10 Pie chart of support for 2012 Republican presidential candidates 168 8.11 Line chart showing global temperatures by year 169 8.12 Screenshot captured from @NRO Twitter feed 171 8.13 Line graph showing hypothetical changes in weight over time 172 8.14 Line graph showing changes in Planned Parenthood procedures between 2006 and 2013 173 8.15 Bar chart of homicide rates by country 176 8.16 Bar chart of average homicide rates for high (Gini index ≥34) and low (Gini index