Uses and Misuses of Inflation and Price Indices 152757069X, 9781527570696

Citation preview

Uses and Misuses of Inflation and Price Indices

Uses and Misuses of Inflation and Price Indices By

Jeff Ralph, Paul A. Smith and Robert O’Neill

Uses and Misuses of Inflation and Price Indices By Jeff Ralph, Paul A. Smith and Robert O’Neill This book first published 2024 Cambridge Scholars Publishing Lady Stephenson Library, Newcastle upon Tyne, NE6 2PA, UK British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Copyright © 2024 by Jeff Ralph, Paul A. Smith and Robert O’Neill All rights for this book reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the copyright owner. ISBN (10): 1-5275-7069-X ISBN (13): 978-1-5275-7069-6

CONTENTS

List of Figures and Tables ......................................................................... ix Preface ....................................................................................................... xi Acknowledgements ................................................................................. xvi Abbreviations ......................................................................................... xvii 1

A time of rising prices ....................................................................... 1 1.1 Price rises in context ................................................................. 2 1.2 Overall price change and inflation ............................................ 3 1.3 Inflation and its effects on households ...................................... 7 1.4 Can we avoid inflation? ............................................................ 9 1.5 Inflation adjustment ................................................................ 10 1.6 Responding to inflation in practice ......................................... 11 1.7 Three case studies ................................................................... 14 1.7.1 Measures of inflation ...................................................... 14 1.7.2 The political context ....................................................... 15 1.7.3 Child benefit ................................................................... 16 1.7.4 The State Pension ........................................................... 17 1.7.5 Nurses’ pay..................................................................... 20 1.8 The statistical and the political ................................................ 24

2

What is inflation? ............................................................................. 29 2.1 Inflation and monetary economics .......................................... 29 2.2 Is there a “right” level of inflation? ......................................... 30 2.3 A price index and index numbers............................................ 31 2.4 Users and uses ......................................................................... 34 2.5 An act of measurement............................................................ 36 2.6 Towards the measurement of the level of prices ..................... 37 2.7 Marketplace and consumer challenges .................................... 38 2.8 Complex commodities ............................................................ 39 2.9 An international effort ............................................................. 40 2.10 What are we trying to measure? .............................................. 41 2.11 Is inflation the same for everyone? ......................................... 43 2.12 Concluding remarks ................................................................ 45

vi

Contents

3

How do we measure inflation? ........................................................ 48 3.1 Changing costs and changing value ........................................ 48 3.2 Constructing a price index ...................................................... 49 3.3 Gathering the price information .............................................. 50 3.4 Constant quality and quality adjustment ................................. 52 3.5 Gathering the weighting information ...................................... 55 3.6 Putting the information together ............................................. 57 3.7 An ideal inflation index........................................................... 57 3.8 Assessing sampling and non-sampling errors ......................... 59 3.9 Long-term and short-term change ........................................... 60 3.10 Concluding remarks ................................................................ 61

4

A brief history of inflation measurement ......................................... 63 4.1 A long history of price fluctuations......................................... 63 4.2 Comparing the value of money at two time periods................ 64 4.3 The general level of prices and how to calculate it ................. 65 4.4 Capturing price data ................................................................ 66 4.5 The state intervenes ................................................................. 67 4.6 The First World War and the start of indexation..................... 68 4.7 Post-war wage indexation ....................................................... 69 4.8 The Second World War and a new index ................................ 70 4.9 Revisions to the RPI ................................................................ 72 4.10 Concluding remarks ................................................................ 74

5

Why is inflation measurement controversial? .................................. 78 5.1 A new measure arrives ............................................................ 78 5.2 Differences between the RPI and the HICP ............................ 79 5.3 Elementary aggregates and the RPI ........................................ 80 5.4 Elementary aggregates in the HICP and beyond ..................... 82 5.5 Changing the inflation target ................................................... 84 5.6 Developments in the 2000s ..................................................... 87 5.7 A controversial switch of indexing measure ........................... 89 5.8 Deciding which measure to use............................................... 92 5.9 The status of the RPI and its continued uses ........................... 93 5.10 Owner Occupiers’ Housing in the CPI .................................... 96 5.11 Reviews of consumer price indices ......................................... 97 5.12 The long-term uses of the RPI ................................................ 99 5.13 A mixed position for indexation ........................................... 100 5.14 Legal actions and an uncertain future ................................... 101 5.15 Towards a resolution ............................................................. 101 5.16 Concluding remarks .............................................................. 104

Uses and Misuses of Inflation and Price Indices

vii

6

What can the changing basket tell us? ........................................... 114 6.1 The size of the basket over time ............................................ 115 6.2 What goes in the basket? ....................................................... 115 6.3 Interpreting the changes in the basket ................................... 117 6.4 The basket over the long term ............................................... 118 6.5 Expenditure shares ................................................................ 120 6.6 The price of milk ................................................................... 122 6.7 Price data sources .................................................................. 123 6.8 Consumer behaviour ............................................................. 125 6.9 Concluding remarks .............................................................. 126

7

Quality and trust ............................................................................ 130 7.1 The big picture of official statistics ....................................... 131 7.2 Assessing trust, trustworthiness and quality.......................... 132 7.3 The road to independence for UK official statistics .............. 134 7.4 Understanding statistical quality ........................................... 137 7.5 Official statistics and scientific practice................................ 139 7.6 Official statistics and industrial production........................... 140 7.7 Sharing knowledge and experience ....................................... 142 7.8 Best practice standards .......................................................... 143 7.9 Statistical reviews ................................................................. 145 7.10 Challenges for official statistics ............................................ 147 7.11 Concluding remarks .............................................................. 148

8

Other price statistics....................................................................... 152 8.1 Consumer price inflation for population groups ................... 153 8.2 Variants of CPI and CPIH ..................................................... 154 8.3 Variants of the RPI ................................................................ 155 8.4 Historic price index series ..................................................... 156 8.5 Producer price indices ........................................................... 158 8.6 Housing related price indices ................................................ 159 8.6.1 Owner occupier related house price statistics............... 159 8.6.2 Rental statistics ............................................................. 161 8.7 Alternative measure of consumer prices ............................... 163 8.7.1 Household Costs Indices .............................................. 163 8.7.2 A superlative index version .......................................... 164 8.8 Regional price indices ........................................................... 166 8.9 Concluding remarks .............................................................. 166

viii

9

Contents

Conclusions.................................................................................... 171 9.1 Inflation as a major threat ..................................................... 171 9.2 Inflation as eroding the value of money ................................ 173 9.3 Inflation measures as influential indicators ........................... 174 9.4 Are inflation measures reliable?............................................ 174 9.5 More than one measure of inflation ...................................... 175 9.6 Inflation statistics to inform debate ....................................... 177 9.7 Inflation and fairness ............................................................. 178 9.8 Inflation measurement and the future.................................... 179

Appendix A: A few technical details ...................................................... 181 Appendix B: Glossary of terms .............................................................. 190 Appendix C: Further reading .................................................................. 194 Index ....................................................................................................... 197

LIST OF FIGURES AND TABLES Figures Figure 1.1

Indices of average domestic prices of selected items, January 2010 to August 2023 (2015=100).

Figure 1.2

The level of prices as measured by the Consumer Prices Index (CPI), January 2010 to August 2023 (2015=100).

Figure 1.3

The Consumer Prices Index (CPI) rate of inflation (12month), January 2010 to August 2023.

Figure 1.4

The Retail Prices Index (RPI) rate of inflation (12-month) January 1965 to August 2023.

Figure 1.5

Child benefit for the first child in pounds sterling per week in indexed, nominal and real terms, 2004 to 2023.

Figure 1.6

Basic state pension, pre-2016 scheme, single person in pounds sterling per week, adjusted by prices, wages and the triple-lock, rates starting in April for the years 2004 to 2023.

Figure 1.7

Full-time, weekly, gross, female nurses’ pay, pounds sterling, in nominal and indexed terms, 1988 to 2021.

Figure 1.8

Index series for full-time nurses’ and all private sector services total pay in real terms, 2010 to 2021 using the CPI measure of inflation (2010=100).

Figure 1.9

Index series for full-time nurses’ and all private sector services total pay in real terms, 2010 to 2021 using the RPI measure of inflation (2010=100).

x

List of Figures and Tables

Figure 8.1

CPIH inflation for the lowest (decile 1) and highest (decile 10) equivalised income deciles, democratic weighting, 2020 to 2022.

Figure 8.2

Input and output producer price index values, September 2013 to August 2023.

Figure 8.3

Average house price in pounds sterling, in nominal terms, UK, January 2005 to July 2023.

Figure 8.4

Index of private housing rental prices, 12-month % change, for nations of the UK, January 2016 to June 2023.

Figure 8.5

Comparison of HCI, CPIH and RPI inflation rates, January 2015 to August 2023.

Figure 8.6

Comparison of the CPI calculated with the Lowe and Törnqvist formulas, 2007 to 2009, for locally collected data only.

Table Table 2.1

CPI Index values with annual and monthly rates of change, January to September 2022.

PREFACE Consumer price inflation is arguably the most important of economic indicators. It has a special significance, both through its role in the overall management of the economy and its use in wage negotiations and the adjustment of pensions, benefits and many thresholds. In the UK, inflation statistics are produced and published monthly by the Office for National Statistics and are watched closely. They are always reported in the media and often form the lead item on their day of release. Many newspaper articles are written explaining the significance of the latest values, how they have changed over time and what the future might hold. While modest levels of inflation are considered good for the economy, higher levels are very damaging; governments and central banks strive to keep inflation under control. Inflation statistics are a part of the overall family of official statistics which include data on the economy, the labour market, the population, business and many other topics. New official figures are published almost every day of the year. What the statistics tell us is of widespread interest but the way they are constructed is not. Most official statistics are treated as if they are part of the furniture. They are produced each month, quarter or year, and are accepted at face value by the organisations and individuals who use them. This can be taken as an indication that the producers are considered trustworthy; trust and a high level of quality are what are required of official statistics. However, very occasionally, the way that official statistics are calculated and used attracts public criticism. Inflation statistics in the UK fall into this category. A series of events, some originating in other parts of the world, led to competing measures of inflation in the UK and much argument over which is best; the debate has lasted over twenty years. The disputed use of inflation statistics has resulted in independent reviews, public consultations, the intervention of the Royal Statistical Society, parliamentary inquiries

xii

Preface

and multiple court cases. A proposed resolution is in sight, but won’t be implemented until at least 2030. Even then, it is unlikely to bring the debate to an end. The story behind the events and decisions that led to this undesirable position is a compelling one.

What is this book about? This book has been designed to cover the range of topics relevant to understanding what inflation is, how it has varied over recent times, how it is measured, how inflation measures are used, why they are controversial and their origins. The calculations involved in producing inflation statistics are complex and use much mathematics, economics and statistics; they have benefitted from the combination of extensive fundamental research and many years of practical application. We avoid mathematical details in this book and instead describe the principles behind the construction and usage. For technically minded readers, we provide a short mathematical appendix. We have previously written in-depth accounts of the mathematical foundations of inflation measurement, its historical development and the more recent controversies over which measure to use. These books are more suited to a specialist audience. The importance and influence of inflation measures suggests they deserve to be more widely understood and we feel a relatively short, largely non-technical book will contribute to achieving this. The history of inflation measurement and the uses of inflation measures are themes that run through most chapters. Inflation measures have a long history of development in response to pressing needs. Alongside technical and practical considerations, politics is also part of the story. We dedicate a chapter to a concise description of the development of inflation measurement over a period of around four hundred years. This long view of the development is supplemented by a more detailed chapter describing the recent history. These two chapters explore the development of the conceptual understanding, the practice, the usage and the resulting political implications.

Uses and Misuses of Inflation and Price Indices

xiii

Why is an understanding of the history important? The short answer is that it helps us to appreciate current practice and how the extraordinary extent of the work carried out in the past has advanced and refined the way price change is measured today. Our challenge has been to include as much explanation as possible without getting held up by interesting but obscure technical details. There are other books on inflation which cover different ground. A recent book on the economic background to inflation and the lessons that should be learned from previous episodes of inflation has been written by Stephen D. King. A political history of economic statistics in America, including inflation, by Thomas Stapleford was published in 2009. We give the details of these books in our section on further reading in appendix C.

Organisation of the book We start in chapter 1 by reviewing the recent history of rising inflation from the middle of 2021 through to the summer of 2023. Comparisons are made with a longer period: the last sixty years. The effects of inflation are explored; three case studies illustrate the challenges inflation has presented and how governments have responded. In chapter 2, we consider what we mean by inflation, its place in monetary economics and the challenges that need to be addressed in order to quantify it. Having set the scene in the first two chapters, chapter 3 explains how inflation is measured in practice. We identify the data that are needed, how they are collected and how they are used. The important statistical principles are identified. Our study of the historical development of inflation measurement starts in chapter 4 and covers the long period from the start of the 18th century through to the early 1990s. We describe how the conceptual foundations were established by insightful individuals and early, limited attempts made to find suitable data to produce a measure. The state took over towards the end of the 19th century, and established the first national measure at the beginning of the First World War. We show that there was political interference in inflation measurement between the wars and during the Second World War. A new start was made in the post-war period when what we can call a “modern measure” was first produced. We describe

xiv

Preface

some of the many improvements that were made in the period up to the 1990s. Chapter 5 continues the history, taking a more detailed look at the highly consequential period from 1992 to 2022. The events in this thirty year period led to inflation measurement being put in a very difficult position with an accompanying loss of credibility for these highly important official statistics. This is the longest chapter in the book and attempts to explain the circumstances in some detail. In chapter 6, we take a closer look at the data used in the calculation of inflation and investigate what more it can tell us. We show that the changing basket of goods and services can, if used with care, provide an informal view of social change. In chapter 7 we look at the factors that go into producing statistics of sufficiently high quality to meet the needs of users. We also look at public trust in official statistics and how it has changed over time. Chapter 8 looks beyond the main measures of consumer price statistics to describe other price measures including producer price indices, consumer price indices for special purposes, price statistics for housing, alternative measures and other approaches. We finish the main text with chapter 9 where we summarise the main themes of the book. Our approach is to pose questions on how we should think about inflation and its measurement and suggest answers. We also consider some likely future developments. Three appendices follow. The nine chapters of this book avoid equations to make the material as accessible as possible. However, for those with knowledge of mathematics and economics, appendix A presents a concise subset of the relevant formulas and how they are used. A glossary follows in appendix B and suggestions for further reading in appendix C.

Routes through the book The primary route is, of course, to read all chapters in order. However, for readers with more specific objectives, subsets of chapters will be of

Uses and Misuses of Inflation and Price Indices

xv

particular interest. While individual chapters link together, they still work effectively when standing alone. If the primary interest is in how inflation is currently measured then chapters 3 and 6, with or without appendix A for technical detail would be suitable. If an understanding of the historical development is the main goal then chapters 4 and 5 would suffice. For a view of what inflation is and its importance, then chapters 1, 2, 7 and 8 are recommended. The language of inflation is discussed in chapters 1 and 2; for the two shorter routes through the book, the important terms are also defined in appendix B, the glossary.

ACKNOWLEDGEMENTS We would like thank our current and former colleagues, both in academia and the wider world of official statistics; much has been gained from discussions with them over many years. A wide variety of books, research papers, official documents and newspaper articles have been consulted in the writing of the book and we are grateful for the support of the library staff from our respective universities. Thanks also to Cambridge Scholars for their expert handling of the manuscript. Finally, thanks also to our partners Bryony, Mila and Sarah for their unwavering support, patience and encouragement.

ABBREVIATIONS COICOP CPAC CPI CPIH CPIY CPI-CT EU HCI HICBC HICP HPI IFS IMF IPHRP ISO GDP GSBPM ONS OSR LCF(S) MP NHS NHSPRB PDCA PPI RPI RPIJ RPIX RPIY SPPI

Classification of Individual Consumption by Purpose Consumer Prices Advisory Committee Consumer Prices Index Consumer Prices Index including owner occupiers’ housing Consumer Prices Index excluding indirect taxes Consumer Prices Index at Constant Taxes European Union Household Costs Index High Income Child Benefit Charge Harmonised Index of Consumer Prices House Price Index Institute for Fiscal Studies International Monetary Fund Index of Private Housing Rental Prices International Standards Organisation Gross Domestic Product Generic Statistical Business Process Model Office for National Statistics Office for Statistics Regulation Living Costs and Food (Survey) Member of Parliament (UK) National Health Service National Health Service Pay Review Body Plan, Do, Check, Act Producer Price Index Retail Prices Index Retail Prices Index Jevons Retail Prices Index excluding mortgage interest payments Retail Prices Index excluding mortgage interest payments and indirect taxes Services Producer Price Index

xviii

UK UKHPI UN VOA

Abbreviations

United Kingdom United Kingdom House Price Index United Nations Valuation Office Agency

1 A TIME OF RISING PRICES The period from the summer of 2021 to the end of 2022 saw the economic news in the UK dominated by the rate at which prices were rising. Between August 2021 and August 2022, households faced gas and electricity prices rises of 54% and 96% respectively1. Food prices increased 14%, with milk, cheese and eggs rising by 22% over the same period (ONS, 2023a). The drivers for these price rises included the recovery from the covid-19 pandemic and the war in Ukraine (BBC, 2023). With prices rising at rates not seen for 40 years, many families struggled to cope, despite cutting their spending. As the Bank of England raised interest rates in an attempt to control inflation, so mortgage rates increased, adding further pressure on those homeowners who don’t own their own homes outright. Those renting saw rents rise as some landlords passed on higher costs. The media have an expression for times like these … the country was facing a “cost of living crisis”. Businesses were also affected by the rise in prices for the energy, fuel and raw materials they use. When energy costs rise, the effects on businesses feed into price rises for products in the consumer marketplace. In some cases businesses saw inflation affect both the costs they face and the demand for their products. Sharply rising prices had implications for the labour market too. Workers went on strike as pay offers from employers fell short of the levels required to match the rise in prices. In this chapter, we look at the effects of changing prices and introduce some of the relevant economic concepts, including the general level of prices, the rate of inflation and the value of money. We look at three examples of how governments have approached adjusting benefits, pensions and wages in response to rising prices. In chapter 2 we look in 1

These figures are averages across households; some households were on variable price arrangements and others on fixed rate tariffs.

2

Chapter 1

more detail at the language of prices and inflation and the challenges that need to be addressed to move towards producing a measure which is useful and reliable. The actual practice of measuring the general level of prices in the UK is described in chapter 3.

1.1

Price rises in context

How did price rises between 2021 and 2023 compare with price behaviour in recent years, say back to 2010? Figure 1.1 shows the levels of average domestic prices of electricity, gas and some food items from January 2010 to August 2023, where the average prices have been set to be 100 in 2015 for all three items so that we can more easily identify the trends. For electricity and gas, prices rose in March 2021 as the world’s economies emerged from covid-19 lockdowns, with demand exceeding supply. In addition, the response to the war in Ukraine had consequential effects, with Russia restricting the supply of gas. The stepped nature of the change in energy prices reflects the operation of energy price caps which effectively store up price changes, with consumers seeing prices updated only periodically as the price cap was increased (UK Parliament 2022). The price rises in energy were so extreme that many households were unable to cope. The UK government intervened to limit the effects of these dramatic rises in price with a range of measures including: limits on prices for electricity and gas, rebates on energy bills, cost of living payments for pensioner households and those in receipt of benefits and the freezing of duties (UK Government 2022). For food producers, the increase in energy costs together with higher costs for inputs such as animal feed resulted in increases in the price of food items for retailers and then consumers. Although food price rises weren’t as dramatic as those for electricity and gas they also contributed to the pressure on family budgets.

A time of rising prices 300.0

3

Indices of prices for food, electricity and gas

250.0 Milk, Cheese and Eggs 200.0

Electricity Gas

150.0

100.0

50.0 2010

2012

2014

2016

2018

2020

2022

2024

Figure 1.1: Indices of average domestic prices of selected items, January 2010 to August 2023 (2015=100). Data source: Office for National Statistics 2023a, series D7D8, D7DT, D7DU.

1.2

Overall price change and inflation

If we look at Figure 1.1 and focus on the period from the June 2015 to June 2017 we see little change in average prices; from March 2017, the price of electricity starts to rise but the price of milk, cheese and eggs shows little change and the price of gas shows a very slight fall. In periods outside of major global economic events, most prices change little over time with some items rising a little in price and some falling. If we consider all of the items in the consumer marketplace, most people would say that, in their experience, prices tend to rise more than fall, so that over a long period, say a decade, a significant rise in prices will be seen. While we might notice price changes in items that we buy regularly, it is much less easy to recognise price changes for irregular purchases such as furniture or computers. With a consumer marketplace consisting of millions of items, no individual, however much they like to shop, can form

4

Chapter 1

a balanced judgement of how prices are changing overall. Is the perception that prices rise overall correct? It is not just the sheer number of items in the consumer marketplace that complicates judging the overall movements of prices. The marketplace is dynamic, with new products appearing while others disappear. Individual consumers have their own preferences resulting in differences in purchasing choices and these consumer preferences change over time. An example of change in the marketplace is the rapid improvement of camera functionality in smartphones; this has put pressure on the demand for entry level digital cameras. To make sense of this complex picture of consumer preferences and market changes we have to turn to official statistics. Consumer price statistics take a view across the whole of the consumer marketplace and account for the preferences of the whole purchasing population. By using statistical methods, measures of an overall level of prices, a type of average of prices, can be estimated. Such measures are known formally as the general level of prices, or sometimes just the level of prices. Official estimates are produced every month, so we can track whether overall prices are rising or falling. There are several official measures currently in use; for now, we will just look at the Consumer Prices Index (CPI), which is the main measure used by the government to track price change. An actual average of prices isn’t useful; what is important is the change in prices2. A price index quantifies an overall measure of prices relative to a reference period. To make the degree of change easy to see, the Consumer Prices Index has been set to be 100 in the year 2015; figure 1.2 shows how the CPI has changed since over the period from January 2010 to August 2023. We can see in figure 1.2 that the trend of the general level of prices over this period, as measured by the Consumer Prices Index, was almost always

2 An average over prices of a range of different items isn’t usually very informative. However, when we are considering the price of one type of item, such as property, it is can be useful. With the fascination with house prices (at least in the UK), there is broad public interest in the average house price and averages for different property types and regions of the UK.

A time of rising prices

5

upward, which won’t be a surprise to most people. We can also see a sharper rate of increase from March 2021. 140.0

Consumer Prices Index

130.0 120.0 110.0 100.0 90.0 80.0 2010

2012

2014

2016

2018

2020

2022

2024

Figure 1.2: The level of prices as measured by the Consumer Prices Index (CPI), January 2010 to August 2023 (2015=100). Data source: Office for National Statistics 2023a.

As overall prices rise, so a fixed sum of money will buy less and less, a phenomenon that economists refer to as the diminishing purchasing power of money. This tells us something important: the value of money changes over time. Measures of the general level of prices are calculated by the Office for National Statistics each month. However, it is not the general level of prices that is reported widely but the percentage change over a 12-month period; this is called the (12-month) rate of inflation, or just inflation. A typical news report will say: inflation, as measured by the Consumer Prices Index (CPI), reached 11.1% in October3, the highest value for over 40 years. How has the rate of inflation changed over time? Figure 1.3 shows the rate of inflation, as measured by the CPI, between January 2010 and August 2023.

3

This was the value of 12-month change in the CPI in October 2022

Chapter 1

6

12.0

CPI Inflation

10.0 8.0 6.0 4.0 2.0 0.0 -2.0 2010

2012

2014

2016

2018

2020

2022

2024

Figure 1.3: The Consumer Prices Index (CPI) rate of inflation (12-month), January 2010 to August 2023. Data source: Office for National Statistics 2023a.

Inflation tells us the rate at which prices are changing. Over the period from January 2010 to December 2020, the average value of inflation as measured by the Consumer Prices Index was 2.1%4. In March 2021 inflation stood at 0.7% but by October 2022 it had reached 11.1%, a rapid increase over a relatively short period. Note that even when inflation had fallen to 6.7% in August 2023, it still meant that prices overall were rising. There are times when the general level of prices falls and inflation is negative, but they are rare. Between January 2010 and August 2023, inflation was negative for only three months out of 164 and then by only 0.1% on each occasion. There is a very important aspect of inflation that sometimes confuses people. When inflation is falling, as it was between October 2022 and August 2023 (apart from a 0.3% rise in January), prices are still rising, but just at a lower rate. It is understandable that falling inflation is a cause for celebration but it’s not as welcome as stable prices.

4

Economists are usually comfortable with this slow and steady growth in prices as it allows for the reorganisation of relative prices, a practice that is considered positive for the economy.

A time of rising prices

1.3

7

Inflation and its effects on households

It is not difficult to see how an inflation rate of 10% would be harmful for households. For those on a fixed, relatively low income, sharp rises in the prices of essential items such as heating and food might require them to cut back on one or the other (or both). Some households will be able to manage by adjusting their spending but others will be pushed beyond the point at which they can cover all of their necessary expenses. If a household has a fixed income, then as prices rise, so less and less can be bought with the same sum of money. The effects of inflation are particularly stark when inflation is at high levels, but it has also has a damaging effect when inflation is at lower levels. The example below demonstrates this. Imagine you have an annual income of £29,500, which was the median annual pay for Great Britain in June 2022, before tax and other deductions, excluding bonuses, rounded to the nearest hundred pounds (ONS 2022). Let’s say you spend £26,000 a year and save £3,500. Assume also that your income remains unchanged over the following years. If your purchases taken together move with a rate of inflation of 2% per year, then over a period of 10 years your spending would need to increase to around £31,700 to purchase the same items. Your spending would match your fixed income in year 7; you would no longer be able to save anything. Beyond that point, you would need to cut back on what you buy as well. Now consider the same scenario but with inflation at 5%. After 10 years, your equivalent spending would need to rise to £42,351. In this scenario, your expenditure would match your income in year 3. There is an alternative way of representing this. We can say that as prices rise, the purchasing power of our fixed income declines. Our income of £29,500 is fixed as a sum of money; after 10 years with annual inflation at 2% its purchasing power would fall to £24,200 in today’s money. If annual inflation was 5% then after 10 years the purchasing power of our income would be around £18,100 in today’s money. No-one would want to see their income decline from £29,500 to £18,100 over ten years. The effects of inflation are damaging.

Chapter 1

8

Is an inflation figure of 5% or more lasting over a period of ten years or so realistic? We don’t have to go back very far in UK economic history to find a long period of high inflation. The 1970s and early 1980s are often used as the classic example of high inflation in the UK. For the period from January 1974 to December 1981, the rate of inflation was above 10% for 81 out of 96 months with an average value of 15.6%. Inflation peaked at 26.9% in August 1975. Figure 1.4 shows inflation for the period 1965 to 2023, based on the Retail Prices Index, which was the official measure for most of the period5. 30.0

RPI Inflation

25.0 20.0 15.0 10.0 5.0 0.0 -5.0 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020 2025 Figure 1.4: The Retail Prices Index (RPI) rate of inflation (12-month) January 1965 to August 2023. Data source: Office for National Statistics 2023a.

If we look beyond the UK, it is less rare for countries to experience sustained high rates of inflation; for example, Ghana has only had a single year with inflation less than 5% since 1970 (World Bank n.d.). Is there a positive side to inflation? Just as fixed incomes (in money terms) reduce in value as the level of prices rise, so do debts. An individual or couple buying a property with a mortgage would see the size of their

5

We explain the difference between the CPI and the RPI in chapter 5.

A time of rising prices

9

borrowing reduce, as a multiple of salary borrowed, if income was increased in line with inflation.

1.4

Can we avoid inflation?

From a household point of view, it would be helpful if inflation was at or near zero. However, in a globalised world, prices can be affected by distant events beyond the control of the UK government, so some rises are inevitable. A number of countries task their central banks with keeping inflation at low levels; in the UK, the target value is 2% (as measured by CPI inflation) and the Bank of England has the responsibility to keep inflation at or around this level. If inflation deviates from the target level by more than a set amount, the central bank will intervene, usually by raising (or lowering) interest rates. This approach is not a precise science and there is invariably a lag between interventions and their effects on inflation. Why is the target for the Bank of England (and the central banks of other countries) 2% and not zero? Annual inflation of around 2% is considered healthy for the economy as we will discuss in the next chapter. We can conclude that UK households can at best expect relatively low values of inflation for most of the time with occasional periods of higher inflation; the impact of the latter being far more dramatic than the former. The result will be a reduction in the value of money over time and households will be at risk of damaging consequences. The fall in the value of money due to price rises is an issue that was recognised a long time ago. An appreciation of the concept of the value of money changing over time was recognised at the start of the 18th century. By the early decades of the 19th century, the problems caused by gradual price increases were described clearly, together with an outline of the way to estimate overall price change and what this information could be used for. It was proposed that wage rates should be adjusted by a measure of the level of prices to compensate for the falling value of money. We will see that it took a long time for a practical, national measure of the general level of prices to be produced. It wasn’t until 1914 that such a measure

10

Chapter 1

appeared. It was used to inform changes to wage rates for essential workers during the First World War; its use grew rapidly after the war6.

1.5

Inflation adjustment

Adjusting pay in response to rising prices was the first application of a measure of the level of prices for official purposes. However, in today’s world, pay is part of a wider picture. There are other sources of household income, such pensions and benefits. To compensate for rising prices, adjusting these sources of income by an inflation measure is required. A number of other financial quantities such as tax thresholds are specified in money terms and also require adjustment if they are to maintain their value. The costs of providing services which extend over many years are affected by changes in the value of money too; examples include mobile phone contracts and regulated rail fares. To preserve financial values, a whole range of incomes, thresholds and costs need regular updates. The adjustment of financial amounts in line with the growth in an index is called indexation. The growth in the general level of prices, or inflation, is the most commonly used indicator for indexation, but growth in average wages is also sometimes applied. Adjusting financial amounts in line with inflation is sometimes known as uprating (Masala 2022). In this book we use the term “indexation” to mean adjustment by inflation; we use “wage indexation” when adjustment is made by a measure of the growth in average wages. In principle, indexation can be applied by employers and providers of private pensions each year. The government can adjust public sector pay and pensions, the state pension, benefits and thresholds. What happens in practice? Isn’t it tempting to either under-index, that is, to adjust less than an inflation measure suggests, or not index at all? When inflation is at relatively low levels, like 2%, inflation adjustments can usually be made without serious implications for the government or for most employers. For higher values of inflation, the position is not straightforward as we will see in the following sections.

6

We explore the early history of inflation measurement in chapter 4.

A time of rising prices

11

Earlier in the chapter we examined the changes in the price of electricity, gas and some foods; we saw that the degree of price change is not the same across commodities. The official figures estimate inflation for all goods and services that are bought by households. Depending on the actual items that households buy, they may find their monthly expenditure rises higher than the overall inflation figure suggests. If their income, through wages, benefits and pensions is adjusted by the official rate of inflation, they may still end up paying more for what they buy, though it’s also possible they might spend less. Although inflation adjustment is highly beneficial, it is not expected to compensate all households for all possible combinations of goods and services they buy. In addition there is a concern that simply increasing people’s wages in this manner will fuel a secondary inflation effect and cause a potential inflation spiral. This risk was raised during the discussions on suitable pay rises in 2022 and 2023; it is an issue hotly debated in economics. When inflation rates reach high levels, such as occurred in 2022 and 2023, employers and the government are usually reluctant to match adjustments to inflation. This leads to disputes and industrial action. Employers including the government stress that pay rises have to be “affordable”; for employees it is understandable that they don’t want their incomes to fall in real terms. For the government, it is a balance between protecting the public, especially those on low incomes, against protecting the state of public finances. It is a contention that is ever present in budget documents, manifestos, parliamentary debates and parliamentary inquiries.

1.6

Responding to inflation in practice

An important function of official statistics is to enable the public to judge the performance of the government. With inflation statistics readily available, we can use them to explore whether pay, benefits and pensions are keeping up with overall price change. If we collect data year by year, we can plot graphs of the actual sums of money received each year through wages, benefits or pensions. However, whenever we want to study the variation of financial quantities over periods of time, care is needed. There are two ways of proceeding. Where data series include the effect of the change in the value of money we say we are representing the variation

12

Chapter 1

in money or nominal terms. If we want to see the change over time without the change in the value of money, we can use a measure of the level of prices to remove it and redraw our graphs. We call this the variation in real terms7. For state benefits and pensions, inflation adjustments are usually made but the picture is mixed. There is a statutory requirement to increase some benefits by at least the rate of inflation while others rely on the discretion of the relevant minister. For example, benefits administered by the Department for Work and Pensions include Disability Benefits (Disability Living Allowance, Attendance Allowance and Personal Independence Payment), Carer’s Allowance and Incapacity Allowance which have statutory indexation while Universal Credit does not. Child benefit, administered by HM Revenue and Customs, is also indexed at the discretion of ministers (Kirk-Wade et al., 2022). When negotiating pay, the level of inflation is taken into account by both employers and employee representatives. Employers are often keen to make pay offers conditional on the workforce accepting revisions to working practices to improve productivity. Where indexation is under the control of government, which is the case for benefits, public sector pay and pensions, the state pension and also for thresholds including those for direct taxes and duties, there is inevitably a degree of political judgement involved. In periods of recession, or more generally when public expenditure is considered to be too high, ministers may freeze payments by not applying inflation adjustments, so reducing the value of the benefit, pay or pension in real terms. While this reduces household incomes it strengthens public finances, again in real terms. A student of HM Treasury budget documents will be very familiar with Chancellors of the Exchequer freezing payments and thresholds, or underindexing them. It is a major mechanism used by governments to control public expenditure. Why is it so attractive to ministers and used widely? One reason is that it doesn’t result in the levels of attention that direct tax rises attract but can be very effective when applied over a number of years. 7

Appendix A gives a formula for doing this.

A time of rising prices

13

Given its prominence, it is perhaps not surprising that leaving a payment or a threshold unchanged in money terms while prices and/or wages increase is given a name. It is called fiscal drag, or a stealth tax. When the Chancellor of the Exchequer delivers a budget or financial statement to the House of Commons, observers of government finances will take careful note of any new fiscal policies and changes to headline rates of taxes and duties. They will also listen carefully to what is said about how inflation is accounted for across existing benefits, thresholds and public pensions. Analysts and financial journalists will combine all these factors to reveal the net effects on typical households. The freezing of thresholds figure prominently in analyst reports (Institute for Fiscal Studies 2022). An example of a stealth tax is provided by the freezing of personal tax thresholds. The Chancellor of the Exchequer announced in his March 2021 budget that the personal tax allowance and the higher rate threshold (plus a National Insurance threshold) would be frozen at the 2021-22 levels for the four following financial years 2022-23 to 2025-26. Subsequently, this was extended by two years to the 2027-28 financial year. In the 2021 budget, the additional government revenue for the 2025-26 financial year was estimated to be £8.2bn, but this assumed a lower rate of inflation that we have subsequently experienced. In The March 2023 budget, this figure was revised to £23.4bn, a very substantial figure. The current prediction is that if the freeze continues to 2027-28 the additional revenue in that year will be about £52bn. The Institute for Fiscal Studies estimates that to achieve this level of additional review would require basic and higher rate tax to increase by 6p and VAT to increase from 20% to 26%. If the freeze does extend all the way to the 2027-28 financial year it will be a stealth tax like no other (Institute for Fiscal Studies 2023, 187-8). To illustrate how the rate of inflation enters into government decision making for adjusting public sector benefits, pensions and pay, we examine three case studies, each extending back over several decades. We have chosen child benefit for the first child, the basic state pension and pay for female, full time nurses. In these brief case studies, we will examine changes over time in money terms and how this compares with the level of

Chapter 1

14

prices, to see whether the levels of income have increased or declined in real terms. We will also make comparisons with average pay, where appropriate. The period we have chosen is to an extent arbitrary, but the range from 2004 to 2022 is a useful period for our purposes as we will see, and covers a period of economic growth which was halted first by a global financial crisis and then a global pandemic.

1.7

Three case studies

Before we proceed, there are two important aspects we need to explain briefly. Firstly, we describe the different measures of consumer price inflation produced by the Office for National Statistics. We will say a lot more about them in later chapters so only the basic differences are required at this point. Secondly, the political context is needed, so we explain a little of the positions that governments found themselves in when they came to power over the period of interest.

1.7.1

Measures of inflation

The oldest measure of the level of prices still produced today is the Retail Prices Index (RPI) which began in interim form in 1947 and in full form in 1953. It was the measure used by governments for inflation adjustment until the 2010/11 financial year. A new measure, the Consumer Prices Index (CPI), was introduced in 1997 and the government switched to using it for inflation adjustment in the 2011/12 financial year. The CPI doesn’t include the contribution from owner occupiers’ housing. This was added in 2013 to form the Consumer Prices Index including owner occupiers’ housing costs, usefully abbreviated to the CPIH. The CPI remains the government’s measure for consumer price inflation at the time of writing, though the CPIH is expected to take its place at some point in the future. In these case studies, we want to examine how a benefit, a pension and a wage have changed relative to inflation, that is, in real terms. To do this, we need to decide which measure or measures of inflation to use. We use the official measure as specified by the government at the time. So, for the financial years 2004/5 to 2010/11, that means the RPI and from 2011/12

A time of rising prices

15

onwards the CPI. We have made this choice to be able to compare a benefit, pension and salary actually paid against what would have been paid if the government had applied an increase as specified by inflation each year. Some commentators choose other approaches, for example, they may use the RPI for all years which means our results differ a little from theirs8.

1.7.2

The political context

To understand government actions over the period of interest, it is helpful to recall a little of the economic and political background. The period from 1993 to 2007 was a time of moderate economic growth, with national debt stable at about 40% of GDP; indexation by the RPI was normal practice. Responding to the 2007-8 recession led to pressure on debt and public finances; debt rose sharply to about 75% of GDP in 2010. In 2010, the incoming Conservative-Liberal Democrat coalition government judged that public spending was excessive and needed to be brought under control. A period characterised as a time of austerity extended from the financial year 2010/11 to 2014/15 where expenditure was tightly controlled; it continued under the Conservative government elected in 2015 and lasted until the 2017/18 financial year. Post-2015 governments also faced the challenges of Brexit, the covid-19 pandemic and the impact of the war in Ukraine, which strongly influenced their economic policies and spending priorities. An important change was made in 2010 by the Conservative-Liberal Democrat coalition government which is highly relevant to this chapter and the rest of this book. As we noted in the previous section, before 2010, the Retail Prices Index (RPI) was the measure used by the government to adjust public sector pay, pensions and benefits. In 2010, the new government announced that, from the 2011/12 financial year, the Consumer Prices Index (CPI) would be used instead. CPI inflation was

8

Using the CPI for the whole period would also be a reasonable choice, though later in this chapter, we will look back to a time before the CPI started. We could still use the CPI though we would need the ONS “back calculated” version. We discuss this historic series in chapter 8.

Chapter 1

16

lower in magnitude than RPI inflation and the change was made to reduce government spending as we will explain in chapter 5.

1.7.3

Child benefit

Child benefit is a non-taxable payment that families can claim for each child with a lower rate for children beyond the first child. It was introduced in 1977, replacing family and child tax allowances. It is paid to the main carer, usually the mother. Historically, it has seen a high take-up, around 96%. It was available to everyone regardless of income until January 2013 when the government introduced the High Income Child Benefit Charge (HICBC), which reduced the benefit for families where the highest earner has an income that exceeds £50,000, removing the benefit completely where that person’s income exceeds £60,000. It is collected through the self-assessment tax process, though families can elect not to receive it. The change from a universal benefit to a means tested approach was one of the decisions the coalition government took to reduce the budget deficit (Seeley 2022). Child benefit is one of the benefits that do not have uprating on a statutory basis, meaning that increases in line with inflation are not set out in law but are subject to ministerial discretion. While the early practice was to index it (using the RPI), the austerity years of 2011/12 to 2017/18 saw it frozen or subject to below inflation, 1% increases. After the austerity period ended, it rose at least with inflation using the CPI measure. Without the imposition of the austerity approach, it would have been expected to increase in line with the CPI, following the policy decision to move to the CPI for indexation of benefits and public sector pensions from the 2011/12 financial year. The HICBC threshold values have not been indexed at all, thereby increasing the number of families losing the benefit (Rutherford, 2013; Seeley, 2022). Figure 1.5 shows the value of child benefit for the first child from 2004 to 2023 both in nominal and real terms. Also included is the indexed amount, that is, the money needed to maintain the value of the benefit at its 2004 level.

A time of rising prices

17

Child benefit, first child

45.00 40.00 35.00 30.00 25.00 20.00 15.00

Indexed Nominal Real

10.00 5.00 0.00 2004

2008

2012

2016

2020

2024

Figure 1.5: Child benefit for the first child in pounds sterling per week in indexed, nominal and real terms, 2004 to 2023. Source: Office for National Statistics 2023a; UK Parliament 2019; UK Government 2023a.

In nominal, or money terms, child benefit increased from £16.50 a week in 2004/5 to £24 in the 2023/24 financial year. If it had been fully indexed by the government’s inflation measure, it would be over £40 in 2023, and at 2023 prices. The effect of the austerity period, with nine years of the level being frozen or with a sub-inflation adjustment, was to reduce its value to £14.08, at 2004 prices. The freezing of the threshold at which child benefit begins to be withdrawn has also had a notable effect, with 26% of families losing some or all of child benefit in 2022; that is double the percentage when it was first introduced (Institute for Fiscal Studies 2022).

1.7.4

The State Pension

The state pension provides an income for individuals who have reached state pension age. For those who don’t also have a private pension it is supplemented by other benefits. It is contributory through national insurance payments made via payroll systems with credits applied in certain circumstances. Before 2016, the state pension came in two tiers, a basic state pension and an additional state pension; this was replaced by a flat rate

18

Chapter 1

scheme, the new state pension, for those reaching their state pension age on or after 6th April 2016. A qualifying year is earned if sufficient national insurance is paid in that financial year. In the new scheme, a minimum of ten qualifying years is needed to gain any state pension and 35 qualifying years are needed to get a full state pension9 (UK Government n.d.). The adjustment of the state pension has been through many changes since the 1970s. The Social Security Act of 1973 introduced a statutory duty to adjust the basic state pension in line with the level of prices. The incoming Labour government in 1974 changed this, deciding that the state pension should be adjusted by the greater of average wages and prices. The new Conservative government in 1979 decided this approach was unsustainable and that linking to prices was appropriate. A low level of inflation in 1999 with a correspondingly small increase in the state pension attracted adverse media coverage and persuaded the Chancellor of the Exchequer to change the approach. Indexing by prices remained but with a minimum increase of 2.5% (Thurley, 2010). Whether to adjust the state pension by the rise in average wages or prices was much debated. In circumstances where wages rose at a faster rate than prices, applying an adjustment just based on prices was criticised, being described as allowing pensioners to “fall behind” the working population. Towards the end of the 2000s, the political consensus moved towards a combined measure using prices, wages and a floor of 2.5%. Increasing pensions by the greater of prices, average wages or 2.5% became known as the “triple-lock” and was introduced in 2010; it was applied despite the restrictions of the austerity period. With an election approaching in 2019, the triple-lock became a sensitive political issue. The incoming Conservative government in 2019 had promised to maintain the triple-lock until 2024 in a manifesto commitment. However, in a controversial move, the triple lock was suspended for the 2022/23 financial year. The government’s justification was that the recovery from the covid-19 pandemic had driven the growth in average wages to what was considered 9

People who paid a reduced rate of national insurance through contracted out arrangements with their employer receive a reduced state pension unless they make up the loss through working or buying additional years.

A time of rising prices

19

an artificially high level of 8%. Instead, a double-lock consisting of the higher of inflation and 2.5% was applied, giving an increase of 3.1% which was the value of the CPI10 (Thurley and McInnes 2021). The government promised to restore the triple lock for the 2023/24 year which meant an increase of 10.1% which was the highest ever uprating of the state pension. The effect of the triple-lock can be seen by comparing how the state pension would have changed if it had been adjusted by just prices, just wages and then through the triple lock; this is shown in Figure 1.6. The effect of the triple-lock is to raise the basic state pension above both the growth of either prices or wages with the exception of its suspension for the 2022/23 financial year (Thurley and McInnes 2021). The new state pension, which started in April 2016, is not shown; its growth also follows the triple-lock. 180.00

Basic state pension (pre-2016 scheme)

160.00 140.00 120.00 100.00 80.00

Actual Wages Prices

60.00 40.00 2004

2008

2012

2016

2020

2024

Figure 1.6: Basic state pension, pre-2016 scheme, single person in pounds sterling per week, adjusted by prices, wages and the triple-lock, rates starting in April for the years 2004 to 2023. Source: McInnes 2021; Office for National Statistics 2023a, b, UK Government 2023. 10

The value of the CPI in the September of the previous year is used as the prices measure. The average wages measure is the value in the three months to July of the previous year.

Chapter 1

20

The House of Lords Select Committee on Intergenerational Fairness in 2019 heard representations that the triple-lock is too generous and is unfair to younger working people. The committee’s report, and a previous report from the House of Commons Work and Pensions Committee, notes that in the period 1980-2010, where the state pension was adjusted by prices, it fell behind the growth in wages. In 1980, the basic state pension was worth 28% of average earnings, falling to 16% in 2000-2008. The subsequent application of the triple-lock has seen its value rise to almost 19% of average wages11 (UK Parliament 2016, UK Parliament 2019). Their Lordships, in their committee report, recommend that in future the state pension should be aligned to increases in wages with some additional protection in times of high inflation12.

1.7.5

Nurses’ pay

Nurses are, of course, a vital core group of NHS staff and careful consideration has to be given to recruitment and retention; pay is an important element in such considerations. The task of assessing the appropriate level of pay is given to an independent panel, the NHS Pay Review Body (NHSPRB), which makes annual recommendations on pay to government. The NHSPRB takes evidence submitted by government, employers and trades unions as part of the process of producing a recommendation. While the government will take note of the NHSPBR’s report, it is not obliged to implement its recommendations (Buchan, Shembavnekar and Bazeer 2021, 7-10). In this case study, there are complicating factors, with NHS pay supplemented by additional allowances and different arrangements made for lower paid staff. There have been staged awards in some years with 11

It is interesting to consider what is fair in these circumstances. Wage rises may result from increases in productivity, perhaps following reforms in working practices, which cannot apply to pensioners. However, where wage growth exceeds inflation, one could argue that all of society should benefit, which would mean pensioners should be included. 12 This committee, formed specifically for this one inquiry, has now been disbanded. Parliament has said that the issue of intergenerational fairness will be taken up through other channels.

A time of rising prices

21

part of a pay rise starting in April and a second part following in November; there have been multi-year deals and changes to salary scales and points. We have used a simplified approach here following the work of Buchan and colleagues (Buchan, Shembavnekar and Bazeer 2021). This case study uses average (mean) full-time, weekly, gross female nurses’ pay (in GBP) based on ONS survey sources13. Nurses’ pay is a complex area and other studies provide a more detailed look at the issues (e.g. Propper, Stockton and Stoye 2021). We start with a long-term view of nurses’ pay, considering the period from 1988 to 2021. Figure 1.7 shows average, weekly, full-time, gross pay for female nurses in both nominal and indexed terms. 800.0 700.0

Nurses' pay, 1988 - 2021

600.0 500.0 400.0 300.0 200.0 Nominal pay Indexed pay

100.0 0.0 1988

1993

1998

2003

2008

2013

2018

2023

Figure 1.7: Full-time, weekly, gross, female nurses’ pay, pounds sterling, in nominal and indexed terms, 1988 to 2021. Source: Buchan, Shembavnekar and Bazeer 2021, Office for National Statistics 2023a, b.

13 These are the Annual Survey of Hours and Earnings and its predecessor, the New Earnings Survey.

22

Chapter 1

If we take average, weekly, full-time, female nurses’ pay in 1988 and maintain its value using the official measures of inflation each year; it reaches £421 in 2021 which is considerably less than the actual amount at £743. The growth of nominal pay exceeded indexed pay for almost every year up to 2010. There was a particularly sharp rise in the period around 1988 to 1990 which was the result of a new pay structure and a reevaluation of responsibilities and pay. By 2021, nominal pay was almost 80% higher than if pay had been adjusted by inflation alone14. A different picture emerges if we just consider the period from 2010 to 2021; this encompasses the austerity period and the four subsequent years. Over the austerity years, public sector pay was either frozen in nominal terms or rose by no more than 1%; there were larger increases after austerity ended. How did nurses’ pay fare in real terms? To add some context, we include average private sector pay for the services sector15. The price index we use is important when removing the effect of inflation; our preference is the CPI, which has been the government’s official measure since 201116. However, we recognise that in making pay claims, employee representatives often use the RPI, so we will show real terms pay using both price indices. Figure 1.8 shows nurses’ pay in real terms for the period 2010-2021, together with average private sector pay for services, using the CPI. Pay is represented in index number form, that is, it has been scaled to be 100 in 2010. This provides an effective way of comparing data series.

14

In the Report by Buchan et al., the authors choose a modelled version of the CPIH to produce a real data series of long term nurses pay; we choose the RPI as it was the official measure of consume inflation over the relevant period. 15 This is total pay, that is, regular pay plus any bonus. 16 As we will see in chapter 5, the National Statistician recommended in 2013 that as the RPI didn’t meet international standards, its use should cease as soon as possible. All calculations of real terms financial data series for recent years should use either the CPI or the CPIH.

A time of rising prices 115

23

Nurses' real pay (using the CPI)

110 105 100 95 90 85 80 75 2010

Private sector pay Nurses' pay 2012

2014

2016

2018

2020

2022

Figure 1.8: Index series for full-time nurses’ and all private sector services total pay in real terms, 2010 to 2021 using the CPI measure of inflation (2010=100). Sources: Office for National Statistics 2023a, b, c.

Over the 2010-2021 period, average private sector pay for service occupations rose by about 8% in real terms. In contrast, average nurses’ pay fell by 2%. Figure 1.9 is the corresponding picture but with the RPI measure of inflation used to convert nominal to real pay. The degree to which pay has risen relative to inflation is different when using the RPI; the RPI was higher than the CPI over the time period. Using the RPI measure, nurses’ pay fell about 10%. Private sector services pay just matched RPI inflation.

Chapter 1

24 115

Nurses' real pay (using the RPI)

110 105 100 95 90 85 Private sector pay

80

Nurses' pay

75 2010

2012

2014

2016

2018

2020

2022

Figure 1.9: Index series for full-time nurses’ and all private sector services total pay in real terms, 2010 to 2021 using the RPI measure of inflation (2010=100). Sources: ONS 2023a, b, c.

A similar exercise for secondary school teachers’ pay shows a greater decline in real terms than for nurses, with a 10% fall using the CPI and a 19% using the RPI. The decline varies across grades with the greatest decline in real terms for teachers in the upper three grades (Sibeita 2023). The use of the RPI in these calculations of the loss of pay in real terms right up to 2023 is controversial. It is attractive for employee organisations to use it as it produces a higher drop in real terms income than the CPI. However, since the RPI has been officially declared a poor measure of consumer price inflation we don’t recommend its use in this way; in fact, we count this as a misuse.

1.8

The statistical and the political

The three case studies we have presented show how indexation has sometimes been applied and sometimes withheld. While it is the responsibility of the Office for National Statistics to provide official

A time of rising prices

25

measures of inflation (and average wages), it is the government’s role to decide whether or not to adjust public sector pay, pensions and benefits to take account of inflation. These brief case studies are part of a broader picture which shows that failing to fully adjust for inflation, or not adjusting at all are approaches used widely by governments to raise the tax take or reduce public spending17. As we noted above, this approach, known as fiscal drag, or as imposing a stealth tax, provides a convenient means of managing expenditure. One viewpoint is that this is a responsible way of managing the overall fiscal position of the country and at times we all have to receive less than we’d like. A caveat might be that special considerations should always apply to the most vulnerable in society. An alternative viewpoint might be that if pay, pensions and benefits fall behind during challenging times that it is only fair that rises above inflation should make up any lost ground in better times. What is clear from these examples is the important role that inflation statistics play in political debates. They tell us how the value of money has changed over time and give us the data needed to adjust for it, if we choose to do so. They allow us to judge whether the offers of employers and governments leave employees and households better or worse off over time in real terms. While the use of inflation statistics is particularly important, statistics on average wages are also valuable, not only as a requirement for operating the triple-lock, but as an additional reference when adjusting benefits and pensions. Both the change in the level of prices and in average wages also provide us with indicators of how the wider economy is performing. Having introduced the important roles that inflation measures play we now turn in the next chapter to understanding more about the economic context in which they are often discussed and how they are calculated.

17 Similar activity occurs in the private sector, where businesses wish to maximise profit and minimise costs.

26

Chapter 1

References BBC. 2023. “What is the UK inflation rate and why is the cost of living rising?” Accessed May 27, 2023. https://www.bbc.co.uk/news/business-12196322 Buchan, James., Shembavnekar, Nihar and Bazeer, Nuha. 2021. “Nurses’ pay over the long term: what next?” The Health Foundation. Accessed January 2, 2023. https://www.health.org.uk/publications/nurses-payover-the-long-term-what-next Institute for Fiscal Studies. 2022. “Reforms, roll-outs and freezes in the tax and benefit system”. In IFS Green Budget 2022. Accessed May 27, 2023. https://ifs.org.uk/publications/reforms-roll-outs-and-freezes-taxand-benefit-system Institute for Fiscal Studies. 2023. “Policy risks to the fiscal outlook”. In IFS Green Budget 2023. Accessed October 23, 2023. https://ifs.org.uk/collections/ifs-green-budget-2023 Kirk-Wade, Esme and Harker, Rachael. 2022. “Benefits uprating 2023/24”. House of Commons Library. Accessed January 2, 2023. https://researchbriefings.files.parliament.uk/documents/CBP9680/CBP-9680.pdf Masala, Francesco and Seeley, Anthony. 2022. “Fiscal drag: An explainer”. House of Commons Library. Accessed January 2, 2023. https://commonslibrary.parliament.uk/research-briefings/cbp-9687/ McInnes, Roderick. 2021. “Benefits uprating 2021”. House of Commons Library. Accessed January 2, 2023. https://commonslibrary.parliament.uk/research-briefings/cbp-9131/ Office for National Statistics. 2022. “Average weekly earnings in Great Britain: August 2022”. Accessed September 25, 2023. https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/e mploymentandemployeetypes/bulletins/averageweeklyearningsingreat britain/august2022 Office for National Statistics. 2023a. “Consumer price inflation dataset – September 2023”. Accessed October 2, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/datasets/co nsumerpriceinflation

A time of rising prices

27

Office for National Statistics. 2023b. “Average weekly earnings in Great Britain: September 2023”. Accessed October 2, 2023. https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/e mploymentandemployeetypes/bulletins/averageweeklyearningsingreat britain/september2023 Office for National Statistics. 2023c. “Average weekly earnings. K54G total pay, private sector services”. Accessed October 2, 2023. https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/ea rningsandworkinghours/datasets/averageweeklyearnings Propper, Carol., Stockton, Isabel and Stoye, George. 2021. “Cost of living and the impact on nursing labour outcomes in NHS acute trusts”. Institute for Fiscal Studies Report R185. Accessed January 2, 2023. https://ifs.org.uk/sites/default/files/output_url_files/R185-Cost-ofliving-and-the-impact-on-nursing-labour-outcomes-in-NHS-acutetrusts-3.pdf Rutherford, Tom. 2013. “Historical Rates of Social Security Benefits”. House of Commons Library. Accessed January 2, 2023. https://researchbriefings.files.parliament.uk/documents/SN06762/SN0 6762.pdf Seeley, Anthony. 2022. “The High Income Child Benefit Charge”, House of Commons Library. Accessed January 2, 2023. https://researchbriefings.files.parliament.uk/documents/CBP8631/CBP-8631.pdf Sibieta, Luke. 2023. “What has happened to teacher pay in England?” Institute for Fiscal Studies. Accessed January 17, 2023. https://ifs.org.uk/articles/what-has-happened-teacher-pay-england Thurley, Djuna. 2010. “Pension uprating – background”. House of Commons Library. Accessed January 2, 2023. https://commonslibrary.parliament.uk/research-briefings/sn02117/ Thurley, Djuna and McInnes, Rod. 2021. “State Pension triple lock”. House of Commons Library. Accessed January 2, 2023. https://researchbriefings.files.parliament.uk/documents/CBP7812/CBP-7812.pdf UK Government. n.d. “The new state pension”. Accessed September 9, 2023. https://www.gov.uk/new-state-pension

28

Chapter 1

UK Government. 2022. “Overall government support for the cost of living: factsheet”, May 2022. Accessed January 2, 2023. https://www.gov.uk/government/publications/government-support-forthe-cost-of-living-factsheet/government-support-for-the-cost-of-livingfactsheet UK Government. 2023a. “Tax credits, child benefit and guardian’s allowance”. Accessed September 9, 2023. https://www.gov.uk/government/publications/rates-and-allowancestax-credits-child-benefit-and-guardians-allowance/tax-credits-childbenefit-and-guardians-allowance UK Government. 2023b. “Benefit and pension rates 2023-24”. Accessed September 9, 2023. https://www.gov.uk/government/publications/benefit-and-pensionrates-2023-to-2024 UK Parliament. 2016. “Intergenerational Fairness”. Third Report of Session 2016-17. House of Commons Work and Pensions Committee. Accessed January 2, 2023. https://publications.parliament.uk/pa/cm201617/cmselect/cmworpen/5 9/59.pdf UK Parliament. 2019. “Tackling Intergenerational Unfairness”. Report of Session 2017-19”. House of Lords Select Committee on Intergenerational Fairness and Provision. Accessed January 2, 2023. https://publications.parliament.uk/pa/ld201719/ldselect/ldintfair/329/32 9.pdf UK Parliament. 2022. “Domestic energy prices. Research Briefing”. House of Commons Library. Accessed January 2, 2023. https://researchbriefings.files.parliament.uk/documents/CBP9491/CBP-9491.pdf World Bank. n.d. “Inflation, consumer prices (% annual)”. Accessed January 2, 2023. https://data.worldbank.org/indicator/FP.CPI.TOTL.ZG?locations=GH

2 WHAT IS INFLATION? In chapter 1, we introduced the concepts of the general level of prices and inflation. We showed that the value of money changes over time; usually prices rise, so the value of money declines. An example showed that inflation will erode the purchasing power of a fixed sum of money. With a measure of inflation, the government and employers can, at least in principle, compensate households for a decline in the value of money by increasing their income. Three case studies showed that, in practice, sometimes adjustments are made and sometimes not. In this chapter we will take a brief look at inflation in the context of the wider economy and the how it is managed. We examine how the general level of prices and inflation are reported by the Office for National Statistics each month. There are many applications of a measure of inflation and we take a brief look at who uses them and some of the uses. We then move on from the concepts and language of inflation to begin to look at the concepts behind inflation measurement. We describe several approaches to measurement and consider some of the challenges that need to be addressed; this sets the scene for chapter 3, which explains some of the aspects of how inflation is measured in practice.

2.1

Inflation and monetary economics

Why is the level of prices so important in the management of the economy? To answers this, we need to look at a little economic theory. As the name suggests, monetary economics is a branch of economics that comprises theories of the role money plays in economies. It has grown in importance over the last few decades to become highly influential in the development of economic policies enacted by governments and central banks. The focus placed on money has been instrumental in the understanding of inflation, with the quantity theory of money providing a conceptual underpinning. The theory proposes that an increase in the money supply, or the quantity of money in an economy, will lead to an increase in the general level of prices. An increase in the level of prices

30

Chapter 2

will be accompanied by a fall in the value of money and therefore the purchasing power of a unit of currency (O’Neill, Ralph and Smith 2017, 24-25). Through the application of monetary policies, the supply of money in an economy can be managed by central banks and governments. There are a variety of mechanisms that can be applied. The money supply can be increased through central banks buying assets from commercial banks in exchange for cash which can then be lent to businesses and households. Other approaches in the armoury of central banks include raising or lowering the reserve requirements for commercial banks and raising or lowering interest rates. Reserve requirements are the amount of money a bank holds in reserve to be able to meet sudden withdrawals. Increasing the reserve requirement will decrease the money supply. Raising interest rates makes borrowing more expensive and encourages saving; lowering interest rates promotes the opposite. The application of these tools is not simple as there will be a lag between an issue being identified and the central bank medicine working (or not). The timing and magnitude of an intervention are hard to judge (International Monetary Fund n.d.).

2.2

Is there a “right” level of inflation?

The natural assumption to make is that any inflation is bad as rising prices will be damaging for households on a fixed income, particularly those on low incomes. In chapter 1, we saw an example of the effect of small and large values of inflation on a fixed level of income, when inflation extended over a number of years. In theory, benefits, pensions and wages could be adjusted each year in line with the change in the level of prices. This would mean that employees, pensioners and recipients of benefits would not need to be concerned about rising prices. In contrast, the providers of household income, private sector employers and the government, would be concerned at rises in prices. However, as our three case studies in the previous chapter illustrated, in practice, there are periods where governments have decided that the public finances aren’t in a state to permit adjustment in line with prices.

What is inflation?

31

Are there any positive aspects to inflation? It depends on the circumstances and viewpoints of the affected parties. We noted in section 1.3 that while a fixed income will be eroded by rising prices, levels of debt will fall too. A person or couple buying a property with a mortgage would see the size of their borrowing as a multiple of salary reduce if pay was increased in line with inflation while their debt stayed the same. There are also benefits on a whole economy scale from low values of inflation; a common target for central banks is to keep inflation around the 2% mark. A small amount of inflation gives the government some flexibility in managing the public finances by freezing thresholds or under-indexing. In the labour market, businesses employing differential responses to inflation across industries can drive transfers of labour from less productive to more productive industries and roles (O’Neill, Ralph and Smith 2017, 26-29). In examining historic rates of inflation in chapter 1, we drew on the experience of the UK where the highest level of inflation, 26.9%, as measured by the RPI, occurred in August 1975. Most of the time, the rate of inflation has been much lower, with occasional peaks and troughs. The importance of the careful management of the supply of money can be seen from looking across the world where there have been instances of inflation reaching extremely high levels, in the thousands of per cent. The classic example comes from 1920s Germany where the value of a unit of currency was so low that paper money had to be carried in wheelbarrows to buy basic items or bricks of money were used as children’s playthings, such was their low value. Extreme levels of inflation have occurred more recently in Zimbabwe (in 2008) and Venezuela (in 2019). Such situations of extreme levels of price increases are described as hyperinflation. These extreme examples show that keeping control of the level of prices is vital to allow society to keep functioning in a stable and desirable way (O’Neill, Ralph and Smith 2017, 32-33).

2.3

A price index and index numbers

Measures of the general level of prices and the rate of inflation derived from them are produced each month by the Office for National Statistics. In this section we look at how they are reported. For the general level of prices, we could use an actual money amount; however, this has little

Chapter 2

32

meaning and is not a useful piece of information. Instead, we set the value to be 100 at a chosen time, usually the period of a year, and measure the change in the general level of prices relative to that reference year. It is the change in the level of prices that is useful. The period of time we choose as a reference point and the value of 100 are both, to an extent, arbitrary; we can choose both to be convenient. Table 2.1 is an extract from the statistical bulletin from the Office for National Statistics for consumer price inflation for September 2022 reporting on the Consumer Prices Index (CPI).

2022

CPI Index (UK, 2015 = 100)

CPI 12-month rate

CPI 1-month rate

Jan

114.9

5.5

-0.1

Feb

115.8

6.2

0.8

Mar

117.1

7.0

1.1

Apr

120.0

9.0

2.5

May

120.8

9.1

0.7

Jun

121.8

9.4

0.8

Jul

122.5

10.1

0.6

Aug

123.1

9.9

0.5

Sep

123.8

10.1

0.5

Table 2.1: CPI Index values with annual and monthly rates of change, January to September 2022. Source: Office for National Statistics, 2022c.

The column headed “CPI Index” shows the general level of prices relative to the value in the year 2015 where it was set to 100. This is called an index number representation and is a very effective way to display change. For example, we can use the index numbers as a quick route to determine percentage changes. From the table, we can see that the level of prices has increased from 100 in 2015 to 123.8 in September 2022; this is an increase of 23.8%. Setting the value of data series of index numbers to be 100 in a

What is inflation?

33

reference period allows us to just read off the percentage change from the index numbers18. The series of index numbers in this column is the value of the Consumer Prices Index, one of several measures produce by the Office for National Statistics and the preferred measure used by the government19. A price index is a useful tool; it tells us how the level of prices has changed over time. We can use it to convert the value of money from one time period to another The use of a price index to convert the value of money from one time period to another may seem strange at first. One way to look at it is to consider money from different periods of time as if it was from different countries with different currencies. If we travel abroad, we are familiar with converting pounds to euros or dollars using an exchange rate. A price index is like an exchange rate but over time rather than between countries20. There are many occasions where it is useful to convert the value, or purchasing power of money from one time period to another. Comparing the prices of items in the past to the price today is often revealing. Some goods have risen faster than the level of prices, such as property, and some have fallen, such as electronic goods21. Let’s look at a specific example drawn from a classic novel.

18

We provide an introduction to index numbers and their uses in a previous book (Ralph, O’Neill and Winton 2015). 19 The Office for National Statistics’ main measure is the CPI including owner occupiers’ housing, CPIH. It is expected to become the government’s preferred measure in the future. 20 We can create “spatial” indices as well as the “temporal” indices. For example, average house prices are published for the countries of the UK and for regions in England. We could set the value for England to be 100 and calculate the averages for Scotland, Wales and Northern Ireland relative to England expressed as spatial index numbers. 21 A portable DVD player in 2000 cost around £1000, equivalent to almost £3000 in 2023 prices. Although they have been largely replaced by downloading films to a phone or tablet, a new portable DVD player can be bought for around £50 today.

Chapter 2

34

Example: In Jane Austen’s famous novel Pride and Prejudice, Mr Darcy is said to have an income of £10,000 a year. What would be the equivalent sum of money today? We can use a price index to answer this question. Firstly, we need to determine when the novel was set. As a rough guess, we can say around 1800 as the novel mentions the Napoleonic Wars. Next we need a price index that extends all the way back to 1800. The Bank of England provides a calculator that takes a sum of money in one year and produces the equivalent sum in another; a price index provides the data behind the calculator (Bank of England, n.d.). Inputting £10,000 in 1800 yields £689,000 in today’s money – a tidy sum. It is only a rough guide of course as it is hard to determine the level of prices so far back in time. The Bank’s webpage explains how the long run price index is constructed. The Office for National Statistics provides the necessary data from 1750 onwards, with other sources providing data for earlier periods. We look at historic price indices in section 8.4. The three case studies in chapter 1 illustrate the usefulness of a consumer price index. In those case studies, we examined how a benefit, a pension and a level of pay had changed over time both in nominal and real terms. When a financial quantity is adjusted by the prevailing value of inflation, we call this (full) indexation. For child benefit and nurses’ pay, we saw that over the austerity period from 2010/11 to 2017/18, in some years there was no indexation and in others a below inflation adjustment was given. For the basic state pension, the triple lock provided a level of adjustment that either matched or exceeded inflation. The variation of pay, benefits and pensions relative to the level of prices is how we find out whether people are better or worse off as the value of money changes. We can use a consumer price index to remove the effect of inflation; this gives a data series in real terms. The process of removing the effect of inflation from a (nominal) data series is called deflation.

2.4

Users and uses

Developing an understanding of who uses official statistics and what they are used for is an important activity for organisations that produce them. In

What is inflation?

35

chapter 7, we will see that the quality of an official statistic is defined as its “fitness for purpose”. Clearly, producers of official statistics need an understanding of the uses and therefore the users if high quality is to be achieved. For high profile statistics like inflation, the main uses are easy to see. We saw in chapter 1, the use made by employers and the government in deciding on pay, benefits and the state pension. However, there are many uses beyond these high profile examples. The Office for National Statistics produces inflation measures and they publish a document which sets out examples of organisations that use inflation statistics and summaries of how they are used (Office for National Statistics 2018). We give just a few examples here. For the overall management of the economy, governments monitor inflation as it is an important ingredient in setting broad economic policy. Many countries set inflation targets as part of a monetary policy approach to managing their economies. We noted in chapter 1 that the inflation target in the UK is 2% (as measured by the CPI) and it is the Bank of England’s responsibility to meet this target. By publishing statistics for inflation, a wide range of organisations from businesses to think tanks, to special interest groups and charities can make their own assessments of the state of the economy and how it affects their areas of interest. The public can use inflation statistics as a way of assessing the performance of central banks and the government and as a factor in their own decision making. The government will take inflation into account when setting pay for public sector workers, usually via independent panels such as the one for health sector workers which was mentioned in chapter 1. Similarly, public sector pensions and a wide range of benefits usually receive increases each year. As we saw in chapter 1, some benefits are indexed by statute and some at the discretion of ministers. Private sector employers and trades unions will also take inflation into account when negotiating pay. Private sector pensions usually have automatic uprating specified in their pension agreements, up to a celling value (often 6%, sometimes 10%) with trustees having discretion to uprate by a higher amount if circumstances allow. Governments set thresholds for financial items such as personal tax bands,

36

Chapter 2

inheritance tax thresholds and national insurance contributions; these may or may not be adjusted each year by the rate of inflation. An interesting case is the personal tax allowance. The Chancellor of the Exchequer announced in his 2021 budget that the personal tax allowance and higher rate threshold would be maintained at the 2021-22 levels up to and including the 2025-26 financial year (UK Government 2021). With inflation at relatively high values, this represents a substantial increase in tax income for the government. If the personal tax threshold had been uprated by inflation for the 2022-23 and 2023-24 financial years, it would be over £13,000 rather than the frozen value of £12,570. Governments raise money through the issuing of bonds, also known as gilts. They sell bonds in return for regular interest payments (the coupon) and a capital repayment on a maturity date (the principal). Some bonds have a fixed return and others, about a third, have returns that are indexlinked. These index-linked bonds, or gilts, attract regular payments dependent on the level of inflation. Index-linked gilts are attractive to organisations with financial commitments that are inflation dependent, like providers of final salary pension schemes. We will see in chapter 5, that this application of inflation has been highly consequential in how inflation is measured in the UK. Financial amounts in contracts, such as for the cost of services provided, particularly for long-term arrangements, can specify adjustment by the rate of inflation in the contract terms. An example is mobile phone contracts from telecoms companies where the monthly cost is often uprated by the RPI. Student loan interest payments have an inflation adjustment specified as do regulated rail fares. These examples are just a selection of the many uses of inflation statistics. We now begin our look at the measurement of inflation, starting with a brief look at measurement in general.

2.5

An act of measurement

We measure, or at least, try to measure very many things. The performance of schools and hospitals, the speed of light, well-being, the

What is inflation?

37

number of vacancies in the economy and the extent of deprivation are all quantified in approximate forms. Although the context for each of these areas of interest and the mechanisms used are very different, we use the same word: measurement. We all have a degree of familiarity with measurement. From time to time we weigh ingredients for a recipe or use a tape measure or a laser measuring device for estimating the size of a room. On the surface, measuring may appear relatively straightforward; however, once we look a little deeper, we find that measurement is rarely simple. In the physical sciences, measurement is highly technical. A recent example of this is provided by the project to standardise the measure of the kilogram (National Institute of Science and Technology n.d,a). Before 2019, a kilogram was referenced to the weight of a single artefact housed in the Archives Nationale in Paris, the International Prototype of the Kilogram. However, every physical object is subject to some deterioration and in 2019 a new definition, free of ties to artefacts was approved. The kilogram is now defined in terms of an invariant of nature, the Planck constant. In a similar way, the metre is now defined in terms of the invariant speed of light in a vacuum (National Institute of Science and Technology n.d,b). Economics is at root a social science and therefore most of the ideas which are important to its development are conceptual with a complex route to practical applications that requires approximation and simplification. The broad definitions of the level of prices and inflation are helpful in an informal way, but like measurement of the mass of an object, if we look a little deeper into its meaning and how it is measured we encounter complexity. The rest of this chapter and the next will explain the route from the conceptual to the practical for inflation measurement.

2.6

Towards the measurement of the level of prices

We can readily imagine capturing prices for a small selection of goods in shops but what we need is data that encompasses all the goods and services that consumers buy and an appropriate way of summarising it. This needs careful thought and the answers to a range of challenging

38

Chapter 2

questions. The number of items in the consumer marketplace is enormous; how do we account for the sheer size and variety of items? Do we need to include the same items from different types of retail outlets? How do we include goods and services where there is no readily available price, such as education and health services? How do we combine the prices for different items and how do we reflect the different amounts that consumers spend on different types of consumer item? What formulas do we use to combine data to calculate the general level of prices? Beyond the challenge of collecting prices for the substantial number and diversity of products and services available to consumers there are other factors to consider. The marketplace is dynamic with new products appearing and others disappearing. To promote specific items, retail outlets run sales promotions with some items discounted for a period of time. Consumers respond to relative prices and change the items they buy and the quantities accordingly. The simplest approach to measuring price change would be to select a sample of commodities and collect prices for the identical items every month. We can think of this as a very large shopping basket with goods and services which we price up every month. This is relatively straightforward for items such as milk and bread which don’t change much over time but will not be simple for more complex items such as electronic goods. For electronic products, new models appear regularly with improved specifications and often at similar or even lower prices than older models (Office for National Statistics 2022b).

2.7

Marketplace and consumer challenges

In order to illustrate the potential issues involved in the measurement of inflation we can conduct a small scale thought experiment where we focus on inflation for a single household. We can simplify this further by considering only household spending on food and assuming that the household does almost all of its food shopping in one large shop on the same day every month.

What is inflation?

-

-

-

39

The household’s shopping list contains items they would like to buy every month but then part-way through a year, some are unavailable. This could be due to seasonal availability of fruit, high demand for some items or issues with supply chains. The household will either buy an alternative item to compensate or not buy that type of item at all. How do we account for missing and replacement items? Some items may be on special offer in some months and not in others. Do we record the undiscounted or the discounted price? A new item has appeared on the shelves, promising all the taste and half of the calories of an existing product the household previously bought; do we incorporate this into our basket or continue with the old product? Some items are bought in amounts which mean they can last more than one month; for example a large bag of rice may be used over several months. How would we account for the purchase in one month of an item that is consumed over several months?

This simple example for one household and one area of spending of a budget illustrates some of the challenges around consumer and retailer behaviours that need to be considered when producing measures of inflation.

2.8

Complex commodities

In measuring price change our ideal is to find identical items each month. While this is possible for many items which don’t change frequently, other products do change on a regular basis. What items present particular difficulties? Clothing is a particularly complex category of item. Clothes are produced with many variations of colour, style and material. They also possess difficult to quantify attributes which affect price, such as whether a garment is fashionable or not. Some clothing items are long lived in the marketplace and stable in their attributes; for example, men’s formal work shirts. Other types of clothing are more transient. Fashion clothes can have several seasons in a calendar year, with a high turnover of stock and

40

Chapter 2

discounting to make way for new season products. Highly fashionable items command high prices but their attractiveness can decline over time; the degree of “fashionability” is a factor in the price. Finding identical items of fashion clothing over the course of a year is almost impossible. Statisticians have struggled to devise effective ways of accounting for the effect of “fashion” on price. Owner occupiers’ housing is also a difficult to measure commodity. Firstly, it is a durable item which is “used” over long periods of time. A property may be bought outright, or with a mortgage. Mortgage payments may include capital repayments as well as payments of interest with the former excluded from a measure of consumption. One approach to including housing in a basket of goods and services is to think of owner occupiers’ housing as providing a flow of housing services to the owners. This flow can be measured by approximating the rental cost for an equivalent property; the concept is simple but requires complex analysis to put into practice. There are other possible approaches to measuring owner occupiers’ housing costs (O’Neill, Ralph and Smith 2017, 301-310). We noted in the previous section that electronic goods also present challenges. Computers are available in a wide range of models with options for components like hard disk capacity and memory, all of which contribute to the overall price. New models are introduced frequently which resemble older models but with upgraded components. Finding identical or comparable models across a calendar year can be difficult. To accommodate the changing nature of electronic goods, statistical models have been developed that indicate the contribution of each component to the overall price. These models can be used to price adjust a new model to match the specification of a previous model, this use of statistical models is called hedonics (O’Neill Ralph and Smith 2017, 181).

2.9

An international effort

This brief look at the challenges of compiling a measure of the level of prices suggests it is not an easy task. However, the subject has benefited from the concerted efforts of a worldwide community of statisticians and economists over many years. As inflation measures are required by all

What is inflation?

41

countries, the problems that need to be solved can be shared. Combining the expertise and experience of practitioners from many countries has provided a powerful approach to devising optimal solutions. When addressing the many issues, a balance has to be struck between theoretical ideals and practicality. The international experience gained from developing and producing inflation measures over many years has been summarised into a best practice methodology to guide new people coming into statistical posts. Supranational bodies such as the International Labour Organisation and the International Monetary Fund have taken on a guiding role in establishing and documenting this best practice.

2.10 What are we trying to measure? Before specifying how to go about measuring inflation, we need to be clear as to what we are trying to measure. In this section we identify two approaches to the measurement of overall price change and while doing so, we will clarify the meaning of an expression that appears frequently in discussions of prices. We noted in chapter 1 that prices started to rise from March 2021 and continued to rise throughout 2022 and 2023. As rising prices began to affect households, it became common for media sources, politicians and the general public to refer to households facing a “cost of living crisis” in the UK and in other countries too. This is the popular expression of choice to briefly summarise the situation in which household budgets are squeezed as the price of essential goods and services, such as food and energy, increase significantly. With the rate of inflation reaching its highest level for 40 years and therefore becoming a central issue in public life, so the “cost of living crisis” has become a familiar expression. In the media and in general conversation, the cost of living is equated with the general level of prices and the increase in the cost of living with the rate of inflation. However, the cost of living has a specific technical meaning in economics and it is not what is calculated each month by the Office for National Statistics (ONS). Before we embark on describing how

42

Chapter 2

the ONS calculates the general level of prices in the next chapter, it is important to specify the target for measurement. In economics, a “cost of living” measure will attempt to quantify the cost of maintaining a constant standard of living for an individual or household. This concept of a “standard of living” is not a simple one. In the broadest sense, it can take into account all the physical and nonphysical aspects of our lived experience; for example, it might include subtle elements such as job security, or feelings of personal safety. If we narrow the definition to just the contribution to our standard of living from goods and services, the definition of a cost of living measure would be the minimum cost of maintaining a defined standard of living as relative prices change. In response to price changes, changes in what is available in the marketplace and our individual tastes, we may decide to change what we buy so as to maintain our level of satisfaction, or utility. A cost of living measure therefore tries to answer the question: how has the cost of maintaining a constant standard of living changed between two time periods? The inflation statistics produced each month by the Office for National Statistics are not cost of living measures; they are what are known as “cost of goods” measures. A cost of goods measure estimates the price change for a fixed basket of goods and services between two time periods. It is what almost all national statistics institutes around the world produce, including the ONS. The fixed basket approach is an effective method but only over a relatively short period of time. The marketplace changes with products appearing and disappearing and consumer tastes change. The cost of goods approach allows the basket to be updated each year to reflect these changes. In this way, the cost of goods approach does accommodate an element of the changing marketplace and consumer tastes. There is a degree of overlap in the “cost of living” and “cost of goods” approaches but there are important differences as well. If the costs of all goods and services increase then we will see an increase in both a cost of goods and a cost of living measure; indeed, a rapid increase in one measure is likely to be associated with a large increase in the other. A key difference is in the overall treatment of consumer behaviour. For example,

What is inflation?

43

the cost of living concept accommodates substitution behaviour, which occurs when consumers switch from a good for which prices are rising rapidly to using other goods which are cheaper. An example of this arose in 2022 when sunflower oil became much more expensive as Ukraine, one of the main exporters of the oil, was unable to maintain supply levels due to the conflict with Russia. As sunflower oil prices increased markedly, consumers switched to other cooking oils whose price increased more slowly. The cost of goods approach would only be able to reflect this by reducing the contribution of sunflower oil and increasing other oils once a year when the basket was reviewed. A cost of living index would aim to include in-year changes. As a cost of living measure more closely reflects real consumer behaviour, why don’t we set this as a goal when measuring inflation? The US has adopted the cost of living approach as the target for its inflation measures; however, most countries have continued with the fixed basket of goods and services approach as their target. With limited budgets and limited time to collect data and perform the necessary calculations, a cost of goods approach is more practical for national statistical institutes. We give a more formal and thorough description of the differences and the theoretical background of the two approaches in one of our previous books (O’Neill, Ralph and Smith 2017, 266-284).

2.11 Is inflation the same for everyone? Inflation is reported as a single, headline figure every month for the UK as a whole. In September 2022, CPI inflation was 10.1%; this was the aggregate rate across the four nations of the UK (Office for National Statistics 2022c). However, people might feel that this UK-wide measure does not reflect their own personal experience of price changes over the past twelve months. Could it be that inflation varies between regions of the country, or for households with different levels of income? Investigation does indicate that different households experience different rates of inflation. For example, survey evidence provided by the Office for National Statistics shows that the ability of households to deal with changes in prices of essentials, such as food and energy, varies both by

44

Chapter 2

economic status, disability status, ethnicity and the security of their housing situation (Office for National Statistics 2022d). At the same time, ONS reported that in many cases the price of lower priced items was rising more quickly than the general measure of inflation. For example, the price of vegetable oil rose 46% percent between April and September 2022, while food in general rose 15% (Office for National Statistics 2022e). Variations in the rate of inflation also apply when households are categorised by income and expenditure decile. For the period June to August 2022, inflation for lower income households, those in the second decile, was 8.7%, while those in higher income households, those in the ninth decile, faced inflation at a lower value of 7.8%22 (Office for National Statistics 2022f). We look at inflation measures for categories of household in section 8.1. When inflation is at low levels, the inflation experience for different households, while interesting, will probably not be of great significance. As we noted in chapter 1, through most of 2021 and 2022, energy and food prices increased sharply in the UK contributing to high levels of inflation. Lower income households, which include some pensioners and those reliant on state benefits, spend a greater proportion of their income on energy and food than other households. If the national rate of inflation is used to uprate their incomes then it is possible this will result in a situation in which these households are left without enough money to maintain their purchases of essential items. Wealthier households can absorb rising costs or re-arrange their expenditure so that the impact may be just inconvenient. Households who are only just managing, and who receive an increase in income matching the national, all-items, rate of inflation may still need to reduce their expenditure on food and energy. These more detailed statistics for household types can assist governments in targeting support where it is most needed. Of course, the single, national headline rate of change in the general level of prices will always be the most widely used measure. In theory, it is possible for each individual to calculate their own inflation rate. To do this accurately would require a considerable amount of record 22

This is using the CPIH measure.

What is inflation?

45

keeping and some arithmetic. Tools are available to simplify such a calculation (BBC 2023). While this is an interesting exercise and helps to educate people in what is meant by inflation, it is of limited consequential use.

2.12 Concluding remarks In this chapter, we have taken the seemingly simple concept of a general level of prices and started to explore what is required to actually produce a measure. We found that challenges arise very quickly. However, the many benefits of having high quality measures of the level of prices have provided ample motivation for statisticians and economists to develop effective, practical measures. By taking a carefully balanced approach between respecting theoretical ideals and embracing practicality, measures of sufficient quality have evolved. We will see how the development progressed in chapter 4, but before that, chapter 3 will describe how modern inflation measures are calculated in practice.

References BBC. 2023. “UK inflation rate calculator: How much are prices rising for you?” Accessed October 2 2023. https://www.bbc.co.uk/news/business-62558817 Bank of England. n.d. Inflation calculator. Accessed October 2 2023. https://www.bankofengland.co.uk/monetary-policy/inflation/inflationcalculator International Monetary Fund, International Labour Organization, Statistical Office of the European Union (Eurostat), United Nations Economic Commission for Europe, Organisation for Economic Co-operation and Development, The World Bank. 2020. “Consumer Price Index Manual: Concepts and Methods”. Accessed January 2, 2023. https://www.imf.org/en/Data/Statistics/cpi-manual International Monetary Fund. n.d. “Monetary Policy: Stabilizing Prices and Output”. International Monetary Fund, Finance and Development. Accessed October 2 2023. https://www.imf.org/en/Publications/fandd/issues/Series/Back-toBasics/Monetary-Policy

46

Chapter 2

National Institute of Science and Technology. n.d,a. “SI Redefinition: Kilogram”. Accessed October 2 2023. https://www.nist.gov/si-redefinition/kilogram-present National Institute of Science and Technology. n.d,b. ”SI Units”. Accessed October 2 2023. https://www.nist.gov/pml/owm/metric-si/si-units#:~:text=The%20 International%20System%20of%20Units,globe%20as%20World %20Metrology%20Day%20 O’Neill, Rob., Ralph, Jeff and Smith, Paul A. 2017. Inflation: history and measurement. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-319-64125-6 Office for National Statistics. 2018. “Users and uses of consumer price inflation statistics: July 2018 update”. Accessed January 2, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/methodolog ies/usersandusesofconsumerpriceinflationstatisticsjuly2018update Office for National Statistics. 2022a. “UK House Price Index: August 2022”. Accessed January 2, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/bulletins/ho usepriceindex/august2022 Office for National Statistics. 2022b. “Consumer Price Inflation, basket of goods and services: 2022”. Accessed January 2, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/ukc onsumerpriceinflationbasketofgoodsandservices/2022 Office for National Statistics. 2022c. “Consumer Price Inflation UK: September 2022”. Accessed January 2, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/bulletins/co nsumerpriceinflation/september2022 Office for National Statistics. 2022d. “Impact of increased cost of living on adults across Great Britain: June to September 2022”. https://www.ons.gov.uk/peoplepopulationandcommunity/personalandh ouseholdfinances/expenditure/articles/impactofincreasedcostoflivingon adultsacrossgreatbritain/junetoseptember2022

What is inflation?

47

Office for National Statistics 2022e. “Tracking the price of the lowest-cost grocery items, UK, experimental analysis: April 2021 to September 2022”. Accessed January 2, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/trac kingthelowestcostgroceryitemsukexperimentalanalysis/april2021tosept ember2022 Office for National Statistics. 2022f. “CPIH-consistent inflation rate estimates for UK household groups: April to June 2022”. Accessed January 2, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/cpi hconsistentinflationrateestimatesforukhouseholdgroups20052017/aprilt ojune2022 Ralph, Jeff., O’Neill, Rob and Winton, Joe. (2015). A practical introduction to index numbers. Chichester UK: John Wiley and Sons. https://doi.org/10.1002/9781118977781 UK Government. 2021. “Budget 2021”. Accessed January 2, 2023. https://www.gov.uk/government/publications/budget-2021-documents

3 HOW DO WE MEASURE INFLATION? In chapter 2, we considered what we are trying to measure and identified the cost of goods approach as the standard which almost all countries use when constructing measures of inflation. We also explored some of the challenges that need to be overcome. In this chapter, we look at how inflation is measured in practice.

3.1

Changing costs and changing value

Everyone has a view of what inflation means for them, affected at any particular point in time by the way prices for some specific products change. Petrol prices change frequently; in 2022 in the UK, they rose in the first half of the year and fell in the second half. Energy bills went up, largely as a consequence of the war in Ukraine. These are costs that we interact with regularly so we recognise when prices change. At the same time new rules on insurance renewal prices meant that these costs became smaller for some people, but this was less visible because insurance renewal typically comes round only annually. It is unlikely that any overall insurance price falls would balance out the large rises in the cost of energy, but these effects should offset each other to some extent in a measure of inflation. So, how do we go about measuring inflation in a systematic and objective way? As we saw in chapter 1, there is a duality between two views of inflation. We can think of prices rising, and we often do think in these terms because we see prices changing. But we can also think of the value of money falling; one pound today buys fewer goods and services than one pound ten years ago. Some of the first steps in inflation measurement in the 19th century were driven by the need to understand the value of gold, and the collection of price information was a way to measure that value (Smith and Ralph 2021). So, one of the key components is information about the prices of goods and services.

How do we measure inflation?

49

There are nevertheless considerable challenges in gathering prices in a way which makes them usable. For example, how do you decide the contribution to inflation from the changing price of a cup of coffee and the contribution from the changing price of a rail ticket? In order to obtain a balanced assessment of the price changes, we need to know the relative expenditure on coffee and rail tickets (and many other products).

3.2

Constructing a price index

A price index is calculated from two main data sources: prices of goods (and services) and information about how much we spend on the things to which those prices relate. In price index terminology the data on relative expenditures are known as the weights. And this makes sense – we should not allow the changes in prices of cutlery, which is something bought relatively infrequently, or equivalently by a small proportion of the population each month, to have the same influence on a measure of inflation as energy, which is bought by nearly everyone more or less all the time, even if that is at different rates in the summer and winter. A price index is put together from information about the prices and weights in two time periods (with some exceptions, see below) – the current period about which we are interested, and a base or reference period which is used to compare the current period information to. There are different ways in which this can be done. The standard approach is based on a method introduced by Joseph Lowe, and later taken up by the German economist Étienne Laspeyres, in which the prices in the current period and the prices in the base period are both multiplied by the weights in the base period – a base-weighted price index23. This assumes that the quantities purchased are the same in both periods, and fixed at the amounts of the base period. By fixing the amounts, we are able to construct a price index which reflects only the changes in the prices, and which is not affected by the 23 We will explain the contribution of Joseph Lowe in the next chapter. In the Lowe approach, the weights may come from a different time period to the base prices. The formulas for combining price and expenditure data are given in appendix A.

50

Chapter 3

changes in the quantities of goods and services sold. This is known as a “cost of goods” index (where ‘goods’ implicitly includes ‘services’) as we saw in chapter 2. By international convention, almost all countries produce consumer price indices constructed in this way. In fact there is a very wide range of ways to put together the price and weighting information in an index number formula, and therefore there has been considerable debate about which is the best formula to choose in any given situation. These debates still continue, and one of the key differences between the Retail Prices Index and the Consumer Prices Index in the UK is in one of the formulas used in their construction; this will be explored in chapter 5.

3.3

Gathering the price information

Collecting price information is a major undertaking. In the nineteenth century, at the beginning of price index calculation in the UK, indices were constructed from a few prices gathered by interested individuals from readily available sources, such as published price lists. But this was not an effective process for a national price index, and the only way to collect sufficient information was to utilise the resources of the state. So early in the twentieth century, after some reviews of the availability of price information, a regular series was started by the a government body, the Board of Trade, and became a very important tool during the First World War for linking wages and prices. We explore the history of inflation measurement in chapters 4 and 5. Many changes have been made in the century since then, and new developments continue. In 2023, the main part of the price capture process is carried out by price collectors who visit shops and other outlets providing goods and services. They record price and product information on hand-held devices. Not every retail outlet can be visited and not every item priced, so how are items and outlets selected? There is a complex sampling process which defines locations and outlets from which prices are collected, again related to expenditure. There is such a large expenditure in some locations that it makes sense to always

How do we measure inflation?

51

include them in the sample of areas where prices are collected, but only a random sample of smaller locations is included. The sample is rotated so that these locations are periodically replaced, and when a new location is included, the retail outlets are first listed, a process that records the outlets and the products they sell and (in some cases) an estimate of the shelf space for a product, as a proxy for the sales. A sample of outlets is selected from this listing, to give a full range of prices in that location (Office for National Statistics 2019, 23-30). We also need to decide which goods and services to collect price information for. The start of this process is related to the weights – only goods and services which contribute sufficiently to expenditure (across the whole population) are included. In the UK, only goods and services which make up approximately 0.5 parts per thousand of expenditure are considered, so that resources are not used on measuring prices which will have a negligible effect on the final index. There are always some goods close to the threshold for inclusion, so additional data and professional judgement are used to decide which of these to include (generally those where the expenditure share seems to be increasing). All the goods and services together form the basket for the CPI. It is not what a typical household buys, but rather a reflection of the expenditure of all households on goods and services. The contents of the basket are a regular topic of interest when they are updated each year, and we look at how the basket has changed over time in chapter 6. Further steps are taken to specify which representative items are selected to reflect the price changes in a category of goods or services (e.g. apples, strawberries and oranges are examples of the representative items used in the “fruit” category, among a range of other fruits which are also priced), and a specific item is selected within each shop to price. The specific item is normally chosen to be one of the most purchased items in that category from that store, and research has shown this to be a good practice (de Haan, Opperdoes and Schut 1999, Dorfman, Lent, Leaver and Wegman 2006). In some other countries even this part of the sampling involves a random selection process, but this is not currently the case in the UK.

52

Chapter 3

The basic principle behind the collection of price data to construct a price index is that the price should refer to the same product (or service) on each occasion, in all of its characteristics. The description of an item to be priced must therefore be recorded in some detail when a new product is first selected. The price collector then checks the product against the description each time a price is collected. If there is a difference, then an adjustment is made, called a quality adjustment because it reflects a difference in the quality of the good or service being priced. We describe this process in more detail in section 3.4 below. There are also some centrally collected prices; for example, from retail chains with national pricing policies, or where goods have a standard price everywhere (e.g. newspapers), or where the prices can be obtained directly from the internet (e.g. flights). Around 180,000 prices are collected each month to feed into the calculation of the Consumer Prices Index. Each needs a base period price to which it can be compared (so that we know how prices have changed). In most cases the base period price has been collected previously and the calculation is straightforward, but for some new or replacement items, it can be necessary to impute a base price, using information from the part of the overall price index to which the price belongs. This process allows new price quotes to be included in the index quickly.

3.4

Constant quality and quality adjustment

It is important to record a range of information about a product: packet size, weight, the place the item is sold (often a shop, but could also be online, for example), and so on. A change in any of these characteristics may alter the quality of the item, that is, an expression of the overall value to a consumer. For example, a different shop may be open different hours, or be located locally, which may affect the quality even if the product itself is unchanged. So in price collection, these details must be checked, and an adjustment to the price may be needed if any of the attributes changes. So the first task when a product is chosen is to record a description. When the size of the product being sold changes, this adjustment is often relatively straightforward. It is well-known that the size of chocolate bars

How do we measure inflation?

53

fluctuates according to the state of the economy. “Our biggest bar ever” is a sure sign that the economy is doing well, and when there is a recession, the size of the chocolate bar shrinks. The size shrinks, but the price doesn’t change. Let’s imagine a situation where a 60g chocolate bar was priced at 90p but is reduced to 55g with the price unchanged at 90p. The price per unit of chocolate bar has gone up, so at constant quality we want to include this in the price index. This kind of inflation even has a name – it is called shrinkflation. We can easily calculate that after the change, a 60g bar would cost 60/55 × 90 = 98.2p, and this new price is used in the calculations, producing a series of prices for a constant quality of chocolate bar (or a series of prices per unit of chocolate, if you prefer to think of it that way). Other quality adjustments are not so easy. The continual increase in computer capability makes it difficult to make an adjustment when a particular computer is replaced by a different one as we described in chapter 2. We noted that it is possible to construct a statistical model from the collected price and characteristic data across many computers as a means to calculate a price at a constant specification or quality. A statistical model works by reflecting the impact of each characteristic on the price of a given computer which is then used to construct a price for a computer of constant quality. This approach is called hedonic regression (Office for National Statistics 2019, 52-54). While a price change may be related to an improved specification it can also reflect a pure price change from the increased price of a component; of course, both may happen at the same time. When a new model of computer is introduced with changes in specifications it is an opportunity for the underlying price to be revised as well, so the overall change in price must be divided into an element that is due to the change in quality, and an element that is a pure price change. The statistical model can make this distinction. Periodically, the statistical model needs to be re-fitted as the relative input prices of computer components will change over time. The effect of this is to produce a price index for computers which generally falls – you get more computer capability (e.g. processing power) for your money as time goes on, or equivalently the cost of a unit of

54

Chapter 3

processing power goes down. So the price index for computers may fall, even though the price of a computer may not change much. There has been some debate over whether this is the right approach. If you need a computer for tasks that require limited processing power, it’s not possible to buy a computer that only does that limited processing, so to obtain the utility from the computer you would need to pay for one that is more powerful than you require. Utility is a concept from economics which has been much used in the development of price measurement. The idea is that a consumer should choose the cheapest way to obtain the same utility, or benefit, from a range of goods and services, and that price measurement should allow for the kinds of choices where a cheaper product is bought in greater quantity than a relatively more expensive one. This is called substitution since one product may be substituted for another; we return to this in the next section. For technology goods we accept for now that the price of a constant quality product reduces, even if it eventually becomes impossible to buy a product with a particular specification. We noted in chapter 2 that there is a further question about how to define quality in the case of goods where fashion has an effect on a consumer’s decision to buy. These kinds of clothing goods typically first become available at a high price, and the price reduces as they become more commonplace or more generally accepted; this can be thought of as a typical product lifecycle. The point of the lifecycle at which a purchase is made affects its quality; a consumer pays a premium to obtain a product early in its lifecycle. However, there is no easy way to measure this effect, or to make a suitable adjustment for this component of quality. If no adjustment is made, and the prices at the end of the lifecycle of one good are linked to the prices of a new good which acts as a replacement in the calculation of the price index, the effect is to make a price series which is continually decreasing. This has been discussed in detail for clothing prices in the UK, since some types of clothing are very prone to changes in fashion. But the same kind of effect can apply in technology, where there is a cachet associated with owning the latest tech. In Sweden a hedonic model is used for clothing prices, but this is an unusual approach. It has been suggested that fashion items should be omitted from the index

How do we measure inflation?

55

calculation in favour of clothing items that are more stable and which therefore generate a continuous prices series. This would avoid the difficulty in trying to measure quality changes in fashion (O’Neill, Ralph and Smith 2017, 362-3). The need to price items at constant quality requires a lot of information on the characteristics of products to be collected as well as just the price. It takes a lot of effort to produce a measure of underlying or pure price change which is not associated with changes in quality. The number of goods to which this approach can be applied has to be small given the effort required.

3.5

Gathering the weighting information

The weights are derived from estimates of how much is spent on different types of goods and services. Most of the detail is derived from the Living Costs and Food Survey, a survey of households in the UK which collects information on households’ spending on different products. The survey is based on a two-week diary for small expenditures, and a one-year recall for expenditure on larger items. Some supplementary sources provide additional information. For example, it is known that cigarettes and alcohol tend to be underreported in expenditure surveys; administrative data from the collection of excise duty provide the total expenditure on these products, enabling an adjustment to be made. Another requirement for the CPI is that it should cover expenditure by everyone in the UK – so omitting expenditure by UK residents abroad, but including spending by foreign visitors. UK residents’ holiday expenditure is measured, but there is no survey which covers (at a detailed commodity level) spending by foreign visitors. We have estimates of the total expenditure of everyone in the UK, gained from surveys of businesses; this can be used as extra information to ensure that all spending in the UK is covered. All of this information, from the survey and from other available sources is put together in the national accounts, where a balancing process trades off the differences between the sources to produce the best estimate of each element of household expenditure.

56

Chapter 3

It is often stated that the CPI represents inflation “for an average household”, but in fact this is not correct, because households do not contribute equally to the patterns of expenditure. Some households have more resources and therefore (in general) spend more than others, and it is the total spend which is used to calculate the weights. Therefore households which spend more money have a larger influence on the CPI and therefore on the rate of inflation. It would be more accurate to say that the CPI represents the inflation for “the average pound spent”. The ONS is developing Household Costs Indices (HCI) which, among other differences, use a weighting approach which gives each household an equal contribution in the calculation of the weights. So the HCIs do represent a measure of inflation for the average household, though they also include several other differences from the CPI; we look at Household Costs Indices in chapter 8. The legacy measure, the Retail Prices Index (RPI), excludes households with the largest expenditures, and also households which derive most of their income from the state pension, that is, some of the households with the smallest expenditures. In this way it tries to get closer to the average household concept, but apart from these adjustments, the weighting still has the property that households with higher spending contribute more. The coverage of products in the RPI is also slightly different, so the RPI and CPI calculations use different sets of weights. We saw in chapter 2 that inflation is different for different groups of people, and this is most often approximated by calculating a new set of weights based only on the expenditure patterns of households or people that belong to the group of interest. This, however, assumes that the group of interest buys goods and services across the whole range of collected prices. This may be a reasonable assumption in many cases, since the prices collected are for the most commonly purchased goods and services within each category. But for some specific groups, particularly those with very limited resources, it may be that only some specific types of goods are affordable. During the period of high inflation in 2021-3 there has been particular interest in inflation for the lowest-priced goods, in a movement instigated by the journalist and campaigner Jack Monroe and supported by

How do we measure inflation?

57

some experimental analyses from the ONS (Office for National Statistics 2022). We look at inflation for different types of household in chapter 8.

3.6

Putting the information together

Once all the necessary components have been gathered, they need to be put together to produce the price indices themselves, from which inflation measures are calculated. This is a hierarchical process, and first step is to combine price information for different items within a store. At this level there is usually no weighting information, and the data are averaged to create an index at the lowest level, called the elementary aggregate level. The type of average used is different in different consumer price indices. In the RPI, elementary aggregate indices are calculated using an arithmetic average of the prices changes for the different items, whereas in the CPI it is mostly the geometric average. In a few cases in both the CPI and the RPI a different calculation is used, the ratio of the arithmetic means of the prices (not the price changes). The difference in choice of elementary aggregate formulas drives a lot of the difference between CPI and RPI, and remains a matter of controversy, which we explore further in chapter 5. The subsequent step sees the elementary aggregate indices combined. From this stage on we have weighting information, so the lowest level indices are multiplied by their weights (presented as a proportion of the total weight) and added together. Component indices with higher weights represent a greater proportion of spending and so appropriately have more influence on the top level index. There is a classification of products called COICOP (classification of individual consumption by purpose) which is hierarchical, going from a range of very detailed products at the lowest level to groups of products at higher levels and to a single overall grouping at the top level. This classification is the basis of the way the components of the CPI are put together (O’Neill, Ralph and Smith 2017, 206-7).

3.7

An ideal inflation index

There has been a long-running debate over the best way to create a price index, extending back into the nineteenth century, but becoming increasingly

58

Chapter 3

sophisticated. There are many formulas for calculating a price index, and the choice of which to use is influenced by many factors, including the availability of the data. This is not the place to review all the arguments, which are complex. The choice of formula we make defines what concept of inflation we are estimating. Almost all National Statistical Institutes (NSIs) use the Lowe index with weighting information taken from a period prior to the price reference period. It takes time to collect the information to calculate the weights. Inasmuch as there is general agreement over the best approach to be used, the target for an inflation measure is a superlative index, so-called because it uses quantity information from both the base period and the current period, and so accommodates some of the changes in spending patterns (the kinds of changes caused by substitution, as discussed in the previous section). However, it is not possible to calculate a superlative index in a timely way, because it needs information on quantities of each product in the latest period. The USA calculates a superlative index retrospectively when quantity data are available, to demonstrate how the target index moves. Such a superlative index has been calculated for the UK and we look at this in chapter 8. We could consider how to best approximate a superlative index using only the weighting information available to us. Studies would suggest using geometric formulas for all stages of calculation from the lowest, elementary aggregate level, right to the top, all-items level. Most NSIs use a geometric formula for their elementary aggregates but above that level, arithmetic formulas are used; almost all use the Lowe formula. Further research is needed to better understand the effects of using geometric formulas at all levels. If we accept a superlative index as a target, then we can assess how closely the index which is actually calculated approximates a superlative index. This is a measure of the quality of the index itself, but a measure different to quality adjustment. This kind of assessment using a calculated superlative index is complex and at best undertaken only occasionally as a special exercise or as a research project.

How do we measure inflation?

3.8

59

Assessing sampling and non-sampling errors

The prices collected as the inputs to the calculation of an overall index are derived from a sample. It is simply not possible to collect data for all products, still less all transactions24. This leads to the question of how much the selection of a particular sample affects the inflation rate actually calculated – what is the sampling error for the price index? To answer this question properly we need a sampling design which uses a probabilistic procedure to select at each stage of the design. The stages involve selections for areas, shops, representative items and the actual products to price. In the UK only some of these stages are probabilistic, so even this kind of calculation involves some approximations. Approximating sampling errors for UK inflation is a work in progress and there are no clear answers yet. In the US however, the whole design is probabilistic, and they do make a regular calculation of the sampling errors. Note that “error” here is used in a statistical sense to mean the difference between what we can measure and some underlying true value which we don’t know, and does not mean that there is anything inherently incorrect in the calculated index. As well as sampling errors there are non-sampling errors, including the errors caused by the variability of the data used as the basis for quality adjustments. Special projects are also needed to assess these errors, so they are calculated infrequently, and there are many components, so it is difficult to gather together all of this evidence on the same price index. Nevertheless there is general agreement that the sampling errors are small compared with the other kinds of errors in the construction of the price index. It has also been known for a long time that the price information has more effect on the quality of the index than the weighting information. There is a framework for putting all this information together, but only in Sweden has an attempt been made, so we are not in a position at this stage to give a view of the overall quality of inflation measures in the UK.

24

The emerging practice of using transaction information from store scanner datasets could increase the volume of price information in some parts of price collection.

60

3.9

Chapter 3

Long-term and short-term change

The methods for constructing a price index are tailored so that they are as good as possible for estimating the price change from period to period, separating out the quality changes from the pure price changes, dealing with changes in the availability of products and dealing with changes in spending patterns and the basket of goods. The extent to which the methods succeed in accounting for these differences is the overall quality that we discussed in the previous section. The tailored methods aim to work as well as possible for changes in prices over relatively short periods. The index is therefore relatively accurate for measuring inflation, that is, the twelve-month change in the price index. Price indices have other uses too, and there are plenty of examples of expressions such as “£3000 in 1880 is worth £200,000 in today’s money” which demonstrate the use of a price index for adjustments over much longer periods. We saw this in the previous chapters, where we described a price index as a means of converting the value of money from one time period to another. This is a different kind of challenge altogether – if there are nearly 150 years between the time of the “£3000” and “today” then the basket of goods and services would be very different in the two periods being compared. How do we make meaningful comparisons over such long periods? We can’t price an identical basket over such a time interval. The basket is updated every year, to ensure it reflects the products in the marketplace and consumer preferences. We measure the price change for a year then update the basket and weighting information and measure price change for another year. Each year’s worth of index values are joined together to make the long-run, continuous series. What change is being measured over a long period is a subtle question. It depends on the ways in which the index has been constructed and joined together to make a longrun series over the comparison period. Essentially we are relying on all the measures of prices relatively close to each other linking together to make a good estimate of the change in prices over a longer period. The possibilities of statistical errors having an effect is much larger over long periods, so these kinds of comparisons would be better regarded as ballpark only, rather than accurate assessments of long-term price change.

How do we measure inflation?

61

The pattern of inflation, the up and down over longer periods, rather than the actual level is probably well estimated.

3.10 Concluding remarks The construction of a price index is a large and complex operation, and a series of checks and quality measures are used to ensure that the calculated index is as good as it can be commensurate with the cost of constructing it. This chapter has given a brief overview of the process of data collection and construction of the CPI. It necessarily leaves out many details of what is included and excluded at different stages of the processing. The Office for National Statistics provides a comprehensive technical manual of the methodology (ONS, 2019). The approach to calculating official statistics changes over time. There are many small changes to ensure the methodology keeps up to date; there are also more fundamental changes which arise occasionally. Recent investigations into the use of data sources which can be collected with less labour-intensive processes, such as store scanner data and price information from retailer websites have been shown to be effective for some products, and it is likely that these new sources will gradually be introduced into the CPI in the future.

References De Haan, Jan., Opperdoes, Eddie and Schut, Cecile M. 1999. “Item selection in the Consumer Price Index: cut-off versus probability sampling”. Survey Methodology, 25, 31-42. Accessed January 2, 2023. https://publications.gc.ca/collections/collection_2016/statcan/12001/CS12-001-25-1-eng.pdf Dorfman, Alan H., Lent, Janice, Leaver, Sylvia G. and Wegman, Edward. 2006. “On sample survey designs for consumer price indexes”. Survey Methodology, 32, 197–216. Accessed October 2, 2023. https://www150.statcan.gc.ca/n1/en/pub/12-001x/2006002/article/9554-eng.pdf?st=ZHk85ov9

62

Chapter 3

O’Neill, Rob., Ralph, Jeff and Smith, Paul A. 2017. Inflation: history and measurement. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-319-64125-6 Office for National Statistics. 2019. “Consumer Prices Indices Technical Manual, 2019”. Accessed January 2, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/methodolog ies/consumerpricesindicestechnicalmanual2019 Office for National Statistics. 2022. “Tracking the price of the lowest-cost grocery items, UK, experimental analysis: April 2021 to September 2022”. Accessed January 2, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/trac kingthelowestcostgroceryitemsukexperimentalanalysis/april2021tosept ember2022 Smith, Paul A. and Ralph, Jeff. (2021). “Measuring the general level of prices in the UK in the long-nineteenth century: from individual innovation to state production”. Romance, Revolution and Reform, 3, 10-35. Accessed October 2, 2023. https://www.rrrjournal.com/_files/ugd/ec6761_d5bf37115b764c93bbb 2de84e35c0f16.pdf

4 A BRIEF HISTORY OF INFLATION MEASUREMENT

The language of inflation is familiar to most of us today. Many articles in the media debate issues around prices, inflation and the likely responses of employers and the government. However, the conceptual basis and the means to measure inflation had to be worked out and it took a long time to achieve. A few key questions arise naturally. When did we start to measure a level of prices and when was it first used to inform wage rates? Are fluctuating prices and their effects a recent phenomenon? In this chapter we give a brief history of the development of the concepts behind inflation and how its measurement evolved. The story extends over a long period of time and in this chapter we will take it from its early beginnings up to the year 1997 when a European measure of inflation was introduced. Chapter 5 takes the story forward to the end of 2022. With just a few pages to dedicate to a long-term history, only a summary is possible. Two of our previous books give a more detailed account (O’Neill, Ralph and Smith 2017; Ralph, O’Neill and Smith 2020).

4.1

A long history of price fluctuations

Examples of efforts to control prices extend back thousands of years. Statutes specifying controls on wages and prices were part of the wellknown Code of Hammurabi (around 2150 BCE) and the Roman Emperor Diocletian’s Edict of 301 CE set out maximum prices for a long list of items. These two examples show that that the damaging effects of price rises were known far into the past and rulers understood that action was required to try to minimise their effects. Two types of event were primary drivers of harmful prices rises – bad weather and war, both of which occurred frequently. Neither of these two famous attempts to manage prices succeeded and the difficulty of controlling prices is evident from many other instances over the centuries (Schuettinger and Butler 1979).

64

Chapter 4

The variation in prices, its causes and whether prices could be controlled were topics that naturally aroused interest from time to time. In order to explore the subject objectively, records of prices and their changes over time were needed. To what extent were such historical records available? Prices were part of accounting records of manors and religious houses extending back to the late 12th century. Estate farming satisfied most needs but some of the yields were sold to pay for desirable items that could not be produced locally, such as wine and salt. These accounts record details of income and expenditure (Burnett 1969, 13-18). Although historical price information was fragmentary, it was sufficient to prompt the interest of a few inquisitive people from a variety of backgrounds, who enquired into what was known as “the course of prices”.

4.2

Comparing the value of money at two time periods

William Fleetwood, the Bishop of Ely, was one of these people with a keen interest in prices. He published an account of English coinage, Chronicon Precosium, in 1707, examining how prices had changed over the course of 600 years. His book contains a solution to a question posed to him. A regulation of an Oxford college stipulated that a fellow would lose his position if his income exceeded £5 in a year. This regulation originated in 1440 at a time when prices were significantly lower than in 1707. The Bishop was asked whether the figure of £5 was still appropriate given the change in prices. To provide an answer, the Bishop chose four items relevant to a fellow: “Corn, Meat, Drink and Cloth”, and examined typical prices in both 1440 and 1707. He found that prices for all four items had risen around five or six times. He concluded the equivalent threshold for income for 1707 would be an amount between £25 and £30 (Fleetwood 1707). We can recognise some important elements in the Bishop’s approach. His four items could be thought of as a very basic basket of goods for which he found prices in two time periods. He worked out the ratio of prices for each of his four items. As the ratios were roughly the same, he didn’t need to consider how to combine them, though it is likely he would have worked out an effective way of doing so. He then converted the threshold sum of money from its value in 1440 to its equivalent value in 1707 using

A brief history of inflation measurement

65

the price ratios. It is an impressive piece of work. If we set it in context, we can say he compared the value of money over two time periods for a specific set of circumstances, though his writing doesn’t indicate that he thought his approach could have a wider significance (O’Neill, Ralph and Smith 2017, 51-52).

4.3

The general level of prices and how to calculate it

We now move from the contribution of a bishop to that of a Member of Parliament: Sir George Shuckburgh Evelyn. Sir George was the MP for Warwickshire between 1780 and 1804. Outside of his work in Parliament, Sir George was interested in what today we call physics, but then was known as natural philosophy. His interests encompassed standards of weights and measures and astronomy; he established an observatory on his estate. Beyond his scientific studies he shared an interest with Bishop Fleetwood: the change of prices over time, which he called “economical researches”. In collecting prices, he chose a broader basket: “the necessities of life together with that of day labour …. at different periods from the Conquest to the present time”. For each time period, he averaged prices creating a measure of the level of prices; collectively, these averages could be used to examine the variation over time. He went further and set the average price to be 100 for the year 1550 and scaled other average price values at each time period, creating an index number series, which is a startlingly modern representation of change. His index series showed a relatively small increase in the level of prices from 1050 to 1550 and a faster increase up to 1790 (Schuckburgh Evelyn 1798). The importance of this work identifying the concept of a level of prices was recognised, though some commentators were critical, suggesting the items he selected to price were insufficiently described and that items bought frequently should be counted more than once (an early form of weighting). Alternative estimates of the level of prices were calculated by others, which showed different rates of increase over time. While the work of Shuckburgh Evelyn and others introduced important aspects of inflation measurement, the foundational elements were brought

66

Chapter 4

together as a coherent whole by Joseph Lowe, a Scottish economist working in the early decades of the 19th century. In his book from 1822, “The Present State of England in Regard to Agriculture, Trade and Finance” (Lowe 1822), he described clearly the importance of a measure of the level of prices and how it could be used as well as how to estimate it. He explained how fluctuations in the value of money arose and caused great harm. He used an alternative term for the level of prices: “the power of purchase” and described how a measure of it would enable the correction of “a long list of anomalies in regard to rents, salaries, wages etc.”. In order to calculate it, he noted that a wider selection of items was required, in his terms: “a standard of more comprehensive character”. His most recognisable contribution comes from his description of the formula for calculating the level of prices, combining price and expenditure data. Today it is called the Lowe formula, and is still used by almost all statistical offices around the world when calculating inflation as we noted in the previous chapter. The mathematical form is given in appendix A.

4.4

Capturing price data

The work of Lowe, Schuckburgh Evelyn and others established the concept of a level of prices, the uses to which it could be put and the means by which it could be calculated. However, the data they collected was on a very small scale. For the concept to be effective, it was clear that recording and collecting data on a much bigger scale was required. Again, it was a few insightful individuals who sought out and collated prices found in trade journals and commercial price lists for a broader range of commodities, including precious metals and wool. It wasn’t a one off activity; it had to be established as a repeated, regular activity. Two of these individuals were Thomas Tooke and William Newmarch who captured and published price data; Newmarch went on to calculate an index series of prices for commodities (Newmarch 1861). The Economist magazine continued this work from 1864 creating both individual commodity price index series and a combined index comprising 22 items25. In Hamburg, the economist Adolf Soetbeer instigated the collection and publication of price data for 300 commodities which was 25

The Economist magazine still publishes commodity price indices.

A brief history of inflation measurement

67

extended back to 1847 by the Chamber of Commerce. Using such data, William Jevons, Augustus Sauerbeck and others calculated measures of the level of commodity prices. Sauerbeck, a London wool merchant, was particularly notable, first publishing an article in the Journal of the Royal Statistical Society on commodity prices and a price index; he published annual updates all the way to 1913 (Sauerbeck 1886; Balk 2008, 9-12). There are two important points to make at this point in our history. Firstly, the data discussed above represented wholesale prices, not retail prices; for a consumer price index we need the latter. Secondly, none of the work involved the state. The state, in the form of the Board of Trade, did start to collate prices for commodities, publishing them in “Miscellaneous Statistics in the United Kingdom” between 1855 and 1883. However, this did not include calculating and publishing price indices.

4.5

The state intervenes

The difficulty in finding useful statistical information in the mass of government, Royal Commission and select committee reports in the early part of the 19th century led to the creation of a statistical department in the Board of Trade in 1832. While this made some improvements in the assimilation and presentation of available data, many of the statistics that were required to inform debate were not produced as the data needed was not being collected. Concerns about the extent of poverty, wages, unemployment and the relative economic performance of the country drove the need for more and better data (Davidson 1995). Pressure for action came from parliamentarians, the trades unions, employers and the Royal Statistical Society. Eventually, in 1886, funding was made available for the creation of a labour department in the Board of Trade, with the Board promising statistics on wages, hours of work, retail prices, household expenditure and the cost of living (Hansard 1886, 73). Initially, the effort of the new bureau was focused on wages and unemployment and progress in developing the collection of retail price and household expenditure information was slow. The first survey of household expenditure in 1889 questioned just 36 men. In 1902, a further survey investigated the income and household expenditure of 114

68

Chapter 4

agricultural workers. Further political pressure was applied to the Board of Trade for data to better inform debate on tariff reform; the Board responded by producing two substantial reports in 1903. In the first report, previous work on wholesale prices was summarised and a long-running wholesale price index series produced using the latest data combined with series from earlier research. The report also presented retail price data, the first time such data had been published in an official document. The second report combined recent price and expenditure data to create a retail price index for the period 1877-1901 (Board of Trade 1903a, 1903b). The pressure to produce these reports on a short timescale was reflected in weaknesses in the data, particularly on household expenditure, of which the Board of Trade was aware. It corrected this by running a more extensive survey in 1904 and published the results in the following year (Board of Trade 1905). The Board made further improvements to household expenditure data in the following years by running further surveys. By introducing arrangements to collect retail prices on a regular basis and improving expenditure data, the Board was able to start publishing an index of retail prices on a regular basis from September 1914. The expenditure data was collected from working class households and the retail prices from “shops in possession of working class custom”. It was called the (working class) Cost of Living Index (O’Neill, Ralph and Smith 2017, 112-7).

4.6

The First World War and the start of indexation

The years from 1900 to 1913 saw a 14% price rise in food prices, with steeper rises up to the start of the First World War. In the early months of the war, the government was concerned that these sharp price rises would trigger industrial action by essential workers demanding improved pay. Wage boards were the bodies tasked with setting minimum pay for their respective industries. Some wage boards already existed and others were formed quickly; the government encouraged them to improve pay, taking into account the recently created Cost of Living Index. This was another important step – the use of a price index to inform pay was the start of what we now call indexation. Adjustments took the form of occasional pay

A brief history of inflation measurement

69

increases and not the regular changes we are used to today (Ralph, O’Neill and Smith 2020, 24-5). The post-war government commissioned a review of how prices had been affected by the conflict. A committee of inquiry was established and it found an almost doubling of food prices between 1914 and 1918. Perhaps surprisingly, there hadn’t been a drop in food consumption, though what was bought did change. The combination of wage rises and restrictions on rents had brought relative stability to what might have been a very difficult period of time. One further point the committee made was that they had found it hard to find all the data they needed; they recommended a central statistical office be formed (Working Class Cost of Living Committee 1918). This wasn’t the first time such a call had been made but not acted on and it wouldn’t be the last.

4.7

Post-war wage indexation

Once instituted, the link between price changes and pay became firmly established after the war. Instead of needing to meet to discuss how to adjust wages as prices changed, the link began to be formalised by what became known as “sliding scales” which set out how wages would be adjusted in response to different levels of increase or decrease in the Cost of Living Index. Unlike today, there was an unattractive aspect of the scales, at least for workers: if prices fell, so did wages. At first, prices rose in the 1920s. In September 1920, about half a million workers were on sliding scales; by 1922 this number had risen to 3 million. The Cost of Living Index then fell and the numbers on the scales declined to 2.5 million by 1925 and 1.5 million by 1933. Reductions in wages were understandably unpopular. The situation then changed back as prices rose from 1934 and trades unions pressed for corresponding wage rises. In the 1930s, the Cost of Living Index was starting to be used for other purposes; for example, a portion of civil service and armed forces’ pensions were adjusted using it (Searle, 2015).

70

Chapter 4

As the Cost of Living Index became more prominent, so it began to attract criticism26. The household expenditure data from before the First World War continued to be used in the index and was increasingly recognised as being unrepresentative of post-war spending. At the time, it was called the “basis” of the index. The index attracted criticism from both business and the trades unions during the 1920s (Hansard 1921). However, sensitivities to the possible implications of updating the “basis” meant a new household expenditure survey was repeatedly deferred. A cabinet paper from 13th February 1931 discussed a recommendation from the Economic Advisory Council that a new “basis” for the Cost of Living Index be produced. The minutes of the meeting record that it was decided to defer any decision as the prospect of determining a new basis might create a disturbance in industrial circles which would be undesirable at the current time (National Archives 1931a, 1931b). The lack of action only increased the degree of criticism. After much discussion across government departments, agreement to proceed with an improved basis was reached in April 1936 (National Archives 1936; Ward and Doggett 1991, 139). A large scale household expenditure survey was eventually carried out in 1937/38. Not surprisingly, a much changed expenditure profile was found. The percentage of household expenditure on food was found to be 39%, down from 60% in 1914. The onset of the Second World War meant that the new estimates could not be incorporated into the Cost of Living Index until after the war had ended (O’Neill, Ralph and Smith 2017, 124-5).

4.8

The Second World War and a new index

Like the period at the start of the First World War, prices rose sharply in the month after war was declared. Trades unions demanded increases in pay to compensate for price rises. The government considered breaking the link between the Cost of Living Index and wages but it was so well established in the minds of workers that an alternative approach was needed. The route the government took was to impose price controls,

26

This is precisely the position we find today, with the debate about whether or not the Retail Prices Index is a poor measure of inflation still going on, as we describe in chapter 5.

A brief history of inflation measurement

71

limiting price rises of basket items to between 125 and 130 percent of prewar levels by applying subsidies. This stabilised the Cost of Living Index and consequently wage increases. What the price subsidies didn’t do was control prices for all items that households bought, just those in the basket. This meant that households experienced price rises that weren’t compensated for. Added to this was the use of the unrevised “basis” from the previous war which was widely recognised as being substantially unrepresentative of spending patterns. By the end of the war, the index was thoroughly discredited and a substantial revision was required urgently (Searle 2015). A fresh start for inflation measurement would require a new household expenditure survey, bypassing the one carried out in 1937/38 which was now almost a decade old. However, subsidies and rationing persisted after the end of the war and it was argued that a new expenditure survey would reflect abnormal expenditure patterns. The alternative was to wait until conditions returned to normal; this presented a difficult problem as there was pressure for change. The Minister for Labour and National Service appointed a Cost of Living Advisory Committee in August 1946 to advise him on the best course of action. The committee comprised representatives from government departments, the trades unions, employers and academics, supported by a technical group. It was clear from comparing the 1914 and 1937/38 expenditure patterns that the use of the old “basis” from the beginning of the century must end as quickly as possible. Using sales data, it was possible to show that spending patterns were volatile suggesting that conditions were not sufficiently stable to justify a new household expenditure survey. To resolve the deadlock, the advisory committee recommended the creation of an interim index using the 1937/38 “basis”, that is, expenditure weights; the weights could be updated via a new household expenditure survey when conditions allowed (O’Neill, Ralph and Smith, 2017, 132-136). Although the interim index wouldn’t contain up to date weighting data, other improvements could be made. The basket was expanded with the number of food items increasing from 14 to 80. The data for expenditure came from all households and prices were taken from outlets serving the whole population; other methodological revisions were made as well. The

72

Chapter 4

advisory committee recommended that, in future, regular updates to the expenditure weights (the “basis”) be determined and applied. This was important point; without regular revisions, the expenditure weights would become increasingly out of date and would diminish public confidence as had happened before. The Interim Index of Retail Prices began in January 1947. By 1951, the advisory committee decided that conditions were sufficiently stable that a new expenditure survey could be run. Despite some reservations, it was decided to proceed and a new expenditure survey was carried out in 1953/54. The use of revised weights was combined with other improvements to create the Retail Prices Index which began in January 1956 (O’Neill, Ralph and Smith 2017, 136-142). The advisory committee continued to meet on an occasional basis to consider options for developing the index further27. One suggestion it considered was whether separate versions of the index should be produced for different social and economic groups. A specific index for professional workers was one option, though it was rejected by the committee. Similarly, regional indices were considered, but also rejected, with the committee being concerned that producing multiple monthly indices would just lead to public confusion. These questions were considered again in 1967 and the advisory committee did recommend separate indices for one and two-pensioner households with the weights calculated using expenditure data taken from these types of households; they were produced from 1969. The need for further social and economic indices together with regional indices was considered at regular intervals by the advisory committee. The current position is discussed in chapter 8.

4.9

Revisions to the RPI

By the 1960s, changes in society needed to be reflected in the calculation of the index. Expenditure on meals bought and eaten outside of the home was increasing. While this had been partly accounted for across different parts of the index, its growing importance meant it justified its own group

27

The advisory committee decided to change its name in 1968 from the Cost of Living Advisory Committee to the Retail Prices Index Advisory Committee. Their report from 1971 was the first to use the new name.

A brief history of inflation measurement

73

within the index. In 1962, the advisory committee recommended starting price collection, which the Ministry of Labour enacted in 1963 and 1964. This proved effective and a new group in the index, “meals bought and consumed outside the home”, was included from 1968. Price change for owner occupiers’ housing was recognised as important but difficult to measure, as we noted in chapter 2. It had been considered by the committee in 1952 and 1962 and was examined again in 1968. It was originally included in the index through a proxy – the price change of equivalent rents; this was an internationally recognised method. An alternative approach was to use a measure of the costs associated with buying a property and its maintenance; however, this presented difficulties. Some properties were bought outright while others were bought with a mortgage. While some of the money paid represented the cost of providing essential “housing services” to owners, payments could also be thought of as representing an investment; savings and investments were excluded from what was a consumption index. The advisory committee reconsidered the way owner occupiers’ housing was measured in 1968 and decided that “rental equivalence”, though not ideal, was still the best method. Perhaps not surprisingly, owner occupiers’ housing was revisited by the committee in 1974. This was motivated by concerns that movements in imputed rents were not in line with observed rents; this, together with the growth in owner occupation and mortgages, prompted reconsideration. A different and more direct approach was considered which would include mortgage interest payments. This had the advantage that it was a more understandable method but had the drawback in that some parts of a payment were more like savings and investments which should be excluded from a price index. After some consideration, the advisory committee devised a satisfactory approach based on mortgage interest and it was introduced in 1975, replacing the rental equivalence method. The treatment of owner occupiers’ housing would continue to be re-examined periodically and is controversial to this day (O’Neill, Ralph and Smith 2017, 149-150).

74

Chapter 4

An interesting inclusion in the report of the committee from 1986 was a list of the uses to which it was appropriate for the Retail Prices Index to be put. These included: for pay bargaining, to monitor the effectiveness of counter-inflation policies, the uprating of social security benefits, pensions and tax thresholds and removing the effects of price change from other economic indicators such as retail sales, a process known as deflation. This list provides a good indication of the growth in the use of an inflation measure from its origins in 1914. In the 1990s a new use arose from the practice of inflation targeting. As we noted in chapter 2, the adoption of monetary economics meant that a target was set for inflation. The measure used to monitor performance against the target was a variant of the RPI: RPIX, which was the RPI excluding mortgage interest. As mortgage interest was affected by the rate of inflation it could not be part of a measure used to compare inflation to the target. The inflation target value was set to be 2.5% as measured by RPIX. The Retail Prices Index Advisory Committee continued to meet periodically to discuss what improvements to the methodology of the RPI were required. Over the years, the changes proposed by the advisory committee were discussed in government. Occasionally, matters arose that were of interest to employers’ organisations and trades unions (O’Neill, Ralph and Smith 2017, 145-158). In contrast to the regular debate in government and employment circles, there was little discussion in the public realm, a position that would change after 2010 as we will see in the next chapter.

4.10 Concluding remarks The long history of development of measures of inflation saw the conceptual basis put on a firm footing in the 1820s through the work of Joseph Lowe. Although he had set out the need for a measure, how to calculate it and the important uses to which it could be put, it took 90 years before the first national measure was produced and used. Given its importance, why did it take so long?

A brief history of inflation measurement

75

Two reasons have been suggested. Firstly, much effort went into Natural Philosophy which was seen as a noble calling. In contrast, studying prices was seen as being somewhat less noble. Indeed, several early students of prices felt they had to apologise for their “economical researches”. Secondly, a robust national measure needed significant resources which were beyond the capability of individuals and needed the resources of the state. The start of the working class, Cost of Living Index coincided with the growing role of the state in society. Improvements in inflation measurement were slow at first with little change occurring until after the Second World War. In contrast, the Retail Prices Index saw regular development from its beginning in 1956. The RPI was the UK’s sole measure of consumer price inflation until 1996, a period of 40 years. Over this period, there was debate over how the RPI was constructed, but this was also entirely carried out in professional circles. It was only the need for an updated basis that was discussed more widely. This would change in 2010. The conjunction of a technical choice hidden deep in the index with economic and political moves outside of the UK would lead to long-term controversy. Our story of the development of inflation measures and their uses continues in chapter 5.

References Balk, Bert. M. 2008. Price and Quantity Index Numbers. Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CBO9780511720758 Board of Trade. 1903a. “Report on Wholesale and Retail Prices in the United Kingdom in 1902, with comparative statistical tables for a series of years”. London: HMSO. Board of Trade. 1903b. “Memoranda, Statistical Tables and Charts prepared in the Board of Trade with reference to various matters bearing on British and foreign trade and industrial conditions, Cd. 1761”. London: HMSO. Board of Trade. 1905. “Second series of memoranda, statistical tables, and charts prepared in the Board of Trade with reference to various matters bearing on British and foreign trade and industrial conditions, Cd. 2337”. London: HMSO.

76

Chapter 4

Burnett, John. 1969. A History of the Cost of Living. Middlesex, England: Penguin Books. Davidson, Roger. 1995. “Official labour statistics: a historical perspective”, Journal of the Royal Statistical Society, Series A, 158, 165-173. https://doi.org/10.2307/2983410 Fleetwood, William. 1707. Chronicon Precosium, or an account of English money, the price of corn and other commodities for the last 600 years. London: Charles Harper. Hansard. 1886. Journals of the House of Commons. Jan-Jun and Aug-Sept 1886. Vol. 141. Accessed October 2, 2023. https://assets.parliament.uk/Journals/HCJ_volume_141.pdf Hansard. 1920. Debate in the House of Commons, 1st December 1920. Vol. 135 cc1284-5W. Accessed October 2, 2023. https://api.parliament.uk/historic-hansard/written-answers/1920/ dec/01/cost-of-living#column_1285w Hansard. 1921. Debate in the House of Commons, 20th June 1921. Vol. 143 cc885-7. Accessed October 2, 2023. https://api.parliament.uk/historic-hansard/commons/1921/jun/20/costof-living Lowe, Joseph. 1822. The present state of England in regard to agriculture, trade and finance: with a comparison of the prospects of England and France. London: Longman, Hurst, Rees, Orme and Brown. National Archives. 1931a. “Economic Advisory Report: Committee on Revision of Cost-of-Living Index Number, 13th February 1931”. CAB 24/219/44. National Archives. 1931b. “Minutes of the Cabinet, 18th February 1931”. CAB 23/66/15. National Archives. 1936. “Memorandum from the Minister for Labour, 21st February 1936”. CAB 24/260/22. Newmarch, William. 1861. “Results of the trade of the UK during the year 1860; with statements and observations relative to the course of prices since the year 1844”. Journal of the Statistical Society of London, 24, 74-124. https://doi.org/10.2307/2338413 O’Neill, Rob., Ralph, Jeff and Smith, Paul A. 2017. Inflation: history and measurement. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-319-64125-6

A brief history of inflation measurement

77

Sauerbeck, Augustus. 1886. “Prices of commodities and the precious metals”. Journal of the Statistical Society of London, 49, 581-648. https://doi.org/10.2307/2979294 Schuettinger, Robert L. and Butler, Eamonn F. 1979. Forty centuries of price and wage controls – how not to fight inflation. Washington D.C.: The Heritage Foundation. Searle, Rebecca. 2015. “Is there anything real about real wages?” The Economic History Review, 68, 145-166. Section 4.8. https://doi.org/10.1111/1468-0289.12059 Shuckburgh Evelyn, George. 1798. “An account of some endeavours to ascertain a standard of weight and measure”. Philosophical Transactions of the Royal Society of London, 88 133-182. https://doi.org/10.1098/rstl.1798.0010 Ward Reg. and Doggett Ted. 1991. Keeping Score. Central Statistical Office, London: HMSO. Working Classes Cost of Living Committee (1918). “Report of the committee appointed to inquire into and report upon (i) the actual increase since June, 1914, in the cost of living to the working classes and (ii) any counterbalancing factors (apart from increases of wages) which may have arisen under War conditions, Cd. 8980”. London: HMSO.

5 WHY IS INFLATION MEASUREMENT CONTROVERSIAL? All official statistics require choices to be made in the way they are constructed. During the development or enhancement of these statistics the options are discussed by experts and a consensus or majority position reached. If further study reveals new information, then changes can be made to ensure the best methodology is being used, as far as existing knowledge allows. For complex statistics like inflation, there are very many of these choices to be made. In most cases, decisions are straightforward and attract little dispute; discussions take place in specialist forums and rarely extend into the public sphere. However, an obscure, but highly consequential aspect of how inflation measures are constructed in the UK burst into the public eye in 2010 and has remained there ever since. It has resulted in protests from pensioners’ groups, court cases, inquiries by parliamentary committees, interventions from the Royal Statistical Society and put the governing body of UK official statistics in a difficult position. In this chapter, we continue the history of inflation measurement in the UK from chapter 4, exploring the events that led up to this extraordinary situation and the consequences which are still being played out. Its origins can be found in choices made many years ago and uses which hard-wired a specific price index and its methods into long term applications which have made change difficult.

5.1

A new measure arrives

The era where the Retail Prices Index was the sole UK measure of inflation came to an end in the 1990s with the introduction of a European measure. The creation of the common currency for the European Union, the Euro, was agreed in 1992. In order for countries to be allowed to join the currency union, their economies were required to show a degree of economic convergence, with criteria set to measure the extent to which

Why is inflation measurement controversial?

79

convergence had been achieved. One criterion concerned the level of inflation. To make a consistent assessment across countries, a common measure of inflation was required. As each country had its own measure which had evolved over many years, the prospect of every country having to change its domestic measure to a newly defined common index was daunting. Instead, it was decided that a new common inflation measure would be created to run alongside the many existing domestic measures. Preparations for a harmonised consumer price index had already started in 1991, but after 1992, an urgent programme of development started under the management of Eurostat, the statistical organisation of the EU. The new measure was called the Harmonised Index of Consumer Prices (HICP)28 and the story of its development is well documented by John Astin who headed up the development work (Astin 2021). Development of the new index in each country began in the mid-1990s with experimental versions produced before full implementation at the start of 1997. As required by Eurostat, the UK, like the other EU countries, produced its version of the HICP which ran alongside the RPI and initially attracted little public attention.

5.2

Differences between the RPI and the HICP

The RPI and the HICP differed in a number of aspects of their methodology. The population base of the RPI excluded high and low income households; the expenditure weights did not include households in the top 4% of income and at the other end of the income scale, those who earned at least 75% of their income from the state pension. It was thought that these groups were unrepresentative of the general population. Both were included in the HICP. There were also differences in the commodity coverage with some items included in scope for the RPI but excluded from the HICP (e.g. buildings insurance) and vice versa (e.g. stockbroker fees). The treatment of owner occupiers’ housing also differed, with it included in the RPI but not in the HICP29. The coding scheme, that is, the allocation

28

Initially, there was some concern over the name – it sounded like “hiccup”. The choice of the best method for including owner occupiers’ housing took much further study; it was added in 2013 in the UK. 29

80

Chapter 5

of items to categories, for the HICP was based on an international standard and was consistent with the National Accounts, while the RPI used a longstanding, custom approach (Office for National Statistics 2011a). All of these differences contributed to a gap between inflation measured using the RPI and the HICP. However, the difference that came to public prominence involved the mathematical formulas used in the calculations at the lowest level of the aggregation structure known as the elementary aggregate level. As we noted in chapter 3, weighting information is not available for most elementary aggregates; about 60% of the categories at this low level use unweighted combinations of prices. For these unweighted elementary aggregates, the RPI uses arithmetic formulas and the HICP uses a geometric formula30. The mathematical forms are given in appendix A. The choice of formula at the lowest level of a price index should be a technical detail that is only of concern to specialists who study and develop the methodology. If the magnitude of the difference in inflation calculated using the different formulas was small, say around 0.1 percentage points, then any debate over the best formula to use would have remained in the expert domain. However, in the UK, the difference between inflation calculated from the domestic measure, the RPI, and the HICP, resulting from the use of different formulas, averaged 0.54 percentage points for the period January 2005 to December 2009, a difference that was much more than “a technical detail”. However, at first, this difference didn’t attract much public attention as the HICP was only used for reporting inflation on a common basis to the EU and for a specific, technical purpose, as we will see in section 5.5.

5.3

Elementary aggregates and the RPI

The different choices of formula at the lowest, elementary aggregate level of the RPI and the UK version of the HICP would go on to become a major issue for inflation statistics. How did it come about that the choices of formula were different in the two indices? There is a long history 30 An arithmetic index, the Dutot, is used in about 5% of elementary aggregates in the UK version; a geometric mean is unsuitable for small or zero prices.

Why is inflation measurement controversial?

81

behind this question and we will give only a brief account. Investigations into the relative merits of different formulas have been going on for more than a century. For elementary aggregates, where weighting information is unavailable, the choice is between the Carli, Dutot and Jevons formulas31. The Carli is the arithmetic average of price relatives32; the Dutot is the ratio of arithmetic averages of prices and the Jevons is the geometric mean of price relatives. The Dutot and Jevons formulas tend to produce very similar index numbers but the Carli formula can produce quite different values in certain circumstances. When the RPI was first calculated, most prices were collected locally within towns. The collected prices were averaged for each town and the price change between two periods calculated as a ratio of averages, which is a use of the Dutot formula. For centrally collected prices, received by post, the ratio of prices was first calculated and then averaged, which is a use of the Carli formula. The calculation of the RPI was computerised in the 1960s, at which point all elementary aggregate price changes were calculated using the Carli formula (Ministry of Labour and National Service 1959; Carruthers, Sellwood and Ward 1980). In the 1970s, average prices for a number of goods were published regularly in the Department of Employment Gazette33. Comparisons of price change between these averages (a Dutot calculation) and averaged price relatives in the RPI (a Carli calculation) showed differences, in some cases magnified by the relatively high levels of inflation at the time. This was a concern and the question of the choice of formula was studied in the Department of Employment and by a technical group reporting to the RPI Advisory Committee in 1976-77. The recommendation on formulas was to switch to the Dutot; however, this was only implemented for elementary aggregates containing homogeneous items (Department of Employment 1977)34. Although the recommendation was to use the Dutot for nonhomogeneous items as well, the advisory committee felt that other 31

There are other choices but we won’t explore them here. A price relative for an item is the ratio of its current price to the reference price. 33 In the 1970s, the RPI was produced by the Department of Employment 34 Elementary aggregates containing homogeneous items tend to show a small variation in price change. 32

82

Chapter 5

considerations were important and the RPI went ahead with a mix of the Dutot and the Carli (Carruthers et al. 1980). The documentary record doesn’t show any further work on formulas in the UK until the end of the 1990s. A number of papers from National Statistical Institutes around the world appeared in the 1980s examining the choice of elementary aggregate formula. In the 1990s, a number of events brought the topic to prominence. A paper from Statistics Canada presented at the first meeting of a specialist conference of price statisticians in Ottawa35 in 1994 discussed potential problems when using the Carli formula in a consumer price index (Schultz 1994). In 1995, in the US, a commission was appointed by the Senate to examine the construction and use of inflation measures. It was led by the economist Michael Boskin; the Boskin Commission reported in December 1996. One of its recommendations was to use the geometric mean for elementary aggregates, which it claimed better represented substitution behaviour. This occurs when consumers switch between brands or products in response to relative price changes. By 1997, the UK had started to produce its version of the HICP, which sat alongside the RPI. In response to the differences between the ways the indices were constructed, the ONS started a three year programme to review the methodology of the RPI. This work included reviewing the use of elementary aggregate formulas and recommended either not using the Carli or minimising its use. This resulted in a reduction in the use of the Carli in the RPI, with revised criteria for choosing between the Carli and Dutot based on the variance of the price quotes. Homogeneous items, that is, those with low variance, would use the Dutot with the Carli used otherwise (Baxter and Camus 1999).

5.4

Elementary aggregates in the HICP and beyond

At Eurostat, work on specifying the formula for elementary aggregates for the HICP began in 1993. Before the 1990s, the Jevons, the geometric average of price relatives, was not used regularly in price indices. Why 35 The conference has continued and meets every few years – it is known as the Ottawa Group after the location of its first meeting.

Why is inflation measurement controversial?

83

was this? It may have been because of the situation where a zero price occurred, when the geometric mean couldn’t be used, or that in the past it was more complex to calculate than an arithmetic mean. The Dutot and the Jevons formulas were both were considered acceptable for inclusion in the HICP. Other formulas were allowed as long as they produced index numbers that didn’t deviate much from the index numbers produced by the Dutot and Jevons. A specific comparability criterion was specified and this effectively ruled out the use of the Carli which in certain circumstances produces quite different index numbers (Astin, 2021, pp.124-133). In the UK version of the HICP, the Jevons is used for 64% of the elementary aggregates, with the Dutot used for 5% and the remainder having weights available, so using a weighted formula. In the same period, other EU countries were compiling their versions of the HICP to run alongside their domestic measures. It is interesting to compare and contrast their experiences to what occurred in the UK. As we noted above, in the UK, the difference between inflation calculated with the domestic measure (the RPI) and the UK version of the HICP resulting from different formulas at the elementary aggregate level averaged 0.54 percentage points. Was this size of difference found in other EU countries? No. When the HICP was first introduced, most EU countries reported a difference between their domestic measures and the HICP of 0.1 percentage points, a much smaller gap (O’Donoghue and Wilkie 1998). Why was this? For their domestic measures of consumer price inflation, a number of EU countries originally used the Carli in at least some of their elementary aggregates, including Denmark, Italy and Luxembourg; however, they all swapped to using the Jevons. Other countries moved from the Dutot to the Jevons. If we look wider than the EU, Canada moved from the Carli to the Dutot in 1978 and to the Jevons in 1995, Australia from the Carli to the Jevons in 1998 and Switzerland from the Carli to the Jevons in 2000. Only the UK retained the use of the Carli index in its domestic measure. Those countries that used the Dutot in their domestic measures found that the HICP calculated with the Jevons differed by around 0.1 percentage points (Evans 2012).

84

Chapter 5

Why did the Carli and Jevons differ much more than the Dutot and the Jevons? Mathematical analysis provides important insights. Using analytical techniques, it can be shown that the Carli/Jevons difference is related to the variability of the price relatives; this means that the greater the differences between items in an elementary aggregate, the greater the variability and therefore the greater the difference will be. In contrast, the Dutot/Jevons difference is proportional to the variability of the price relatives at the current period minus the variability of the price relatives at the base, or reference period. The variability of the price relatives at the two time periods is usually similar which explains why the Jevons and Dutot difference is small (O’Neill, Ralph and Smith 2017, 243-259)36. The move away from the Carli to the Jevons or Dutot that almost all countries made over the period 1978 to 2000 was motivated mostly by concerns about the behaviour of the Carli which can over-estimate the effects of price change relative to other formulas in certain circumstances. A further motivation came from economic theory that suggested the Jevons better represented consumer behaviour, a belief that was subsequently rejected, as we will see.

5.5

Changing the inflation target

In 1992, a new use was found for the Retail Prices Index. With the adoption of inflation targeting as part of a monetary economics approach, RPIX, a variant of the RPI which excluded mortgage interest, was selected as the measure to assess economic performance against a target initially set at 2.5%. For the RPI, which was designed as a compensation index, it was a use in a different context. In June 2003, the Chancellor of the Exchequer announced the intention to change the inflation measure used to assess performance relative to the inflation target. The UK’s version of the HICP would replace the RPIX, but with a revised target of 2% to reflect the difference between the two inflation measures. This change of measure was confirmed in a pre-budget report published in December 2003 (HM Treasury 2003).

36

We give the formulas for the differences in appendix A.

Why is inflation measurement controversial?

85

The reasons behind the change included: better international comparability in the use of formulas, wider population coverage and a better representation of consumer behaviour. The document was accompanied by a paper from the Office for National Statistics in December 2003, in which the National Statistician, Len Cook, announced that the UK version of the HICP would be renamed the Consumer Prices Index (CPI). The ONS paper looked at the choice of measure in more detail. It provided a brief history of the origins of the RPI and compared it with the recently arrived HICP. It also included an overview of how the measures are constructed and the differences in their methodologies (Office for National Statistics 2003). Of particular interest to our story is the treatment of consumer behaviour. The annual updates to the basket and the expenditure weights account for changes in the consumer marketplace and consumer preferences. If relative prices have changed and consumer purchasing changes in response, this can be accounted for in the annual updates. However, as we noted in chapter 2, during the year the methodology treats consumer behaviour as unchanging; just the changes in prices are recorded. The ONS report from 2003 describes the resulting measure as a “fixed quantity” price index; an alternative description is a “pure price” index or a “cost of goods” index. How do consumers behave when relative prices change? We know from analysis of detailed consumer purchasing data obtained from commercial market research companies that customers do change their purchasing behaviour in response to relative price changes and it happens throughout the year37. The behaviour is complex with the willingness to swap between products and product types varying from product to product. Including this complex behaviour in a consumer price index outside of the annual updates would be very difficult. In section 2.9, we described two approaches to constructing an inflation measure: the economic cost of living approach and the cost of goods approach. If our aim was to produce an economic cost of living index then at least in principle we would try to include substitution behaviour and it would give a lower value for inflation than the cost of goods approach. Typically, a cost of goods index is said to overestimate inflation because of this.

37

Research by the ONS on alcohol purchases shows this (Winton et al. 2013).

86

Chapter 5

The important point here is that the aim of the RPI was to produce a cost of goods index and not a cost of living index. This point was made when the RPI was first produced and was restated by the Retail Prices Index Advisory Committee in its review of the RPI methodology in 1986; it was very clear in stating that the RPI was a pure price index and not a cost of living index. The HICP was also clearly specified as being a pure price index (Department of Employment 198638; Astin 2021, 29-30). The ONS document from 2003 seems confused as it clearly states that neither the RPI nor the CPI is a cost of living index but then suggests that a benefit from using the CPI as the target measure is that it would better account for in-year substitution behaviour, through the use of the Jevons index. However, subsequent study showed that this belief about the use of the Jevons index is based on an unrealistically simple economic model (O’Neill, Ralph and Smith 2017, 230-233). The other arguments the Treasury and ONS reports present for preferring the CPI were more soundly based. The CPI, the UK version of the HICP, did more closely match international standards; it was designed more recently than the RPI and embodied more up to date understanding of best practice. It was designed as an index for macroeconomic purposes. In addition, the use of the Carli index exhibits an upward bias relative to other formulas (in certain situations). It is interesting to wonder what the RPI Advisory Committee would have said about the CPI and the arguments for preferring it for inflation targeting. Unfortunately, we will never know as the committee did not meet after 1994 and a new group, the Consumer Prices Advisory Committee (CPAC), was not convened until 2009. Both the pre-budget report of 2003 and the accompanying ONS announcement made important statements about the intentions going forward. The pre-budget report stated that the use of the RPI for the purpose of adjusting pensions, benefits and index-linked gilts would remain. The National Statistician wrote that the CPI was now the main 38

As chapter 4 noted, the original consumer price index launched in 1914 was called “the Cost of Living Index”; this name was considered misleading and so was changed in 1947.

Why is inflation measurement controversial?

87

domestic measure for macroeconomic purposes. This was a very clear statement and presented a sensible approach going forward. At the time, this switch of the measure for use in inflation targeting didn’t attract much public attention. Perhaps this was because of the reassuring statement that the RPI would continue to be used for uprating purposes. For observers and commentators, the switch of measure for inflation targeting could be seen as a purely technical matter.

5.6

Developments in the 2000s

A major change to the governance of official statistics in the UK occurred with the formation of a Statistics Commission in 2000 as part of the Labour government’s commitment to make official statistics more independent of government. The RPI was an exception to this independence with the governance including special arrangements for its management. The use of the RPI in important applications such as index-linked gilts meant that the scope and definition of the RPI was under the control of the Chancellor of the Exchequer while the National Statistician advised on the methodology. Any potential change to the methodology would need to be discussed with the Bank of England and the Chancellor in case it would result in a detrimental effect on holders of index-linked gilts. In the event of a proposed change being classed as detrimental, it would be the Chancellor of the Exchequer who would decide whether it could go ahead. The terms of the issue of the gilts included an option for holders to redeem them should a detrimental change occur, which could be difficult for a government to finance (Statistics Commission 2004). This consultation process with the Bank of England and the Chancellor was triggered in 2004 when changes were made to the treatment of electronic goods in both the RPI and the CPI. We noted the difficulty in measuring price change for these goods in chapters 2 and 3; in order to accommodate the complexity of these commodities, use was made of a statistical technique for quality adjustment called hedonics. Hedonic models were introduced for some electronic goods firstly in the CPI in 2003 and then in the RPI in 2004. The changes were discussed with the Bank of England and the Chancellor and were deemed not to adversely affect gilt-holders. In the past, these changes would have been discussed in

88

Chapter 5

an advisory committee, but the RPI advisory committee had not met for a decade. In its report from 2004, the Statistics Commission expressed concern regarding the governance of the RPI with the continuing lack of an advisory committee and insufficient public awareness of methodological changes being made. It recommended that the ONS make a greater effort to publicise changes in methodology (Statistics Commission 2004). For the years up to 2009, the Office for National Statistics was engaged in a busy schedule of work on price statistics. A multi-year review of the RPI had started in 1999 and reported in 2003. In addition, a series of improvements to the CPI were being made. The ONS carried out further work on the difference between the RPI and the CPI after 2000. A short history of the use of formulas was published by the ONS in response to a freedom of information request. It stated that an internal report from 2005 re-examined the use of the Carli formula and the criteria for deciding between the Carli and the Dutot in the RPI. It concluded that the Carli should not be used. However, this was not implemented and the ONS note stated that no further documents on the matter were produced in the years following (What do they know? 2012). In this period there were also a number of administrative and governance changes. The production of the RPI moved from ONS London to ONS Newport; the Statistics and Registration Service Act gained Royal assent in July 2007 and the UK Statistics Authority was established, replacing the Statistics Commission, thereby further separating the management of official statistics from government. Although there was no advisory committee, the ONS met regularly with the Bank of England and the Treasury to discuss matters of common interest. At the meeting in January 2009, the minutes record three items of discussion. First, the overall methodology of the RPI was considered to be out of date and required reviewing, second that the choice of elementary aggregate formulas was a priority and third there was concern that the lack of an advisory group might have inhibited change in the indices (What do they know?, 2012). The lack of an advisory group was resolved with the formation of the Consumer Prices Advisory Committee (CPAC), which first met in July 2009.

Why is inflation measurement controversial?

5.7

89

A controversial switch of indexing measure

In contrast to the switch from the RPI to the CPI for inflation targeting in 2003 which attracted little public attention, a bombshell announcement was contained in the budget report for 2010. It said that “the Government will adopt the CPI for the indexation of benefits, tax credits and public sector pensions from April 2011 and Local Housing Allowance from 2013-14”. This was a different message to that given by the National Statistician in 2003 and a very significant change with long-term consequences. The document also stated that “the Government is reviewing how the CPI can be used for indexation of taxes and duties while protecting revenues” (HM Treasury 2010, 17-18). The presentation of the change in the budget report is highly informative. It comes under a section titled “budget spending measures” with the statement that the change, along with a two year freeze on public sector pay, would contribute to “spending consolidation”. A table of the financial implications of budget policy decisions shows that the change of indexing was expected to contribute over half of the total expected savings from all of the 29 changes being made to welfare measures. It was a primarily a means to reduce government expenditure. The consequences of this move from the RPI to the CPI in 2010 were immediately apparent to financial journalists, think tanks, trades unions and pensioners’ groups. It would mean a lower rate of growth of pensions and benefits with the cumulative effect increasing each year. In 2011, the government’s decision was challenged in court by six public sector unions. At the hearing, senior staff from HM Treasury confirmed that the motivation for the change was to save money and that the changes would deliver significant savings. They also said that the CPI was a “better” measure of inflation (High Court 2011). In a curious and unfortunate turn of events, an unconnected revision to the instructions for collecting prices for clothing items increased the difference between the RPI and CPI. In 2008, the UK version of the HICP was subject to a compliance monitoring assessment by Eurostat, the statistical office of the European Union. Compliance assessments are a mechanism

90

Chapter 5

to assess the quality of the implementations of the HICP across EU countries. A persistent negative rate of price change for clothing in the UK version of the HICP, the CPI, was in contrast to other EU countries and the ONS was asked to investigate. Revised instructions for price collectors were developed and applied to clothing price collection in January 2010; this led to an increased level of clothing inflation. However, the change also resulted in a bigger RPI-CPI inflation gap, which went from 0.59 to 0.85 percentage points. The difference between the CPI and RPI inflation measures resulting from the use of different elementary aggregate formulas became known as the formula effect. Ironically, the change of instructions was made to address a perceived issue in the CPI, but the same price quotes were used for both the CPI and the RPI. The increase in the formula effect only made the government announcement later that year even more consequential and controversial (Ralph, O’Neill and Smith 2020, 84-6). The impact of the change in indexing from the RPI to the CPI was softened by a more generous treatment of the state pension announced in 2010, as we described in chapter 1. The annual uprating of the state pension would be governed by what was christened the “triple-lock”, defined as the greater of the CPI, average wages and 2.5%, a generous adjustment approach39. All three of these indexing mechanisms had been applied to state pensions in the past starting with a link to wages in 1974, a reversion to prices in 1979, then from 2002 the greater of prices and 2.5%. The triple-lock is still in operation at the time of writing, though it was temporarily suspended for the 2022/23 financial year, when the increase was determined by just CPI inflation which gave a 3.1% increase (UK Parliament 2021). The 2010 budget report and subsequent court case made it clear that the motivation for the change in indexing measure was to reduce public expenditure. The financial and political context in the UK saw the new coalition government inheriting a challenging financial position following the recession of 2007-8; it responded by imposing a period of austerity. While it is likely there was some consideration of the relative statistical 39

The adjustment for the tax year 2011/12 used the RPI.

Why is inflation measurement controversial?

91

merits of the RPI and CPI, it was clearly a secondary concern. Many observers just saw the change as an opportunistic move from an index with a higher value to one with a lower value with a token effort to justify the change on a partial statement of the perceived merits of the CPI. What statistical and economic justification was offered for the change of measure? The budget report gave three reasons. First, it claimed that the CPI provided a better reflection of the inflation experiences of benefit and pension recipients because it excluded most of the housing costs faced by homeowners; specifically, the RPI includes mortgage interest payments and most pensioners owned their own homes outright. For those who didn’t own their own homes and were on low incomes, the government provided support in other ways such as through housing benefit. Second, the CPI better accounted for substitution behaviour and third, it provided consistency with the measure used for inflation targeting (HM Treasury 2010). The change to the inflation target measure was justified, at least partly, on the fact that the CPI (HICP) was designed as a macroeconomic measure. This wasn’t a valid argument for the change to indexing of state pensions and benefits. The government’s justification was further stated in a press release from the Department for Work and Pensions announcing the start of the triple- lock for adjusting the state pension. It stated that “the CPI is less volatile providing more stability for pensioners and benefit claimants. It is the headline measure of inflation in the UK and the target measure of inflation used by the Bank of England”. The CPI was considered less volatile than the RPI through the inclusion of mortgage interest payments in the latter but not the former (Department for Work and Pensions 2010). The CPI was indeed less volatile than the RPI. If we take the period from 2000 to the end of 2009 it is clear that the variation in the RPI exceeded that of the CPI. However, this seems a very weak justification. If recipients of pensions and benefits were given the choice between continuing the RPI as the indexing mechanism or swapping it to the CPI and so reducing volatility, they would surely choose the former. Being the headline measure is again a weak argument; preferring the CPI as the target measure for macroeconomic purposes is of little relevance to its use

92

Chapter 5

for compensation purposes. Even a generous interpretation of this reasoning would find it seriously wanting and it is not at all surprising that this argument wasn’t convincing for many observers, including those directly affected such as pensioners groups. There were better reasons for switching to the CPI, but these wouldn’t be clear until 2013, as we will see. The widespread belief that the change of measure was purely a cost saving move was given weight by a series of announcements by the government over the following years showing a tendency to switch the indexing mechanism to the CPI for instances where money was paid out but to retain the RPI when money was taken in. In fact, changes to both followed although switching the measure for instances where the government took an income was slower to be enacted in order to “protect government revenue”. In the 2011 budget the government announced that indexation of direct tax thresholds would switch to the CPI and in 2012 a switch for capital gains tax allowances was announced. In the 2013 budget, the government said it would not make any further changes for the time being but keep the use of the RPI under review.

5.8

Deciding which measure to use

It is worth pausing in the story at this point to consider the roles and responsibilities of both the users and the producer of inflation statistics. One might think that the ONS, as producer of the RPI and CPI, would be in an ideal position to advise users on which measure to use for a particular purpose. However, this is not what happens in practice. The ONS, as producer, has the responsibility for producing official statistics that are fit for purpose, that is, to meet user needs as much as is practical given time and money constraints. For inflation measures, ONS has published a number of documents setting out the differences between the RPI and CPI and their respective strengths and weaknesses. However, it is for users to decide which measure best suits their needs; it follows that it is the responsibility of the government to choose the measure(s) best suited to meet its needs (Office for National Statistics 2011b, 12). The reasoning for the ONS position was set out in a UK Statistics Authority

Why is inflation measurement controversial?

93

assessment report from 2010. It noted that the ONS “would not wish to contradict policy adopted by other government departments or private companies in contracts or legal agreements since, by doing so, it could be considered liable for any loss of income resulting from its advice” (UK Statistics Authority 2010, p13). Two examples show how different applications of inflation adjustment led to different outcomes. The first example comes from the indexing arrangements for private sector pensions. Some private sector pensions have the use of the RPI written explicitly into their pension agreements. We will see later that attempts to change the index in several of these agreements failed in court. The second example is the indexing provision for public sector pensions. As noted in section 5.7, after the government changed the indexing of public sector pensions in 2011 from the RPI to the CPI, six trade unions took the government to court to challenge the decision. In a part of the relevant legislation, Section 150(1) of the Social Security Administration Act 1992, the Secretary of State is obliged to “review certain sums annually in order to determine whether they have retained their value in relation to the general level of prices obtaining in Great Britain estimated in such manner as the Secretary of State thinks fit". This clearly gives flexibility for the government to choose what they consider to be an appropriate measure. The court case also rested on whether it was appropriate for the government to choose an indexing measure based on the state of government finances, rather than purely on technical merits. The trade unions lost the court case and the subsequent appeal (High Court 2011).

5.9

The status of the RPI and its continued uses

Public disquiet over the presence of two measures for consumer price inflation persisted with many interested parties disagreeing that the CPI was the better measure to use. The existence of two competing measures of the same widely used, important economic quantity that differ by such a degree was an extraordinary situation. Concern was expressed by Professor David Hand, the president of the Royal Statistical Society, who wrote to the head of the UK Statistics Authority, Sir Michael Scholar, in 2010. In response, the Office for National Statistics agreed to publish more

94

Chapter 5

information on the history and differences between the two measures. This appeared in 2011; one document examined the differences between the indices and another set out the uses of the respective measures (Office for National Statistics 2011a, 2011b). The importance of the differences between the RPI and CPI and their uses justified a detailed investigation. The ONS carried out a programme of research on the issues, publishing the results in a number of papers. The increased size of the formula effect resulting from the change in price collection instructions for clothing was revisited to see if it could be reduced without affecting the overall rate of clothing inflation as measured by the CPI. A greater level of outlet stratification for clothing prices was also considered. Amended price collection instructions including tighter specifications were applied in a pilot exercise carried out in early 2012. Analysis of the results didn’t show a significant reduction in the formula effect; neither did the additional stratification of the clothing component. While the pilot exercise continued, it became clear that efforts to reduce the size of the formula effect were unlikely to be successful (Office for National Statistics 2012a). It has remained at about 0.9 percentage points ever since. In its research programme, the ONS explored the claim that the Jevons index better represented substitution behaviour in response to price change than other index formulas; they used consumer panel data for this work. Such data are found from capturing detailed purchasing information from many households over extended periods, where some households stay in the “panel” for many months and sometimes years; it is a rich source of consumer data. This showed that no elementary aggregate formula adequately represented consumer behaviour. The original claim that the Jevons better reflected substitution behaviour was based on very simple economic models which didn’t represent real consumer behaviour. Other work explored statistical properties of the different formulas to see if that provided evidence to identify which is best (in a defined statistical sense) but the results were inconclusive (Winton, O’Neill and Elliott 2012; Elliott, O’Neill, Ralph and Sanderson 2012).

Why is inflation measurement controversial?

95

To assist the ONS in their investigations, a leading expert in index numbers and price statistics, emeritus Professor Erwin Diewert of the University of British Columbia, was commissioned in early 2012 to assess the suitability of the RPI and CPI to meet user needs and to identify weaknesses in these indices. In his report, presented in the summer of 2012, he recommended removing the Carli formula from the RPI, replacing it with the Jevons; this would eliminate the upward bias in inflation as measured by the RPI. He also recommended removing the restriction on the lowest and highest paid from the RPI weight calculations. For clothing, he suggested either removing fashion items from the basket or treating them as strongly seasonal items (Diewert 2012). Advice was also sought from expert groups including the Government Statistical Service’s Methodology Advisory Committee. The results of the various strands of investigation related to the formula effect were brought together at the September 2012 meeting of the Consumer Prices Advisory Committee. It concluded that the use of the Carli formula was no longer appropriate. The committee’s advice was presented to the National Statistician, Jil Matheson. With the prospect of making a highly significant decision to change the RPI methodology, the National Statistician decided to hold a public consultation, which was in accordance with statistical best practice. After the consultation the public responses would be considered, and the National Statistician would decide whether or not to recommend to the UK Statistics Authority, that the RPI methodology should change; if so, it would have serious consequences. If the National Statistician decided to change the RPI methodology, the process was clear. The recommendation would go to the UK Statistics Authority, who would advise the Bank of England. They would assess the change to see whether it would result in material, detrimental consequences for gilt holders, which it certainly would, and the Chancellor of the Exchequer would then need to decide whether to proceed. If the Chancellor agreed, the change could go ahead as soon as March 2013 (Office for National Statistics 2012b). The consultation ran in October and November 2012 and the public were invited to comment on four options for improving the RPI, which included the option to not change the methodology and options for replacing the

96

Chapter 5

Carli in several ways. The ONS held public meetings around the UK to explain the position and take questions. Over four hundred written responses to the consultation were received with a substantial majority preferring no change to the RPI. The reasons given included the adverse implications for gilts and private sector pensions and the possibility of legal action over contracts with the RPI explicitly included. Others disagreed with the technical arguments (Office for National Statistics 2013). The National Statistician considered all the expert advice, the research findings and the responses from the consultation. She decided that there was significant value in maintaining the RPI in its current form, which would enable it to continue to meet user needs, including its long term use for index-linked gilts and pensions. However, she stated that the use of the Carli formula in the RPI did not meet international standards and a new index, the RPIJ, with the Carli replaced by the Jevons would be created. The conclusion that the RPI with the Carli formula didn’t meet international standards led to it being re-assessed by the UK Statistics Authority against the Code of Practice for Official Statistics (UK Statistics Authority 2013). This was an important step. Almost all of the most important official statistics go through this assessment process which is carried out by the regulatory arm of the UK Statistics Authority – the Office for Statistics Regulation. This process evaluates whether the statistics comply with the statement of best practice documented in the Code of Practice for Official Statistics. Those that do are declared “National Statistics”. It is an important indicator of statistical quality. The reassessment report noted that the use of Carli formula in the RPI was not consistent with internationally recognised good practice. This resulted in the RPI losing its National Statistics status. The official advice was for uses of the RPI to cease as soon as was practical.

5.10 Owner Occupiers’ Housing in the CPI While the formula effect work was prominent in the public eye, another important issue was being resolved. The omission of owner occupiers’ housing from the CPI was a recognised weakness and work had been in

Why is inflation measurement controversial?

97

progress across Europe for some years to evaluate a number of options to include it. The Consumer Prices Advisory Committee (CPAC) provided oversight of the work in the UK to evaluate the respective options. When the work completed, the committee was invited to assess the merits of the options and make a recommendation to the National Statistician. It concluded that the rental equivalence approach was the best option. This method uses the change in the cost of renting an equivalent property as a proxy measure; it is a well-established approach and we noted in chapter 4 that it was the original method used in the RPI. The recommendation was made to the National Statistician. As this was a significant matter, the National Statistician decided it should be subject to a consultation to invite the views of stakeholders. This took place in 2012 and generated twenty responses, a small number in contrast to the RPI consultation, though more in line with other consultations. Some responses supported the rental equivalence method and others preferred one of the alternatives, the net acquisitions method. The National Statistician accepted the recommendations of CPAC and the rental equivalence method was adopted. The CPI including owner occupiers’ housing using rental equivalence was named the CPIH and it was first produced in 2013. The intention was for the CPIH to become the ONS’ main measure in the future.

5.11 Reviews of consumer price indices The period from 2010 to 2015 had seen major changes to the landscape of consumer price statistics. The declaration that the RPI was not a good measure of inflation through its use of the Carli formula and its loss of National Statistics status was a dramatic fall from its 40 year position as the UK’s sole measure of consumer price inflation. A short, independent review of the decision on the Carli formula was carried out in 2014 by Peter Levell, a senior economist at the Institute for Fiscal Studies. He concluded that there was a sound case for the finding that the Carli formula was flawed and that the Jevons formula was to be preferred (Levell 2014).

98

Chapter 5

The change in the choice of indexing measure by the government, the loss of National Statistics status for the RPI and the start of the CPIH, represented a significant revision of consumer price indices and their uses. In 2013, Sir Andrew Dilnot, the chair of the UK Statistics Authority, asked Paul Johnson, the head of the Institute for Fiscal Studies, to carry out a review of consumer price indices to see whether they were fit for current and future needs. A small team of experts with a wide experience of official statistics was assembled to assist, together with supporting staff from the ONS. The review commissioned original research as part of its work, though this was necessarily limited given the timeframe of the review. The Johnson review reported in January 2015. It recommended that there should be one main measure of inflation which would be the CPIH once firmly established and the CPI in the meantime. It also agreed that the RPI was flawed through its use of the Carli formula and its use should be limited to long-term applications in index-linked gilts, private pensions and contracts. It recommended that the ONS develop a new inflation measure that included payments that households make, such as capital repayments, even if they are normally excluded from inflation measures. This is called a “payments” approach and was originally proposed by the Royal Statistical Society. It would assess the impacts of price changes on different household types. This set of measures would be used to inform debate. A further recommendation was to produce an annual experimental measure using a different type of index formula, a so-called “superlative” index (Johnson, 2015). These additional price indices are described in chapter 8. After the review, the National Statistician, John Pullinger, launched a consultation to gain input from the public on the recommendations of the review. At the end of 2016, the National Statistician announced that the CPIH would become the ONS preferred measure from March 2017 and the reduced role for the RPI would be reinforced by discontinuing

Why is inflation measurement controversial?

99

production of many of its variants40 (UK Statistics Authority 2015a; UK Statistics Authority 2015b; Pullinger 2016).

5.12 The long-term uses of the RPI Private pensions and index-linked gilts are two of the important long-term uses of the RPI. Soon after the 2010 announcements of the change of indexing measure for state pensions, public sector pensions, housing benefits and tax allowances, the government looked at whether these two long term uses might also be changed. A public consultation at the end of 2010 explored whether the government should introduce legislation to allow pension trustees to change the index measure for private pensions. This “legislative override” would remove the barrier to change where the RPI was explicitly included in pension agreements. The government reviewed the responses and replied to the consultation in June 2011 noting that most stakeholders were against a change; the government decided not to go ahead as they concluded that there was no fair way to proceed (Department for Work and Pensions 2011). To follow the consultation on indexing private sector pensions, another consultation was launched in June 2011 on whether the government should launch CPI indexed gilts. These new gilts could provide a means of managing risks for providers of CPI indexed financial products. In particular, they would be relevant if the government decided to proceed with a legislative override for private pension indexing at some point in the future. The consultation was run by the Debt Management Office. In considering the responses, the Office decided that introducing CPI indexed gilts presented too great a risk to financing government debt and would not be cost effective. There was also a consideration that waiting for the CPIH to become the government’s main measure before proceeding would be wise (Debt Management Office 2011). Although neither proposal went ahead, these moves showed that the government was considering the practicality of moving away from the RPI for its more entrenched applications. 40

We look at variants of the all-items consumer price indices in chapter 8.

100

Chapter 5

5.13 A mixed position for indexation Although long-term uses of the RPI would remain for the time being, the government continued to move away from the RPI in other areas. It announced that the indexation of capital gains exempt allowances would move from the RPI to the CPI from April 2013. Overall, it was a gradual shift as many uses remained and some new ones began. Student loan repayments and repayments under “right to buy” schemes were indexed by the RPI41. Commentators who believed that there was a deliberate government position to switch to the CPI when money was paid out and keep the RPI for money coming in still had much evidence to support their view. The budget of 2013 noted the National Statistician’s decisions on the RPI and stated that the government would keep the use of the RPI “under review” and the issuance of RPI-indexed gilts would continue. No further changes were announced in 2014 or 2015. Announcements of indexation changes began again in 2016 with the indexation of business rates switching to the CPI, but with implementation deferred to 2020; this changeover date was revised to 2018 in the budget document for 2017 (HM Treasury 2017). The position of the RPI was discussed more extensively in the 2018 budget report. It noted the continuing use of the RPI in both private and public sectors. It reiterated that the move away from the RPI for government purposes would necessarily be slow; this was to ensure the reduction in revenue to the Exchequer was gradual. The CPIH was acknowledged as the best measure of consumer price inflation and it would become the government’s main measure once it was established. The budget report also included an important announcement; the government stated it would not introduce any new uses of the RPI (HM Treasury 2018).

41

Some student loan rates were RPI+3%. This high rate was designed to ensure higher payments from higher earners as a redistributive mechanism, a reason not well-publicised by the government.

Why is inflation measurement controversial?

101

There were also moves taking in place in the private sector where the RPI was replaced by the CPI in price controls for regulatory organisations. In an interesting development in the gilt market, the University of Cambridge sold £300 million of CPI index-linked bonds along with a similar value of fixed rate bonds (University of Cambridge 2018).

5.14 Legal actions and an uncertain future With the RPI officially declared a poor measure of consumer price inflation, several employers sought to change the indexation measure in their final salary pension schemes. Such schemes were expensive and in some cases were in deficit. Not surprisingly, the desire to change went to court with employers losing the court cases with judgements stating that the official position of the RPI did not justify a change. Verdicts were upheld in the court of appeal. The decisions were dependent on the wording of individual pension agreements which meant each scheme had to be tested in court separately (Ralph, O’Neill and Smith 2020, 115-6). The use of the RPI in many private sector pensions would have to remain and while they were almost all closed to new members they would be active for many years. The position of the UK Statistics Authority was a difficult one. The RPI was the only official statistic they were legally required to produce under the Statistics and Registration Service Act 2007, yet the RPI had been discredited as a measure of consumer price inflation. While the use of the RPI had been discouraged for some years, its use wasn’t declining as quickly as was hoped.

5.15 Towards a resolution Two events in 2018 started a gradual move towards resolving the situation. The first was a meeting at the Royal Statistical Society and the second an inquiry by a parliamentary committee, the House of Lords Economic Affairs Committee. Both the Royal Statistical Society and various parliamentary committees from both houses keep a watching brief over the integrity and effectiveness of official statistics.

102

Chapter 5

The unsatisfactory position with inflation measures prompted the Royal Statistical Society to hold a public meeting in June 2018. At the meeting, the National Statistician summarised the position with inflation measures and identified two restrictions on the UK Statistics Authority. Firstly, it could not formally advise users on which measure to use for which purpose; this was for users to decide as we noted in section 5.8. Secondly, the methodology of the RPI could not be changed as there would be considerable financial consequences which the Treasury would not accept (Royal Statistical Society 2018). Several speakers at the event described the current position as a mess and that more should be done to resolve the issues. A pressing need for clarity was illustrated by pension providers; they were anxious to understand the long term position as they had to plan for many years ahead. They were seeking a clear, long-term stable position with fit for purpose indexing measures. The trigger for the House of Lords inquiry was a regular session of the Economic Affairs Committee with the Governor of the Bank of England. The Governor suggested the committee use its influence to reduce the use of the RPI. The committee had already scheduled a short look at the RPI but decided to carry out a full inquiry. It started in July 2018 and reported in January 2019. Many stakeholders were invited to appear before the committee in public hearings and written evidence from interested parties was sought (Ralph, O’Neill and Smith 2020, 119-121). The committee heard evidence from a range of stakeholders including the Chief Secretary to the Treasury, Liz Truss, who explained that while a transition away from the RPI was needed, its use in pension agreements, gilts and contracts meant it would be a long term goal. The head of the Debt Management Agency, Sir Robert Stheeman, explained his reluctance to see a mixed market of RPI and CPI indexed bonds. He acknowledged that while such a mixed market might be possible at some point, the current position with only RPI indexed bonds would continue for the foreseeable future.

Why is inflation measurement controversial?

103

The committee heard representations that despite the official view that the RPI was not a good measure of inflation, its entrenched position in longterm uses meant little could be done in the short term. However, continuing the situation with users being able to choose between the CPI and RPI as alternative indexing measures presented its own problems. Many stakeholders believed the choice was a case of “inflation-shopping” where a decision was made not on methodological grounds but on which produced a higher or lower value of inflation. This approach was not only undermining the integrity of inflation statistics but also running the risk of affecting the reputation of all official statistics. The House of Lords committee report was highly critical of the situation and of the UK Statistics Authority’s failure to act sooner. In particular, it said that it was not the responsibility of the Authority to decide that the RPI methodology could not change because of the financial implications. It was their role to produce high quality, fit for purpose statistics. They should have acted sooner to produce an improved RPI that met international standards and then asked the Chancellor for permission to replace the old version. It was the Chancellor’s role to decide whether to proceed (UK Parliament 2019). Following the committee’s report, the UK Statistics Authority responded quickly. In March 2019, the head of the UK Statistics Authority, Sir David Norgrove, wrote to the Governor of the Bank of England and the Chancellor of the Exchequer suggesting two options: cease producing the RPI or amend its methodology. The Chancellor declined both in the short term, but proposed that change could go ahead in 2030 using the second option, with the possibility of bringing forward the date to between 2025 and 2030 (UK Statistics Authority 2019-20). The government and UK Statistics Authority carried out a joint consultation in 2020 seeking views on the proposal to change the RPI to be aligned with the CPIH and a possible earlier date for changing the RPI. The consultation document contained details of how the methodology of the RPI would be changed; a response to the consultation followed. Following consideration of the responses and further communications between the UK Statistics Authority, the Bank of England and the

104

Chapter 5

Chancellor, the Chancellor decided that the change in methodology would go ahead, but not before 2030, the date of maturity of the final specific index-linked gilt42. This would minimise the impact on gilt holders (UK Government 2022). An end may be in sight for the uncomfortable position of inflation measurement, though there is plenty of time before 2030 for debate to continue. If the proposed change in the methodology of the RPI goes head in 2030, holders of private pensions with the RPI explicitly included will see a reduced rate of growth in their pensions. It would not be surprising if there were further court cases.

5.16 Concluding remarks This chapter has set out the events from the introduction of the HICP in 1997 as the UK’s second measure of consumer price inflation, alongside the RPI, through to the RPI being declared a poor measure of consumer price inflation in 2013 and the intention for the RPI to be changed to have its methods brought into line with the CPIH in 2030. Looking back at the sequence of events and re-reading many of the documents produced by the ONS, the UK Statistics Authority and international statistical organisations, together with research papers, UK budget documents, consultations and responses is highly informative. A few episodes stand out. The arrival of the HICP, with its up to date design developed from scratch, and its use of the Jevons formula at the elementary aggregate level, provided an alternative measure of consumer price inflation across EU countries starting in January 1997. At this time, the UK was alone among EU countries in having the Carli index present in its domestic measure, the RPI; all other countries had moved away from its use. This meant that the magnitude of the difference between the HICP and the RPI was much larger in the UK than elsewhere. It was unfortunate that the change to

42 Index-linked gilts are the subject of section 21 of the Statistics and Registration Service Act 2007.

Why is inflation measurement controversial?

105

price collection instructions in order to address a problem in the UK’s HICP led to a widening of the gap in 2010. Could the Carli have been removed from the RPI at an earlier date? Potential issues with the Carli formula were noticed in the UK in the 1970s. Statistics Canada was concerned in the 1980s and published research to that effect and removed the Carli from their consumer price index in 1987. The ONS examined the use of the Carli formula several times and recommendations were made to remove the Carli, but instead its use was only reduced. Why wasn’t the Carli formula removed in the UK many years ago? While we have found no clear documentation to explain the reasons, we can speculate. The benefits from using the index that could have replaced it, the Jevons, were overplayed in the 1990s. It was promoted as better representing substitution behaviour by the influential Boskin Commission in the US, which also encouraged its use as part of a move towards a cost of living approach. The better substitution behaviour argument was only valid under unrealistic assumptions. Both the RPI and CPI are cost of goods indices in which quantities are fixed during each year, so that reflecting in-year substitution behaviour is not appropriate. The ONS was right to reject the Jevons formula on those grounds. Later research suggested that the alleged better representation of substitution behaviour, which was based on simple economic models, doesn’t hold for real consumer data. The documentation from 1970-2000 suggests that ONS research work had concluded that the Carli should be removed, but there was a reluctance to use the Jevons. The second possible reason was the implications for index-linked gilts. We cannot know what response the government would have made if the ONS had proposed changing the formula in the 1990s or had the Statistics Commission done the same in the early 2000s. We can imagine the Chancellor of the Exchequer at the time might well have said no and deferred a decision to a later date. The change in inflation target in 2003 to the CPI from RPIX had a reasonable justification that the CPI was designed as a macroeconomic measure and the accompanying ONS document stated that the RPI would

106

Chapter 5

continue to be used for compensation purposes. This was entirely sensible, but the position changed seven years later. The switch of indexing from the RPI to the CPI announced in 2010 was a way of reducing public expenditure on a long-term basis. There was no carefully crafted advance document setting out the statistical reasoning, no public consultation and there was no advisory committee to discuss the change – it was just announced. The ONS was asked to examine the issues but only after the decision had been made. There is no doubt that the motivation for the change was to save money; the transcript of the 2011 court case makes this very clear. The budget document of 2010 and accompanying documents did attempt to justify the change but this was very much a secondary consideration. After 2010, the government made a series of changes of indexing measure in cases where they were paying out money. The intention was to replace the RPI with the CPI for cases where income was received but in a carefully managed way to preserve government finances. Only the longterm uses in indexed linked gilts, private pensions and contracts would need special treatment. The government did examine a legislative override for private pensions and issuing CPI linked gilts but neither went ahead. The evidence suggests the government was serious about replacing the RPI for all official uses. It is fascinating to wonder what would have happened if the HICP hadn’t been created. It is reasonable to think that there would have been internal and external pressure from influential stakeholders on the ONS to bring the RPI methodology up to date and more closely align it with the CPI (and CPIH). This may have entailed changing the Carli index. The ONS research work and consultation on improving of the RPI examined the issues in considerable depth. Expert opinion was sought, advisory groups and committees consulted. The decision that the RPI was not a good measure was examined independently by Peter Levell of the Institute for Fiscal Studies (IFS) and commented on by Paul Johnson, the head of the IFS, in his review. Both agreed the RPI was not a good measure. The weight of expert opinion was that the official decision on the

Why is inflation measurement controversial?

107

RPI was the correct one. There were dissenting views which questioned the conclusions of the ONS research, though none of these opinions were based on new detailed work. Once the verdict on the RPI had been made, the hope was that the use of RPI would decline quickly. However, progress was slow. The House of Lords inquiry in 2018/19 criticised the UK Statistics Authority for not pushing ahead earlier with recommending removing or amending the RPI to the Bank and the Chancellor. However, if that had happened five years sooner the situation may have been the same as now, with the change being deferred until 2030. The change of indexing for government bonds and private pensions will have to wait until the methodology of the RPI is changed. However, there are many other uses for which it would be easy to change to the CPI and a stronger message could come from the UK Statistics Authority and the government. Unfortunately, there are still uses where an indexing measure is being chosen not on methodological grounds but simply on the value of inflation derived from it. The RPI is still being used by employment groups when stating the extent of real terms pay reductions since 2010 and calling for improved pay; this counts as a misuse. It is hard to see this changing in the foreseeable future.

References Astin, John. 2021. Measuring EU inflation – the foundations of the HICP. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/9783-030-68806-6 Baxter, Michael and Camus, Dawn. 1999. “Three Year Research Programme on RPI Methodology”. Economic Trends, 543, 25-29. Accessed January 5, 2023. https://escoe-website.s3.amazonaws.com/wp-content/uploads/2020/01/ 01234444/ET-543-Three-Year-Research-Programme-on-RPIMethodology-Michael-Baxter-Dawn-Camus-Feb-1999.pdf Carruthers, A., Sellwood, D. and Ward, P.W. 1980. “Recent Developments in the Retail Prices Index”. The Statistician, 29, 1-32. https://doi.org/10.2307/2987492

108

Chapter 5

Debt Management Office. 2011. “CPI-linked gilts: Response to Consultation”. Accessed January 5, 2023. https://www.dmo.gov.uk/media/14572/cons20111129.pdf Department of Employment. 1986. “Retail Prices Index Advisory Committee: methodological issues affecting the Retail Prices Index, Cmd. 9848”. London: HMSO. Department for Work and Pensions. 2010. “More help for pensioners as Basic State Pension set to rise in 2011”. Accessed January 5, 2023. https://www.gov.uk/government/news/more-help-for-pensioners-asbasic-state-pension-set-to-rise-in-2011 Department for Work and Pensions. 2011. “Government response to consultation: The impact of using CPI as the measure of price increases on private sector occupational pensions schemes. Consultation of government proposals”. Accessed January 5, 2023. https://assets.publishing.service.gov.uk/government/uploads/system/up loads/attachment_data/file/185036/cpi-private-pensions-consultationresponse.pdf Diewert, W. Erwin. 2012. “Consumer price statistics in the UK”. Accessed January 5, 2023. https://unece.org/fileadmin/DAM/stats/documents/ece/ces/ge.22/2014/ WS1/WS1_1_Diewert_on_Diewert_Consumer_Price_Statistics__in_th e_UK_v.7__06.08__Final.pdf Elliott, Duncan., O’Neill, Rob, Ralph, Jeff and Sanderson, Ria. 2012. “Stochastic and sampling approaches to the choice of elementary aggregate formula”. Accessed January 5, 2023 https://webarchive.nationalarchives.gov.uk/ukgwa/20121204150708/ht tp://www.ons.gov.uk/ons/guide-method/user-guidance/prices/cpi-andrpi/discussion-paper--results-of-ons-research-into-the-application-ofthe-stochastic-and-sampling-approaches-to-the-choice-of-elementaryaggregate-formula.pdf Evans, Bethan. 2012. “International comparison of the formula effect between the CPI and RPI” CPAC(12)07. Consumer Prices Advisory Committee. Accessed January 5, 2023 https://webarchive.nationalarchives.gov.uk/ukgwa/20150123172945/http: //www.ons.gov.uk/ons/guide-method/development-programmes/otherdevelopment-work/prices-development-plan/consumer-prices-

Why is inflation measurement controversial?

109

development-work.html?translation-component=&translationcomponent=&calling-id=77-6743-4&calling-id=77-67434&currLang=English&currLang=English&format=contrast High Court. 2011. “FDA, Prospect and Others vs the Secretary of State for Work and Pensions, HM Treasury and Others. EWHC 3175 (Admin)”. Accessed January 5, 2023. https://www.bailii.org/ew/cases/EWHC/Admin/2011/3175.html HM Treasury. 2003. “Pre-budget report: Seizing the opportunities of the global recovery: The strength to take the long-term decisions for Britain, December2003, Cm 6042”. London: HMSO. HM Treasury. 2010. “Budget 2010”. London: HMSO. Accessed January 5, 2023. https://www.gov.uk/government/uploads/system/uploads/attachment_d ata/file/248096/0061.pdf HM Treasury. 2017. “Budget 2017”. London: HMSO. Accessed January 5, 2023. https://www.gov.uk/government/publications/autumn-budget2017-documents HM Treasury. 2018. “Budget 2018”. London: HMSO. Accessed January 5, 2023. https://www.gov.uk/government/publications/budget-2018-documents Johnson, Paul. 2015. “UK consumer price statistics: a review”. UK Statistics Authority. Accessed January 5, 2023. https://uksa.statisticsauthority.gov.uk/reports-andcorrespondence/reviews/uk-consumer-price-statistics-a-review/ Levell, Peter. 2014. “Is the Carli Index flawed? Assessing the case for the RPIJ”. Journal of the Royal Statistical Society, Series A, 178(2), 303336. http://www.jstor.org/stable/43965476 Ministry of Labour and National Service. 1959. “Method of construction and calculation of the index of retail prices”. 2nd Edition. London, HMSO. National Archives. 1977. “Technical proposals on the combining of price quotations in the RPI”. Department of Employment. File T374/256. O’Donoghue, Jim and Wilkie, Colin. 1998. “Harmonised indices of consumer prices”. Economic Trends, 532, 34–43. Accessed January 5, 2023. https://escoe-website.s3.amazonaws.com/wp-content/uploads/2020/ 01/01234550/ET-532-Economic-Trends-March-1998.pdf

110

Chapter 5

O’Neill, Rob., Ralph, Jeff and Smith, Paul A. 2017. Inflation: history and measurement. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-319-64125-6 Office for National Statistics. 2003. “The new inflation target: the statistical perspective”. Accessed January 5, 2023. https://webarchive.nationalarchives.gov.uk/ukgwa/20160108054407/ht tp://www.ons.gov.uk/ons/guide-method/methodquality/specific/economy/cpi-rpi/1998---2004/index.html Office for National Statistics. 2011a. “History of and differences between the Consumer Prices Index and Retail Prices Index”. Accessed January 5, 2023. https://webarchive.nationalarchives.gov.uk/ukgwa/20160108003710/ht tp://www.ons.gov.uk/ons/rel/cpi/consumer-price-indices/history-ofand-differences-between-the-consumer-prices-index-and-retail-pricesindex/index.html Office for National Statistics. 2011b. “Implications of the differences between the Consumer Prices Index and Retail Prices Index”. Accessed January 5, 2023. https://webarchive.nationalarchives.gov.uk/ukgwa/20110911123624/ht tp://www.ons.gov.uk/ons/taxonomy/index.html?nscl=Consumer+Price +Indices Office for National Statistics. 2012a. “The formula effect gap between the CPI and the RPI”. CPAC(12)24. Consumer Prices Advisory Committee. Accessed January 5, 2023. https://webarchive.nationalarchives.gov.uk/ukgwa/20150123172945/http:// www.ons.gov.uk/ons/guide-method/development-programmes/otherdevelopment-work/prices-development-plan/consumer-pricesdevelopment-work.html?translation-component=&translationcomponent=&calling-id=77-6743-4&calling-id=77-67434&currLang=English&currLang=English&format=contrast Office for National Statistics. 2012b. “National Statistician’s consultation on options for improving the Retail prices Index”. Accessed January 5, 2023. https://webarchive.nationalarchives.gov.uk/ukgwa/20150123205449/ht tp://www.ons.gov.uk/ons/about-ons/get-involved/consultations/ archived-consultations/2012/national-statistician-s-consultation-on-

Why is inflation measurement controversial?

111

options-for-improving-the-retail-prices-index/index.html?translationcomponent=&calling-id=77-6854-4&currLang=English&format=hi-vis Office for National Statistics. 2013. “Response to National Statistician’s consultation on options for improving the Retail Prices Index”. Accessed January 5, 2023. https://webarchive.nationalarchives.gov.uk/ukgwa/20150123205449/http:// www.ons.gov.uk/ons/about-ons/get-involved/consultations/archivedconsultations/2012/national-statistician-s-consultation-on-options-forimproving-the-retail-prices-index/index.html?translation-component =&calling-id=77-6854-4&currLang=English&format=hi-vis Pullinger, John. 2016. “Statement on future of consumer price inflation statistics in the UK”. Office for National Statistics. Accessed January 5, 2023. https://www.ons.gov.uk/news/statementsandletters/statementonfutureo fconsumerpriceinflationstatisticsintheuk Ralph, Jeff., O’Neill, Rob and Smith, Paul A. 2020. The Retail Prices Index – A Short History. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-030-46563-6 Royal Statistical Society. 2018. “The Future of the Retail Prices Index”. Accessed January 5, 2023. https://rss.org.uk/news-publication/newspublications/2020/general-news/rss-open-meeting-on-rpi%E2%80%93-event-report/ Schultz, Bohdan. 1994. “Choice of price index formulae at the microaggregation level: the Canadian empirical evidence”. Accessed January 5, 2023. https://www.ottawagroup.org/Ottawa/ottawagroup.nsf/home/Meeting+ 1/$file/1994%201st%20Meeting%20-%20Schultz%20Bohdan%20-% 20Choice%20of%20Price%20Index%20Formulae%20at%20the%20M icro-Aggregation%20Level%20The%20Canadian%20Empirical %20Evidence%202nd%20Edition.pdf Statistics Commission. 2004. “Changes in the Calculation of the RPI and RPI Governance, Report 20”. Accessed January 5, 2023. https://uksa.statisticsauthority.gov.uk/publication/report-20-changesin-the-calculation-of-the-rpi-and-rpi-governance-september-2004/ UK Government. 2022. “Consultation on the Reform to the Retail Prices Index Methodology”. Accessed January 5, 2023.

112

Chapter 5

https://www.gov.uk/government/consultations/a-consultation-on-thereform-to-retail-prices-index-rpi-methodology UK Parliament. 2017. “The Economics of Higher, Further and Technical Education”. House of Lords, Select Committee on Economic Affairs. Oral Evidence session with Rt. Hon the Lord Willets, Rt. Hon the Lord Adonis and Paul Johnson. 10th October 2017. Accessed January 5, 2023. https://data.parliament.uk/writtenevidence/committeeevidence.svc/evid encedocument/economic-affairs-committee/the-economics-of-higherfurther-and-technical-education/oral/71229.html UK Parliament. 2019. “Measuring Inflation”. House of Lords Economic Affairs Committee, 5th Report of Session 2017-2019. Accessed January 5, 2023. https://publications.parliament.uk/pa/ld201719/ldselect/ldeconaf/246/2 46.pdf UK Parliament. 2021. “State Pension Triple Lock”. Briefing Paper, Number CBP-07812. Accessed January 5, 2023. https://researchbriefings.files.parliament.uk/documents/CBP7812/CBP-7812.pdf UK Statistics Authority. 2010. “Consumer Price Indices—Assessment Report 79”. Accessed January 5, 2023. https://osr.statisticsauthority.gov.uk/wp-content/uploads/2015/12/ images-assessment-report-79-consumer-price-indices_tcm9735275.pdf UK Statistics Authority. 2013. “The Retail Prices Index – Assessment Report 246”. Accessed January 5, 2023. https://osr.statisticsauthority.gov.uk/wp-content/uploads/2020/07/ images-assessmentreport246theretailpricesinde_tcm97-42695.pdf UK Statistics Authority. 2015a. “Measuring consumer prices: The options for change”. Accessed January 5, 2023. https://uksa.statisticsauthority.gov.uk/wp-content/uploads/2015/12/ images-consumerpricesconsultationdocumen_tcm97-44662-2.pdf UK Statistics Authority. 2015b. “Summary of responses—Measuring consumer prices: The options for change”. Accessed January, 5 2023.

Why is inflation measurement controversial?

113

https://www.statisticsauthority.gov.uk/wp-content/uploads/2015/12/ images-summaryofresponsesconsumerpricesconsultatio_tcm9745027.pdf UK Statistics Authority. 2019-20. “RPI Statements and Correspondence”. Accessed January, 5 2023. https://uksa.statisticsauthority.gov.uk/rpistatements-and-correspondence/ University of Cambridge. 2018. “University of Cambridge raises £600 million in pioneering bonds issue”. Accessed January 5, 2023. https://www.cam.ac.uk/news/university-of-cambridge-raises-ps600million-in-pioneering-bonds-issue What do they know? 2012. “Consideration by the ONS of elementary aggregate formulae in the Retail Prices Index: a short history”. Accessed January 5, 2023. https://www.whatdotheyknow.com/request/correspondence_with_govt _formula#incoming-352140 Winton, Joe., O’Neill, Rob and Elliott, Duncan. 2013. “Elementary aggregate indices and lower level substitution bias”. Statistical Journal of the IAOS. 29, 11-19. Accessed January 5, 2023. https://content.iospress.com/journals/statistical-journal-of-the-iaos/29/1

6 WHAT CAN THE CHANGING BASKET TELL US? We saw in chapter 3 how the specification of the representative items to price is an essential part of the way inflation is calculated. It is not practical to collect prices for every item in the consumer marketplace; instead, we choose a sample that represents the full range of goods and services. The items chosen can be conveniently thought of as constituting a large shopping basket; the basket in 2022 contained about 730 items. Each month, prices are found for this collection of items from a variety of locations and outlets, and it is the change in prices from a base, or reference period, when combined with the proportion of expenditure on each representative item (and the many other items it represents) that are used to calculate a consumer price index. While the calculation contains many complex elements, the basket is one aspect that is more readily related to our actual shopping experiences. Changes to the basket are announced each year and the changes are always picked up by the media and reported widely. The items in the marketplace change over time as new products appear and older ones disappear; consumer tastes change as well. Adding a few items into the basket and removing others is a way of reflecting marketplace and consumer preference trends. Although an item appearing or disappearing from the basket for these reasons will occur in one particular year, this will often be the result of underlying changes that have occurred over many years. Consumer tastes change slowly and new products, if successful, usually take a number of years to gain market share. However, as we will see, there are other reasons why changes are made to the basket. Of course, most products persist in the retail marketplace for many years, so it is a relatively small number of changes that are required each year. The update to the basket announced in March 2022 saw 19 items added and 15 removed. The additions included meat-free sausages, sports bras and antibacterial surface wipes. Items removed included doughnuts, men’s suits and coal (ONS, 2022a).

What can the changing basket tell us?

115

In this chapter we explore what we can learn from the data collected as part of the calculation of a consumer price index in the UK. With data extending over a long period of time we can examine what has changed and what it tells us. The data include the contents of the basket, the changes in expenditure shares for different types of goods and services, and the price quotes.

6.1

The size of the basket over time

The first consumer price index was the working class Cost of Living Index, produced at the start of the First World War, as we described in chapter 4. It was used to inform wage rises to compensate essential workers for the sharp rises in prices that accompanied the start of the conflict. The categories of goods included in the basket were food, clothing, rent and fuel, with 23 items in total. The modern era of inflation measurement began after the Second World War, when the Retail Prices Index started in 1956. The basket contained 200 items at this point; by 1993 there were 600 items and today the basket contains over 700 items. The increases were partly driven by the growing complexity of the retail environment and partly to improve the degree to which the basket represented the whole range of goods and services available. Any growth in the size of the basket is slow; over the five years from 2018-2022, the (net) size of the basket increased by just twenty items.

6.2

What goes in the basket?

The principal reason for items to go into the basket is to represent the changing range and popularity of consumer goods and services. For example, in 2022, there were six items in the household textiles category of goods, including curtains, duvets and bed sheets. Within each of these types of textile item there are many variations in individual products, and prices are collected from a selection of these products. The role of the six items is to represent all household textile items. That is, it can reasonably be assumed that the price movements of these six items provide a good approximation to the price movements of all household textile items in the marketplace.

116

Chapter 6

There is no sampling frame from which products could be chosen in a systematic manner; instead judgement is used, supported by a number of guiding principles. The level of household expenditure on a commodity grouping is one such factor. An expenditure of £400-500m or greater will mean that a grouping will be explicitly represented, unless other items are judged to represent this group43. Similarly, where annual expenditure falls below £100m an item is likely to be removed, unless other considerations apply (Office for National Statistics 2020a). Where items are chosen to represent a category of products, a number of specific choices of item can be made without affecting the resulting inflation measure. Practicality is a consideration in the choice. The product needs to be easy to find in retail outlets and to price; also, the ideal situation is for the identical product to be present throughout the calendar year. As we described in chapters 2 and 3, the aim is to price as near identical products at a constant quality as possible on each collection occasion. The variability of prices within a category is also a consideration. Where there is higher variability a greater number of items is desirable. For example, the alcohol and tobacco division is observed to have a small degree of variability while the clothing and footwear category has a higher level. The former has 4% of the representative items allocated to it and the latter has 12% in the 2022 edition of the basket (Office for National Statistics 2022a). The annual process of reviewing and updating the basket involves analysis of data from a variety of sources and includes the degree of expenditure on a type of item and the degree of variability. However, it also requires a level of professional judgement.

43

These levels of expenditure are for the whole of the UK. The European HICP regulations specify an item should be included if expenditure on it exceeds 1 part per thousand of expenditure covered by the CPI; this was about £1 billion in 2020.

What can the changing basket tell us?

6.3

117

Interpreting the changes in the basket

The update of the basket of goods and services is announced in March each year. The items going into the basket and being removed are highlighted. The popular interpretation is that these updates are indicative of changes in consumer tastes and what’s available in the marketplace. Is this true? The basket of goods and services serves a specific purpose – it is to act as a representative sample of the range of consumer products. As we saw in the previous section, there are several reasons why an item might be put into the basket, or taken out, to better represent the whole consumer marketplace. A commodity going into (out of) the basket doesn’t necessarily mean a significant increase (decrease) in consumer spending on it. Does this mean that the popular interpretation of basket changes is incorrect? The annual statistical publication that describes the changes to the basket provides the reasons for the changes, so it is possible to identify and separate out those items added or removed for reasons other than significant changes in consumer expenditure. In the 2022 update of the basket, meat free sausages were added; the ONS note on this addition states that: “the growth in vegetarianism and veganism is widely covered by the press and is being driven by the younger generations as a result of growing social responsibility and health awareness. Additionally, the item increases our coverage of free-from food products and is widely available in supermarkets”. Climbing wall sessions were also added, this item falls in the “recreational and sporting services” class. The ONS notes that: “Recreational and sporting services is an under-represented part of the basket and a climbing session has been introduced to help improve the overall estimate of price movement. The particular service has been chosen to further diversify the range of items in this class and reflects increased popularity in the sport”. A decreasing expenditure on doughnuts has seen them removed from the basket (Office for National Statistics 2022b). The addition of meat-free sausages and the removal of doughnuts can be taken as indicating changes in consumer preferences while the addition of

118

Chapter 6

climbing wall sessions is not. If we are careful to check the reasons for commodities being added to or removed from the basket, it is possible to use the basket updates as indicators of changes in consumer preferences and the marketplace.

6.4

The basket over the long term

With the need for caution in mind, we can use the changing contents of the basket as an informal indicator of social and economic change through consumer preferences and the appearance and the development of new products. The data can tell us what has changed but we need a wider knowledge of the factors behind social and economic change to understand why those changes occurred. If we look over the period from 1947 onwards, what changes do we see and what factors drove those changes? Starting with food, we can see the impact of two factors – health concerns and the rise of convenience foods. White bread alone was in the 1947 basket; wholemeal bread entered in 1987; today we also have garlic bread and crumpets. Butter has always been in the basket with lard appearing in 1962 but disappearing in 1987. Today, spreadable butter, margarine and low-fat spreads are also included. Dried mashed potato appeared in 1974 and disappeared in 198744. Ready cooked mashed potato is still available, but now in chill cabinets so it can be heated in a microwave quickly. 1987 also saw frozen chips appear in the basket, an item we expect to be present for a long time45. Turning to electrical appliances, we can see changes in the kitchen with the emergence of kitchen gadgets making food preparation and cooking easier. Other devices appeared in order to ease the labour needed for 44

Older readers in the UK might recall a famous TV advert from the 1970s where aliens have visited earth and are highly amused to find that earth beings boil potatoes and mash them. The aliens know better; adding hot water to dried mashed potato is so much quicker and easier. 45For the benefit of younger readers, what did we do before frozen chips? One option was to buy them from a fish and chip shop. Otherwise, potatoes were cut into chips and they were fried either in a shallow pan, or a dedicated deep fat fryer; care was needed as chip fat fires were not uncommon.

What can the changing basket tell us?

119

cleaning. Domestic vacuum cleaners were available to buy after the First World War, but their high cost meant they weren’t widely used; also consumers preferred to buy a radio or new furniture (Trentman 2016, 247). It wasn’t until after the Second World War that the vacuum cleaner became widespread in the UK (Science Museum 2020). The vacuum cleaner was present in the 1947 basket along with the electric iron. Washing machines were included in the basket in 1952; refrigerators and cookers in 1962, toasters and microwaves in 1987 then tumble dryers and dishwashers in 1995. Audio visual equipment changes show advances in technology for sound and video, though not everyone would agree that sound quality has improved. The 1947 basket included the radio set and gramophone record. The introduction of personal, portable music in 1979 was a massive advance for teenagers at the time46. The personal cassette player went into the basket in 1987 and was replaced by the personal CD player in 1998. The personal CD player lasted until 2006 when MP3 players replaced it. 1987 saw colour TVs and VHS recorders added with digital set-top boxes included in 2008. Other changes illustrate marketplace developments. Gin had disappeared from the basket in 1996 as sales declined; it reappeared in 2017 following the rise of new varieties of flavoured gins first developed by small craft distilleries. The growing popularity of cycling saw the introduction of the cycle helmet, also in 2017. The rise in spending on “free-from” products was the reason for gluten-free cereal being included in 2020. This is just a small selection of the changes that have occurred in the period since the end of the Second World War. Details of the changes year by year can be found on the Office for National Statistics website from 2017 onwards, with descriptions for earlier years back to 1956, available from the archived ONS website hosted by the National Archives (Office for National Statistics ONS 2022c, 2022d). Several articles are dedicated

46

One of the authors went through the following sequence: compact cassette player, portable CD player, minidisc player, hard disk player, mobile phone with a Bluetooth link to a portable speaker using streaming services. What will be next?

Chapter 6

120

to describing changes over time; an entertaining approach is provided in an ONS Visual History feature (Office for National Statistics 2022e).

6.5

Expenditure shares

Another aspect of the data used in the calculation of inflation is useful as an indicator of social change. The proportion of household expenditure on categories of goods and services tells us how households have allocated their income. There is a long history of exploration of the allocation of household income to different expenditure categories, how it has changes over time and the degree to which households were judged to be in poverty. Household budget investigations can be divided into five phases. In the first phase in the 17th century, hypothetical budgets were created for typical workers. Towards the end of the 18th century, the second phase saw the start of data collection directly from households. The third phase in the latter decades of the 19th century saw large scale, privately funded inquiries carried out in York and areas of London. The state began to collect household budget data at the end of the 19th century and the start of the 20th century in the fourth phase. The final phase was the start of modern household budget surveys in the 1950s, with annual surveys starting in 1957 (Deeming, 2010). In our short history of inflation measurement in chapter 4, we described the household budget surveys that were conducted by the Board of Trade, particularly the important one from 1904 with some updates in 1914. These were part of the fourth phase and were needed for the first consumer price index, the working class, Cost of Living Index (they provided what was known as the “basis” for the index). We noted that, remarkably, another household budget survey didn’t take place until 1937/38, with the results not used until 1947 in the interim Index of Retail Prices. A further expenditure survey followed in 1953/54 and they were soon to become annual. These surveys are a useful source of household expenditure data for historians.

What can the changing basket tell us?

121

Spending on food is an important indicator. When the proportion of household expenditure on food is high, increases in the price of essential items can have severe impacts on households. What historic data do we have? Early records are hard to find, but one household kept weekly spending records over a seven year period in the 15th century. Analysis of these records shows that about 90% of income was spent on food (PhelpsBrown and Hopkins 1956, 297). It we jump forward to the 20th century, the 1904/14 expenditure survey (restricted to working class households) indicated that about 60% of household income was spent on food. At the time of the next survey in 1937/38, this time for the whole population, the estimated percentage spent on food had fallen to 39%. From 1957, a household expenditure survey was carried out every year. This shows that the proportion spent on food was estimated to be 25% in 1970, 16% in 1990 and 11% in 2004, a consistent fall in the share of spending. It is only from 1957 that the data are comparable, but we can get a broad picture from earlier data. Other trends are apparent from the data. The percentage of household spending on clothing and footwear has also declined over time as has the spending on tobacco products. In contrast, spending on housing and travel has risen (O’Donoghue, McDonnell and Placek 2006). Data on how households spend their money has many official uses beyond the highly important matter of providing the expenditure weights for inflation measures. The Living Costs and Food (LCF) Survey is the official household income and expenditure survey, run by the Office for National Statistics. It is a continuous survey which provides high quality data which are used for a variety of purposes both within government and beyond. The results are published annually as an output called Family Spending in the UK. This examines the pattern of spending on a range of categories of goods and services including food, transport, clothing and footwear, education and health (Office for National Statistics 2022f). Analysis of the data from the LCF survey, focused on food, is provided by the Department for Environment, Food and Rural Affairs. Its annual report, Family Food, provides commentary on food and drink purchases, expenditure and derived nutrient content and their variation over time

Chapter 6

122

(Department for Environment, Food and Rural Affairs 2022). Other uses across government include the monitoring and forecasting of levels of car ownership and use; the effects of motoring taxes; and spending on energy across household income deciles. The data are also used in a wide variety of academic research.

6.6

The price of milk

The data collected to calculate inflation measures can provide insights into individual products; an example is milk. Milk is, not surprisingly, a constant item in the basket of goods and services. Fresh milk was in the original basket from 1914 when the first consumer price index started and has been there ever since. Today we have dairy milk in a variety of forms, including whole milk, semi-skimmed and skimmed milk, which are defined in terms of fat content. In 2021, the value of sales of dairy milk was estimated at £4.4bn (Uberoi 2021). Beyond these products, there are flavoured milk drinks and a range of non-dairy milks including soya, almond and oat milk. The 2022 basket includes four types: four pints of whole milk, two pints of semi-skimmed milk, flavoured milk and nondairy milk. Flavoured milk was added to the basket in 2014 and non-dairy, plant-based milk entered in 2017. We saw in chapter 1 that the prices of some items are volatile; petrol and energy are examples. In contrast, some prices are almost constant. Milk is in the latter category, or it was until the end of 2021, when its price started to rise. The price quotes that are gathered each month as part of the ONS data collection to calculate consumer price indices can be used to investigate trends in prices for specific items. Over the period from the start of 2019 to the end of 2021, the average price for a two-pint carton of semi-skimmed milk was stable at about 85p. By May 2022 it had risen to 98p and by July 2022 it was £1.10; at the end of August 2022, several major UK supermarkets were charging £1.25. The effects of the pandemic, plus the Russia-Ukraine conflict has caused increased prices for animal feed, fuel for transport and electricity, all of which affect the price of milk (Financial Times, 2022).

What can the changing basket tell us?

123

In the consumer marketplace, milk can seem a rather pedestrian commodity when compared to the allure of electronic goods (at least to some of us). However, milk is considered an essential item, bought regularly and is used by most people every day. While it doesn’t have the excitement of an occasional purchase of a new TV, or camera, it has a story of its own. A major component of the change in the market for milk is the proliferation of non-dairy or plant-based milks, which extends beyond a taste preference to a choice based on ethical, health and environmental concerns (BBC, 2019). Milk is notable for another, quite different reason. It has a particular importance for politicians who would be advised to be familiar with its cost particularly when a general election is approaching. The suspicion that politicians are out of touch with the everyday concerns of citizens has been put to the test by interviewers asking if they know the price of milk. There are plenty of everyday items that interviewers could choose to quiz politicians on, but there is something iconic about milk as an everyday product. The question has been asked over decades on both sides of the Atlantic and doesn’t usually go well for politicians (BBC 2012).

6.7

Price data sources

The 180,000 price quotes collected each month to use in the calculation of inflation are carefully selected to represent a sample of the prices of basket items across regions and different types of retail outlet. Beyond their primary use, they have value as a source of data for research purposes as we saw in our brief look at the variation in the price of milk in the previous section. The ONS publishes these price quotes each month, though without identifying the retail outlets from which the prices were collected; interested parties can download the price quotes from the ONS website (ONS, 2023). They are one of most extensive publicly available sources of price data and have been used by researchers from across the world. There are many other sources of prices, some on a much larger scale and they have attracted the attention of official statisticians as well as academic researchers. Point of sale scanner data from retail organisations are used to record the purchase of products by customers, adding up the

124

Chapter 6

bill they need to pay. It is also used for stock control, recording how many items have been sold and are remaining to be sold. There are a number of benefits that statisticians can gain from using these data for producing price statistics. The level of detail is much greater than is available with the current price collection – there are price data for more items and at a much higher frequency. They can also provide more detailed expenditure data and so improve the accuracy of expenditure weights and reveal insights into consumer behaviour; we will look at this in the next section. Scanner data have been more readily available in some countries than others. For example, they have been available for research in Australia since 2011 where they are used to produce experimental indices and have some limited use in the production of the official consumer price index. New Zealand, Sweden, Switzerland and other countries have all been using this type of data for some years. In contrast, it has only recently become available to the Office for National Statistics in the UK who plan to start its use in production. Initial work with new data sources will focus on rail fares and second hand cars in 2023 and groceries and private rents in 2024 (Office for National Statistics 2022g). The second source of data is from retailer websites. The price and product information data from websites can be extracted by software that parses the retailer web pages at quiet times with the consent of the retailers. The ONS has been collecting these data on a regular basis for research and development purposes for a number of years. In conjunction with point of sale scanner data, there is also an intention to include these data in the production of inflation measures from 2023 (Office for National Statistics 2020b). Using these data sources presents challenges. The classification of items to expenditure categories is an essential part of estimating the weights used in inflation calculations. The current processing of household records of expenditure is mostly automated with some human intervention required when product descriptions are not clear. In future, this will need to be fully automated as the volumes of data from new sources will be substantial. Machine learning techniques are being investigated to try to move towards full automation. A further complication comes from index number

What can the changing basket tell us?

125

formulas. Traditional formulas used to combine the price and expenditure data don’t work well for the high-frequency nature of these new data sources, so different methods will be required.

6.8

Consumer behaviour

Market research companies collect prices in a similar way to the Office for National Statistics but for different purposes. Their commercial customers are interested in monitoring market shares and brand performance. As we noted in chapter 5, in order to investigate consumer behaviour and preferences, market research companies construct panels of consumers who are asked to record their purchases, much like the Living Costs and Food (LCF) survey. However, while the LCF requires purchasing diaries to be kept for two weeks only, consumer panels can see participants continue for much longer periods. Consumer purchasing behaviour in response to product and price changes is not only of interest to commercial organisations – it can provide useful insights for price statisticians in National Statistics Institutes. The ONS carried out a study in 2012-13 using a panel dataset investigating the degree to which consumers change their choice of product in response to relative price changes. A range of alcoholic drinks was chosen as the products to study. The study found that consumers buying new world wines were more likely to switch products in the face of relative price change than consumers buying old world wines47. The study used economic models as a basis for estimating the propensity to change the choice of product and the degree to which different elementary aggregate price index formulas could replicate the behaviour of more complex formulas as part of the work comparing the properties of elementary aggregates discussed in chapter 5 (Winton, O’Neill and Elliott 2013).

47

The expression “new world wines” refers to wines from Australia, New Zealand, South Africa and California. Old world wines are from France, Spain and Italy, plus others.

Chapter 6

126

6.9

Concluding remarks

In this chapter we have looked at the data that go into estimating a consumer price index and have explored what value they have beyond their primary use. The changing content of the basket, when treated carefully, can provide an informal picture of changes in consumer tastes and the availability of products in the marketplace. The data for household income and expenditure have many uses beyond providing weighting information; they are used widely in government departments and beyond for analysing spending on areas such as healthy food and modes of transport. The collection of price data is going through a major transition with greater use being made of alternative sources including store scanner data and data scraped from websites. The data from these sources can be combined with the conventional price collection to improve the way inflation is calculated. The large volumes of data from these new sources will allow for much more research in consumer behaviour in response to relative price change.

References BBC. 2012. “Should politicians know the price of a pint of milk?” Accessed January 5, 2023. https://www.bbc.co.uk/news/magazine17826509 BBC. 2019. “Plant-based milks on the rise: a quarter of Britons are drinking them”. Accessed January 5, 2023. https://www.bbc.co.uk/news/newsbeat-49030175 Deeming, C. 2010. “The historical development of household budget standards in Britain, from the 17th century to the present”. Social Policy and Administration, 44, 765-788. https://doi.org/10.1111/j.1467-9515.2010.00743.x Department for Environment, Food and Rural Affairs. 2022. “Family Food 2019/20”. Accessed January 5, 2023. https://www.gov.uk/government/statistics/family-food-201920

What can the changing basket tell us?

127

Financial Times. 2022. “UK consumers face ‘bleak’ winter as food inflation hits new high”. Accessed January 5, 2023. https://www.ft.com/content/c0992ca9-0a5f-4264-b122-654086e8cb75 Office for National Statistics. 2020a. “Consumer price inflation basket of goods and services: 2020”. Accessed January 5, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/ukc onsumerpriceinflationbasketofgoodsandservices/2020 Office for National Statistics. 2020b. “Using statistical distributions to estimate weights for web-scraped price quotes in consumer price statistics”. Accessed January 5, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/usin gstatisticaldistributionstoestimateweightsforwebscrapedpricequotesinc onsumerpricestatistics/2020-09-01 Office for National Statistics. 2020c. “New index number methods in consumer price statistics”. Accessed January 5, 2023 https://www.ons.gov.uk/economy/inflationandpriceindices/articles/new indexnumbermethodsinconsumerpricestatistics/2020-09-01 Office for National Statistics. 2022a. “Consumer price inflation basket of goods and services: 2022”. Accessed January 6, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/ukc onsumerpriceinflationbasketofgoodsandservices/2022 Office for National Statistics. 2022b. “Consumer price inflation basket of goods and services dataset”. Accessed January 6, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/datasets/co nsumerpriceinflationbasketofgoodsandservices Office for National Statistics. 2022c. “Consumer price inflation basket of goods and services articles: 2017-22”. Accessed January 6, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/ukc onsumerpriceinflationbasketofgoodsandservices/previousReleases Office for National Statistics. 2022d. “Consumer Prices Index (CPI) and Retail Prices Index (RPI) basket of goods and services”. Hosted by the National Archives. Accessed January 6, 2023. http://www.ons.gov.uk/ons/guide-method/user-guidance/prices/cpiand-rpi/cpi-and-rpi-basket-of-goods-and-services/index.html Office for National Statistics. 2022e. “What's in the basket of goods? 70 years of shopping history”. Accessed January 6, 2023.

128

Chapter 6

https://www.ons.gov.uk/economy/inflationandpriceindices/articles/wha tsinthebasketofgoods70yearsofshoppinghistory/2016-07-21 Office for National Statistics. 2022f. “Family spending in the UK: April 2020 to March 2021”. Accessed January 6, 2023. https://www.ons.gov.uk/peoplepopulationandcommunity/personalandh ouseholdfinances/expenditure/bulletins/familyspendingintheuk/april20 20tomarch2021 Office for National Statistics. 2022g. “Transformation of consumer price statistics: April 2022”. Accessed January 6, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/intr oducingalternativedatasourcesintoconsumerpricestatistics/april2022 Office for National Statistics. 2023. “Consumer price inflation item indices and price quotes”. Accessed May 28, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/datasets/cons umerpriceindicescpiandretailpricesindexrpiitemindicesandpricequotes O’Donoghue, Jim., McDonnell, Carol and Placek, Martin. 2006. “Consumer price inflation 1947-2004”. Economic Trends, 626, 38-54. Accessed October 4, 2023. https://escoe-website.s3.amazonaws.com/wp-content/uploads/2020/09/ 20175140/ET-626-Economic-Trends-Jan-2006-1.pdf Phelps-Brown, Ernest H. and Hopkins, Sheila V. 1956. “Seven centuries of the price of consumables, compared with builders’ wage rates”. Economica, New Series, 23, 296-314. https://doi.org/10.2307/2551457 Science Museum. 2020. “The invention of the vacuum cleaner, from horse-drawn to high tech”. Accessed January 6, 2023. https://www.sciencemuseum.org.uk/objects-and-stories/everydaywonders/invention-vacuum-cleaner Trentman, Frank. 2016. Empire of Things. UK: Allen Lane, Penguin Random House. Uberoi, Elise. 2021. “UK Dairy Industry Statistics”. House of Commons Library. Accessed January 6, 2023. https://researchbriefings.files.parliament.uk/documents/SN02721/SN0 2721.pdf

What can the changing basket tell us?

129

Winton, Joe., O’Neill, Rob. and Elliott, Duncan. 2013. “Elementary Aggregate Indices and Lower Level Substitution Bias”. Statistical Journal of the IAOS, 29, 11-19. Accessed January 6, 2023. https://doi.org/10.3233/SJI-130758

7 QUALITY AND TRUST Previous chapters have described the many uses of inflation measures and the importance of these statistics. They include: acting as a target within monetary policy, informing wage negotiations, determining the adjustment of benefits and pensions, tax thresholds, contract terms and the indexation of some government bonds. The inflation figures produced by the Office for National Statistics influence the flows of very large sums of money. When inflation statistics are reported each month, economic commentators and journalists debate the significance of the numbers, particularly the implications for government policy and for households; they don’t question the quality of the numbers produced. It is perhaps a tribute to the reputation of official statistics that they are rarely doubted. If we stand back a moment, shouldn’t we be surprised by this? Statistics in general don’t have a great reputation and many people get their news from lightly or un-regulated social media channels. We also have to contend with the pernicious effects of post-truth politics (Prutsch 2019, 1-6). How do these highly influential inflation numbers merit such trust? This chapter will attempt to answer this question. We will see that a lot of ingredients go into producing high quality official statistics that are trustworthy. They include: extensive statistical expertise, careful processing and management, a commitment to statistical quality and regular independent reviews. In addition, statisticians and supporting staff are required to act with objectivity and integrity. Achieving and maintaining high levels of quality is the means by which official statistics are set apart from most other sources of information.

Quality and trust

7.1

131

The big picture of official statistics

What are official statistics for? We can look at what the UK Statistics Authority48 says about the role of official statistics: “Statistics are a foundation of our society, supporting the decisions we make at home and at work, as individuals and collectively. They are part of the lifeblood of democratic debate” (UK Statistics Authority 2022, 4).

This statement makes it very clear that official statistics have a very important role to play and to function effectively they need to be authoritative. At this point, it is worth just taking a moment to consider the extent of official statistics in the UK. What counts as an official statistic and who are the producers? An official statistic is formally defined in the Statistics and Registration Service Act 2007. To summarise the legislation: official statistics are produced by the Office for National Statistics, government departments, the devolved administrations and arms-length bodies acting on behalf of government. The UK Statistics Authority website contains a list of national statistics and their producers (Office for Statistics Regulation 2022a). National statistics are a subset of official statistics that have been through an assessment process, which evaluates whether the statistics comply with a set of best practice standards: the Code of Practice for Official Statistics. We will explore the Code and explain its importance later in this chapter. In 2022, the UK Statistics Authority list contained 825 separate sets of statistics, a large number of statistical outputs from many producers. This illustrates the magnitude of the challenge to ensure that all are trustworthy and of high quality.

48

The Office for Statistics Regulation, part of the UK Statistics Authority, is the regulator for official statistics. It promotes and safeguards the publication of official statistics that serve the public good.

132

7.2

Chapter 7

Assessing trust, trustworthiness and quality

How do we decide whether we trust the organisations that produce official statistics? It is only sensible to place trust in an organisation (or a person) when they are worthy of that trust. In order for users of official statistics to trust the organisations that produce them, they must first decide that the producers are trustworthy. How do they do this? A rational approach would be to make a careful, informed assessment of the activities of each organisation that produces statistics. This is clearly not practical. Few people have the time and the necessary skills; statistical producers carry out highly specialised technical operations. This challenge of deciding who we should trust is, of course, a common problem that applies to all the organisations we deal with in our lives. We have to use other ways of assessing trustworthiness. One way is to judge the reliability, integrity and capability of an organisation based on what they do. We can make a judgement on the reliability of statistical producers by seeing whether they do what they are required to do and what they say they will do. Statistical outputs are published to a specified timetable, issued by the producer; for inflation, the producer is the Office for National Statistics. Inflation statistics appear on the second or third Wednesday of each month, with confirmed dates for about a year ahead and provisional dates and times for the year after that. Any changes to a declared publication date are required to be identified and communicated49. The unexpected late publication of inflation figures would be highly newsworthy and would attract much criticism. How about the capability of producers to create statistics of sufficient quality to meet the high standards expected; how can we assess that? Members of the public get most of their information about official statistics from the media. Official statistics are a great source of articles for journalists and commentators, with new statistical releases appearing almost every day. Within the mainstream media, there are many highly knowledgeable journalists who comment on the latest official statistics and also report the views of a range of expert users. For economic statistics, 49

This is a requirement of the Code of Practice for Official Statistics.

Quality and trust

133

these expert users include economists from the Bank of England and from business, think tanks, special interest groups, and international bodies such as the World Bank, plus many others. With such intense scrutiny, any errors would be found quickly unless very subtle. If a statistical release contained a significant error, this would be a newsworthy event. Much effort goes into making sure official statistics are of high quality, so such errors are rare. If we turn now to assessing the integrity of an organisation, this is perhaps a little more subtle. What does it mean to say that an organisation acts with integrity? For statistics, it means that producers will not deliberately publish false or biased statistics. To meet this standard, they need to be free of any influence from outside organisations that might want to gain from the statistics having certain values. For official statistics, most of the producers are within government departments or in government agencies. The obvious risk is that this proximity could allow government ministers to influence statistical outputs to make government performance look better than it actually is. If this was the case, it would seriously undermine the integrity of official statistics and they would no longer be able to achieve what is required of them. How is this risk managed? The essential ingredient is that official statistics are independent of government. In the UK they are overseen by the UK Statistics Authority which reports to Parliament, not government. The statistics that are produced and the methods used are not determined by government ministers but by the National Statistician. This does not mean that government has no role. The government is an important user of official statistics and will expect its needs to be met. However, it sits alongside all other users including the general public, who use official statistics to assess the performance of the government. This separation of statistical production from government influence is a crucial principle not just in the UK, but across the world. For the UK, the history of official statistics from the 1960s onwards illustrates why independence is important and we look at this history in the next section.

134

7.3

Chapter 7

The road to independence for UK official statistics

It may seem an obvious requirement for official statistics to be independent of government. However, sixty years ago, it wasn’t considered to be important. The value of independence had to be learnt, and like many important lessons, it was learnt the hard way. In this section we tell the story of how UK official statistics went through a dark period of distrust before recovering. We start in the 1960s, which was a period of expansion for official statistics. The number of statisticians working in official statistics grew and a community of government statisticians was created with the founding of the Government Statistical Service in 196650. Harold Wilson was Prime Minister at the time and he had worked as an economic statistician during the Second World War. A new director of UK official statistics started in 1967, Claus Moser51, who had a vision for an expanded provision of official statistics that would serve the public good as well as government needs. Although official statistics weren’t independent of government, the political climate was conducive to developing this broad role for official statistics. A dramatic change occurred after the 1979 general election. Control of public expenditure was an important aim for the new Conservative government and a wide ranging review of public expenditure commenced with the aim of improving efficiency under the leadership of a business expert, Derek Rayner52. It led to substantial cuts in budgets and staff across the public sector, including for official statistics. In addition, the vision for official statistics was changed to being just what was required for government use; this was a highly controversial move (Levitas and Guy 1996, chapter 1). The modernisation of official statistics through the use of technology to improve efficiency was a driver at the time and was applied to the production of unemployment statistics in the early 1980s. However, an 50

It is still active today. He went on to become Baron Moser of Regent’s Park in 2001. 52 He went on to become Baron Rayner of Crowborough in 1983. 51

Quality and trust

135

accompanying change in how the statistics were produced saw a political element introduced. The previous practice to determine the number of people unemployed was based on a clerical count of those registered as seeking work. This was replaced by a computerised system which recorded not the numbers seeking work, but those claiming benefits. This meant that the official figures for unemployment were dependent on the rules for who was eligible to claim benefits (Levitas and Guy 1996, chapter 3). The number of unemployed people was particularly politically sensitive in the late seventies and eighties; the unemployment rate rose from 5.5% in May 1979 to 11.4% at the start of 1984. Over this period, a number of changes were made to the criteria for eligibility for claiming unemployment benefit which fed into the unemployment statistics, mostly reducing the number who could claim. This linkage caused serious doubts over the integrity and value of the statistics (UK Government 1998, Section 3.5). Other practices and quality issues also raised concerns including delays to releases of important figures and the perceived poor quality of economic statistics, particularly the National Accounts, where discrepancies and significant revisions were occurring (Jenkinson and Brand 2000). By the end of the 1980s, criticisms of the UK statistical system were mounting, with calls for changes from organisational users and the Royal Statistical Society. An independent review carried out by the economist Stephen Pickford recommended improvements to economic statistics and stronger governance. Extra money was provided for economic statistics and the vision for official statistics was changed back to providing for the public good as well as for government use. Further changes followed in the 1990s with the aim of strengthening official statistics. A code of practice for official statistics was published in 1995 and several separate organisations producing official statistics were merged to create the Office for National Statistics, which became the biggest producer of official statistics in the UK. It still reported to government, but it was at arm’s length as an executive agency of the Treasury. The Labour opposition in the 1990s promoted the need for improved quality of official statistics and freedom from actual or perceived

136

Chapter 7

government interference. A commitment to the reform of official statistics was included in their election manifesto for the 1997 general election, which they won. A green paper, Statistics – A Matter of Trust, appeared in 1998 and a white paper, Building Trust in Statistics, followed (UK Government 1998, 1999). The titles of these papers highlighted the important role of trust. A new governance model was created with a Statistics Commission overseeing official statistics. While it reported to a minister of statistics, the Chancellor of the Exchequer, it was independent of other ministers. An improved code of practice set out professional standards which applied to all producers of official statistics (UK Government 2000). It was hoped that these changes, which were the most significant for thirty years, would lead to higher levels of confidence from users and the general public. However, surveys showed that public confidence in official statistics was still low. Progress had been made but it was not enough. Important bodies including the Royal Statistical Society and a parliamentary committee, the Treasury Select Committee, recommended putting the independence of official statistics into legislation. What improvements would this bring? A statutory footing would require Parliament to pass further legislation to make amendments in future, ensuring wide political scrutiny of any proposed changes. Also, under the Statistics Commission, powers were delegated from ministers, whereas with legislation, officers in the statistical system would have powers specified by statute (UK Parliament 2001; Holt, 2003, p. 356). This further step followed in 2007 with the UK statistical system being put on a legislative footing with the passage of the Statistics and Registration Service Act 2007. Under these new arrangements, which still hold in 2022, the Statistics Commission was replaced by the UK Statistics Authority, which reports to Parliament. A strengthened code of practice was introduced together with an assessment process for official statistics. The Authority was also given a role to intervene if it received a complaint over the misuse of official statistics (UK Government, 2020). In practice, it could not follow up every minor misuse; however, it could intervene when a high profile misuse occurred. Recent history has seen it writing to politicians on a regular

Quality and trust

137

basis when statistics aren’t used in an appropriate manner; such letters are made public. Since the creation of the UK Statistics Authority, surveys of public opinion have shown a much improved level of confidence in official statistics. The UK Statistics Authority continues to commission independent surveys every few years to check whether confidence is being maintained at high levels.

7.4

Understanding statistical quality

Quality and trust are not distinct attributes. Producing statistics of poor quality or containing errors will cause people to question the trustworthiness of a statistical producer. Of course, all organisations make mistakes from time to time. A very occasional error which is openly acknowledged and corrected promptly will attract criticism, but not affect an organisation’s reputation much. A series of mistakes or mistakes occurring on a regular basis are likely to be much more damaging. For official statistics in the UK, mistakes are rare; when they occur they are acknowledged and corrected. We have mentioned the quality of official statistics a number of times, but what does quality mean in the context of official statistics? We start with a high-level answer: for official statistics, quality is defined as “fitness for purpose”. A statistical output that is fit for purpose is one which meets the needs of its users. An obvious implication of this definition is that a statistical producer will need to understand the purposes to which their statistical outputs are put. In order to build this knowledge, developing and maintaining good relationships with users is an important activity for a producer. As we have seen, for inflation statistics, there are many users and uses and considerable effort is required to gain and maintain sufficient understanding. The Office for National Statistics, which produces inflation measures, publishes a report on users and uses, which is updated from time to time (Office for National Statistics 2018). We can now start to take a more detailed look at aspects of quality that sit under the general definition. The natural question for people to ask is: are the numbers right? For inflation statistics, and most other official statistics, this is not a simple question with a yes or no answer. As we saw in chapter

138

Chapter 7

3, consumer price indices are sample statistics, based on a sample of goods and services, with prices collected from a sample of locations with price changes weighted by expenditure shares which themselves are partly derived from a sample of households. We use a sample to estimate some parameter, or parameters of interest from a population, saving time and money. A well-chosen sample will produce accurate estimates which are representative of the underlying population. If we took different samples we would get slightly different results because each sample reflects the underlying populations of products, locations, prices and households imperfectly. However, we know that samples of sufficient size will ensure this variation is small, and a well-chosen sample which is representative of the underlying population from which it is drawn will not lead to biased outputs. Producing accurate and unbiased statistics is an important aspect of statistical quality, but there are other components too. These have been specified as dimensions of quality and are codified in the European Statistical System (Eurostat 2017); they are: x Relevance: this is the degree to which the statistical output meets user needs. x Accuracy and reliability: accuracy is the proximity between an estimate and the (unknown) true value; reliability is the closeness of early estimates to subsequent estimated values. x Timeliness and punctuality: timeliness refers to the time interval between publication and the period to which the data refer; punctuality is the gap between planned and actual publication dates. x Accessibility and clarity: accessibility is the ease with which users are able to access the data; this includes the format in which the data are available and the provision of supporting information. Clarity refers to the quality and sufficiency of the supporting information (metadata), illustrations and accompanying advice. x Comparability and coherence: comparability is the degree to which the data can be compared over time and region. Coherence is the

Quality and trust

139

degree to which data, derived from different sources or methods, but referring to the same topic, are similar. These are very helpful criteria; they are considerations that all statistical producers need to keep in mind throughout the process of creating statistical outputs. These quality criteria not only provide guidance to statistical producers, the have a wider value. An independent review of a statistical output is a helpful way of finding out what is being done well and what can be improved. Reviewers can assess statistics against the quality criteria. We will say more about independent reviews later in this chapter. The quality dimensions are helpful but care is needed when applying them. There are potential conflicts between different dimensions. For example, improving accuracy and the extent of the supporting information will take more time, which reduces the timeliness. A careful balance is required. As well as the quality dimensions, there are other considerations, or supporting principles, that a producer has to bear in mind: x Cost: the cost of producing an output will impose constraints and must be managed. x Respondent burden: collecting data from businesses, households and individuals imposes a burden on them, which must be proportionate. x Confidentiality and security: data must be protected. The quality dimensions and principles are a mixture of the statistical and the practical; they have evolved through the experience of producing statistics across the world over many years. They are part of good practice standards for official statistics.

7.5

Official statistics and scientific practice

Official statistics has borrowed practices from other activities to build its best practice standards. The scientific method holds a highly esteemed

140

Chapter 7

place in the building of knowledge. It is the means by which competing theories are subject to severe testing; through its application knowledge is created. It has been called a knowledge machine (Strevens 2020). Its use has expanded across a wide range of disciplines from its traditional bases in the physical sciences to social, political and behavioural science. Some aspects of the method differ between disciplines; the techniques used in virology differ dramatically from those in particle physics. However, important aspects are the same. Typical common characteristics and values are: researchers should approach inquiry with an open mind, be objective, describe the data and methods used fully, welcome independent scrutiny and be willing to accept criticism when it is justified. Indeed, one of the defining characteristics of the scientific method is that researchers invite challenge to what they do and accept that their work may be wrong. The integrity of knowledge sits above any commercial interests or personal and organisational reputation; it is paramount53. Official statistics have adopted these common scientific principles and embedded them into best practice standards. What does this mean for producers of official statistics? It means that the statistics they produce should be objective, that is, free from any influence from external organisations including: government, commercial organisations or special interest groups. The data and methods should be described in detail so that they can be examined by any interested party to make it clear how the statistics were produced. Statistical producers should invite independent reviews and be willing to make changes where they would improve the statistics.

7.6

Official statistics and industrial production

There is another area of human activity that has provided help to producers of official statistics: general industrial production. Producing statistics involves a series of processing steps and can be thought of as a kind of business production process which takes inputs (data), processes them 53 What constitutes the scientific method is much debated. Some commentators claim there is no single method or even a method at all. However, the main principles are generally accepted and the success of science is beyond (reasonable) question.

Quality and trust

141

(applies statistical methods) and produces outputs (the statistics and commentary). What are the lessons that can be taken from the experience gained from several centuries or more of industrial practice? We can start to answer this by looking at what a typical statistical production process looks like. A standard has been produced which specifies a model of an idealised process. It is called the Generic Statistical Business Process Model, usually abbreviated to the “GSBPM” (United Nations Economic Commission for Europe 2019). It identifies a typical set of steps: x x x x x x x x

Specify needs Design Build Collect Process Analyse Disseminate Evaluate

Reaching the “Evaluate” stage doesn’t mean we are at the end; the output of the evaluation feeds back into the start of the overall process so improvements can be made. The model is sometimes represented as a circle to emphasise the continuous nature of the overall process. The model divides each step into sub-steps, creating 44 sub-processes. For example, the “collect” stage contains “create frame and select sample” and the “process” stage contains “calculate weights” (United Nations Economic Commission for Europe 2019). Not all statistical outputs use all of the model steps; for example, statistics derived from administrative data won’t create a sample frame. The process model encourages producers to engage in “process thinking”. What are the benefits from a business process comprising clearly defined, distinct processing steps? At the end of each step, quality criteria can be defined, and tests carried out so that progressing to the next step doesn’t occur until the current step has been completed correctly. If an error occurs, it is easier to see where it happened. In addition, the performance

142

Chapter 7

of each step can be assessed separately. Perhaps one step takes a long time to run; attention can then be focused on improving it. Individual steps may have specific skill requirements from the staff managing them; staff can be trained for specific roles on particular process steps. Analysis and optimisation of processes leads to a reduction in errors and greater efficiency. Quality in business process environments is clearly important and has been much studied. A number of well-known names (“gurus of quality”) have written extensively on the subject, including Deming and Juran. Quality methodologies have been created from the simple but powerful “Plan, Do, Check, Act (PDCA)” approach, to Six Sigma, Lean and Total Quality Management. These generic approaches have been applied to statistical production to gain the benefits that more general business processes have realised (Ross 1999). One way to demonstrate a commitment to quality standards is to gain accreditation from an internationally recognised standards body. The International Standards Organisation standard ISO9001 sets out criteria for quality management where an organisation can seek certification of its activities. The standard has a strong focus customer focus and a commitment to continual improvement (ISO, 2015). The CPI and CPIH production process used by the ONS have been certified compliant with ISO9001:2015 (Office for National Statistics 2017).

7.7

Sharing knowledge and experience

Most countries across the world produce a very similar range of statistical outputs regardless of their political system. Knowledge of the size of a country’s population, the level of economic growth, rate of inflation and many other statistics are universally beneficial. It makes sense for statisticians and economists around the world to get together and decide on what constitutes the best methods, to document them, and make the knowledge freely available. Learning from each other and coordinating research activities provides an efficient use of resources. A number of bodies such as the International Labour Organisation, part of the United Nations family of international bodies, promotes communication and the

Quality and trust

143

development of common standards and practices; they also provide assistance for developing countries to improve their statistical capabilities. An important factor in collaboration and sharing is that official statistics are produced using public money. As a public good, they are made available, usually without cost, for anyone to use. The statistics are usually accompanied by descriptions of the data sources and the methods used. Often the collected data are made available, though without the details of who provided each data record, which is protected by a confidentially promise. We saw in chapter 6 that monthly, anonymised price quote data are provided for free by the Office for National Statistics (Office for National Statistics 2023). Commercial organisations also produce statistics though they tend not to explain how they were created and usually limit the extent of explanatory documentation. The open approach used in official statistics is very similar to the way science operates. The scientific method promotes the open sharing of research, methods and results. Publicly funded research, with the publication of reports and journal articles, follows this approach. Commercial research is a little less open, but still follows many of the principles. Conferences and other expert gatherings are a regular part of scientific research and official statistics follows this approach. National and international gatherings are facilitated by National Statistics Institutes, supra-national bodies such as the United Nations and organisations such as the Royal Statistical Society. Special interest groups are particularly valuable; we mentioned one of these groups in chapter 5. The Ottawa Group is an international working group of price statisticians who get together every two years to present and discuss papers and issues (Ottawa Group 2022). Following the practice of science, statisticians and economists working in official statistics are strongly encouraged to writeup and share their work.

7.8

Best practice standards

The knowledge built up through many years of development and practice in official statistics around the world is enormously valuable. It has been

144

Chapter 7

captured in a number of forms which help to maintain and promote highquality standards. An iconic core set of good practice statements were created in 1994; they are deliberately concise to maximise their impact. The United Nations Fundamental Principles of Official Statistics is a set of ten core principles (United Nations Statistics Division 2014). It was first produced to aid the transition between centrally planned and marketoriented economies of countries in Central Europe and the former Soviet Union who were setting up their own statistical offices. Two examples of the principles are: Principle 3. To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods and procedures of the statistics. Principle 5. Data for statistical purposes may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies choose the source with regard to quality, timeliness, costs and the burden on respondents. To aid the understanding of the principles and how to put them into practice, the UN also provided two supporting documents “Implementation Guidelines” and a “Handbook of Statistical Organisation – the operation and organisation of a statistical agency” (United Nations Statistics Division 2014). Other sets of good practice principles have been produced which are either derived from or are consistent with the United Nations Principles. Examples are the European Statistics Code of Practice (Eurostat 2017) and the UK Code of Practice for Official Statistics (UK Statistics Authority 2022). In the UK, the Code of Practice for Official Statistics sets the standards for UK statistics to follow. It is hierarchically structured into 3 pillars, 14 principles and 87 practices. The three pillars are trustworthiness, quality and value. For example, under the trustworthiness pillar is the following principle and two of its four practices: Principle T.1 Honesty and integrity: People in organisations that release statistics should be truthful, impartial and independent, and

Quality and trust

145

meet consistent standards of behaviour that reflect the wider public good. Practice T1.1 Everyone that works in organisations producing official statistics should handle and use statistics and data with honesty and integrity, guided by established principles of appropriate behaviour in public life. Practice T1.4 Statistics, data and explanatory material should be presented impartially and objectively. There are also standards for specific official statistics. For inflation measurement, there is an international manual: Consumer Price Index Manual: Concepts and Methods (International Monetary Fund et al. 2020). A number of influential international bodies have been involved in its development and updating, including the International Monetary Fund, the International Labour Organisation and the World Bank. Its fourteen chapters and seven appendices provide an extensive guide. Standards and guides have been produced for other areas of statistics, including the National Accounts and the Labour Market. The comprehensive provision of best practice standards and general supporting material is a reflection of the overall importance of official statistics and fact that almost all the countries in the world produce a similar range of statistics. By pooling resources across countries much more has been achieved than any one country could produce on their own.

7.9

Statistical reviews

However well a statistical producer attempts to follow best practice standards, there are always compromises to be made. Data will be incomplete, a practical set of methods will always fall some way short of the ideal, resource will be limited and time pressing. Carrying out an independent review from time to time is a highly effective way of exploring what could be improved. An extensive programme of assessments has been carried out by the Office for Statistics Regulation, the regulatory arm of the UK Statistics

146

Chapter 7

Authority. An assessment of a statistical output explores the degree to which an output adheres to the Code of Practice. Each assessment takes a few months to complete; a report is produced and published. Where improvements are needed they are documented in the review report. An official statistic that is judged to meet the requirements of the code is labelled a National Statistic. So far, over 360 assessments have been made (Office for Statistics Regulation 2022b). There are other types of statistical review and they come in a variety of shapes and sizes. The Office for National Statistics has used a short review approach, the Regular Quality Review, which lasts less than a day (ONS, 2016). It has also carried out much longer and more in depth reviews through the National Statistics Quality Review format. These are usually internally managed but with external experts and can take a year or so to complete. Two examples are the review of the Living Costs and Food Survey and the review of Foreign Direct Investment; they produced 30 and 20 recommendations respectively (Ralph and Manclossi 2016, James 2016). Occasionally an externally led review is deemed appropriate, where an eminent figure is asked to review a whole topic compromising multiple outputs. In chapter 5, we mentioned the review of Consumer Price Statistics led by Paul Johnson, the head of the Institute for Fiscal Studies. This review was assisted by a small team of external, special advisors. The review report included 24 recommendations for improvement (Johnson 2015). A final type of review has also been mentioned before: the parliamentary inquiry. Several parliamentary committees examine aspects of the working of inflation statistics and official statistics more generally. In chapter 5, we described the inquiry into the use of the RPI by the House of Lords Economic Affairs Committee (UK Parliament 2019). Parliamentary inquiries look at issues arising from the use of statistics and the operation of the UK statistical system. As noted in chapter 5, their reports are usually frank and hard hitting and, as for the RPI inquiry, consequential.

Quality and trust

147

Collectively, these different types of review provide an important contribution to the overall quality of official statistics and are welcomed by statistical producers. They are an illustration of the commitment to improvement that is required of official statistics.

7.10 Challenges for official statistics Although much effort is expended to ensure official statistics achieve high levels of quality there are considerable challenges. One of these is the decline in the achieved sample size for social surveys, where participation is voluntary. In some cases, the achieved sample can be less than half the drawn sample and over time, fewer and fewer households are willing to take part in a survey. A simple response might be to increase the size of drawn sample, but the resources and time available to attempt to contact sampled households is largely fixed. There are a number of reasons for what is known as sample attrition, this includes the growth of on-line, customer “surveys” or requests for feedback which can overwhelm recipients. We encountered the Living Costs and Food Survey in chapter 6. It provides important data on household expenditure which is included in the calculation of inflation as well as having a number of other uses across government. A recent assessment report noted that this survey has an achieved sample of around 5,000 from a drawn sample of 13,000 households; the response rate in 2008 was 51% (Office for Statistics Regulation 2021). This decline affects the quality and robustness of the survey outputs. Work has been underway to address this and other issues (Office for National Statistics 2022). An important area of development in official statistics is the increasing use of administrative data to either supplement or replace survey data. This is data collected as part of administrative processes and subsequently used for statistical purposes. For inflation statistics, there already many such data sources being used to supplement the household expenditure and price collections. Although there are potential advantages in using administrative sources which may contain records for a large percentage of the desired statistical population there are disadvantages too. For example,

148

Chapter 7

the data definitions may not match was is required, data may be missing or incorrect and changes to an administrative system may disrupt the supply of data for statistical purposes. The assessment of quality for administrative sources has required new approaches and this is an area of official statistics that is still developing.

7.11 Concluding remarks In this chapter, we have reflected on the importance of official statistics, and of inflation statistics in particular and how they need to be trustworthy and of high quality to fulfil their function. We have asked two questions. Firstly, what it means for an organisation to be considered trustworthy and particularly a statistical producer. Secondly, how is high quality for official statistics achieved? We have shown that a whole range of factors contribute. Independence from external influence, openness and objectivity, the use of scientific principles and practices, working collaboratively around the world, adopting appropriate values and behaviours, and extensive scrutiny through independent reviews all play a part. Once a high standard has been achieved, hard work is required to maintain the position; it can’t be taken for granted.

References Eurostat. 2017. “European Statistics Code of Practice. Principles 11-15”. Accessed January 6, 2023. https://ec.europa.eu/eurostat/web/productscatalogues/-/KS-02-18-142 Holt, David (“Tim”). 2003. “The need for new statistical legislation for the UK”. Journal of the Royal Statistical Society, Series(A, 166, 349-367. https://doi.org/10.1111/1467-985X.00281 International Monetary Fund, International Labour Organization, Statistical Office of the European Union (Eurostat), United Nations Economic Commission for Europe, Organisation for Economic Cooperation and Development and The World Bank. 2020. “Consumer Price Index Manual: Concepts and Methods”. Accessed January 6, 2023. https://www.imf.org/en/Data/Statistics/cpi-manual

Quality and trust

149

James, Gareth. 2016. “NSQR Series (2) report Number 4 – Foreign Direct Investment”. Office for National Statistics. Accessed January 6, 2023. https://www.ons.gov.uk/methodology/methodologytopicsandstatistical concepts/qualityinofficialstatistics/qualityreviews Jenkinson, Graham and Brand, Martin. 2000. “A decade of improvements to economic statistics”. Economic Trends. 558, 45-50. https://escoe-website.s3.amazonaws.com/wpcontent/uploads/2020/01/01234300/ET-558-A-decade-ofimprovements-to-economic-statistics-G-Jenkinson-M-Brand-May2000.pdf Johnson, Paul. 2015. “UK consumer price statistics: A review”. UK Statistics Authority. Accessed January 6, 2023. https://uksa.statisticsauthority.gov.uk/reports-and-correspondence/ reviews/uk-consumer-price-statistics-a-review/ International Standards Organisation. 2015. “Quality Management Principles”. Accessed January 6, 2023. https://www.iso.org/publication/PUB100080.html Levitas, Ruth and Guy, Will (editors). (1996). Interpreting Official Statistics. Routledge, London and New York. Office for National Statistics. 2016. “Quality Reviews – Regular Quality Reviews”. Accessed January 6, 2023. https://www.ons.gov.uk/methodology/methodologytopicsandstatistical concepts/qualityinofficialstatistics/qualityreviews Office for National Statistics. 2017. “Quality assurance of administrative data used in consumer price inflation statistics”. Accessed January 6, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/methodolog ies/qualityassuranceofadministrativedatausedincpih Office for National Statistics. 2018. “Users and uses of consumer price inflation statistics: July 2018”. Accessed January 6, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/methodolog ies/usersandusesofconsumerpriceinflationstatisticsjuly2018update Office for National Statistics. 2022. “Progress report in response to OSR Assessment of LCF, March 2022”. Accessed October 27, 2023. https://www.ons.gov.uk/news/statementsandletters/progressreportinres ponsetoosrassessmentoflcfmarch2022

150

Chapter 7

Office for National Statistics. 2023. “Consumer price inflation indices and price quotes”. Accessed January 6, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/datasets/cons umerpriceindicescpiandretailpricesindexrpiitemindicesandpricequotes Office for Statistics Regulation. 2021. “Living Costs and Food Survey; Assessment Report 358”. Accessed October, 27, 2023. https://osr.statisticsauthority.gov.uk/wp-content/uploads/2021/07/ Assessment-Report-The-Living-Costs-and-Food-Survey.pdf Office for Statistics Regulation. 2022a. “List of National Statistics”. Accessed January 6, 2023. https://osr.statisticsauthority.gov.uk/national-statistics/ Office for Statistics Regulation. 2022b. “Office for Statistics Regulation Assessment”. Accessed January 6, 2023. https://osr.statisticsauthority.gov.uk/our-regulatory-work/assessment/ Ottawa Group. 2022. “International working group on price indices”. Accessed January 6, 2023. https://www.ottawagroup.org/ Prutsch, Markus J. 2019. Science, Numbers and Politics. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-030-11208-0 Ralph, Jeff and Manclossi, Sylvia. 2016. “NSQR Series (2) report Number 3 – Living Costs and Food Survey”. Office for National Statistics. Accessed January 6, 2023. https://www.ons.gov.uk/peoplepopulationandcommunity/personalandh ouseholdfinances/incomeandwealth/methodologies/nsqrseries2reportnu mber3livingcostsandfoodsurvey Ross, Joel. E. 1999. Total quality management. New York, United States: CRC Press. https://doi.org/10.1201/9780203735466 Strevens, Michael. (2020). The Knowledge Machine. London, UK: Allen Lane. UK Government. 1998. “Statistics – A Matter of Trust”. Cm 3882. Accessed January 6, 2023. https://www.gov.uk/government/publications/statistics-a-matter-oftrust

Quality and trust

151

UK Government. 1999. “Building Trust in Statistics”. Cm 4412. Accessed January6, 2023. https://uksa.statisticsauthority.gov.uk/wp-content/uploads/2015/12/ images-buildingtrustwhitepaper_tcm97-18285.pdf UK Government. 2000. “Framework for National Statistics”. Accessed January 6, 2023. https://uksa.statisticsauthority.gov.uk/wp-content/ uploads/2015/12/images-framedoc1_tcm97-18289.pdf UK Government. 2020. “Memorandum of Understanding between Cabinet Office and the UK Statistics Authority”. Accessed January 6, 2023. https://www.gov.uk/government/publications/memorandum-ofunderstanding-cabinet-office-and-uk-statistics-authority UK Parliament. 2001. “Treasury Select Committee – Second Report”. Accessed January 6, 2023. https://publications.parliament.uk/pa/cm200001/cmselect/cmtreasy/137 /13702.htm UK Parliament. 2019. “Measuring Inflation”. House of Lords Economic Affairs Committee, 5th Report of Session 2017-2019. Accessed January 6, 2023. https://publications.parliament.uk/pa/ld201719/ldselect/ldeconaf/246/2 46.pdf United Nations Economic Commission for Europe. 2019. “Generic Statistical Business Process Model”. Accessed January 6, 2023. https://unece.org/fileadmin/DAM/stats/documents/ece/ces/ge.58/2019/ mtg2/MWW2019_GSBPM_Munoz_Presentation.pdf United Nations Statistics Division. 2014. “Fundamental Principles of Official Statistics”. Accessed January 6, 2023. https://unstats.un.org/unsd/dnss/gp/fundprinciples.aspx UK Statistics Authority. 2022. “Code of Practice for Official Statistics V2.1”. Accessed January 6, 2023. https://code.statisticsauthority.gov.uk/wp-content/uploads/2022/ 05/Code-of-Practice-for-Statistics-REVISED.pdf

8 OTHER PRICE STATISTICS So far in this book, we have focused on consumer price inflation. We have seen there are several measures in use, with the Retail Prices Index (RPI), Consumer Prices Index (CPI) and the CPI including owner occupiers’ housing (CPIH) playing a part. Eventually, it is expected that we will see just one main measure of consumer price inflation, the CPIH; this will simplify what is currently an uncomfortably fragmented situation. In this chapter we will look beyond the all-items headline measures to consider inflation for household types, to see whether they experience price changes to the same degree. Household types include: working and retired households, households with and without children and households with different levels of income, Consumer price measures are important, but not the whole story for price statistics in the economy. The level of prices for domestic property also commands a high profile which is reflected in the frequency of articles on house prices in the business and money sections of newspapers. Price statistics for both domestic property and private housing rentals are important economic indicators; we will look briefly at what they have shown over the last decade or so. Beyond consumer prices and housing, changes in prices paid or charged by businesses are important indicators of the state of the economy too and can provide an early indication of likely trends in consumer prices. As we saw in chapter 4, wholesale prices were collected before retail prices were readily available. Price indices for imports and exports together with price indices for goods leaving the factory gate form part of the portfolio of business price statistics. We will look at whether business price statistics have displayed the rising trends we have seen in consumer prices. We include other examples of price statistics and explain what they can tell us.

Other price statistics

8.1

153

Consumer price inflation for population groups

The main measure of consumer price inflation includes data which represents all households; this gives a single monthly figure for the whole population. However, there is much interest in how different household groups are affected by changes in prices. The Office for National Statistics publishes the CPIH for several household categories, including by deciles of equivalised income, retirement status, child status and housing tenure54. Figure 8.1 shows consumer price inflation as measured by the CPIH for two income groups. Household income has been split into deciles with decile 1 being the lowest household income group and decile 10 being the highest55. Inflation for a specific income group is estimated by calculating weights from expenditure information for the subgroup and applying to the full set of price quotes. Ideally, the subgroup measure would account for individuals choosing differing retail outlets and goods ranges, so facing different prices, but this information is not readily available. Between April 2020 and September 2021, the estimated inflation rate for the lowest income decile was below the rate for the highest decile. However, this situation reversed from October 2021 when the rapidly rising prices of energy and food made inflation for households in the lowest income decile more than 2% higher than for those in the highest income decile. This effect results from lower income households spending a higher proportion of their income on food and energy than high income households.

54 More precisely, equivalised disposable income and equivalised expenditure. Equivalisation is a standard method for adjusting household income or expenditure to take account of resource requirements of different sizes of household (Office for National Statistics 2014, chapter 3). 55 A decile is a division of a population into tenths.

Chapter 8

154 14.0

CPIH inflation by decile

12.0 10.0

Decile 1 Decile 10

8.0 6.0 4.0 2.0 0.0 Jan-20

Jul-20

Jan-21

Jul-21

Jan-22

Jul-22

Jan-23

Figure 8.1: CPIH inflation for the lowest (decile 1) and highest (decile 10) equivalised income deciles, democratic weighting, 2020 to 2022. Source: Office for National Statistics 2022a.

How does inflation affect households with different tenure types? Analysis shows a higher rate for families in subsidised rental properties than for private renters or those who own their own homes (Office for National Statistics 2022a). A comparison between retired and non-retired households shows little difference in their inflation experiences. What is the value of this type of sub-population analysis? It provides an additional insight into how different types of households are being affected, particularly in times of relatively high inflation. It helps to inform both special interest groups and the government when policy is being debated. Although it is unlikely that a different rate of inflation would be applied to specific benefits, it could influence the overall provision of support to particular household types.

8.2

Variants of CPI and CPIH

The headline measures of consumer price inflation will address almost all user needs. However, there are some situations where a variant is required.

Other price statistics

155

Variants tend to omit certain types of goods or services and are produced to address specific needs. There are two variants of the CPI and the CPIH; they are: CPIY, CPI-CT and the corresponding CPIHY and CPIH-CT. CPIY and CPIHY, aim to measure movements in “underlying prices”; in this instance it means without the influence of changes in indirect taxes, including VAT, excise duties and stamp duty. Changes in indirect taxes occur only occasionally, but when they do, they influence prices paid but don’t represent the underlying price changes of a purchased item (Office for National Statistics 2019, 93-5). A related index is CPI-CT (CPIH-CT); it is the all-items CPI at constant taxes; the tax arrangements in place at the “base” or index reference period are held constant56. By comparing it with the headline measure, the effect of changes in indirect taxes can be seen. It is a measure in the HICP family calculated according to European regulations and comparable measures are produced across the EU (Office for National Statistics 2019, 96-7). Core inflation is a variant of the all-items CPI (or CPIH) that is valuable to institutional users such the Bank of England. Although there are different definitions of what constitutes core inflation, a common definition excludes what are considered to be the more volatile components of the all-items index: energy, food, alcoholic beverages and tobacco. It is thought that by removing these components, the remaining part gives a view of the longer-term, underlying trend. Using this definition, for August 2022, CPIH core (12-month) inflation showed a rise of 5.6% as compared to inflation derived from the all items index which was 8.6% (Office for National Statistics 2022b).

8.3

Variants of the RPI

Over the course of the long lifetime of the RPI, a number of variants were produced. The first two were pensioner indices for one and two-person pensioner households on low incomes, introduced in 1969. Low income 56 The index reference period is the time period where the index value is set be 100; in 2022 the index reference period for the CPI was 2015.

156

Chapter 8

meant that state benefits comprised at least three quarters of household income. Next was the Tax and Price Index introduced in 1979. It was a measure that was not affected by movements between direct and indirect taxes. It arose in response to the government increasing VAT while lowering income tax. The Rossi Index was a variant used to uprate income related benefits. It was named after the Minister of State for Social Security (Sir Hugh Rossi) who introduced it in 1983. Recipients were thought unlikely to be incurring housing related costs. It excluded items in the “Housing” subgroup of the RPI, including: rent, mortgage interest, rates and housing depreciation; it lasted until 2011. Two further variants, RPIX and RPIY were first published in the 1990s. We saw in chapter 2 that inflation targeting was introduced in 1992 as part of the transition to monetary economics. A target of 2% was set to be measured by the RPI excluding mortgage interest payments which was called RPIX. RPIY started in 1995 as an “underlying inflation” measure which excluded items affected by interest rates and indirect taxes; mortgage interest payments, local authority taxation, VAT and insurance taxes were all excluded. The RPIJ was a short lived measure which was introduced in 2013. It was the RPI with the Carli formula replaced by the Jevons formula. It followed the 2012 consultation on the RPI and illustrated the degree to which the two formulas affected the magnitude of inflation as measured by the RPI. In 2016, the Office for National Statistics announced that, from March 2017, a number of RPI variants would no longer be published. These included: RPIY, the Tax and Price Index, the RPI pensioner indices and the RPIJ. This was in response to the RPI being downgraded as a measure of inflation so that only what was considered essential was continued (Office for National Statistics 2016a).

8.4

Historic price index series

In chapters 4 and 5, we saw that the RPI, which started in 1956, was joined by a new index, the CPI, in 1997. The CPI, which is the UK’s

Other price statistics

157

version of the European HICP, was designed from scratch and differs in a number of ways from the RPI. The official view is that the CPI methodology is an improvement on that of the RPI. It is, of course, beneficial to make improvements to produce the best version of a statistical measure with the knowledge available at the time. However, there is also a value in maintaining a long-running series calculated in as identical a fashion as possible. These two considerations provide a natural tension. One way of trying to get the best of both worlds is to back calculate a new or improved measure using past data. For the CPI (and the CPIH), the detailed data were retained from 1988 onwards; earlier data had been deleted. This meant that an historic series was readily achievable back to 1988, but not before that. Soon after the CPI was introduced in 1997, an historic series was calculated, extending back to 1988, the limit of data availability. However, by making assumptions and using statistical techniques, a “broadly indicative” series was created which extended back to 1975 (O’Donoghue 1998). In 2012, the Office for National Statistics was asked to revisit the modelling of an historic series and produce historic series for the 12 component sub-series of the all-items CPI57. These could be combined into a modelled, historic all-items series. The request was to extend the modelled series back to 1950. This presented a considerable challenge, but it was achieved58 (O’Neill and Ralph 2014). The work has subsequently been revisited and corresponding historic series produced for the CPIH. Clearly, such modelled series are approximations and have to be treated as such. This work was useful for journalists in 2022 when CPI inflation reached double figures with newspaper articles declaring that inflation as measured by the Consumer Prices Index had reached the highest value seen for over forty years.

57 The CPI all-items index can be sub-divided into 12 high-level categories; examples are “food and non-alcoholic beverages”, “transport”, “clothing and footwear” 58 The difference between the RPI and CPI was modelled where they overlapped and time series techniques used to project back using a regression approach.

158

Chapter 8

A different type of approach can be used to produce approximate price index series for much longer time spans. Researchers have produced a long running price index series from 1750 to 2004 by combining previous work by other researchers which covered the periods 1750 to 1850, 1850 to 1870, 1870 to 1947; this was combined with the RPI up to the time of publication in 2004. This showed a price rise of 140 times over the entire period. Of course, the quality of the data varies significantly over the entire period, with the early periods using wholesale prices for a limited number of commodities (O’Donoghue, Goulding and Allen 2004).

8.5

Producer price indices

Consumer price inflation, as the name suggests, represents the change in prices faced by consumers. In contrast, producer price inflation reflects changes in prices from the viewpoint of producers. One can think of producer price inflation as a forerunner of what might happen to consumer price inflation at a later date, though some price pressures may be absorbed along the way by producers or retailers rather than being passed on to retail customers. Producer price indices used to be called wholesale price indices and a few countries still use this name. Producer price statistics are published for goods bought and sold by UK manufacturers with separate price indices for inputs and outputs. In chapter 4 on the historical development of inflation measurement, we saw that price series for commodities were started in the 19th century, with wholesale prices being more readily available than retail prices. A more recent addition to the price index family is the index for services, the Services Producer Price Index (SPPI) which started in 1995 in the UK. It is an important addition especially as services account for about 80% of UK economic output. Its relatively late development is a reflection of the difficulty in defining standardised services for many industries (O’Neill, Ralph and Smith 2017, 347-8). Figure 8.2 shows input and output Producer Price Indices (PPIs) for the period September 2013 to August 2023. Both input and output PPIs have shown rapid growth since the start of 2021 resulting from the combination of the effects of the covid-19 pandemic and the conflict in Ukraine.

Other price statistics

160.0

159

Input and output PPIs

150.0 140.0 Input PPI 130.0

Output PPI

120.0 110.0 100.0 90.0 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 Figure 8.2: Input and output producer price index values, September 2013 to August 2023. Source: Office for National Statistics 2023a.

8.6

Housing related price indices

In the UK, there is considerable interest in the state of the housing market. High prices in certain locations, the difficulty young people face when trying to buy their first property, the degree to which government targets for house building are met (or not) are all discussed regularly in the media. For those renting, there is a similar interest, with concerns around renting costs rising and the increasing competition for properties. Official statistics contribute to the evidence base through the publication of a number of housing related statistics, including the UK House Price Index, the average house price and an experimental Index of Private Housing Rental Prices.

8.6.1

Owner occupier related house price statistics

The UK House Price Index (HPI) measures the price change of residential housing relative to a reference time period. It is produced jointly by HM

160

Chapter 8

Land Registry, the Office for National Statistics, Land and Property Services Northern Ireland and Registers of Scotland (Office for National Statistics 2023b). Before 2016, several offices within the public sector published separate measures of house price change which differed in their methodology and consequently in their results. In 2010, the National Statistician, Jil Matheson, initiated a review which led to a single public sector index being produced. As part of the review work, a public consultation gathered information on who used the statistics, what they were used for and how they could be improved. This identified a desire for disaggregated statistics, with UK level statistics broken down into region, sub-region and property-type (Office for National Statistics 2010). Following development work and further user consultation, agreement was reached on a single, UK-wide, public sector house price index which started in 2016 (Office for National Statistics 2016b). House price statistics are derived from sales data for residential transactions including purchases with cash or through a mortgage. Across time periods, the relative proportions of different types and sizes of houses sold vary and this variation would be included with the underlying price change in statistics derived from sales data. In order to produce a pure price index, the sales data are mix-adjusted, which standardises the distribution of properties. A model is used containing cells which are defined by combinations of characteristics; example characterises might be “first-time buyer, old dwelling, one bedroom flat purchased in London”. The model contains around 100,000 of these cells. Each month, estimated prices for all cells are produced by the model using the latest data and are then combined with their appropriate weight to produce mix-adjusted average prices (HM Land Registry 2022). It takes time for sale registrations to complete; typically between two and eight weeks. Initial estimates are made using 40% of the total volume of transactions in the most recent month, with second and third revisions made as more data become available, resulting in revisions in subsequent months. By the third estimate, about 90% of the relevant data are available (HM Land Registry 2022).

Other price statistics

161

Figure 8.3 shows how the UK average house price has changed since 2005. The fall from the start of 2008 corresponds to the period of the “great recession” of 2008-9, where GDP shrank by 6% between the first quarter and the second quarter of 2008, and took five years to return to its pre-fall value. Following that period, the average price has risen consistently with some volatility arising from lower than usual numbers of transactions during the covid-19 pandemic. The value in December 2022 was slightly down on the record average price seen in November 2022. 350000

UK Average House Price

300000 250000 200000 150000 100000 2005 2007 2009 2011 2013 2015 2017 2019 2021 2023 2025 Figure 8.3: Average house price in pounds sterling, in nominal terms, UK, January 2005 to July 2023. Source: Office for National Statistics 2023b.

8.6.2

Rental statistics

The Office for National Statistics publishes statistics on rental prices using data from the Valuation Office Agency (VOA); prior to 2019, the VOA was responsible for publishing these statistics. Median and interquartile range rental prices by property type for regions, down to local authority level are presented in a statistical bulletin with an accompanying dataset to allow further analysis. The data are collected from landlords and lettings agencies by Rent Officers from the Valuation Office Agency (Office for National Statistics 2023b).

Chapter 8

162

The index of private housing rental prices (IPHRP) tracks the prices paid for renting property from private landlords. It is currently described as an experimental measure as it is undergoing development following the granting of access to the ONS of VOA microdata. The statistics are available for the whole of the UK, its constituent countries and for regions in England. These data comprise price and property characteristics, which allows for adjustments for changes in the mix of property types (Office for National Statistics 2022c). 11.0

Private rental housing % change

9.0 7.0 5.0

England Wales Scotland Northern Ireland

3.0 1.0 -1.0 2016

2018

2020

2022

2024

Figure 8.4: Index of private housing rental prices, 12-month % change, for nations of the UK, January 2016 to June 2023. Source: Office for National Statistics 2023c.

Figure 8.4 shows the 12-month percentage change in the IPHRP for England, Wales, Scotland and Northern Ireland. Growth has been strongest for Northern Ireland since the end of 2017 with sharp rises in Northern Ireland from August 2020 and all nations from August 2021.

Other price statistics

8.7 8.7.1

163

Alternative measure of consumer prices Household Costs Indices

In chapter 2, we considered two different conceptual approaches to measuring inflation: the cost of goods and the cost of living. There is a third possibility. A “household index” aims to measure the changes in the actual costs faced by households without being restricted by the normal rules of what should and shouldn’t be included in a consumer price index. The original proposal for such a measure came from the Royal Statistical Society in a paper by John Astin and Jill Leyland (Astin and Leyland 2015). The design of a household costs index (HCI) differs from that of a traditional consumer price index. There are differences in the target population, the weighting, the target concept, the commodity coverage and in taking a payments rather than acquisitions approach to consumption (Office for National Statistics 2017). The costs that households face may vary according to a number of characteristics, including: income, whether the occupants are working or retired and whether housing is owned or rented. To reflect these important characteristics, a number of different household costs indices have been produced. Over the period of development, four sets of preliminary series of household costs indices encompassing household characteristics have been produced; a further set was published in December 2023 after which publication will be quarterly (Office for National Statistics 2023d). A comparison of rates of inflation calculated from household costs indices for different occupancy types indicates that those renting experienced higher inflation than owner occupiers over the period from May 2020 to June 2021; after that period, the pattern reversed. The differences between inflation for those renting and owning were driven by tax changes on property purchases and base rate interest changes. Other findings were that high and low income households experienced similar values of HCI inflation from 2017 to 2020, with higher inflation found for wealthier households in the second half of 2021; for the second half of 2023, the rates were similar. Inflation for retired and non-retired households showed little difference (Office for National Statistics 2022d, 2023e).

Chapter 8

164

Figure 8.5 compares the HCI, CPIH and RPI inflation rates for all households over the period January 2015 to August 2023. The RPI produces the highest rates of inflation over the whole period. For the period 2015 to mid-2020, the HCI and CPIH are close, with the HCI being lower than the CPIH until June 2021 when the HCI exceeds the CPIH. 16.0 14.0

HCI, CPIH and RPI Inflation

12.0 10.0

HCI CPIH RPI

8.0 6.0 4.0 2.0 0.0 -2.0 2015

2016

2017

2018

2019

2020

2021

2022

2023

2024

Figure 8.5: Comparison of HCI, CPIH and RPI inflation rates, January 2015 to August 2023. Sources: Office for National Statistics 2023d,e.

8.7.2

A superlative index version

In chapter 3, we considered the way a consumer price index was constructed using price and household expenditure data. The formula used to combine the two types of data is a Lowe index except at the lowest level where weighting data is not available in most cases and unweighted formulas such as the Jevons, Dutot or Carli are used. Although the Lowe formula is almost universally used, it is something of a compromise choice. Most index number specialists would recommend a formula that uses weighting information from the price reference period and the current period. There are mathematical and economic reasons for preferring formulas of this type. Their high status is reflected in their name; such formulas are called superlative. The index formulas proposed

Other price statistics

165

by the economists Irving Fisher and Leo Törnqvist are examples of this class of formula (their mathematical forms are given in appendix A). However, such index formulas are not practical to use for producing inflation figures in a timely manner. When producing an inflation measure, the weighting information takes time to produce; typically it comes from a period of a year or so prior to the price reference period. This makes the Fisher and Törnqvist formulas impractical and is the reason why the Lowe formula is used so widely. While a superlative price index cannot be calculated in as timely a fashion as a price index using the Lowe formula, it can be calculated retrospectively, that is, for a year or two in the past when all the data needed are available. By calculating a retrospective, superlative version of a consumer price index, we can at least see how far the Lowe version differs from it. There are a number of technical complications involved in such a calculation but approximations can be made. A few National Statistical Institutes have calculated such retrospective, superlative index versions of their consumer price indices, including the US and New Zealand. The Office for National Statistics followed suit, publishing a Törnqvist version of the CPI in 2014. The expectation was that a superlative version would grow more slowly than the production Lowe version which is what was found (Clews, Sanderson and Ralph 2014). Figure 8.6 shows the comparison between the CPI calculated with the conventional Lowe formula and the superlative Törnqvist formula. One observation that can be drawn from the superlative versions calculated in New Zealand, the US and the UK, is that the superlative CPI gives a lower value of inflation than the normal CPI. The alternative viewpoint is that the Lowe version of the CPI gives a higher value of inflation than the superlative CPI version. A proposal to use a superlative formula is likely to be very unpopular with everyone who currently benefits from the uprating of benefits, wages, pensions and thresholds using the Lowe version of the CPI.

166 109 108 107

Chapter 8

CPI calculated with the Lowe and Törnqvist formulas

106 105 104 103 102

CPI - Tornqvist

101

CPI - Lowe

100 99 Jan-07 May-07 Sep-07 Jan-08 May-08 Sep-08 Jan-09 May-09 Sep-09 Figure 8.6: Comparison of the CPI calculated with the Lowe and Törnqvist formulas, 2007 to 2009, for locally collected data only. Source: Clews, Sanderson and Ralph 2014.

8.8

Regional price indices

Chapter 4, on the history of inflation measurement, noted that the practicality of producing regional inflation statistics had been discussed several times by advisory committees but each time it was decided not to proceed. However, interest in regional measures has remained. In 2017, the Office for National Statistics asked the University of Southampton to carry out a feasibility study to see whether the current data would support the estimation of regional measures. The study concluded that the use of regional weights derived from the Living Costs and Food Survey would be unsatisfactory; the relatively small volumes of data per region would lead to high levels of volatility. A statistical technique called small area estimation could reduce the problem, though not eliminate it. For now, regional inflation measures aren’t a practical proposition (Dawber et al. 2022).

8.9

Concluding remarks

In this chapter we have looked beyond the all-items measures of consumer price inflation. We have used examples of other price statistics to show

Other price statistics

167

that the family of price statistics encompasses many other measures of price change which help us to understand the complex nature of the economy. The field of price statistics doesn’t stand still. There are new developments like Household Costs Indices and research continues to explore interesting topics such as the possibility of producing regional inflation and alternative approaches using superlative formulas.

References Astin, John and Leyland, Jill. 2015. “Towards a Household Inflation Index”. Royal Statistical Society. Accessed January 6, 2023. https://rss.org.uk/RSS/media/News-and-publications/Publications/ Reports%20and%20guides/Towards_a_Household_Inflation_Index_ May_2015.pdf?ext=.pdf Clews, Gareth., Sanderson, Ria and Ralph, Jeff. 2014. “Calculating a Retrospective Superlative Consumer Prices Index for the UK”. Office for National Statistics. Accessed January 6, 2023. https://webarchive.nationalarchives.gov.uk/ukgwa/20160106041638/ht tps:/ons.gov.uk/ons/guide-method/user-guidance/prices/cpi-andrpi/calculating-a-retrospective-superlative-consumer-prices-index-forthe-uk.pdf Dawber, James., Würz, Nora, Smith, Paul A., Flower, Tanya, Thomas, Heledd, Schmid, Timo and Tzavidis, Nikos. 2022. “Experimental UK regional consumer price inflation with model-based expenditure weights”. Journal of Official Statistics, 38, 213-237. https://doi.org/10.2478/jos-2022-0010 HM Land Registry. 2022. “UKHPI Quality and Methodology”. Accessed January 6, 2023. https://www.gov.uk/government/publications/aboutthe-uk-house-price-index/quality-and-methodology O’Donoghue, Jim. 1998. “Harmonised index of consumer prices: historical estimates”. Economic Trends, 541, 49-54. Accessed January 6, 2023. https://webarchive.nationalarchives.gov.uk/ukgwa/20040723014313/ht tp://www.statistics.gov.uk/cci/article.asp?id=31

168

Chapter 8

O’Donoghue, Jim., Goulding, Louise and Allen, Grahame. 2004. “Consumer Price Inflation Since 1750”. Economic Trends, 604, 38-46. Accessed October 4, 2023. https://escoe-website.s3.amazonaws.com/wp-content/uploads/2020/09/ 20173911/ET-604-Economic-Trends-Mar-2004.pdf O’Neill, Rob and Ralph, Jeff. 2014. “Modelling a Back Series for the Consumer Prices Index, 1950-2011”. Office for National Statistics. Accessed January 6, 2023 https://webarchive.nationalarchives.gov.uk/ukgwa/20160107031523/ht tp:/www.ons.gov.uk/ons/rel/cpi/modelling-a-back-series-for-theconsumer-price-index/1950---2011/index.html O’Neill, Rob., Ralph, Jeff and Smith, Paul A. 2017. Inflation: history and measurement. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-319-64125-6 Office for National Statistics. 2010. “National Statistician’s Review of House Price Statistics”. Accessed January 6, 2023. https://uksa.statisticsauthority.gov.uk/wp-content/uploads/2015/12/ images-national-statisticians-review-of-house-price-statistics_tcm9735564-1.pdf Office for National Statistics. 2014. “Family spending in the UK: calendar year 2014”. Accessed September 25, 2023. https://www.ons.gov.uk/peoplepopulationandcommunity/personalandh ouseholdfinances/incomeandwealth/compendium/familyspending/2015 Office for National Statistics. 2016a. “Clarification of publication arrangements for the Retail Prices Index and related indices: November 2016”. Accessed January 6, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/clar ificationofpublicationarrangementsfortheretailpricesindexandrelatedind ices/november2016 Office for National Statistics. 2016b. “Development of a Single Official House Price Index”. Accessed January 6, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/methodolog ies/developmentofasingleofficialhousepriceindex Office for National Statistics. 2017. “Household Costs Indices Methodology”. Accessed January 6, 2023.

Other price statistics

169

https://www.ons.gov.uk/economy/inflationandpriceindices/methodolog ies/householdcostsindicesmethodology Office for National Statistics. 2019. “Consumer Prices Indices technical Manual, 2019”. Accessed January 5, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/methodolog ies/consumerpricesindicestechnicalmanual2019 Office for National Statistics. 2022a. “CPIH-consistent inflation rate estimates for UK household groups: November to December 2022”. Accessed September 25, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/cpi hconsistentinflationrateestimatesforukhouseholdgroups20052017/nove mbertodecember2022 Office for National Statistics. 2022b. “New estimates of core inflation, UK: 2022”. Accessed January 6, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/articles/new estimatesofcoreinflationuk/2022 Office for National Statistics. 2022c. “Private rental prices development plan, UK: updated February 2022”. Accessed January 6, 2023. https://www.ons.gov.uk/peoplepopulationandcommunity/housing/articl es/privaterentalpricesdevelopmentplan/updatedfebruary2022 Office for National Statistics. 2022d. “Household Costs Indices, UK: fourth preliminary estimates, 2005 to 2021”. Accessed 06 01 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/bulletins/ho useholdcostsindices/householdcostsindicesukfourthpreliminaryestimate s2005to2021 Office for National Statistics. 2023a. “Producer price inflation, UK: April 2023”. Accessed September 26, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/bulletins/pr oducerpriceinflation/april2023 Office for National Statistics. 2023b. “UK House Price Index: July 2023”. Accessed September 26, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/bulletins/ho usepriceindex/july2023 Office for National Statistics. 2023c. “Index of Private Housing Rental Prices, UK: August 2023”. Accessed September 26, 2023.

170

Chapter 8

https://www.ons.gov.uk/economy/inflationandpriceindices/bulletins/in dexofprivatehousingrentalprices/august2023 Office for National Statistics. 2023d. “Consumer price inflation time series, MM23, September 2023”. Accessed September 26, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/datasets/co nsumerpriceindices Office for National Statistics. 2023e. “Household Costs Indices for UK household groups: January 2022 to September 2023”. Accessed December 20, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/bulletins/hou seholdcostsindicesforukhouseholdgroups/january2022toseptember2023

9 CONCLUSIONS In the preceding eight chapters we have covered much ground in relatively few pages. We have explored the conceptual basis behind consumer price inflation, how it is calculated, the history of its measurement and the broader picture provided by other price statistics. As important is how the knowledge of consumer price inflation is applied. We looked at three case studies exploring how child benefit, the basic state pension and nurses’ pay have changed over time and whether they have maintained their value. With all these aspects of our subject in mind we can now stand back and consider the major themes in our story. We do this by posing questions and providing at least partial answers. By taking this approach, we illustrate how one might think about inflation and inflation measurement; in addition, we explore what our knowledge of inflation can and can’t tell us. Much more could be said about the questions we pose, but we need to be concise, so our views have been summarised into just a few pages. Before we proceed, we should make it clear that in the following sections we provide suggestions on how aspects of inflation should be viewed. It is, of course, up to the reader to decide what they think, hopefully informed by the preceding chapters.

9.1

Inflation as a major threat

How much of a threat is inflation to our way of life? Until recently, we had to look to the past to see the damaging effects of high levels of inflation in the UK. However, we saw in chapter 1 that in the period from the start of 2021 to the end of 2022, the rate of inflation, as measured by the CPI, rose from under 1% to over 11% giving us all a more direct experience of its impacts. Through the year 2023, the fall in inflation has proved slower than most people would like. The harmful effects of inflation have been clear with many households struggling to pay for essentials such as power and food, and with governments in the UK and abroad intervening to provide emergency financial support to families and

172

Chapter 9

businesses. There is no doubt that elevated levels of inflation are highly undesirable and we should do everything we can to bring inflation down as quickly as possible and maintain it at modest levels. Maintaining low levels of inflation is a primary objective for governments and central banks. Stable prices, together with economic growth and a low level of unemployment are seen as elements of an ideal economic and social situation. Economic tools have been developed to help to achieve this desirable position and, to a certain extent, are successful. However, there is a limit to what can be achieved in practice; the covid-19 pandemic, Brexit and the war in Ukraine have provided disruptive pressures that overwhelmed our normal stabilising mechanisms. We can think of inflation as if as it were a slumbering giant who we try to pacify as much as possible but occasionally it gets angry and wreaks havoc for a while before returning to sleep. Our economic tools cannot stop occasional periods of high inflation resulting from large scale events brought about by natural or human agency, so we must expect difficult episodes. In crisis situations, governments and central banks can only try to constrain the effects and speed a recovery; opinions vary on the effectiveness of their interventions. The medicine applied to try to control inflation can be unpopular. In the recent episode of high inflation, the Bank of England raised interest rates to stem borrowing which led to mortgage lenders responding with higher mortgage rates, so adding pressure to households already facing widespread price rises. For those households with savings and without a mortgage, interest rate rises can be seen as beneficial. How we think about the response to inflation depends on our circumstances. High levels of inflation are a direct threat to households with low and medium levels of income and many businesses too. While the UK government intervened and provided some financial support in 2022 and 2023, this then added to government borrowing, on which interest is paid

Conclusions

173

and in the future the borrowing will have to be repaid59. We can say that high inflation is a real threat and it has a lasting impact.

9.2

Inflation as eroding the value of money

What of the periods of time where inflation is low? Can we disregard it then? When we do our regular shopping, we may not notice small, gradual price rises in everyday items. When we buy higher priced goods like furniture, we probably won’t remember typical prices when we last bought such items. These price changes tend to occur under our radar. The media will not see much of a story in periods of low inflation and we are unlikely to see much of a response from governments and central banks. Does this mean that we should not be concerned by low levels of inflation? In chapter 1, we saw that while price changes may be relatively small and gradual they still erode the value of money and the effects build up over time. If we are on a fixed level of income in money terms, we will be poorer year by year. However, we also saw in chapter 2 that economists consider low levels of inflation as being healthy for the overall economy. This means that even when economic conditions are good, we can expect inflation to erode household incomes. Chapter 4, on the history of inflation measurement, showed that the idea of compensating for changes in prices was expressed clearly by Joseph Lowe in the 1820s. Despite this clarity of vision, bringing it about in an official manner would take almost a hundred years, with the first application of an official measure of inflation occurring at the start of the First World War. We can think of low levels of inflation as eroding our wealth unless an adjustment is made. Whether we receive income through wages, pensions or benefits, without an adjustment the result will be the same – a lower income in real terms. It is unwise to disregard the effects of a low level of inflation. Instead, we should keep a close eye on whether our income, from wages, pensions or benefits, is being maintained in real terms and encourage the providers to act appropriately.

59

The interest on central government debt for August 2022 was £8.2bn.

174

9.3

Chapter 9

Inflation measures as influential indicators

What can we use inflation measures for? There can be no doubt that they are highly influential. They are a good illustration of the function of official statistics as a whole. Measures of consumer price inflation are important statistics that help decision making by governments, central banks, businesses and individuals. Taken together with statistics on economic growth, public expenditure and the state of the labour market, they are considered essential for the management of a modern economy. Inflation statistics perform other important functions in addition to their role in macroeconomic matters. They are a benchmark against which we can assess of our level of income as prices rise. Without a measure of the level of prices we cannot know the degree to which our purchasing power is changing. Employers, both in the private and public sectors take the rate of inflation into account when making offers of pay settlements. Trades unions make pay claims based on inflation and governments will (usually) adjust benefits, the state pension and public sector pensions in line with the rate of inflation. We should think of inflation measurement as a vital ingredient that feeds into the process for adjusting our income in response to price change.

9.4

Are inflation measures reliable?

Given the importance of inflation measurement, can we be confident that the numbers published each month by the Office for National Statistics are trustworthy and of sufficiently high quality? This is also a question that applies to all other official statistics. Chapter 7 explored how the organisation, methods and procedures around official statistics have evolved to achieve high levels of quality. In addition we asked what is meant by trust in the context of official statistics. Collectively, the governance, methods and procedures used in the production of inflation measures in the UK meet international standards. Two advisory panels provide independent guidance to the National Statistician on the way consumer price measures are constructed, and the statistics have been subject to a variety of reviews. For example, in chapter

Conclusions

175

5, we saw that measures of inflation have been assessed by the Office for Statistics Regulation against the Code of Practice for Official Statistics and an independent review of consumer price statistics was carried out by Paul Johnson, the head of the Institute for Fiscal Studies, and published in 2015. Like all official statistics, a huge effort goes into ensuring that the possibility of error is minimised. Overall, official statistics enjoy a good reputation with their users and the wider public. We noted in chapter 7 how official statistics are almost always accepted by the broad spectrum of users without question, which is mark of the trust placed in them. While we might well agree that this extensive effort together with independent, expert scrutiny ensures that a high level of quality is achieved for inflation statistics, there is a major issue that has to be confronted. If our inflation statistics are so good, why did we end up with the CPI/RPI situation? Doesn’t that represent a failure for official statistics? We look at this question in the next section.

9.5

More than one measure of inflation

Is it appropriate to have two measures for the same important quantity? Doesn’t that lead to confusion, dispute and inappropriate use? It is, of course, acceptable to have more than one measure, if they measure different things. In chapter 8, we explored some of the variants of inflation measures such as core inflation, business price inflation and consumer price inflation for different population sub-groups. Having a range of measures, even if they are for different purposes, does bring the risk of confusion, so each needs to be justified. Having a headline measure, which is the all-items CPI, provides most users with all they need to know, while other measures, such as core inflation can be used by a smaller group of more specialist users. We saw in chapter 5 that after 1997 the UK had two measures of consumer price inflation, the long-standing RPI and the new European HICP. The situation was the same in all EU countries; the European index was designed as an additional measure to run alongside established

176

Chapter 9

domestic measures. It was only in the UK that the two price indices differed by more than a small amount. The typical difference in other countries was around 0.1 percentage points; the difference in the UK was 0.9 percentage points after 2010. The larger difference in the UK arose mainly through the use of the Carli formula in the RPI, a formula not used elsewhere60. This led to the RPI being judged a poor measure of inflation and a gradual switch to the CPI, the name given to the UK version of the HICP. The presence of two competing measures of consumer price inflation is undesirable. If the difference had been about 0.1 percentage points, as in other EU countries, there would have been far fewer concerns. Although the official advice has been to move away from the RPI for all uses, several factors have made this hard. In chapter 5, we saw that for some government gilts and private pensions, the RPI is written explicitly in the specifications as the indexing measure; this makes change difficult. The government’s intention was to switch from the RPI to the CPI in all instances, but they acted slowly for instances where revenue would be reduced by the change. Many private sector uses of the RPI have persisted. We are still in the transition period between the RPI and the CPI nearly 10 years after the official advice to stop using the RPI appeared. It will probably take another 10 years before we see one measure used for all purposes. This episode of competing measures has undoubtedly been damaging for official statistics as a whole. One could argue that such an occurrence is rare and that the circumstances were highly unusual; however, there are lessons that should be learnt61. Perhaps one lesson is to avoid including specific, named measures in specifications of financial instruments and instead to use a more generic expression such as “a measure of the general level of prices” which is the case in other instances.

60 Many countries used the Carli in the past, but they all moved away, except for the UK. 61 One could suggest that the measurement of unemployment in the 1980s was similarly damaging, as we saw in chapter 7.

Conclusions

9.6

177

Inflation statistics to inform debate

Do we now have all the statistical information we need to make appropriately informed economic and political decisions? Does our knowledge of inflation (and other economic statistics) make the government’s choices clear? Do employers, employees, those on benefits and the retired have the information they need to assess what are appropriate levels of pay, benefits and pensions? In chapter 4, we described the situation in the 19th century, where the lack of data on the wealth of the UK relative to competing nations, the state of the labour market and prices inhibited debate and decision making. In parliament, MPs expressed frustration at the inability to decide important questions in the absence of statistical information. Different opinions were expressed, some robustly, but based on limited local knowledge or just political instinct. This motivated calls for better data and led to the allocation of public money for the statistical operations of the Board of Trade to be expanded, though developments were slow at times. These were early steps towards the extensive statistical infrastructure we have today62. We now have a wide range of official statistics to inform our decision making and we can ask whether the vision of those 19th century politicians has been achieved. To a large extent the answer must be yes. We do now have a firm foundation of statistical information on which to base important decisions. However, political debate is as vigorous as ever, with policy choices contested, and difficult decisions on priorities to be made. Debate has not ceased, but the emphasis has shifted from contesting the position to arguing over what to do about it. The statistical base informs choices, but choices still have to be made and whatever is decided is likely to be contested. Extensive though our information base may be, there is much that we don’t know and new data are continually sought. In the UK at the start of 2023, the country saw a wide range of industrial disputes with strikes occurring across the transport, education and health 62 This is sometimes expressed (in rather grand terms) as official statistics providing an epistemic infrastructure through quantification.

178

Chapter 9

sectors with the threat of industrial action in other sectors. The main issue was pay, though proposed revisions to working practices were also a factor. With inflation at, or around 10%, the prospects of a significant erosion of the value of income drove demands for improved pay. The old tension between employees wanting to maintain income in real terms against what is considered affordable by employers was much in evidence. Both the current and historical levels of inflation were important considerations in pay claims and pay offers.

9.7

Inflation and fairness

Can our knowledge of inflation help us decide a fair allocation of resources? Our three case studies in chapter 1 showed different responses to inflation over the austerity period. Nurses’ pay and child benefit were frozen, or limited to 1% annual rises, while the state pension was increased through the triple-lock. Both nurses’ pay and child benefit fell in real terms while the basic state pension exceeded a real terms increase. The choice to treat the basic state pension in a more generous fashion was clearly a political decision to protect pensioners, many of whom are on low incomes. Some parts of the media like to point out that this sector of the population is more likely to turn out to vote at elections than younger citizens. For nurses’ pay, the historical trend is different when considered over a longer term. Between 1988 and 2021, we saw that nurses’ pay had increased by almost 80% above inflation. Clearly, the time period over which we choose to make a comparison is important. If we chose just the period 2010 to 2021, we would say that nurses had seen a real terms pay cut and one could understand their asking for it to be reversed with a greater than inflation pay rise in the current and coming years. Alternatively, if we chose the longer period we would see an overall gain in real terms pay; some commentators might suggest that a temporary real terms pay cut isn’t as bad as it initially seems. The importance of the choice of the comparison period when presenting figures has been known for a long time. Over three hundred years ago, Bishop William Fleetwood noted that selecting data from specific years to

Conclusions

179

influence an outcome was not a good practice. For nurses’ pay, perhaps selecting a long time period for comparison could be considered to be going too far into the past and the more recent experience should be considered as the relevant period. Knowledge of inflation, both the recent experience and over previous years is certainly helpful to inform all parties in pay negotiations but deciding what is fair is very much a matter of judgement. Factors beyond the level of prices enter into consideration. For the public sector, these include the state of overall government finances, shortages of staff and difficulties with recruitment, changes of working practices and comparisons with pay in the private sector. In chapter 1, we described the different ways in which the basic state pension has been adjusted over time, with both prices and wages being used as adjustment measures. Just using prices would maintain the value of the pension in real terms, but if average wages rise faster than inflation, pensioners are being “left behind” relative to working people. The triplelock provides comprehensive protection but is expensive and arguably unfair to younger people, as a House of Lords committee pointed out. It recommended adjustment by average wages, with extra increases in times where prices rise significantly above wages. Knowledge of inflation (and average wages) is highly important in discussing levels of pay, benefits and pensions but many other factors are relevant. Deciding what is fair is hard and highly subjective, even with the assistance of objective statistics. Perhaps one could suggest that a “fair” approach would be to accept lower than inflation pay increases in times when public expenditure is under pressure, but only if above inflation rises happen in better economic times so to at least maintain values over the longer term.

9.8

Inflation measurement and the future

Can we expect improvements in the measurement of inflation in future? Although the methodology behind measuring inflation has been developed over many years, there are always areas where improvements can be

180

Chapter 9

made. The Office for National Statistics runs a development programme for price statistics which includes both small scale changes and more extensive, longer term developments. One of the most significant changes will see the increasing use of price and product data extracted from retailer websites together with retail transaction data. Another area of development is the publication of inflation measures for sub-populations; there is potential for regional variations to be included in future as well. A likely change in the next few years will be the government’s main measure moving from the CPI to the CPIH. The CPIH is the ONS’s primary measure and the intention is that the government will move to the CPIH when it is considered to be sufficiently established. The expectation is that private sector uses will follow in time. Finally, the likelihood is that the RPI in its current form will change in or around 2030. This will mean that those people with private pensions which include the RPI as the indexing mechanism will then see a slower rate of growth. If this change does happen, it will bring to an end a difficult period for inflation measurement.

APPENDIX A: A FEW TECHNICAL DETAILS The previous eight chapters have avoided equations to ensure that those without a mathematical background aren’t disadvantaged. However, as always, a study of the underlying equations brings dividends to a subject, providing greater precision and helping to secure a deeper understanding. This chapter provides a few of the relevant equations for those with the appropriate mathematical background, and we provide references for those motivated to learn more. A modest level of knowledge of mathematics and statistics is assumed, including basic mathematical notation and the concepts of the mean and variance63. A more complete description of the mathematics of price statistics is provided in one of our previous books (Ralph, O’Neill and Winton 2015). In chapter 3, we describe how a consumer price index is constructed, built up from the elementary aggregate level, which estimates price change for specific items. An example for bread is “800g sliced white bread bought from an independent store in the south east of England”. In this case, the definition of the specific item, a large loaf of sliced white bread, also includes the shop type and location. In most cases, weighting information isn’t available for such a detailed specification of item and the price index formulas used will contain only prices. Chapter 5 explained how the choice of formula at the elementary aggregate level became controversial. Above the elementary aggregate level, the formula changes to a weighted one, about which there is much less debate. We start with unweighted formulas for the elementary aggregate level. A.1

Notation

We choose a notation which is used widely. A basket of goods and services contains N commodities. pti is the price of the ith commodity at time period t and qti is the quantity of the ith commodity at time period t. 63 What counts as a “modest level” of mathematics and statistics is, of course, a matter of opinion.

Appendix A: A few technical details

182

We usually calculate price change between a reference period and the current period; we label the two time periods “0” and “t” respectively. A capital “P” represents a price index and “Q” a quantity index64. A price relative is the ratio of the price of an item in time one period to the price in another; we use “R” for the price relative:

R0ti

pti p0i

A.2

The unweighted formulas

The three best-known unweighted formulas are the Dutot, Carli and Jevons; however, there are others and we include the harmonic mean as well. Dutot Price Index The Dutot price index is the ratio of averages of prices in two time periods:

PDutot

1 N ¦ pti N i1 1 N ¦ p0i N i1

Carli Price Index The Carli price index is the average of price relatives:

PCarli

64

1 N

N

pti

¦p i 1

0i

A quantity index measures pure quantity change, with prices fixed.

Uses and Misuses of Inflation and Price Indices

183

Jevons Price Index This Jevons price index is the geometric mean of price relatives, which can also be written as the ratio of geometric means of prices.

§ N pti · ¨– ¸ © i 1 p0i ¹

PJevons

1

N

§ N · ¨ – pti ¸ ©i1 ¹ § N · ¨ – p0i ¸ ©i1 ¹

1

N

1

N

Harmonic price index The harmonic price index is the harmonic mean of price relatives65:

ª1 « «¬ N

PHarmonic A.3

1 § pti · º ¨ ¸ » ¦ i 1 © p0 i ¹ » ¼ N

1

Weighted formulas

For the whole of the aggregation structure apart from the lowest level, weighting information is available, so there is no need to use the Carli, Jevons or Dutot formulas. That doesn’t mean that there isn’t a choice to be made between competing formulas; there are many options for weighted formulas. The weights are expenditure shares, where vti is the expenditure on commodity i at time t. The expenditure on a commodity is the unit price (pit) times the quantity (qit):

wti

vti

pti qti

N

¦v

tj

j

N

(1)

¦p q

tj tj

j 1

65 The harmonic mean is perhaps less well known than arithmetic and geometric means; it is useful for combing rates.

Appendix A: A few technical details

184

Generally, the individual quantities qti are not known; it is the expenditure shares that can be estimated from household surveys (and other sources) and are used in practice. For weighted measures of price change, the two well-known contenders are the Laspeyres and Paasche formulas; both named after the people who first proposed them. They can be written in two forms, one involving prices and quantities and the other in terms of price relatives and weights. Laspeyres price index N

0,t L

P

¦p q ti

0i

i 1 N

¦

N

¦R

0 ti

p0 i q 0 i

i 1

i 1

w0 i

where w0 i

p0 i q0 i n

¦

(2)

p0 j q0 j

j 1

This formula combines price quotes from the two time periods but with the quantities fixed at the base period66. In the “price relative, weight” form, it combines the price relatives with weighting information taken from the base period. It is important to note that this formula is a measure of overall price change in a situation where both prices and quantities may have changed. It is an estimate of pure price change achieved by fixing the quantities at the base period. Laspeyres quantity index We can produce a corresponding measure of overall quantity change in an equivalent way by fixing the prices at the base period and allowing the quantities to change.

66 The base period is a term often used though care is needed. It can mean either the price, weight or index reference periods. Often these time periods are the same.

Uses and Misuses of Inflation and Price Indices N

0,t L

Q

¦p

N

q

0 i ti

i 1 N

¦ R

v

0 ti 0 i

i 1

¦p

q

0i 0i

i 1

N

¦ R

0 ti

N

¦v

w0i

185

where

i 1

0i

i 1

R0ti

qti are quantity relatives. q0i

Paasche price index For the Laspeyres price index, we fixed quantities at the base period. We could instead have fixed quantities or weights at the current period; if we do that we get the Paasche formula: N

0,t P

P

¦p q

ti ti

i 1 N

¦p

q

0 i ti

i 1

1 wti ¦ i 1 R0 ti N

where

wti

pti qti

(3)

N

¦p q tj

tj

j 1

In the “price, quantity” form, it looks very similar to the Laspeyres formula; however, using a little algebra to change to the “price relative, weight” form, it looks a little different. It is a weighted harmonic mean of price relatives. Lowe price index Both the Laspeyres and Paasche formulas have the disadvantage of requiring weights which aren’t available in a timely fashion in practice. It takes a year or so to collect the data to calculate the weights, so a different formula is required. The Lowe formula makes a small but important change to solve this problem.

Appendix A: A few technical details

186

N

0,t PLowe

¦p q

ti bi

i 1 N

bi

¦p

q

§ pti · ¸, © 0i ¹

¦ wc .¨ p i

0 i bi

with wbic

i 1

ªp º pbi qbi « 0i » ¬ pbi ¼ N ªp º pbj qbj « 0 j » ¦ j 1 ¬« pbj ¼»

This formula looks very similar to the Laspeyres formula but it has added flexibility in that the time period “b” can be before the time period “0” and have a different duration, which is just what is needed in practice. The Laspeyres formula can be thought of as a special case of the Lowe. The Lowe formula is used in almost all countries in the calculation of their measures of inflation. A.4

Superlative formulas

Chapter 8 described so-called superlative formulas as having desirable properties though they share the disadvantage of the Laspeyres and Paasche price indices in needing weights that are not available in a timely fashion. However, they can be calculated retrospectively. These formulas use weighting information from both the base and current periods. Fisher price index

PF0,t

PL0,t ˜ PP0,t

It is the geometric mean of the Laspeyres and Paasche Price Indices (equations (2) and (3)). Törnqvist price index:

0,t T

P

n

§ pti · ¨ ¸ – i 1 © p0 i ¹

w0 i  wti 2

n

–R

w0 i  wti 2

0 ti

i 1

The weights are calculated using equation (1) as before; it is a base and current period weighted, geometric mean of price relatives.

Uses and Misuses of Inflation and Price Indices

A.5

187

The value of money and constant prices

In the previous chapters, we noted the important fact that the value of money changes over time. We may have a data series of annual incomes for the period of 2010 to 2022, where each data point is the income in pounds at the value of money in that particular year. This data series will include both changes in income and the change in the value of money. In this case, we say that the data series is presented in money or nominal terms. It is often useful to remove the effect of inflation to see just the underlying variation; we call this presenting the data series in real terms, or at constant prices, which means at a constant value of money. How do we do this? We’ll use the example of annual income for the period of 2010 to 2022 where the series is in nominal or money terms. If we wanted to adjust the series so that the data for each year is at the (constant) value of money in 2015, we would use a price index to remove the effect of inflation. Using the standard notation, our time period “0” represents the year 2010 and our variable time period we label “t”, so t runs from 2010 to 2022. We want to adjust the data series to be wholly in the value of money from 2015, which we label as a reference period “rc”. Our constant price income series m is given by:

mt , rc

P 0, rc mt . 0,t P

P 2010, 2015 mt . 2010,t P

(4)

It is often the case that we want to create a constant price series at the value of money in the base year, so “rc=0” in our notation. Equation (4) then simplifies:

mt ,0

mt .

P 0,0 P 0,t

mt .

1 P

2010,t

The price index we choose will depend on the data series we are working with. In the case of income, it would be appropriate to use the all-items

Appendix A: A few technical details

188

CPI. If instead we were aiming to remove the effect of price change from military expenditure, then a price index relevant to military goods would be appropriate. Ideally, the scope of the price index should match the data series as closely as possible. A.6

Relationships between the elementary aggregate formulas

We noted in chapter 5 that calculating overall price change for a collection of items using different elementary aggregate formulas leads to different results. In some instances the differences are small but in others they can be relatively large. Mathematical analysis helps to understand what drives the differences. For the difference between the Jevons and the Carli we can use a Taylor series expansion of the Jevons to show that: 0,t PJevons

1 V2 0,t PCarli  . R 2 R0,t

1 V2 0,t PCarli  . 0,Rt 2 PCarli

(5)

Where the terms:

R0,t

1 N ¦ R0ti N i1

V R2

1 N §1 N · R0ti  ¨ ¦ R0ti ¸  ¦ N i1 N © i1 ¹

and 2

2

2

1 N ¦  R0ti  R0ti N i1



2

These are the mean and variance of the price relatives. Equation (5) says that the difference between the Jevons and the Carli is related to the variance of the price relatives, so the greater the degree of heterogeneity in the price changes of items in an elementary aggregate, the greater the difference will be (O’Neill, Ralph and Smith 2017, 252-255). If we now turn to the difference between the Jevons and the Dutot, we first define an error term which expresses an individual price quote as a deviation from the average price for an item in an elementary aggregate at a time period:

Uses and Misuses of Inflation and Price Indices

PiW

PiW (1  eiW )

where W

189

0, t

With some mathematical manipulation, the following relationship can be derived:





0,t 0, t ª PJevons 1  V e20  V e2t º | PDutot ¬ ¼

This tells us that the Jevons and Dutot difference (ratio) is related to the difference in the variances of the price quotes in the two time periods. In almost all cases, these will not differ greatly, so the Jevons is usually close to the Dutot (O’Neill, Ralph and Smith 2017, 252-255).

References O’Neill, Rob., Ralph, Jeff and Smith, Paul A. 2017. Inflation: history and measurement. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-319-64125-6 Ralph, Jeff., O’Neill, Rob and Smith, Paul A. 2020. The Retail Prices Index – A Short History. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-030-46563-6

APPENDIX B: GLOSSARY OF TERMS Note: the formulas for the indices in the glossary can be found in Appendix A. Aggregate (index) Aggregation

Base period

Basket

Characteristics Carli price (quantity) index Commodity Constant price series

Consumption goods and services

Cost of goods index

A price (quantity) index for a collection of consumer products. The combination of price or quantity indices for smaller groups of consumer products to give an index for a larger group. A reference time period. It can be the index reference period, the price reference period or the weight reference period. A collection of goods and services and their associated quantities. It can be thought of as a very large “shopping basket” containing services as well as physical goods. A product’s attributes by which it can be identified; they usually influence its price. An unweighted price (quantity) index; the arithmetic mean of price (quantity) relatives. A general term for a good or service. A data series of economic indicators such as prices or wages at a constant value of money pertaining to a specified time period. Goods and services that are purchased and consumed by consumers. They are not used to create other goods and services. In the National Accounts, they are known as “final consumption goods and services”. The ratio of the price of a representative basket of goods and services from two time periods.

Uses and Misuses of Inflation and Price Indices

Cost of living index

Coverage

Current, or comparison period Current prices Deflation Dutot price (quantity) index Elementary aggregate

Expenditure weights General level of prices Indexation Index reference period Item Jevons price (quantity) index Laspeyres price (quantity) index

191

The ratio of the minimum expenditure for a consumer to maintain their utility or satisfaction when responding to changes in prices in two time periods. The customer’s tastes are assumed to be unchanged and they seek to maximise their utility. The extent to which a basket of goods and services includes representatives of all the items which incur expenditure. The time period that we are comparing to the base period. The prices that pertain to the current time period. The process of removing price changes from a time series of financial values. An unweighted price (quantity) index; the ratio of the arithmetic average of prices (quantities) for two time periods. A basic constituent of a consumer price index. It is created from price and sometimes weighting data though in many cases weights aren’t available just price information is used. The fraction of consumer expenditure on a collection of goods and services. An average measure of the overall prices for a set of goods and services; usually relative to a reference period. The adjustment of an amount of money by an index; usually a price or wage index. The period of time where the value of the index is set to equal 100. General name for a good or a service. An unweighted price (quantity) index; the geometric mean of price (quantity) relatives. A weighted price (quantity) index; it is usually calculated as the sum of price (quantity) relatives weighted by the expenditure share from the base

192

Lowe price (quantity) index

Nominal terms

Owner-occupiers’ housing

Paasche price (quantity) index

Price reference period Price index

Price relative Product Purchasing power Quality adjustment

Appendix B: Glossary of terms

period. A weighted price (quantity) index; it is calculated as the sum of price (quantity) relatives weighted by the expenditure share from a generic time period which can be different to the price reference period. The Laspeyres and Paasche indices are special cases of the Lowe index. A data series of economic indicators such as prices or wages which include the effects of inflation. Housing owned by the occupiers, including those with a mortgage. The costs are often included in a price index by using a proxy: the cost to rent an equivalent house. A weighted price (quantity) index; it is usually calculated as a weighted harmonic mean of price (quantity) relatives. The weights are taken from the current period. The time period to which the current time period is compared. A means of quantifying the change in the general level of prices between two time periods. It can be used for converting the value of money from one time period to another. The ratio of the current and reference (unit) prices of an item. An alternative term for a good or service. The goods and services that can be bought with a fixed sum of money at a defined time period. The process of adjusting the change in the price of a commodity where an important characteristic has changed. It enables a constant quality comparison to be made.

Uses and Misuses of Inflation and Price Indices

Real terms

Reference period

Reference population

Representative product Scope Substitution

Superlative price index

Weights Weight reference period

193

A data series of economic indicators such as prices or wages with the value of money held constant; it excludes the effects of inflation is said to be in real terms. A time period, typically a month or a year. There are three types of reference period. The index reference period is the time period where the index is set to be 100. The price reference period is the time period to which prices are compared. The weight reference period is the time period from which weighting information is taken. The collection of households (or businesses) whose purchases are of interest for creating an index. A product included in the basket; its price behaviour is often used to represent the price change of many similar products. The collection of goods and services that form the target of the index. See: coverage. The act of a consumer changing a purchasing decision from one product to another in response to a relative price change. Alternatively, a change in the quantities of related items bought in the face of price changes. A price index containing weighting information from both current and reference time periods. They have desirable properties and can approximate a cost of living index. See expenditure weights. The time period, usually a year, from which weighting information is taken.

APPENDIX C: FURTHER READING The references at the end of each chapter provide an extensive list of books, papers and manuals for readers who want to follow up on specific topics. In this brief section, we highlight a small number of books and manuals from the references that we feel are particularly useful for the reader wanting to explore the subject in more depth.

Books Our two previous books explore the history of inflation measurement in more depth. The first provides a long term history together with a lot more technical detail. The second focusses on the Retail Prices Index and its rise and fall as a measure of inflation. O’Neill, Rob., Ralph, Jeff and Smith, Paul. A. 2017. Inflation: history and measurement. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-319-64125-6 Ralph, Jeff., O’Neill, Rob and Smith, Paul A. 2020. The Retail Prices Index – A Short History. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-030-46563-6 For more information on the calculations and formulas used in estimating inflation, our first book introduces index numbers as a general statistical topic and its application to price statistics. Although it is a mathematical book, it requires only relatively basic competencies in mathematics. It contains exercises with answers and additional on-line resources including code in SAS® and R. Ralph, Jeff., O’Neill, Rob and Winton, Joe. (2015). A Practical Introduction to Index Numbers. John Wiley and Sons. Chichester UK. https://doi.org/10.1002/9781118977781

Uses and Misuses of Inflation and Price Indices

195

A very comprehensive, but much more technical treatment of index numbers and price statistics is provided by: Balk, Bert. M. 2008. Price and Quantity Index Numbers. Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CBO9780511720758 There are different approaches to writing about inflation. A recent book from Stephen D. King examines the economic background to inflation and what causes it; it identifies lessons from previous instances of inflation. King, Stephen. D. 2023. We need to talk about inflation. Fourteen urgent lessons from the last 2,000 years. Yale University Press, New Haven and London. Thomas A. Stapleford’s book from 2009 provides a political history of economic statistics, including inflation, in the US covering the period 1880-2000. Stapleford, T. A. 2009. The cost of living in America, a political history of economic statistics, 1880-2000. Cambridge University Press, Cambridge.

Manuals The Office for National Statistics publishes a manual on the construction on the Consumer Prices Index which is very readable and provides a good, approachable description of how it is calculated. Office for National Statistics. 2019. Consumer Prices Indices Technical Manual, 2019. Accessed January 2, 2023. https://www.ons.gov.uk/economy/inflationandpriceindices/method ologies/consumerpricesindicestechnicalmanual2019 For a more technical and very comprehensive treatment of how to produce a consumer price index, the international manual is an excellent guide.

196

Appendix C: Further reading

International Monetary Fund et al. 2020. Consumer Price Index Manual: Concepts and Methods. Accessed October 30, 2023. https://www.imf.org/en/Data/Statistics/cpi-manual

Reviews The review of UK consumer price indices carried out by Paul Johnson, the head of the Institute for Fiscal Studies, provides an excellent and very clear explanation of the challenges and issues behind inflation measurement. Johnson, Paul. 2015. “UK consumer price statistics: a review”. UK Statistics Authority. Accessed January 5, 2023. https://uksa.statisticsauthority.gov.uk/reports-andcorrespondence/reviews/uk-consumer-price-statistics-a-review/

The House of Lords Economic Affairs Committee inquiry into inflation measures provides an excellent overview of the recent issues. UK Parliament. 2019. “Measuring Inflation”. House of Lords Economic Affairs Committee, 5th Report of Session 2017-2019. Accessed January 5, 2023. https://publications.parliament.uk/pa/ld201719/ldselect/ldeconaf/246/ 246.pdf

INDEX Arithmetic average, 57, 80, 81, 83 Austerity, 15–19, 22, 34, 90, 178 Basket, 38, 42, 43, 51, 60, 64, 71, 95, 114–20 Board of Trade, 50, 67, 68, 120, 177 Boskin Commission, 82, 105 Carli index, 80–84, 86, 88, 95, 96, 95–99, 104–6, 182, 188–89 Child benefit, 12, 13, 16, 17, 178 Clothing, 39, 54, 55, 94, 93–95, 116, 121 Code of Practice for Official Statistics, 96, 131, 135, 136, 144, 146, 175 COICOP, 57 Consultations Consumer Price Statistics Review, 98 CPI indexed gilts, 99 Improving the RPI, 95, 96, 106 Owner occupiers’ housing, 97 Reform of the RPI, 103 Use of CPI in private sector pensions, 99 Consumer Prices Advisory Committee, 86, 88, 95, 97 Consumer Prices Index (CPI) definition, 4, 14, 55, 56, 85 examples, 5, 6, 33 historic, 157 move to, 86, 89, 90, 91 usage, 15, 16, 19, 22, 23, 24, 32 variants, 155

Consumer Prices Index including owner occupiers’ housing (CPIH) definition, 14, 97 historic series, 157 household categories, 153 usage, 98, 100, 152, 164, 180 variants, 155 Cost of goods approach, 42, 43, 50, 85, 105, 163 Cost of Living Advisory Committee, 71, 72, 73 Cost of living approach, 42, 43, 85, 86, 163 Cost of Living Index (working class), 68–70, 115, 120 Dutot index, 80–84, 88, 182, 188 Electronic goods, 33, 38, 40, 87 Elementary aggregates, 57, 58, 80– 84, 88, 94, 104, 125, 181, 188 Fiscal drag. See Stealth tax Formula effect, 90, 93–97 General level of prices definition, 4, 5, 6, 8, 10, 31, 32 economic theory, 29 measurement, 38, 65 usage, 12, 14, 25, 30, 32, 34, 44, 93, 152, 174 Generic Statistical Business Process Model, 141 Geometric average, 57, 58, 81, 82, 83 Government bonds, 36, 107, 130 Government Statistical Service, 134

198 Harmonised Index of Consumer Prices (HICP), 83, 78–86, 104, 105, 106, 175 Hedonics, 40, 53, 54, 87 Household Costs Indices (HCI), 56, 163–64 Household expenditure definition, 51, 55 inclusion in basket, 116 shares, 120–22 survey, 55, 69–72 Indexation application of, 10 benefits, 12 definition, 10, 34 examples, 68 Index-linked gilts, 36, 86, 87, 96, 104, 107 Inflation adjustment by, 10 data series, 11 definition, 5, 6 examples, 6, 7, 8 household groups, 43, 44, 56, 154 measurement, 38 preferred level, 9, 30, 31 target, 35, 74, 84, 87, 91 Intergenerational fairness, 20, 179 Jevons index, 67, 83, 80–84, 86, 94, 95, 96, 97, 104, 105, 156, 164, 181–83, 188–89 Level of prices. See General level of prices Lowe index, 49, 58, 66 Money supply, 29–31 Nurses’ pay, 20, 21, 22, 24, 34, 178, 179 Owner occupiers’ housing, 40, 73, 96, 159–61

Index Pensions private sector, 93, 96, 98, 99, 101, 104, 106 public sector, 93, 99 state, 17, 19, 90, 99, 178 Price collection current, 50, 51, 52, 114, 123–25 historic, 65, 64–68 Price index adjustment, 60 construction, 49, 50, 52, 57 definition, 4, 33 Pure price index. See Cost of goods approach Quality adjustment, 52, 53, 52–55 Quality reviews Consumer Price Statistics (the Johnson Review), 98, 146, 175, 196 National Statistics Quality Review, 146 OSR assessment, 145 regular quality review, 146 Rate of inflation. See Inflation Regional price indices, 72, 166, 180 Retail Prices Index (RPI) definition, 14, 56, 79, 86 examples, 8, 15 governance, 87, 88 interim measure, 72 RPIJ, 96, 156 usage, 23, 24, 82, 87 uses, long-term, 99, 100, 102 variants, 74, 84, 105, 156 Retail Prices Index Advisory Committee, 74, 81, 86, 88 Royal Statistical Society, 67, 98, 101, 102, 135, 136, 143 Scientific method, 139, 140, 143

Uses and Misuses of Inflation and Price Indices Statistics and Registration Service Act 2007, 101, 131, 136 Stealth tax, 13, 25 Superlative index, 58, 98, 164–66, 186

Triple-lock, 18, 25, 90, 178, 179 Uprating. See indexation Weights. See Household expenditure shares

199