What if I told you that seven million households in this country ‘can’t afford a toothbrush’? It wouldn’t pass the smell test. But what if I told you the figure came from a highly respected research organisation?
Treating contested statistics as national emergencies is a poor basis for making public policy
The Joseph Rowntree Foundation has concluded that 7.4 million ‘low-income families’ have been unable to afford essential items at some point during the past six months. It says this is the highest figure since it began conducting the survey in 2021. Alfie Stirling, the economist behind the research, has described the findings as a ‘national emergency’. But is that what the evidence shows?
It has certainly been presented that way. When Nick Ferrari, the LBC presenter, asked the economist, ‘7.4 million households in this country, Mr Stirling, can’t afford a 50p toothbrush?’, he agreed, replying: ‘That is precisely why I think it should be an emergency, and actually the government is recognising that with policies they’re trying to put out.’
When Ferrari questioned whether the number passed the smell test, Stirling responded: ‘Are you suggesting these numbers aren’t right?’ That made me look at the research for myself. The methodology used to arrive at the figure is far weaker than the headlines suggest.
First, the definition of ‘low income’ is remarkably broad: the bottom 40 per cent of all UK households. That amounts to roughly 11.7 to 12 million households. Most readers would probably assume that ‘low income’ refers to a much smaller group of people at the sharpest end of financial hardship, rather than more than a third of households in the country. Why the bottom 40 per cent is the appropriate threshold for this particular analysis is far less obvious than the headlines imply.
The bigger issue, however, is not whether the threshold should be 40 per cent or 20 per cent. It is how poverty is measured. If everyone’s income doubled overnight, the number classified as ‘low income’ under a purely relative measure could remain almost unchanged. Poverty should not be judged solely by where people sit in the income distribution. We should also ask whether they can afford the necessities of everyday life and enjoy a reasonable standard of living.
Secondly, the figures are based not on official government data, such as the Office for National Statistics’ Family Resources Survey, but on an opt-in online panel of 4,121 respondents. The Joseph Rowntree Foundation then extrapolates the survey results, applying the proportion of respondents who reported going without essentials to the estimated number of households in the bottom 40 per cent of the income distribution.
Like many online surveys, the research relies on self-reported income rather than independently verified administrative data. That is not unusual, but it inevitably introduces an additional source of uncertainty. The larger problem, however, is how the findings are presented once they leave the report. Large headline numbers make for compelling news stories, but they can easily create a misleading impression.
Even if we assume the methodology is sound, the definition of being ‘unable to afford essentials’ is itself surprisingly broad. Respondents are asked about a list of roughly a dozen items, including food, heating, clothing and toiletries. A household is counted as going without essentials if someone reports going without ‘often or sometimes’ during the previous six months for any one of those items.
That leaves considerable room for interpretation. Even relatively affluent households might occasionally choose to wear an extra jumper rather than switch on the heating during a period of exceptionally high energy prices. That could still be recorded as having gone without an essential. Combine that definition with a ‘low-income’ population of almost 12 million households, and it is not surprising that the resulting headline figure is so large.
Yet anyone watching the interviews or reading much of the media coverage would come away with a rather different impression: that 7.4 million households literally cannot afford a 50p toothbrush today. That is not what the research itself demonstrates. It is an interpretation of the findings, and an exaggerated one at that.
To return to Stirling’s question: ‘Are you suggesting these numbers aren’t right?’ Yes. Respected organisations are perfectly capable of producing research that deserves critical scrutiny. Questioning a statistic is not the same thing as denying that poverty exists. But presenting a highly contestable figure as established fact, and then declaring a national emergency on its basis, does little to improve the public debate. Instead, it encourages politicians to respond to headlines rather than evidence.
F. A. Hayek, the Nobel Prize-winning economist, once wrote: ‘Emergencies have always been the pretext on which the safeguards of individual liberty have been eroded.’ In the context of poverty research, they can also become the pretext for poorly designed welfare policy. Big numbers generate big headlines. But treating contested statistics as national emergencies is a poor basis for making public policy.
Comments