In March 2025, the Daily Telegraph published a table claiming to show that foreign nationals from certain countries had exceptionally high conviction rates. But in many cases, the sample sizes were so small as to render the data unusable. The Telegraph published anyway – and IPSO let the sample-size problem pass.

“The question of an appropriate sample size was a matter of editorial judgement”

The Telegraph, as recorded in a recent IPSO ruling (000324-26, paragraph 9).

No one would dispute that some decisions are purely a matter of editorial judgement.

But sample size? Really?

Didn’t we all learn at school that some samples are just too small?

The data behind the Telegraph’s analysis recorded around 290,000 convictions a year in England and Wales. So who would think that relying on just two convictions to support an analysis of offender nationality is a good idea?

Who would use this flimsy data to negatively label a small nationality living in the UK, estimated at only 500 people, and prominently place it in a league table of the top five worst offenders for one category of offence?

The Telegraph did exactly this for Chad nationals, ranked 4th out of 200 plus nationalities, in an article published in March 2025, and in two later articles.

If the Telegraph’s editorial judgement considers this an appropriate sample size to rank nationalities in this way, then the quality of editorial judgement is sadly lacking.

And it’s not just Chad.

There’s a table in the article which attempts to rank all 200 plus nationalities in the UK. In the same category, Armenia, Cuba, Djibouti, Gabon and Nicaragua, nationalities all with small populations (each estimated at 1,000), are ranked between 30th and 40th, with rates several times that of UK nationals, but each on the basis of just one conviction.

Judgements like this appear to fall short of the Telegraph’s commitment to the IPSO Editors’ Code, in which the publication has pledged to take care not to publish misleading information.

It’s absurd to suggest that editors are free to judge an appropriate sample size without reference to well established statistical methods that any newspaper can use.

If it wasn’t obvious to the Telegraph that the sample size was too small, they should have asked a statistician to check.

The Telegraph ranked Chad as 4th out of 200. But a standard statistical calculation gives a 95% confidence interval for Chad’s ranking of anywhere between 1st and 60th.

That’s a very wide range, even if the population is accurate.

But that’s a big ‘if’.

It would have been difficult, if not impossible, to know the size of small populations living in the UK with any reasonable degree of accuracy, especially in a time of unusually high immigration. The population figures are likely to be the biggest source of uncertainty in this sloppy data analysis.

I made a complaint to IPSO, and the IPSO Complaints Committee did accept that the population used for Afghan nationals may have severely underestimated their numbers, making the rate appear much higher than it should be (paragraphs 11 and 13 of their ruling).

But this story is not just about the Telegraph’s absurd claim, but also about the surprisingly weak response from the complaints body, which has a responsibility to maintain standards for its member publications.

The IPSO Complaints Committee did not accept my complaint about the sample size for Chad. They took the view that if a reader could find both the number of convictions and the population size, that was enough: they could judge for themselves.

But it was far from easy to find either of these figures.

The Committee said that a map “made clear the number of convictions” (paragraph 16).

Saying the map “made clear the number of convictions” fails to mention that the reader had to click on each country on the map in turn, to see the number of convictions for each nationality. Are readers really expected to click on every feature that might be interactive in order to judge the reliability of data shown earlier in the article?

The Committee goes on to say that the number of convictions and the rate allow the reader to “understand the population size this calculation was based on, and draw their own conclusions as to the reliability of the data”.

But to ‘understand the population size’ the reader would then have to divide the number of convictions by the rate and then multiply by 10,000 to work out each population.

Is this really what the Committee imagines readers typically do?

It seems an article is only misleading if it would mislead a reader who treats every news story like a forensic investigation, perhaps assisted by calculators or spreadsheets.

But the Committee’s view that everything is fine as long as readers can see the sample size is a flawed assumption.

Two convictions from a population of 500 is obviously too small, but many sample sizes that are statistically speaking too small to state a claim with confidence may not look so obviously small to the reader.

Perhaps the Committee expects the reader to do their own calculation of a 95% confidence interval to check?

It’s already clear that the Committee expects that readers should be comfortable crunching numbers.

Such calculations should be the publication’s job, as part of its duty not to mislead. The Telegraph failed to do this for the article in question.

We are left to wonder whether the Telegraph’s position on sample size being ‘a matter for editorial judgement’ applies just to the articles under complaint, or describes their general approach to all quantitative data.

Because if it’s the latter, that’s all the reader needs to know to draw their own conclusions about the reliability of the publication as a whole.

Lastly, what would other newspapers say if readers asked them these questions: how do they decide whether a sample is too small to support a claim, and how do they handle figures, such as small populations, that carry large uncertainty?

If we don’t ask, we won’t know.

And readers may continue to be misled about issues that matter, by editorial judgements that ignore good statistical practice, misinform the electorate and divide our society.

Sources

IPSO ruling 000324-26, 29 October 2026

Telegraph article, 10 March 2025

Jane Cullen is a retired teacher who taught at the Open University and in sixth form colleges, and now campaigns against misleading statistics in the press.