I've mostly tuned out of reading the news. But, once in a while, a headline will be shared on social media and I'll momentarily stare into the abyss.
There has recently been a change in UK Prime Minister. I neither know nor care whether that's a good thing. What I do know is that Senior Political Correspondents collectively shat their pants with excitement at being able to breathlessly report any half-baked gossip which would drive clicks.
I saw "Rumours suggest so-and-so will be appointed…", and "Supporters say that A. N. Other will be fired from…", and "Insiders reveal urgent change of policy around…", and so on.
They were all bollocks.
I spent a dull afternoon going through a couple of dozen "sources say" headlines. About 85% were completely wrong. That person didn't become Minister for Administrative Affairs, that other one wasn't fired, the much ballyhooed policy announcement never came.
A few weeks ago, The Economist ran a searing exposé on the accuracy of… The Economist. In an article called "Is The Economist always wrong?" they used a Large Langue Model to test the accuracy of their forecasts. It made for fascinating reading:
We declared smartphones the future of computing in 2002, said that something like streaming would devour DVDs in 2008, and cautioned in 2011 that a flood of Chinese-made cars would wash over the West. Still, our techno-optimism occasionally got ahead of itself. Distributed electric grids (boosted by The Economist in 2000), cheap bioethanol (2003), open standards for social media (2008) and augmented reality (2016) have yet to bring about the breakthroughs we prophesied.
I'd like someone to run the same process on the political rumours reported by the mainstream press. Just how accurate are the tips journalists receive?
Now, obviously, perhaps the rumour was true at the time - but circumstances changed. Perhaps journalists are hanging around in bars and listening to SpAds boast about what the boss has just said. Perhaps they're just making shit up.
It would be fascinating to see if any political journalist has a better than 50% ratio of hits to misses. I strongly suspect that they have an incentive to post the most outrageous rumour rather than the most accurate one.
So, if you have access to a couple of years of newspaper archives, a local AI model which won't burn the planet, and want to cause chaos - please create a leaderboard of the most- and least-accurate political rumour-mongers in the UK.
5 thoughts on “Are political journalists always wrong?”
@blog About one paragraph of a politician's three page speech might be actual news, on a good day. But political commentators need to fill an hour of airtime and page after page of newspapers with something. I think that inevitably leads to very low standards for whether a rumour is considered credible enough to publish.
| Reply to original comment on en.osm.town
@Edent Twitter made punditry more pronounced in that it became easy for journalists to throw something out, get traction with it, and drive discussion in the 'right' direction. If it didn't work, they could delete it, the idea of 'tomorrow's chip wrapper' taken to an extreme. It's all part of the process now.
| Reply to original comment on gotosocial.grnwds.uk
@Edent this has been annoying me all month. So many totally unsubstantiated rumours stated as planned policies. Often because one policy wonk who once did some advising to Burnham has suggested that the gov should do something.
| Reply to original comment on hachyderm.io
Alex B
A further explanatory theory might be that some names and roles were being briefed to privileged individuals to test if they were leaking, as a https://en.wikipedia.org/wiki/Canary_trap
Any political journalists reporting that information were not "wrong", as the information was authentic, but deliberately inaccurate. Ideally, perhaps, anything they report should be verified by more than one source, but sometimes that is not possible.
This was too good a challenge to leave alone, so I decided to pick it up. I ran a deliberately narrow pilot: I wanted to look at the reporting about whom Andy Burnham would appoint as Chancellor and then compared the claims with the eventual appointment. I used ChatGPT Work (GPT-5.6 Sol, medium reasoning) to help locate, classify and assess the contemporary reports. The results are here: https://faultlinewatch.com/political-rumour-accuracy-pilot/ Part of the reason your challenge appealed to me is that I’m actually fascinated myself by how claims evolve and how they eventually collide with evidence. In fact, I’ve built a website around precisely that idea: the Faultline Observatory, where I track scientific and technological claims over time. Your proposal applies much the same problem to political reporting: distinguishing what insiders want, what journalists think is likely, what is presented as decided—and what actually happens. What I have run for you is only a small pilot, not yet the journalist leaderboard you proposed. But the result is quite revealing...
More comments on Mastodon.