Are political journalists always wrong?

I've mostly tuned out of reading the news. But, once in a while, a headline will be shared on social media and I'll momentarily stare into the abyss.

There has recently been a change in UK Prime Minister. I neither know nor care whether that's a good thing. What I do know is that Senior Political Correspondents collectively shat their pants with excitement at being able to breathlessly report any half-baked gossip which would drive clicks.

I saw "Rumours suggest so-and-so will be appointed…", and "Supporters say that A. N. Other will be fired from…", and "Insiders reveal urgent change of policy around…", and so on.

They were all bollocks.

I spent a dull afternoon going through a couple of dozen "sources say" headlines. About 85% were completely wrong. That person didn't become Minister for Administrative Affairs, that other one wasn't fired, the much ballyhooed policy announcement never came.

A few weeks ago, The Economist ran a searing exposé on the accuracy of… The Economist. In an article called "Is The Economist always wrong?" they used a Large Langue Model to test the accuracy of their forecasts. It made for fascinating reading:

We declared smartphones the ­future of computing in 2002, said that something like streaming would devour DVDs in 2008, and cautioned in 2011 that a flood of Chinese-made cars would wash over the West. Still, our techno-optimism occasionally got ahead of itself. Distributed electric grids (boosted by The Economist in 2000), cheap bioethanol (2003), open ­standards for social media (2008) and augmented ­reality (2016) have yet to bring about the breakthroughs we prophesied.

I'd like someone to run the same process on the political rumours reported by the mainstream press. Just how accurate are the tips journalists receive?

Now, obviously, perhaps the rumour was true at the time - but circumstances changed. Perhaps journalists are hanging around in bars and listening to SpAds boast about what the boss has just said. Perhaps they're just making shit up.

It would be fascinating to see if any political journalist has a better than 50% ratio of hits to misses. I strongly suspect that they have an incentive to post the most outrageous rumour rather than the most accurate one.

So, if you have access to a couple of years of newspaper archives, a local AI model which won't burn the planet, and want to cause chaos - please create a leaderboard of the most- and least-accurate political rumour-mongers in the UK.