How bad is link-rot on my blog?

I read this brilliant blog post by Wouter Groeneveld looking at how many dead links there were on his blog. I thought I'd try something similar.

What is a broken link?

Every day, I look at the On This Day page of my blog and look at that day's historic posts. I click on every link to see if it is still working. If it isn't, I have a few options.

1. If the site is working, but the content has moved, I change the link to point at the content.
2. If the site is dead, or the content isn't there, I stick the URl into the WayBack Machine and pick the archive closest to the publication date.
3. If the archive doesn't have it, I try to find a link from around the same date and point it to that.
4. If not, I remove the link.

Why don't I just cURL the URl and see if it responds with a 2xx HTTP code?

I used to use an automated checker to test my links. But spam sites lie. Lots of the dead sites have been taken over by scammers, spammers, and AI grifters. They return an HTTP OK and then serve up advertising. Yeuch! So now I spend each morning Tending To My Digital Garden and manually checking the links.

So, let's scan through every link within my blog posts and see which ones have been archived.

Results

Here's a stacked chart showing blog posts from 2008 until today:

And, if you prefer percentages, here's another chart:

Rather unsurprisingly, the older posts have the most link-rot.

Here's the data if you want to play with it yourself:

 CSVYear,Total,Archives

2008,159,27

2009,1335,156

2010,1166,91

2011,1130,85

2012,1392,98

2013,1889,84

2014,1858,35

2015,3016,43

2016,2274,37

2017,1309,17

2018,1561,25

2019,1580,18

2020,3362,33

2021,3621,40

2022,2904,22

2023,2358,15

2024,2136,19

Deficiencies

There will be some links that I haven't spotted. There will be dead links in review metadata and in comments. My code might not have accounted for some weird edge cases in the HTML.

But, generally, it should be a roughly accurate assessment.

Generating the data

Using the WP CLI, this can be run with wp eval-file archive.php:

 PHP<?php

//  Load the WordPress environment

require_once( "wp-load.php" );



//  Loop through the desired years

for ( $year = 2008; $year <= 2024; $year++ ) {

        //  Initialise the counters

        $total_links   = 0;

        $archive_links = 0;



        //  Get all the published posts for that year

        $posts = get_posts( [

            "date_query" => [

                [ "year" => $year, ],

            ],

            "post_type"   => "post",

            "post_status" => "publish",

            "numberposts" => -1,

        ] );



        //  Loop through each post

        foreach ( $posts as $post ) {

            $ID = $post->ID;



            //  Render the content of the post

            $content = apply_filters( "the_content", get_the_content( null, false, $ID ) );



            //  Count total occurrences of "<a href="

            preg_match_all( '/<a href=/i', $content, $matches );

            $total_links += count( $matches[0] );



            //  Count total occurrences of "<a href=" which contain "archive.org"

            preg_match_all( '/<a href=["\'][^"\']*archive.org/i', $content, $archive_matches );

            $archive_links += count( $archive_matches[0] );

        }



    //  Display the results as CSV

    echo "{$year},";

    echo "{$total_links},";

    echo "{$archive_links}\n";

}