As a little thank-you for being a member of RSS Club I thought I'd show you some trailers for upcoming blog posts.

I use the brilliant Editorial Calendar Plugin to organise all my scheduled blog posts. Here's what you can expect over the next month:

I tend to write in bursts - rather than once per day - and then spread the posts out. As I'm going on a long break soon, I want to make sure there are plenty of posts in the queue. There are also a bunch of posts scheduled over the next few years on specific dates.

Of course, if I unexpectedly die, I guess they be posthumous…

I'm also working on what will be (I hope) a reasonably big political story. I'm under embargo until my media partner publishes it - but I hope it'll go live in the early hours of Tuesday. Stay tuned 😊

If there's something you'd like to see me write about, please drop me a comment via your favourite method.

I recently read Susam's blog post where they said that "most of the traffic to my personal website still comes from web feeds" - I wondered if that was true for my site.

I've been writing this blog for a while. I've never much bothered with "aggressive" SEO - I have a fairly semantic layout, all my reviews have metadata, and stuff like that - but I'm not cramming in keywords, using AMP, or whatever other chickens Google requires to be sacrificed for a higher ranking. Nevertheless, I do OK.

Last year, I added a bit of local-only, lightweight statistics-gathering to my blog. I can see which sites people click on to reach mine. Google is right up the top, DuckDuckGo is surprisingly high, Bing is lucky to crack the top 20 on any day. Similarly, I can see how much traffic I get from the Fediverse and BlueSky (Twitter has all but vanished).

A few weeks ago I added RSS and Newsletter tracking. These data are very lossy. If someone is subscribed to my RSS feed and opens a post and their client downloads a lazy-loaded image at the end of the post, I get a hit. For email it's broadly the same. If an email is opened and the tracker image is loaded, I get a hit (although Gmail does obfuscate that somewhat).

I'm not looking for super-accurate numbers (although I do block as many AI crawlers and bots as possible). I'm not creepily following people around the web nor am I trying to sell them anything. I just want a rough idea of where people find me.

Here are my blog's views for the last 28 days.

Some months I get a surge of hits from link aggregators like HN or Reddit. Sometimes I'm linked to from a popular site or cited in academic work. But most of the time I bumble along getting hits from here, there, and everywhere. Nevertheless, it's lovely to see so many people choosing to subscribe⁰ (for free!) and astonishing that they provide more traffic than a major search engine.

Obviously, these are two very different types of traffic. People who are searching for a specific thing and stumble upon my blog are different from those who decide to like and subscribe.

But, yeah, about 25% of my traffic comes from people who have chosen to subscribe.

I'm just delighted that so many people read my random thoughts.

Nevertheless, some get through. Here's a particularly pernicious one - it appeared as three comments ostensibly in reply to each other.

At first glance these look like normal comments. They each address the content of the blog post albeit somewhat superficially. The first comment looks like it was from a social media post sharing my link - I get a lot of those as pingbacks, so it initially didn't trigger any suspicions from me.

The second is ostensibly a reply to the first and continues the conversation. Again, a bit shallow, but seems to be engaging in good faith.

The third looks like yet another reply. They all have unique email addresses, none of them have set their username to anything overly odd, and none of the users have filled out their URl.

But notice, in the second one, there's a link to a dodgy casino! There's no https:// so it didn't jump out as a link.

All three came from the same IP address in the Philippines, so easy to block for now.

Each reply is spaced exactly 3 minutes apart which, in retrospect, looks a little odd.

Re-reading them carefully, they all look like AI slop. A plausible sounding summary, written in a casual style, but with very little semantic content. Seeing them as replies to each other primed me to think they were genuine because I'm used to spam coming in individual replies. Having the spam in the middle comment made it easy to glaze over.

Remember, there are no technological solutions to social problems. Sticking more and more barriers in the way of commenting only discourages genuine replies while the profit motive incentivises spammers to work around them.

I like semantics. It tickles that part of my delicious meaty brain that longs for structure. Semantics are good for computers and humans. Computers can easily understand the structure of the data, humans can use tools like screen-readers to extract the data they're interested in.

In HTML, there are three main ways to impose semantics - elements, attributes, and hierarchical microdata.

Elements are easy to understand. Rather than using a generic element like <div> you can use something like <nav> to show an element's contents are for navigation. Or <address> to show that the contents are an address. Or <article><section> to show that the section is part of a parent article.

Attributes are also common. You can use relational attributes to show how a link relates to the page it is on. For example <a rel=author href=https://example.com> shows that the link is to the author of the current page. Or, to see that a link goes to the previous page in a series <a rel=prev href=/page5>.

Finally, we enter the complex and frightening world of microdata.

Using the Schema.org vocabulary it's possible to add semantic metadata within an HTML element. For example, <body itemtype=https://schema.org/Blog itemscope> says that the body of this page is a Blog. Or, to say how many words a piece has, <span itemprop=wordCount content=1100>1,100 words</span>.

There are many properties you can use. Here's the outline structure of a single blog post with a code sample, a footnote, and a comment. You can check its structured data and verify that it is conformant HTML.

Feel free to reuse.

<!doctype html>
<html lang=en-gb>
<head><title>My Blog</title></head>
<body itemtype=https://schema.org/Blog itemscope>

    <header itemprop=headline>
        <a rel=home href=https://example.com>My Blog</a>
    </header>

    <main itemtype=https://schema.org/BlogPosting itemprop=blogPost itemscope>
        <article>
            <header>
                <time itemprop=https://schema.org/datePublished datetime=2025-12-01T12:34:39+01:00>
                    1st January, 2025
                </time>
                <h1 itemprop=headline>
                    <a rel=bookmark href=https://example.com/page>Post Title</a>
                </h1>
                <span itemtype=https://schema.org/Person itemprop=author itemscope>
                    <a itemprop=url href=https://example.org/>
                        By <span itemprop=name>Author Name</span>
                    </a>
                    <img itemprop=image src=/photo.jpg alt>
                </span>
                <p>
                    <a itemprop=keywords content=HTML rel=tag href=/tag/html/>HTML</a> 
                    <a itemprop=keywords content=semantics rel=tag href=/tag/semantics/>semantics</a> 
                    <a itemprop=commentCount content=6 href=#comments>6 comments</a>
                    <span itemprop=wordCount content=1100>1,100 words</span>
                    <span itemtype=https://schema.org/InteractionCounter itemprop=interactionStatistic itemscope>
                        <meta content=https://schema.org/ReadAction itemprop=interactionType>
                        <span itemprop=userInteractionCount content=5150>
                            Viewed ~5,150 times
                        </span>
                    </span>
                </p>
            </header>

            <div itemprop=articleBody>
                <img itemprop=image src=/hero.png alt>
                <p>Text of the post.</p>
                <p>Text with a footnote<sup id=fnref><a role=doc-noteref href=#fn>0</a></sup>.</p>

                <pre itemtype=https://schema.org/SoftwareSourceCode itemscope translate=no>
                    <span itemprop=programmingLanguage>PHP</span>
                    <code itemprop=text>&amp;lt;?php echo $postID ?&amp;gt;</code>
                </pre>

                <section role=doc-endnotes>
                    <h2>Footnotes</h2>
                    <ol>
                        <li id=fn>
                            <p>Footnote text. <a role=doc-backlink href=#fnref>↩︎</a></p>
                        </li>
                    </ol>
                </section>
            </div>
        </article>

        <section id=comments>
            <h2>Comments</h2>
            <article itemtype=https://schema.org/Comment itemscope id="comment-123465">
                <time itemprop=dateCreated datetime=2025-09-11T13:24:54+01:00>
                    <a itemprop=url href=#comment-123465>2025-09-11 13:24</a>
                </time>
                <div itemtype=https://schema.org/Person itemprop=author itemscope>
                    <img itemprop=image src="/avatar.jpg" alt>
                    <h3>
                        <span itemprop=name>Alice</span> says:
                    </h3>
                </div>
                <div itemprop=text>
                    <p>Comment text</p>
                </div>
            </article>
        </section>
    </main>
</body>
</html>

This blog post is entitled "maximally" but, of course, there is lots more that you can add if you really want to.

Remember, none of this is necessary. Computers and humans are pretty good at extracting meaning from unstructured text. But making things easier for others is always time well spent.

Here's this post in plain text - https://shkspr.mobi/blog/2025/12/a-small-collection-of-text-only-websites.txt

Obviously a webpage without links is like a fish without a bicycle, but the joy of the web is that there are no gatekeepers. People can try new concepts and, if enough people join in, it becomes normal. I'm not saying the plain-text is the best web experience. But it is an experience. Perfect if you like your browsing fast, simple, and readable. There are no cookie banners, pop-ups, permission prompts, autoplaying videos, or garish colour schemes.

I'm certainly not the first person to do this, so I thought it might be fun to gather a list of websites which you browse in text-only mode. If you know of any more - including your own site - please drop a comment in the box!

• Terence Eden's blog - add .txt to any URl.
• Daring Fireball - add .text to any URl.
• Zach Flowers - replace .html with .txt.
• Fabien Benetou's PIM - add ?action=source to any URl.
• M0YNG - add .txt to any URl.
• Gwern - add .md to any URl or send an HTTP Accept for Markdown.
• Dan Q's textplain.blog - the entire blog is plain text!
• Matt Hobbs - there is a feed of plaintext which allows you to read recent posts.
• Bananas Playground - add index.txt to any post. Also works with index.md.
• Jorvik Systems - change .html to .md for Markdown.
• Max Glenister's blog - add .txt to any post's URl.
• Philipp Dubach's notes - add .txt to any post's URl.
• Derick Rethans' blog - add .txt to any post's URl.
• Ricardson's blog - add .txt to any post's URl.
• elle's blog - text mode only available via curl elle.sh/blog.
• Benji.dog - add .txt to any note's URl.
• Denis Defreyne's weeknotes - add .md to any weeknote's URl.

If you'd like to add a site, please get in touch. The rules are simple - content which has the MIME type of text/plain. No HTML, no multimedia, no RTF, no XML, no ANSI colour escape sequences.

Emoji are fine though; emoji are cool.

Here's how the admin list of posts looks to me:

I don't want it to look like that. I want it in RFC3339 format.

I know what you're thinking, just change the default date display - but that only seems to work in some areas of WordPress. It doesn't change the column-date format. Here's what mine is set to:

So that doesn't work.

Instead, you need to use the slightly obscure post_date_column_time filter

Add this to your theme's functions.php:

//  Admin view - change date format
function rfc3339_post_date_time( $time, $post ) {
    //  Modify the default time format
    $rfc3339_time = date( "Y-m-d H:i", strtotime( $post->post_date ) );
    return $rfc3339_time;
}
add_filter( "post_date_column_time", "rfc3339_post_date_time", 10, 2 );

And, hey presto, your date column will look like this:

Obviously, you can change that code to whichever date format you prefer.

When zoomed in, it shows the relation between posts and tags.

In this case, I can see that the posts about Small Gods and Pyramids both share the tags of Discworld, Fantasy, and Book Review. But only Small Gods has the tag of Religion.

Isn't that cool! It is a native feature of Quartz's GraphView. How can I build something like that for my WordPress blog?

Aim

Create an interactive graph which shows the relationship between a post, its links, and their tags.

It will end up looking something like this:

You can get the code or follow along to see how it works.

This is a multi-stage process. Let's begin!

What We Need

When on a single Post, we need the following:

• The tags assigned to that Post.
• Internal links back to that Post.
• Internal links from that Post.
• The tags assigned to links to and from that Post.

Tags assigned to that Post.

This is pretty easy! Using the get_the_tag_list() function we can, unsurprisingly, get all the tags associated with a post.

$post_tags_text = get_the_tag_list( "", ",", $ID );
$post_tags_array = explode( "," , $post_tags_text );

That just gets the list of tag names. If we want the tag IDs as well, we need to use the get_the_tags() function.

$post_tags = get_the_tags($ID);
$tags = array();
foreach($post_tags as $tag) {
    $tags[$tag->term_id] = $tag->name; 
}

Backlinks

Internal links back to the Post is slightly trickier. WordPress doesn't save relational information like that. Instead, we get the Post's URl and search for that in the database. Then we get the post IDs of all the posts which contain that string.

//  Get all the posts which link to this one, oldest first
$the_query = new WP_Query(
    array(
        's' => $search_url,
        'post_type' => 'post',
        "posts_per_page" => "-1",
        "order" => "ASC"
    )
);

//  Nothing to do if there are no inbound links
if ( !$the_query->have_posts() ) {
    return;
}

Backlinks' Tags

Once we have an array of posts which link back here, we can get their tags as above:

//  Loop through the posts
while ( $the_query->have_posts() ) {
    //  Set it up
    $the_query->the_post();
    $id                  = get_the_ID();
    $title               = esc_html( get_the_title() );
    $url                 = get_the_permalink();
    $backlink_tags_text  = get_the_tag_list( "", ",", $ID );
    $backlink_tags_array = explode( "," , $backlink_tags_text );
}

Links from the Post

Again, WordPress's lack of relational links is a weakness. In order to get internal links, we need to:

1. Render the HTML using all the filters
2. Search for all <a href="…">
3. Extract the ones which start with the blog's domain
4. Get those posts' IDs.

Rendering the content into HTML is done with:

$content = apply_filters( "the_content", get_the_content( null, false, $ID ) );

Searching for links is slightly more complex. The easiest way is to load the HTML into a DOMDocument, then extract all the anchors. All my blog posts start /blog/YYYY so I can avoid selecting links to tags, uploaded files, or other things. Your blog may be different.

$dom = new DOMDocument();
libxml_use_internal_errors( true ); //  Suppress warnings from malformed HTML
$dom->loadHTML( $content );
libxml_clear_errors();

$links = [];
foreach ( $dom->getElementsByTagName( "a" ) as $anchor ) {
    $href = $anchor->getAttribute( "href" );
    if (preg_match('/^https:\/\/shkspr\.mobi\/blog\/\d{4}$/', $href)) {
        $links[] = $href;
    }
}

The ID of each post can be found with the url_to_postid() function. That means we can re-use the earlier code to see what tags those posts have.

Building a graph

OK, so we have all our constituent parts. Let's build a graph!

Graphs consist of nodes (posts and tags) and edges (links between them). The exact format of the graph is going to depend on the graph library we use.

I've decided to use D3.js's Force Graph as it is relatively simple and produces a reasonably good looking interactive SVG.

Imagine there are two blog posts and two hashtags.

const nodes = [
    { id: 1, label: "Blog Post 1",    url: "https://example.com/post/1", group: "post" },
    { id: 2, label: "Blog Post 2",    url: "https://example.com/post/2", group: "post" },
    { id: 3, label: "hashtag",        url: "https://example.com/tag/3",  group: "tag"  },
    { id: 4, label: "anotherHashtag", url: "https://example.com/tag/4",  group: "tag"  },
];

• Blog Post 1 links to Blog Post 2.
• Blog Post 1 has a #hashtag.
• Both 1 & 2 share #anotherHashtag.

const links = [
    { source: 1, target: 2 },
    { source: 3, target: 1 },
    { source: 4, target: 1 },
    { source: 4, target: 2 },
];

Here's how to create a list of nodes and their links. You will need to edit it for your own blog's peculiarities.

<?php 
// Load WordPress environment
require_once( "wp-load.php" );

//  Set up arrays for nodes and links
$nodes = array();
$links = array();

//  ID of the Post
$main_post_id = 12345;

//  Get the Post's details
$main_post_url   = get_permalink( $main_post_id );
$main_post_title = get_the_title( $main_post_id );

//  Function to add new nodes
function add_item_to_nodes( &$nodes, $id, $label, $url, $group ) {
    $nodes[] = [ 
        "id"    => $id, 
        "label" => $label, 
        "url"   => $url, 
        "group" => $group
    ];

}

//  Function to add new relationships
function add_relationship( &$links, $source, $target ) {
    $links[] = [
        "source" => $source,
        "target" => $target
    ];
}

//  Add Post to the nodes
add_item_to_nodes( $nodes, $main_post_id, $main_post_title, $main_post_url, "post" );

//  Get the tags of the Post
$main_post_tags = get_the_tags( $main_post_id );

//  Add the tags as nodes, and create links to main Post
foreach( $main_post_tags as $tag ) {
    $id   = $tag->term_id;
    $name = $tag->name;

    //  Add the node
    add_item_to_nodes( $nodes, $id, $name, "https://shkspr.mobi/blog/tag/" . $name, "tag" );
    //  Add the relationship
    add_relationship( $links, $id, $main_post_id );
}

//  Get all the posts which link to this one, oldest first
$the_query = new WP_Query(
    array(
        's'              => $main_post_url,
        'post_type'      => 'post',
        "posts_per_page" => "-1",
        "order"          => "ASC"
    )
);

//  Nothing to do if there are no inbound links
if ( $the_query->have_posts() ) {
    //  Loop through the posts
    while ( $the_query->have_posts() ) {
        //  Set up the query
        $the_query->the_post();
        $post_id = get_the_ID();
        $title = esc_html( get_the_title() );
        $url   = get_the_permalink();

        //  Add the node
        add_item_to_nodes( $nodes, $post_id, $title, $url, "post" );
        //  Add the relationship
        add_relationship( $links, $post_id, $main_post_id );

        //  Get the tags of the Post
        $post_tags = get_the_tags( $post_id );

        //  Add the tags as nodes, and create links to main Post
        foreach($post_tags as $tag) {

            $id   = $tag->term_id;
            $name = $tag->name;

            //  Add the node
            add_item_to_nodes( $nodes, $id, $name, "https://shkspr.mobi/blog/tag/" . $name, "tag" );
            //  Add the relationship
            add_relationship( $links, $id, $post_id );
        }

    }
}

//  Get all the internal links from this post
//  Render the post as HTML
$content = apply_filters( "the_content", get_the_content( null, false, $ID ) );

//  Load it into HTML
$dom = new DOMDocument();
libxml_use_internal_errors( true );
$dom->loadHTML( $content );
libxml_clear_errors();

//  Get any <a href="…" which starts with https://shkspr.mobi/blog/
$internal_links = [];
foreach ( $dom->getElementsByTagName( "a" ) as $anchor ) {
    $href = $anchor->getAttribute( "href" );
    if (preg_match('/^https:\/\/shkspr\.mobi\/blog\/\d{4}$/', $href)) {
        $internal_links[] = $href;
    }
}

//  Loop through the internal links, get their hashtags
foreach ( $internal_links as $url ) {
    $post_id = url_to_postid( $url );
    //  Get the Post's details
    $post_title = get_the_title( $id );

    //  Add the node
    add_item_to_nodes( $nodes, $post_id, $post_title, $url, "post" );
    //  Add the relationship
    add_relationship($links, $main_post_id, $post_id );

    //  Get the tags of the Post
    $post_tags = get_the_tags( $post_id );

    //  Add the tags as nodes, and create links to main Post
    foreach( $post_tags as $tag ) {
        $id   = $tag->term_id;
        $name = $tag->name;

        //  Add the node
        add_item_to_nodes( $nodes, $id, $name, "https://shkspr.mobi/blog/tag/" . $name, "tag" );
        //  Add the relationship
        add_relationship( $links, $id, $post_id );
    }
}

//  Deduplicate the nodes and links
$nodes_unique = array_unique( $nodes, SORT_REGULAR );
$links_unique = array_unique( $links, SORT_REGULAR );

//  Put them in the keyless format that D3 expects
$nodes_output = array();
$links_output = array();

foreach ( $nodes_unique as $node ) {
    $nodes_output[] = $node;
}

foreach ( $links_unique as $link ) {
    $links_output[] = $link;
}

//  Return the JSON
echo json_encode( $nodes_output, JSON_PRETTY_PRINT );
echo "\n";
echo json_encode( $links_output, JSON_PRETTY_PRINT );

Creating a Force Directed SVG

Once the data are spat out, you can include them in a web-page. Here's a basic example:

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <title>Force Directed Graph</title>
        <script src="https://d3js.org/d3.v7.min.js"></script>
    </head>
    <body>
        <svg width="800" height="600">
            <defs>
                <marker id="arrowhead" markerWidth="10" markerHeight="7" refX="10" refY="3.5" orient="auto" fill="#999">
                <path d="M0,0 L10,3.5 L0,7 Z"></path>
                </marker>
            </defs>
        </svg>
        <script>

            const nodes = [];
            const links = [];

            const width  = 800;
            const height = 600;

            const svg = d3.select("svg")
                .attr( "width",  width  )
                .attr( "height", height );

            const simulation = d3.forceSimulation( nodes )
                .force( "link",   d3.forceLink( links ).id( d => d.id ).distance( 100 ) )
                .force( "charge", d3.forceManyBody().strength( -300 ) )
                .force( "center", d3.forceCenter( width / 2, height / 2 ) );

            //  Run simulation with simple animation
            simulation.on("tick", () => {
                link
                    .attr("x1", d => d.source.x)
                    .attr("y1", d => d.source.y)
                    .attr("x2", d => d.target.x)
                    .attr("y2", d => d.target.y);   node
                    .attr("transform", d => `translate(${d.x},${d.y})`);
            });

            // Draw links
            const link = svg.selectAll( ".link" )
                .data(links)
                .enter().append("line")
                .attr( "stroke", "#999" )
                .attr( "stroke-width", 2 )
                .attr( "x1", d => d.source.x )
                .attr( "y1", d => d.source.y )
                .attr( "x2", d => d.target.x )
                .attr( "y2", d => d.target.y )
                .attr( "marker-end", "url(#arrowhead)" );

            //  Draw nodes
            const node = svg.selectAll( ".node" )
                .data( nodes )
                .enter().append( "g" )
                .attr( "class", "node" )
                .attr( "transform", d => `translate(${d.x},${d.y})` )
                .call(d3.drag() //  Make nodes draggable
                    .on( "start", dragStarted )
                    .on( "drag",  dragged )
                    .on( "end",   dragEnded ) 
                );

            //  Add hyperlink
            node.append("a")
            .attr( "xlink:href", d => d.url ) //    Link to the node's URL
            .attr( "target", "_blank" ) //  Open in a new tab
            .each(function (d) {
                const a = d3.select(this);
                //  Different shapes for posts and tags
                if ( d.group === "post" ) {
                    a.append("circle")
                        .attr("r", 10)
                        .attr("fill", "blue");
                } else if ( d.group === "tag" ) {
                    //  White background rectangle
                    a.append("rect")
                            .attr("width", 20)
                            .attr("height", 20)
                            .attr("x", -10)
                            .attr("y", -10)
                            .attr("fill", "white"); 
                    // Red octothorpe
                    a.append("path")
                            .attr("d", "M-10,-5 H10 M-10,5 H10 M-5,-10 V10 M5,-10 V10") 
                            .attr("stroke", "red")
                            .attr("stroke-width", 2)
                            .attr("fill", "none");
                }
                //  Text label
                a.append( "text")
                    .attr( "dy", 4 )
                    .attr( "x", d => ( d.group === "post" ? 12 : 14 ) )
                    .attr( "fill", "black" )
                    .style("font-size", "12px" )
                    .text( d.label );
            });

            //  Standard helper functions to make nodes draggable
            function dragStarted( event, d ) {
                if ( !event.active ) simulation.alphaTarget(0.3).restart();
                d.fx = d.x;
                d.fy = d.y;
            }
            function dragged( event, d ) {
                d.fx = event.x;
                d.fy = event.y;
            }
            function dragEnded( event, d ) {
                if (!event.active) simulation.alphaTarget(0);
                d.fx = null;
                d.fy = null;
            }

        </script>
    </body>
</html>

Next Steps

It needs a bit of cleaning up if I want to turn it into a WordPress plugin. It might be nice to make it a static SVG rather than relying on JavaScript. And the general æsthetic needs a bit of work.

Perhaps I could make it 3D like my MSc Dissertation?

But I'm pretty happy with that for an afternoon hack!

You can get the code if you want to play.

//  Add ordering by comments
add_action( 'pre_get_posts', 'pre_get_posts_by_comments' );
function pre_get_posts_by_comments( $query ) {
    //  Do nothing if the post_status parameter in the URL is not "comment-order"
    if ( ! isset( $_GET['comment-order'] ) ) {
        return;
    }

    $query->set( "orderby", "comment_count" );  //  Default: date
    $query->set( "order", "DESC" ); //  Biggest first
}

This makes use of the pre_get_posts hook to rewrite the posts query. That means it works on most WordPress pages.

For example:

• My homepage https://shkspr.mobi/blog/?comment-order
• Posts with a specific tag https://shkspr.mobi/blog/tag/blockchain/?comment-order
• Dates https://shkspr.mobi/blog/2012/?comment-order

Did you find this post useful? Please leave a comment here!

My ideal job involves being employed by a millionaire tech-bro. Just before they get on stage, or moments before they file a lawsuit, or an instant before they publish their thought leadership - I will appear to them. I will be dressed in rags, body smeared with excrement, weeping sores blotching my face. I will sidle up to them, lean down, and whisper into their ear "Sic Transit Gloria Mundi!"

This used to be the way, of course. When a new Pope or Emperor was paraded through the streets, the people cheered and the acolytes coo'd. But someone would chant at them in Latin, "Thus passes all Earthly glory."

The sun sets on every empire. Each god-king was eventually proved to be mortal. The evil that men do lives after them and the good is oft interred within their bones. And so on. We know this. No one is perfect all the time. Every genius has a moment of idiocy.

For small projects, it makes sense to have only one gaffer. All the work passes through him. Too many cooks poison the well.

As projects get bigger, one person doesn't scale. It is functionally impossible to know everything, see everything, please everyone. Yet the sole arbiter remains. We (almost-jokingly) call this position BDFL. The Benevolent Dictator For Life.

But BDFL only works if the D is genuinely B. Otherwise the FL becomes FML.

It must be psychologically difficult being responsible for a mega-project. I certainly couldn't do it. If you're wasting time reading this blog post, you almost certainly couldn't do it. I like my friends to challenge my occasional missteps. I want people to be somewhat honest to my face. I can't imagine what it would do to my ego to receive endless praise. Of course I'd tune out the negative voices.

This isn't to excuse their excesses. Nor to fully demystify their demagoguery. I just want them to have good mental health so their public meltdowns don't reverberate through the æther, infecting us all with their psychic fallout.

*sigh* I'd rather not be blogging about blogging.

It has also become outdated, buggy, and suffers from link-rot.

So, every day, I tend to my digital garden. I go in to old posts and check that the links are still pointing somewhere relevant. Are the embeds still live or do they need replacing? Has my writing somehow been mangled by me buggering about with CSS?

Oh, I could use automated tools. But when a spammer takes over a domain I've linked to, they rarely send an HTTP 410 code.

Sometimes the work is delightful - finding a prescient post from a decade ago. Sometimes it is frustrating - being unable to find a vital-but-long-dead link. And sometimes it is sad - seeing how much or how little the world has changed.

But, mostly, it is meditative. We do our best to fight against decay, but entropy always wins in the end. Every link eventually withers and every truth is eroded by time. Nevertheless, we continue.

I still have a lot of Tweets embedded on this blog. Using WordPress, it was easy to paste in a link and have it converted to an embed. But I don't want to direct people to a dangerous site.

So here's a somewhat automated way to replace embedded Tweets with screenshots.

Shut up and show me the code!

Demo

Use the Embed Platform

Take the ID of the Tweet you want to convert. Add it on to the end of an embed URl like this - https://platform.twitter.com/embed/Tweet.html?dnt=true&embedId=twitter-widget-0&frame=false&hideCard=false&hideThread=true&lang=en&theme=light&width=550px&id=1092852483033055232

Let's make that a bit more readable:

https://platform.twitter.com/embed/Tweet.html?
   hideCard=false
  &hideThread=true
  &lang=en
  &theme=light
  &width=550px
  &id=1092852483033055232

You can change whether to show a card (the attached image or link), show the preceding message in the thread or not, what UI language to show, dark or light mode, and how wide you want the embed to be.

Use Selenium to automate the screenshot

Using Python, we can use Selenium's Chrome Webdriver to open pages, find elements, and take screenshots:

import time

import io
from PIL import Image

from selenium import webdriver 
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

#   Chrome's headless options
chrome_options = Options()
chrome_options.add_argument('--headless=new')
chrome_options.add_argument('--window-size=1920,1080')

#   Turn off everything
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--disable-infobars")
chrome_options.add_argument("--disable-logging")
chrome_options.add_argument("--log-level=3")

#   Start Chrome
driver = webdriver.Chrome(options=chrome_options)

#   Open the page
driver.get("https://platform.twitter.com/embed/Tweet.html?dnt=true&embedId=twitter-widget-0&frame=false&hideCard=false&hideThread=true&lang=en&theme=light&width=550px&id=1092852483033055232")

#   Twitter is slow!
time.sleep(5)

#   Get the Tweet
tweet = driver.find_element(By.TAG_NAME, "article")
#   Use the parent element for more padding
tweet = driver.execute_script("return arguments[0].parentNode;", tweet)

#   Save as an image
print("Save")
image_binary = tweet.screenshot_as_png
img = Image.open(io.BytesIO(image_binary))
img.save("tweet.png")

Get the alt text

Accessibility is important. Getting the text of the Tweet is as simple as:

#   Get the alt text
alt = tweet.text

But that retrieves all the text - including things like "Copy link to post" and "Read more on X" - because Twitter doesn't believe in semantic HTML. There's also no way to easily get the number of likes and retweets - which might be useful information.

If there are images in the post, it's useful to get their alt text. This is simpler:

images = tweet.find_elements(By.TAG_NAME, "img")
print("\nAlt text of images:")
for img in images:
    alt_text = img.get_attribute("alt")
    if alt_text:
        print(alt_text)

Use The API

There is a better way to get all the text, alt text, and metadata. Use the hidden syndication API! But that's a blog post for another time…

Get the Code

Grab the code from GitHub - and if it is useful to you, please star the repo or leave a friendly comment.

Of course, if we can use the API to get the pure data, perhaps it is possible to make some lovely semantic HTML rather than an image...? 😉

Lots of companies encourage their staff to blog. It's free PR! It makes them look like they're on the cutting edge of technology! It helps with recruitment!

It can also be a corporate nightmare. What if the developer says something stupid? What if it accidentally reveals something top secret? What if the CEO doesn't like it?

And so, gradually, any free-wheelin' developer blog gradually succumbs to the tender mercies of the comms team. It starts with help editing and clarifying the writing. Then it turns into massaging the tone-of-voice. Before too long you have to wait for a slot on the corporate comms grid. And then you need seven levels of sign-off just to post something anodyne like "5 reasons we switched to Rust".

*sigh*

So, maybe once or twice, I cheated.

Rather than writing a blog post called "An interesting issue configuring GLaDOS with YAML" I opened an issue on our public GitHub repo.

Oh, sure, it wasn't as "bloggy". And I still made sure not to discuss anything secret or sensitive. For some of the more contentious issues, I ran it by a few trusted colleagues first. But, for all intents and purposes, it was a blog post. Nerds got to reply in the comments, people shared it on HN, people quoted it on social media.

I've since left that sort of restrictive environment. I still see my old team occasionally publishing a useful blog post. But I keep my eyes on their GitHub issues to see the real story!

What's the moral here?

Should you bypass your employer's strict social media policy? No, absolutely not. Should you ensure that your GitHub issues are well-formatted, interesting to read, and useful to others? Yes.

Today, I am very hungover. Although I usually write a bunch of posts a few days and weeks in advance, I find myself looking at my publishing calendar and seeing it blissfully empty.

Part of the reason I blog is that I truly want to learn something new every day - and I want to share that knowledge. Whether it's a cool programming trick, and interesting book, or thoughts on the world - I enjoy lifelong learning.

But I am tired. Also hungover - but mostly tired. Perhaps it is a little burn-out. Perhaps it is a little disillusionment with the state of the tech world. Perhaps I'm spending too much of my energies elsewhere. Either way, despite the little dopamine hit I get when my posts go viral, or when new friends leave comments, I think now is the perfect time to throttle back.

So, this is the 3,094th post that I've published on this site - and it is likely to be the last one for a little while. I'll still be writing when the mood strikes. And I'll still be wittering away on Mastodon.

Thank you for reading, commenting, and sharing.

Be seeing you!

I kept up the blogging for a few months, then it trickled off. I preferred posting on Usenet and other primitive forms of social media. But, by 2007, I was back to blogging on my own site again, and I never really stopped. This blog fluctuates between being a diary, an excuse to rant, and technical writing. It's my site and I can do whatever I want with it. That's rather freeing.

I have an "On This Day" feature of my blog. Every morning I check what I was writing about on this day in years gone by. I find it informative and meditative to see how much I've grown² and what topics I keep returning to.

I'm not big on milestones or anniversaries. But it does feel rather nice to know that I started something a few decades ago that is still a going concern.

Here's a little treat to thank you for reading:

Someone (probably Jeremy Keith) said:

A blog post doesn't need a title.

In a literal sense, he was wrong. The HTML specification makes it clear that the <title> element is mandatory. All documents have title.

But, in a practical sense, he was right. This blog post has an empty <h1> element - the document might be semantically invalid, it might reduce accessibility, but the post is still available.

A blog post can be a plain text document uploaded to a server. It can be an image hosted on a social network. It can be a voice note shared with your friends.

Title, dates, comments, links, and text are all optional.

No one is policing this.

Go create something which doesn't fit properly with the rest of the world.

I first started a blog on Blogger.com in 2004 - twenty years ago. Like all blogs, I managed half a dozen posts before I forgot about it.

Cut to 2007 and I decided to launch shkspr.mobi as a weird site dedicated to rendering Shakespeare's plays in txt spk. Judging by Archive.org I was still using Blogger.

By 2008 I was blogging most months. And then I never really stopped. In early 2009 I switched to WordPress which led me down the path of developing my own theme and plugins.

Along the way, I've added necroposts - blog posts from work blogs which have since become defunct, or letters that I wrote to magazines when I was a kid.

I've also been quite liberal with my use of retroposts - posts written far in advance and then published once the dust has settled. There are a few more of those in the pipeline.

As of today, there are about 6,500 pieces of media in my library - taking up 2.3GB.

Last year I hit 12,000 comments - now I'm on about 13,500.

Since 2009, when I first turned on WordPress stats, my blog has been visited 8.5 million times.

According to some code I copy and pasted the total wordcount across all my posts is...

1,553,953 words.

Bloody hell!

So, because bloggers like nothing more than writing about blogging, I thought I'd do a little look behind my process.

How I write

I don't write every day. But I do write most days. I usually write title and scraps of ideas, then leave them to ferment⁰. They usually start out as social-media posts which have got a little traction.

When I do write a post, I'll quite often leave it in a half-finished state and come back to it later. I usually have several blog posts on the go at any one time. It is incredibly rare that I'll write something and publish it that same day. And it is only occasionally that I publish something in the same week it was written.

I schedule and reschedule as the mood takes me.

I write in MarkDown using the classic editor. I'm too stuck in my ways to switch to Gutenberg blocks.

Why I write

Because I enjoy it.

Someone once asked me how I managed to read so many books and the answer is simple - because I prefer reading books to doing other things.

I could spend more time playing video games, or learning to solder, or cooking from scratch, or brewing beer, or drinking beer, or any of a thousand hobbies¹. But reading and writing are what I prefer doing.

Oh, sure, there's a thrill when a post goes viral. Or when someone you admire says something nice about the writing. Or when someone leaves a comment on an old blog saying how I helped them. Or when I'm cited in academic research.

But the reason I write is to get all the ideas out of my head². I blog daily because, deep down, I want to learn something new and surprising every day - and I want to share that with anyone who happens to pass by.

What I've learned

It is trite, but the blog posts I expected to be lauded were mostly flops. The half-arsed ones often do well³.

People can't read. I'll write as clearly as I can and some yutz will misunderstand me⁴.

Some people are incapable of understanding hyperbole. Perhaps it is a feature of British English which isn't well understood around the world?

Americans don't like swearing in blog posts and get a bit po-faced about it⁵.

My predictions rarely come true, my opinions fluctuate, and my spelling remains poor.

I'm nothing without my unpaid editor.

Thank you for reading

I mostly write for me. But it is lovely to know that people all around the world occasionally stop by to read. It is very sweet of you.

If you've enjoyed my writing or ever found it useful, please go off and start your own blog. Write whatever nonsense comes into your head.

You should publish your words, pictures, songs, reviews on your own site. And then you can choose to share them out to where your audience is. Perhaps that's posting the link on Facebook, or a copy of a photo on Instagram, or sharing the episode on YouTube.

There's no shame in meeting your audience where they are - but the canonical version should be somewhere you control.

But what about the conversation? Social media is about socialising. People who comment on LinkedIn can't see the comments on Mastodon and vice versa.

This is where the concept of "CABOOM!" (exclamation optional) comes in⁰. As far as practical, you should bring the various comment threads together so everyone can see them¹.

There are a few ways to do this - choose whichever is right for you.

Do Nothing

Let people self-discover that you syndicate. Perhaps they'll stumble upon a bunch of people discussing your post over on Slashdot. That's fine, but a little hard work.

Point to your syndication sites

At the bottom of your post, you can say "Click here to read the comments on MySpace!"

Find other sources

Did someone share your post to HackerNews? Add a link on your page to the orange-site so that others can read what's being said.

Convert comments to WebMentions

Using a service like Brid.gy you can "upgrade" external comments to WebMentions. They will appear on your post as normal comments, but with a link back to their original location.

(Ab)use APIs

Services like Reddit have APIs. You could pull in a complete comment thread and present it on your own site.

Make backups

Some of those external sites are going to break one day. Take a backup using Archive.org or snap a screenshots. Perhaps link to those if the original goes away.

Be mindful of others' feelings

I've previously discussed the ethics of syndicating comments using WebMentions. Not everyone wants their comment copied back to your site. Some people regularly delete comments. Or they may have valid copyright concerns. Just because someone has written something in public, it doesn't mean it is in the public domain. Be kind to your commenters and respect their wishes.

Have fun

There is literally no-one policing this. If you don't want to do it, that's fine. If you want to give it your best effort, that's also fine.

I've read and reviewed around 300 books over the last few years. So I wrote a scrap of code which goes through all my book reviews, grabs their cover and rating, and displays them in a nice grid. You can visit it at shkspr.mobi/blog/library

You know how when you're at a friend's home you rummage through their bookshelf? That's the same idea here. You get to poke through the stacks and, hopefully, pick out something entertaining to read. Or, perhaps, you'll be warned off something truly dreadful.

All the code for my blog is open source - and I'd be delighted if you did something interesting with it.

This was heavily inspired by Dave Rupert's bookshelf.

Here's what it looks like:

Code

All we need to do is search WordPress for the URl of the current page. Loop through the results. Then display those links.

$the_query = new WP_Query(
    array(
        's' => get_the_permalink(), // This post
        'post_type' => 'post',      // Only posts, not pages
        "posts_per_page" => "-1",   // Get all results
        "order" => "ASC"            // Oldest first
    )
);

//  Nothing to do if there are no inbound links
if ( !$the_query->have_posts() ) {
    return;
}

//  Loop through the posts
while ( $the_query->have_posts() ) {
    //  Set it up
    $the_query->the_post();
    $id            = get_the_ID();
    $title         = esc_html( get_the_title() );
    $url           = get_the_permalink();
    $comment_date  = get_the_date( "c" );
    $comment_year  = get_the_date( "Y" );

    //  Get the thumbnail
    if ( has_post_thumbnail( $id) ) {
        $thumb = get_the_post_thumbnail( $rid, 'full', 
            array( "class"   => "related-post-img",
                   "loading" => "lazy" //   Lazy loading and other attributes
            ) 
        );
    } else {
        $thumb = null;
    }

    //  Create the HTML for the incoming link
    echo <<<EOT
    <li>
        <a href="$url">
            $thumb
            $title
        </a>
        <time datetime="$comment_date">$comment_year</time>
    </li>
    EOT;

}

In order to reduce duplicates, you may also want to disable self-ping. Here's some code you can stick in your functions.php:

function no_self_ping( &$links ) {

    $home = esc_url( home_url() );

    // Process each link in the content and remove if it matches the current site URL
    foreach ( $links as $l => $link ) {
        if ( 0 === strpos( $link, $home ) ) {
            unset( $links[ $l ] );
        }
    }
}
add_action( 'pre_ping', 'no_self_ping' );

One small annoyance is that it doesn't like multi-line entry. It treats every hit of the enter key as "plz run the codez" - so, at the end of this blog post, I've put the commands in copy-n-paste format.

Once you've installed WP CLIP, go to the command line and run wp shell. You'll be greeted with an interactive prompt wp>

Method One - Quick Search

This command constructs a query which gets all posts, which have been published, where there is no thumbnail ID. Note - this isn't quite the same as not having a featured image.

$args = array(
    'post_type'     => 'post',
    'post_status'   => array('publish'),
    'posts_per_page'=> -1,
    'meta_query'    => array(
        array(
            'key' => '_thumbnail_id',
            'compare' => 'NOT EXISTS'
        )
    ),
);

Next we run that query. It will dump quite a lot of information into the screen, we'll format it shortly.

$query = new WP_Query( $args );

Finally, we loop through all the posts the query has found and print them out in a nice format.

$posts = $query->posts;
foreach ( $posts as $post ) {
    echo $post->post_date . " " . $post->guid . " " . $post->post_title . "\n";
}

That will give you the date, link, and title of every post where there is no featured image set.

Copy and Paste Snippet

$args = array( 'post_type' => 'post', 'post_status' => array('publish'), 'posts_per_page'=> -1,  'meta_query' => array(       array( 'key' => '_thumbnail_id', 'value' => '?', 'compare' => 'NOT EXISTS' ) ),);
$query = new WP_Query( $args );
$posts = $query->posts;
foreach ( $posts as $post ) { echo $post->post_date . " " . $post->guid . " " . $post->post_title . "\n"; }

Method Two - Full Search

Sometimes the WordPress database can get a little confused. It will say there is a post thumbnail, but when you try to retrieve it, there's nothing there!

This method is slightly slower if you have lots of posts. It goes through every single post and checks whether the featured image can be retrieved. If not it will let you know.

$args = array(
    'post_type'     => 'post',
    'post_status'   => array('publish'),
    'posts_per_page'=> -1,
);

$posts = get_posts( $args );

foreach ( $posts => $post) {
    if( get_the_post_thumbnail( $post ) == "" ) {
        echo $post->post_date . " " . $post->guid . " " . $post->post_title . "\n"; }
}

Copy and Paste Snippet

foreach (get_posts( array( 'post_type' => 'post', 'post_status' => array('publish'), 'posts_per_page' => -1,) ) as $post) { if(get_the_post_thumbnail($post)== "") { echo $post->post_date . " " . $post->guid . " " . $post->post_title . "\n"; } }