They're also really easy to create programmatically.

This uses the SVG "polyline" which takes a list of x,y co-ordinate pairs. But can you spot the small problem?

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1024 124">
    <polyline fill="none" stroke="#0074D955" stroke-width="3" 
        points="12,48 83,84 154,79 226,90 297,79 369,65 440,78 512,80 583,88 654,12 726,56 797,92 869,93 940,97 1012,106"></polyline>
</svg>

The SVG co-ordinate system has position 0,0 at the top left. Most graphics formats are like that. That's fine for our x value - but it means higher y values will appear lower on the graph.

Getting the x co-ordinate of each data point is easy. Take the width of the SVG image and divide it by the number of data-points.

The y co-ordinate is harder. The algorithm is:

1. Find the height of the SVG.
2. Find the maximum value in the data.
3. Find the minimum value in the data.
4. Divide the maximum value by the height of the graph.
5. For each data point, either:
   • To have the lowest value at the bottom of the graph, subtract the minimum from the value, then multiply by the ratio in (4).
   • Or, to retain the gap between zero and the lowest value, multiply the value by the ratio in (4).
6. The y co-ordinate is calculated by subtracting the value in (5) from the height in (1).

Here's some code showing how it works. I've added a little padding to the inside of the graph - you'll see why later:

//  Max and min of views.
$max_views = max( $svg_views_data );
$min_views = min( $svg_views_data );
$svg_data_length = sizeof( $svg_dates_data ) - 1;

//  SVG details for scaling.
$svg_padding = 12;
$svg_width_graph  = 1000;
$svg_width  = $svg_width_graph + ( $svg_padding * 2 );
$svg_height_graph = 100;
$svg_height = $svg_height_graph + ( $svg_padding * 2 );

//  Calculate where each point should be.
$x_per = $svg_width_graph / ( $svg_data_length );
$y_per = $svg_height_graph / $max_views;

//  Loop through the data.
foreach ( $svg_views_data as $index=>$views ) {
    //  X is from the left.
    $x_pos = intval( $x_per * $index ) + $svg_padding;
    //  Y is from the top.
    $y_pos = $svg_height - intval( $y_per * $views ) - $svg_padding;

    //  Add a point to the line.
    $polyline_points .= "{$x_pos},{$y_pos}\n";
}

echo <<< SVG
<svg xmlns="http://www.w3.org/2000/svg"
    viewBox="0 0 $svg_width $svg_height" class="chart">
    <polyline
        fill="none"
        stroke="#F00"
        stroke-width="3"
        points="{$polyline_points}"/>
</svg>
SVG;

Suppose someone suggests stupidly simple sparklines suffer seriously so someone should supplement statistics several circles?

Using the same co-ordinates, we can place an SVG circle on top of the point. Give it a "title" attribute and you have a little bit of interactivity.

<circle cx="12" cy="48" r="5" fill="#0074D955"><title>4,707 Views</title></circle>

Here's how it looks (view source to understand how it is constructed).

Hover over any of those little circles and you'll see some pop-up text giving you information about that datapoint.

…that's it! If you have an array of data points, you can easily create a graph with no graphing library, no plugins, no 3rd party dependencies. Just super simple SVG.

How can you quickly and easily find any posts which don't have a featured image?

For this, I use WP CLI - it allows you to run complex WordPress actions and queries using the command line. After you have installed WP CLI you can get started.

Missing Images

On the command line, run:

wp eval 'foreach(get_posts(array("post_type"=>"post","post_status"=>array("publish"),"posts_per_page"=>-1,)) as $post){if(get_the_post_thumbnail($post)==""){$post_type_object=get_post_type_object($post->post_type);$link=admin_url(sprintf($post_type_object->_edit_link . "&action=edit", $post->ID));echo $post->post_date . " " . $link . " " . $post->post_title . "\n";}}'

Here's the code in a slightly more readable format:

foreach ( 
   get_posts( 
       array( "post_type"      => "post", 
              "post_status"    => array("publish"), 
              "posts_per_page" => -1,
       ) 
   ) as $post) { 
      if( get_the_post_thumbnail( $post)== "" ) { 
         $post_type_object = get_post_type_object( $post->post_type ); 
         $link = admin_url( sprintf( $post_type_object->_edit_link . "&action=edit", $post->ID ) ) ;
         echo $post->post_date . " " . $link . " " . $post->post_title . "\n";
      } 
}

That will print out:

2024-05-02 12:34:11 https://example.com/wp-admin/post.php?post=123&action=edit "A post about sausages" 
2023-09-13 20:55:52 https://example.com/wp-admin/post.php?post=456&action=edit "I like cheese"
2021-12-31 15:43:33 https://example.com/wp-admin/post.php?post=789&action=edit "Touching computers"

You can then go and edit each of those posts to add a featured image.

Missing Alt Text

Adding alt text means that people who can't see images will still be able to understand what the picture represents. Here's another one-lines to find all featured images with missing alt text:

wp eval 'foreach (get_posts(array("post_type"=>"post","post_status"=>array("publish"),"posts_per_page" => -1,)) as $post){if(simplexml_load_string(get_the_post_thumbnail($post))["alt"]==""){$post_type_object=get_post_type_object($post->post_type);$link=admin_url(sprintf($post_type_object->_edit_link . "&action=edit",$post->ID));echo $post->post_date . " " . $link . " " . $post->post_title . "\n";}}'

And, in slightly more readable form:

foreach (
   get_posts( 
      array( "post_type"      => "post", 
             "post_status"    => array("publish"), 
             "posts_per_page" => -1,
           ) 
   ) as $post) { 
      if( simplexml_load_string( get_the_post_thumbnail( $post ) )["alt"] == "") { 
         $post_type_object = get_post_type_object( $post->post_type ); 
         $link = admin_url( sprintf( $post_type_object->_edit_link . "&action=edit", $post->ID ) ) ;
         echo $post->post_date . " " . $link . " " . $post->post_title . "\n"; 
      } 
}

Again, that lists the datetime of the post, its edit link, and its title.

Now, if you'll excuse me, I have about 873 posts which need updating 🤯

During the episode, Eddie starts reading this newspaper:

Obviously, the "Hammersmith Bugle" is not a real paper and they never ran a headline "No News Shocker". But judging from all the other shots, the prop is based on a real newspaper.

So I decided to rip off Dirty Feed's shtick and find out what was used to create the fake newspaper. The quest took me o'er hill and dale. Through the rough hinterlands of Hammersmith and into the nether regions of Wimbledon. By which I mean - I used lots of online archive sources.

And it nearly worked! I found all of the internal pages. I also found the back page:

That's from The Surrey Herald - but that's a paper with lots of regional editions. None of which had the right headline.

So I emailed my (frankly asinine ) request to Surrey Museums. They were polite, but unable to help. Their website gave a clue though - the location of the archives of the Surrey Herald:

Surrey Herald: Chertsey, Addlestone and Byfleet edition (also Walton, Weybridge and Hersham edition Feb 1979 to 1999 at Elmbridge Museum)

So I contacted the fine people at Elmbridge Museum who were happy to rummage through their microfiche for me. I expect, much like Indiana Jones, the archivists had to knock down fake walls, find a mystic box containing the treasure, and then dodge various snakes and villains to retrieve the priceless artefact. Or they may have a well designed archival system which is a pleasure to use. I don't know.

Anyway! All of which is to say that they very kindly sent me a quick scan of the front page of Surrey Herald's Walton, Weybridge and Hersham edition from November 3rd 1994.

Here it is in all its glory!

That's a perfect match for what's seen on screen:

Hurrah! Another mystery solved thanks to publicly funded museums!

What have we learned today?

• Archivists are lovely, generous, and helpful people.
• Museums are brilliant.
• Not everything in the world has been digitised.
• There was quite a lot of news that day no matter what the drunken hacks at the Hammersmith Bugle say.
• We do not know if centenarian Elsie Bartlett was aware that her photo featured in this seminal part of British comedy.

I recently read Susam's blog post where they said that "most of the traffic to my personal website still comes from web feeds" - I wondered if that was true for my site.

I've been writing this blog for a while. I've never much bothered with "aggressive" SEO - I have a fairly semantic layout, all my reviews have metadata, and stuff like that - but I'm not cramming in keywords, using AMP, or whatever other chickens Google requires to be sacrificed for a higher ranking. Nevertheless, I do OK.

Last year, I added a bit of local-only, lightweight statistics-gathering to my blog. I can see which sites people click on to reach mine. Google is right up the top, DuckDuckGo is surprisingly high, Bing is lucky to crack the top 20 on any day. Similarly, I can see how much traffic I get from the Fediverse and BlueSky (Twitter has all but vanished).

A few weeks ago I added RSS and Newsletter tracking. These data are very lossy. If someone is subscribed to my RSS feed and opens a post and their client downloads a lazy-loaded image at the end of the post, I get a hit. For email it's broadly the same. If an email is opened and the tracker image is loaded, I get a hit (although Gmail does obfuscate that somewhat).

I'm not looking for super-accurate numbers (although I do block as many AI crawlers and bots as possible). I'm not creepily following people around the web nor am I trying to sell them anything. I just want a rough idea of where people find me.

Here are my blog's views for the last 28 days.

Some months I get a surge of hits from link aggregators like HN or Reddit. Sometimes I'm linked to from a popular site or cited in academic work. But most of the time I bumble along getting hits from here, there, and everywhere. Nevertheless, it's lovely to see so many people choosing to subscribe⁰ (for free!) and astonishing that they provide more traffic than a major search engine.

Obviously, these are two very different types of traffic. People who are searching for a specific thing and stumble upon my blog are different from those who decide to like and subscribe.

But, yeah, about 25% of my traffic comes from people who have chosen to subscribe.

I'm just delighted that so many people read my random thoughts.

Throughout my time working for the UK Government - in GDS, NHSX, i.AI, and others - I championed Open Source. I spoke to dozens of departments about it, wrote guidance still in use today, and briefed Ministers on why it was so important.

That's why I'm beyond disappointed at recent moves from NHS England to backtrack on all the previous commitments they've made about the value of open source to the UK's health service.

It's rare that multiple people leak the same story to me, but that's what gives me confidence that lots of people within the NHS are aghast at this news.

A few days ago, I was sent this quote which was attributed to a senior technical person in NHS England.

We are obviously looking at things like Mythos, which is more sophisticated at finding vulnerabilities. In the next week or so, we will be changing our tack on coding the open and making our code public until we're on top of that risk.

Most of our repos, unless they're essential, will be removed for security reasons.

As I've written before, this is not the correct response to the purported threat by Mythos. Neither the AI Safety Institute nor the NCSC recommend this action. While there may be some increase in risk from AI security scanners, to shutter everything would be a gross overreaction.

Nevertheless, that's what the NHS is preparing to do.

On the 29th of April, guidance note SDLC-8 was sent out. Here's what it says:

The majority of code repos published by the NHS are not meaningfully affected by any advance in security scanning. They're mostly data sets, internal tools, guidance, research tools, front-end design and the like. There is nothing in them which could realistically lead to a security incident.

When I was working at NHSX during the pandemic, we were so confident of the safety and necessity of open source, we made sure the Covid Contact Tracing app was open sourced the minute it was available to the public. That was a nationally mandated app, installed on millions of phones, subject to intense scrutiny from hostile powers - and yet, despite publishing the code, architecture and documentation, the open source code caused zero security incidents.

Furthermore, this new guidance is in direct contradiction to the UK's Tech Code of Practice point 3 "Be open and use open source" which insists on code being open.

Similarly, the Service Standard says:

There are very few examples of code that must not be published in the open.

The main reason for code to be closed source is when it relates to policy that has not yet been announced. In this case, you must make the code open as soon as possible after the policy is published.

You may also need to keep some code closed for security reasons, for example code that protects against fraud. Follow the guidance on code you should keep closed and security considerations for open code.

There's also the DHSC policy "Data saves lives: reshaping health and social care with data":

Commitment 601 – completed May 2022

We will publish a digital playbook on how to open source your code for health and care organisations

And, here's NHS Digital's stance on open source in their Software Engineering Quality Framework:

The position of all three of these documents is that we should code in the open by default.

All of which is reflected in the NHS service standard:

Public services are built with public money. So unless there's a good reason not to, the code they're based should be made available for other people to reuse and build on.

All of which is to say - open source should be baked into the DNA of the NHS by now. There are thousands of NHS repositories on GitHub. The work undertaken to assess all of them and then close them will be massive. And for what?

Even if we ignore the impracticality of closing all the code - it is too late! All that code has already been slurped up. If Mythos really is the ultimate hacker, hiding the code now does nothing. It has likely already retained copies of the repositories.

And if it were both practical and effective to hide source code - that doesn't matter. These AI tools are just as effective against closed-source. They can analyse binaries and probe websites with ease.

There are tens of thousands of NHS website pages which refer to their GitHub repos - will they all need to be updated? What's the cost of that?

I've no idea what led to NHS England making this retrograde decision - so I've send a Freedom of Information request to find out.

I am convinced that closing all their excellent open source work is the wrong move for the NHS. I hope they see sense and reverse course.

Until then, I've helped make sure that every single NHS repository has been backed up and, because the software licence permits it, can be re-published if the original is closed.

In the meantime, you should email your MP and tell them that the NHS is wrong to shutter its world-leading open source repositories.

Don't let them take away your right to see the code which underpins our nation's healthcare.

Further Reading

• I'm quoted in this article from The New Scientist.
• Matt Hancock on the issue
• Discussion by Jessica Morley, PhD
• Free Software Foundation Europe press release
• Further commentary from New Scientist
• Petition - Keep Things Open

Google's Android has now broken that.

On the web, we used to use:

<input type="file" accept="image/jpeg">

That opened the phone's photo picker and let the use upload a geotagged photo. But a while ago Google deliberately broke that.

Instead, we were encourage to use the file picker:

<input type="file">

That opened the default file manager. This had the unfortunate side-effect of allowing the user to upload any file, rather than just photos. But it did allow the EXIF metadata through unmolested. Then Google broke that as well.

Using a "Progressive Web App" doesn't work either.

So, can users transfer their photos via Bluetooth or QuickShare? No. That's now broken as well.

You can't even directly share via email without the location being stripped away.

Literally the only way to get a photo with geolocation intact is to plug in a USB cable, copy the photo to your computer, and then upload it via a desktop web browser?

Why?!?!?

Because Google run an anticompetitive monopoly on their dominant mobile operating system.

Privacy.

There's a worry that users don't know they're taking photos with geolocation enabled. If you post a cute picture of your kid / jewellery / pint then there's a risk that a ne’er-do-well could find your exact location.

Most social media services are sensible and strip the location automatically. If you try to send a geotagged photo to Facebook / Mastodon / BlueSky / WhatsApp / etc, they default to not showing the location. You can add it in manually if you want, but anyone downloading your photo won't see the geotag.

And, you know, I get it. Google doesn't want the headline "Stalkers found me, kidnapped my baby, and stole my wedding ring - how a little known Android feature puts you in danger!"

But it is just so tiresome that Google never consults their community. There was no advance notice of this change that I could find. Just a bunch of frustrated users in my inbox blaming me for breaking something.

I don't know what the answer is. Perhaps a pop up saying "This website wants to see the location of your photos. Yes / No / Always / Never"? People get tired of constant prompts and the wording will never be clear enough for most users.

It looks like the only option available will be to develop a native Android app (and an iOS one?!) with all the cost, effort, and admin that entails. Android apps have a special permission for accessing geolocation in images.

If anyone has a working way to let Android web-browsers access the full geolocation EXIF metadata of photos uploaded on the web, please drop a comment in the box.

In the meantime, please leave a +1 on this HTML Spec comment.

At the turn of the century, it was common to listen to an "acquired" music file only to find it was missing a few seconds at the end due to a prematurely stopped download. Some video formats would refuse to play at all if the moov atom at the end of the file was missing.

I wondered if it would be possible to make a file format which was close to impossible to read unless the entire file was intact. I don't mean including a checksum to detect download errors - I mean a layout which was intrinsically fragile to corruption.

While digging through an old backup CD, I found my original notes. I'm rather impressed at what neophyte-me had constructed. My outline was:

• The file ends with a 32 bit pointer. This points to the location of the first information block.
• The information block describes the length of the data block which follows it.
• At the end of the data block is another 32 bit pointer. This points to the location of the next information block.
• The start of the file may be a pointer, or it may be padded with random data.
• There may be random data padded between the data blocks.

This ensures that a file which has been only partially downloaded - whether truncated at the end or missing pieces elsewhere - cannot be successfully read.

Here's a worked example. Start at the end and follow the thread.

0. Random data.
1. Data block size is 2.
2. Data
3. Data
4. EOF.
5. Data block size is 1.
6. Data.
7. Go to location 1.
8. Random data.
9. Go to location 5.

There are, of course, a few downsides to this idea.

Most prominently, it bloats file size. If the data block size was a constant 1MB, that would pad the size a negligible amount. But with variable data block size, it could increase it significantly. Random padding also increases the size.

If the block size is consistent and there's no random padding data, the files can be mostly reconstructed.

Depending on which parts of the file are missing, it may be possible to recover the majority of the file.

A location block size of 32 bits restricts the file-size to less than 4GB. A 64 bit pointer might be excessive or might be future-proof!

Highly structured files with predictable patterns, or text files, may be easy to recover large bits of information.

A malformed file could contain an infinite loop of pointers.

Perhaps a magic number should be at the start (or end) of the file?

While reading the file is as simple as following the pointers, constructing the file is more complex, especially if blocks have variable lengths.

Code

Here's a trivial encoder. It reads a file in consistently sized chunks of 1,024 bytes. It shuffles them up and writes them to a new file. The last 4 bytes contain a pointer to the first block, which says the data length is 1,024. After that, there is a 4 byte pointer to the next block location.

import random

#   Size of data, headers, and pointers.
data_length = 1024
header_length  = 4
pointer_length = 4

#   Read the file into a data structure.
original_blocks = list()
with open( "test.jpg", "rb") as file:
    for data in iter( lambda: file.read( data_length ), b"" ):
        #   Add padding if length is less than the desired length.
        padding = data_length - len( data )
        data += b"\0" * padding
        original_blocks.append( data )

#   How many blocks are there?
original_length = len( original_blocks )

#   Create a random order of blocks.
order = list( range( 0, original_length ) )
random.shuffle( order )

#   Where is the start of the file?
first_block_index = order.index( 0 )
first_block_pointer = first_block_index * ( header_length + data_length + pointer_length )

#   Loop through the order and write to a new file.
i = 0
#   Open as binary file to add the pointers correctly.
with open( "output.rff", "wb" ) as output:
    while i < original_length:
        #   Where are we?
        current_block = i
        current_block_value = order[i]
        #   Write length of data in little-endian 32 bytes.
        output.write( data_length.to_bytes( header_length, "little") )
        #   Write data
        output.write( original_blocks[ current_block_value ] )
        i = i+1
        #   Last block. Write an EOF header.
        if ( current_block_value + 1 >= original_length ):
            eof = 4294967295
            output.write( eof.to_bytes( header_length, "little") )
        else:
            next_block = order.index( current_block_value + 1 )
            #   Write pointer to next block
            next_block_location = next_block * ( header_length + data_length + pointer_length )
            output.write( next_block_location.to_bytes( pointer_length, "little" ) )
    #   At the end of the file, write the pointer to block 0.
    output.write( first_block_pointer.to_bytes( pointer_length, "little" ) )

And here is a similarly trivial decoder. It reads the last 32 bits, moves to that location, reads the block size, reads the data and writes it to a new file, then reads the next pointer.

import os
#   Size of data, headers, and pointers.
header_length  = 4
pointer_length = 4
#   File name to write to.
decoded_file = "decoded.bin"

#   Create an empty file.
with open( decoded_file, "w") as file:
    pass

#   Function to loop through the blocks.
def read_block( position, i ):
    #   Move to the position in the file.
    input_file.seek( position, 0 )
    #   Read the data length header.
    data_length = int.from_bytes( input_file.read( header_length ), "little" )
    #   Move to the data block.
    input_file.seek( position + header_length, 0 )
    #   Read the data.
    data = input_file.read( data_length )
    #   Read the pointer header.
    next_position = int.from_bytes( input_file.read( pointer_length ), "little" )
    #   If this is the final block, it may have null padding. Remove it.
    if ( next_position == 4294967295 ) :
        data = data.rstrip(b"\0")
    #   Append the data to the decoded file.
    with open( decoded_file, "ab" ) as file:
        file.write( data )
    #   If this is the final block, finish searching.
    if ( next_position == 4294967295 ) :
         print("File decoded.")
    else:
        #   Move to the next position.
        read_block( next_position, i+1 )

#   Open the file as binary.
input_file = open( "output.rff", "rb" )

#   Read the last 4 bytes.
input_file.seek( -4, 2 )

#   Get position of first block
first_block = int.from_bytes( input_file.read(), "little" )

#   Start reading the file.
seek_to = first_block
read_block( seek_to, 0 )

As I said, these are both trivial. They are a bit buggy and contain some hardcoded assumptions.

Here are two files encoded as "RFF" - Random File Format - an image by Maria Sibylla Merian, and the text of Romeo and Juliet.

Have fun decoding them!

I’ve recently taken part in the DriveToImprove Beta. My car now has a small device installed in it, which measures my driving. It reports back to me my location, whether I was speeding, and if I’m braking or accelerating too harshly. It also gives me a great little dashboard showing how well I’m doing and how I compare to others.

Bringing “high scores” to real life is part of a trend known as “gamification“.

DriveToImprove has the usual aspects of modern gamification – badges, a leaderboard, and the ability to track your progress.

As I was driving home one day last week, a child ran out in front of my car. I had only a split second to react – slam on the brakes and risk losing points for “Harsh Braking” or continue on, risk hitting the child, but maintain my perfect score…

It’s a real dilemma – safety vs score.

Of course, no one would think like that, would they? I certainly didn’t!

But gaming incentives have a funny effect on our brain. Games are fun – and that makes them highly emotionally manipulative.

That use of scores, rewards, and fun can manipulate us in all sorts of negative ways. Games can encourage us to hate, to harm our bodies, to join a cult, it can cause you to annoy your friends – in some extreme cases, the pleasure associated with playing a game can take over someone’s life until they literally play themselves to death.

So, when designing systems which utilise gamification, we have to be aware that the human brain is susceptible to all sorts of tricks – and we have to be really careful when we subvert them.

We know from feedback on our community that sometimes rapid acceleration is necessary. So is it always a good idea to penalise users for it?

Although we work hard to make sure that the game aspect of the products we create never take priority over safety, it’s impossible to predict what effect our stimulating of the brain’s pleasure centres will have. After all, earning rewards in a game can have the same effect on the brain as cocaine!

What’s great about the test-and-learn approach we take at The Lab is that it gives us time to see the consequences of our developments. We can remain focussed on adding value to our applications, while remaining aware of any unintended side-effects.

I found this video from Eran May-raz and Daniel Lazo a startling and amusing look at what may happen if we let gamification run away with itself.