Open Source – Terence Eden’s Blog

NHS Goes To War Against Open Source

@edent — Fri, 01 May 2026 11:44:04 +0000

The NHS is preparing to close nearly all of its Open Source repositories.

Throughout my time working for the UK Government - in GDS, NHSX, i.AI, and others - I championed Open Source. I spoke to dozens of departments about it, wrote guidance still in use today, and briefed Ministers on why it was so important.

That's why I'm beyond disappointed at recent moves from NHS England to backtrack on all the previous commitments they've made about the value of open source to the UK's health service.

It's rare that multiple people leak the same story to me, but that's what gives me confidence that lots of people within the NHS are aghast at this news.

A few days ago, I was sent this quote which was attributed to a senior technical person in NHS England.

We are obviously looking at things like Mythos, which is more sophisticated at finding vulnerabilities. In the next week or so, we will be changing our tack on coding the open and making our code public until we're on top of that risk.

Most of our repos, unless they're essential, will be removed for security reasons.

As I've written before, this is not the correct response to the purported threat by Mythos. Neither the AI Safety Institute nor the NCSC recommend this action. While there may be some increase in risk from AI security scanners, to shutter everything would be a gross overreaction.

Nevertheless, that's what the NHS is preparing to do.

On the 29th of April, guidance note SDLC-8 was sent out. Here's what it says:

The majority of code repos published by the NHS are not meaningfully affected by any advance in security scanning. They're mostly data sets, internal tools, guidance, research tools, front-end design and the like. There is nothing in them which could realistically lead to a security incident.

When I was working at NHSX during the pandemic, we were so confident of the safety and necessity of open source, we made sure the Covid Contact Tracing app was open sourced the minute it was available to the public. That was a nationally mandated app, installed on millions of phones, subject to intense scrutiny from hostile powers - and yet, despite publishing the code, architecture and documentation, the open source code caused zero security incidents.

Furthermore, this new guidance is in direct contradiction to the UK's Tech Code of Practice point 3 "Be open and use open source" which insists on code being open.

Similarly, the Service Standard says:

There are very few examples of code that must not be published in the open.

The main reason for code to be closed source is when it relates to policy that has not yet been announced. In this case, you must make the code open as soon as possible after the policy is published.

You may also need to keep some code closed for security reasons, for example code that protects against fraud. Follow the guidance on code you should keep closed and security considerations for open code.

There's also the DHSC policy "Data saves lives: reshaping health and social care with data":

Commitment 601 – completed May 2022

We will publish a digital playbook on how to open source your code for health and care organisations

And, here's NHS Digital's stance on open source in their Software Engineering Quality Framework:

The position of all three of these documents is that we should code in the open by default.

All of which is reflected in the NHS service standard:

Public services are built with public money. So unless there's a good reason not to, the code they're based should be made available for other people to reuse and build on.

All of which is to say - open source should be baked into the DNA of the NHS by now. There are thousands of NHS repositories on GitHub. The work undertaken to assess all of them and then close them will be massive. And for what?

Even if we ignore the impracticality of closing all the code - it is too late! All that code has already been slurped up. If Mythos really is the ultimate hacker, hiding the code now does nothing. It has likely already retained copies of the repositories.

And if it were both practical and effective to hide source code - that doesn't matter. These AI tools are just as effective against closed-source. They can analyse binaries and probe websites with ease.

There are tens of thousands of NHS website pages which refer to their GitHub repos - will they all need to be updated? What's the cost of that?

I've no idea what led to NHS England making this retrograde decision - so I've send a Freedom of Information request to find out.

I am convinced that closing all their excellent open source work is the wrong move for the NHS. I hope they see sense and reverse course.

Until then, I've helped make sure that every single NHS repository has been backed up and, because the software licence permits it, can be re-published if the original is closed.

In the meantime, you should email your MP and tell them that the NHS is wrong to shutter its world-leading open source repositories.

Don't let them take away your right to see the code which underpins our nation's healthcare.

Does Mythos mean you need to shut down your Open Source repositories?

@edent — Fri, 24 Apr 2026 11:34:30 +0000

Much Sturm und Drang in the world of Open Source with the announcement that the "Mythos" AI is now the ultimate hacker and is poised to unleash havoc on every code base.

So should you close all your Open Source projects to make them safe?

No.

Firstly, all your Open Source code has already been slurped up.

It was all ingested for "training purposes" years ago. If it was moderately interesting then it was backed-up by a digital hoarder. It has been archived by various digital libraries. Anyone who wants to do research on your code base can.

Closing now doesn't meaningfully protect you.

Secondly, most of the security holes in your systems are probably not in your code. Vulnerabilities exist throughout your supply chain. All the dependencies - your OS, libraries, and even hardware - are all richer targets for hackers. Finding a CVE in a popular library is almost certainly more worthwhile than investigating your Open Source code.

The bigger risk comes not from subtle logic bugs but from phishers, poor password hygiene, and insider threats. Securing your existing systems provides more protection than rushing to close-source your code.

Finally, closing the source of something doesn't protect you. These new AI models can easily investigate and your closed source systems and potentially penetrate them. It has always been possible to analyse websites and binaries. AI doesn't change that - although it might accelerate it.

Open Source does have risks but AI doesn't upend decades of evidence that closed-source is just as vulnerable to attackers.

In cases where the state creates code using public money, it has a responsibly to share that code. Automated threat analysis - even by hypercapabe AI - doesn't change that.

I would strongly recommend reading the UK's AI Safety Institute's evaluation of Claude Mythos Preview’s cyber capabilities and the NCSC's advice. Neither of them recommend closing down Open Source code.

How Can Governments Pay Open Source Maintainers?

@edent — Sat, 14 Mar 2026 12:34:42 +0000

When I worked for the UK Government I was once asked if we could find a way to pay for all the Open Source Software we were using. It is a surprisingly hard problem and I want to talk about some of the issues we faced.

The UK Government publishes a lot of Open Source code - nearly everything developed in-house by the state is available under an OSI Approved licence. The UK is generally pretty relaxed about people, companies, and states re-using its code. There's no desire and little capability to monetise what has been developed with public money so it becomes public code.

What about the Open Source that UK Government uses?

The state uses big projects like WordPress, as well as moderately popular NPM packages, and small Python libraries and everything in between. But can it pay the maintainers of that software?

A version of this blog post was originally published on Hackernoon.

Fixing The Plumbing

Open Source is facing a crisis. The code that the world relies on is often developed by underpaid engineers on the brink of burn-out. While I don't think anyone wants Open Source to have a paywall, it seems obvious that large organisation should pay their way and not rely solely on volunteer labour.

Here are some of the problems I faced when trying to get the UK Government to pay for OSS and how you as a maintainer can help make it easier for large organisations to pay you.

Firstly, lots of OSS doesn't have a well defined owner; so who gets the money?

I'm not saying that every little library you create needs to be published by a registered company, nor am I suggesting that you should remove your anonymity. But Governments and other organisations need to know who they are funding and where the money is going. The danger of accidentally funnelling money to a sanctioned state or person is just too big a risk for most organisations.

If you want to receive funding - make it really clear who you are.

What Can You Offer?

Even when there is an owner, there often isn't an easy mechanism for paying people. Donation sites like GitHub Sponsors, Ko-Fi, and Patreon are great for individuals who want to throw a small amount of money to creators but they can be problematic for larger organisations. Many OSS projects get around this by offering support contracts. It makes it much easier for an organisation to justify their spend because they're no longer donating to something which can be obtained for free; they're paying for a service.

This doesn't have to be a contract offering a 24/7 response and guaranteed SLA. It can be as simple as offering best-effort email support.

The important thing is to offer an easy way for a larger organisation to buy your services. Many organisations have corporate credit cards for lower-cost discretionary spending which doesn't require a full business-case. How easily could a manager buy a £500 support contact from your site?

Maintainers don't only have to offer support contracts. Many choose to offer training packages which are a good way to raise money and get more people using your product. Some project maintainers will speak at your conference for a suitable fee.

Again, the aim here is for maintainers to offer a plausible reason for a payment to be made.

Playing Well With Others

Open Source has a brilliant culture of allowing multiple (often anonymous) contributors. That's fine when there's no money involved, but how does a moderately sized project decide who receives what share of the funding? Services like OpenCollective can make it easier to show where the money is going but it is better to discuss in advance with all contributors what they expect as a share.

If people think they're being taken advantage of, or that a project maintainer is unjustly enriching themselves, it can cause arguments. Be very clear to contributors what the funding is for and whether they're entitled to any of it.

Finally, we faced the issue that some OSS projects didn't want to take money from the "big bad state". They were worried that if people saw "Sponsored by the Government" they would assume that there were backdoors for spies, or that the developer might give in to pressure to add unwanted features. This (usually) isn't the case but it is easy to see why having a single large organisation as the main donor could give the impression of impropriety.

The best defence against this is to have lot of paying sponsors! Having the state as one of many partners makes it clear that a project isn't beholden to any one customer.

It isn't impossible to get Governments to spend on Open Source. But state spending is heavily scrutinised and, bluntly, they aren't set up to pay ad hoc amounts to non-suppliers, who aren't charging money. While large projects often have the resources to apply for Government grants and contracts, smaller projects rarely have the time or expertise. It is critical that maintainers remove the barriers which make it too hard for organisations to pay them.

In Summary

Make it easy for Governments and other large organisations to pay you.
Be as obvious as possible that you are able to accept payments from them.
Don't be afraid to put a large price on your talents.
Offer multiple paid-for options like speaker fees, support, and feature development funding.
Talk with your contributors to let them know how any funding will be shared.

Book Review: Diversifying Open Source - An Open Standards Playbook for Inclusive and Equitable Tech Projects by Paloma Oliveira ★★★★☆

@edent — Fri, 06 Feb 2026 12:34:05 +0000

It is refreshing to read a political polemic which contains useful actions the reader can take. Too many books about the social problems with technology end up being a diagnosis with no cure.

Paloma Oliveira's new book (with technical review by my friend Dawn Foster) is a deep dive into how we can all make Open Source more inclusive and equitable.

Unlike most tech books, it doesn't follow the usual pattern of restricting itself to the US hegemony. It is very focussed on the EU and the needs of people around the world. It is clear in identifying many of the problems which arise when people say they just want to focus on tech, not politics:

When projects focus purely on technical excellence without considering accessibility, they create implicit barriers. Documentation written only in English, community discussions held during North American business hours, or development environments that require high-end hardware all reflect choices that determine who can participate—though these choices often remain unexamined.

This is profoundly important. The book isn't afraid to be challenging. It links the way companies extract value from the commons to the way colonisers extracted value from the lands they "discovered".

There are a few missteps which I didn't care for. While it starts as very casually written, it quickly finds itself getting into the weeds of political philosophy. I think that's a necessary evil. But I don't know how easily people will be convinced by passages like:

Bratton notes secessionist withdrawal in traditional territories and consolidation domains in stacked hemispheric, the continuing expansions of nebular sovereignties, and the reform of conventional States into regional platforms.

Similarly, there are a few "just-so" stories which are fictional parables. I think they would have been more convincing as actual case-studies.

I did find myself skipping some of the background in order to get to the parts I found more interesting. The chapter on "Political Rhetoric and Institution Validation" felt a bit out of place and I didn't get much from it.

But, after all that theory, there is a lot of practical advice. From how to structure your README to how to communicate change to your community. Even better, all the templates and resources are on GitHub.

It is thoroughly referenced and gave me lots of new rabbit-holes to follow Rather pleasingly, it cites my 2020 blog post "Please Stop Inventing New Software Licences" as an example of the ways in which corporates often try to stifle open source.

If you want to help Open Source succeed, you owe it to yourself to grab a copy of this book.

The Peaceful Transfer of Power in Open Source Projects

@edent — Wed, 19 Nov 2025 12:34:27 +0000

Most of the people who run Open Source projects are mortal. Recent history shows us that they will all eventually die, or get bored, or win the lottery, or get sick, or be conscripted, or lose their mind.

If you've ever visited a foreign country's national history museum, I guarantee you've read this little snippet:

King Whatshisface was a wise and noble ruler who bought peace and prosperity to all the land.

Upon his death, his heirs waged bloody war over rightful succession which plunged the country into a hundred years of hardship.

The great selling point of democracy is that it allows for the peaceful transition of power. Most modern democracies have rendered civil war almost unthinkable. Sure, you might not like the guy currently in charge, but there are well established mechanisms to limit their power and kick them out if they misbehave. If they die in office, there's an obvious and understood hierarchy for who follows them.

Most Open Source projects start small - just someone in their spare room tinkering for fun. Unexpectedly, they grow into a behemoth which now powers half the world. These mini-empires are fragile. The most popular method of governance is the Benevolent Dictator For Life model. The founder of the project controls everything. But, as I've said before, BDFL only works if the D is genuinely B. Otherwise the FL becomes FML.

The last year has seen several BDFLs act like Mad Kings. They become tyrannical despots, lashing out at their own volunteers. They execute takeovers of community projects. They demand fealty and tithes. Like dragons, they become quick to anger when their brittle egos are tested. Spineless courtiers carry out deluded orders while pilfering the coffers.

Which is why I am delighted that the Mastodon project has shown a better way to behave.

In "The Future is Ours to Build - Together" they describe perfectly how to gracefully and peacefully transfer power. There are no VCs bringing in their MBA-brained lackeys to extract maximum value while leaving a rotting husk. No one is seizing community assets and jealously hoarding them. Opaque financial structures and convoluted agreements are prominent in their absence.

Eugen Rochko, the outgoing CEO, has a remarkably honest blog post about the transition. I wouldn't wish success on my worst enemy. He talks plainly about the reality of dealing with the pressure and how he might have been a limiting factor on Mastodon's growth. That's a far step removed from the ego-centric members of The Cult of The Founder with their passionate belief in the Divine Right of Kings.

Does your tiny OSS script need a succession plan? Probably not. Do you have several thousand NPM installs per day? It might be worth working out who you can share responsibility with if you are unexpectedly raptured. Do you think that your project is going to last for a thousand years? Build an organisation which won't crumble the moment its founder is arrested for their predatory behaviour on tropical islands.

I'm begging project leaders everywhere - please read up on the social contract and the consent of the governed. Or, if reading is too woke, just behave like grown-ups rather than squabbling tweenagers.

It is a sad inevitability that, eventually, we will all be nothing but memories. The bugs that we create live after us, the patches are oft interrèd with our code. Let it be so with all Open Source projects.

How to actually test your readme

@edent — Tue, 07 Oct 2025 11:34:08 +0000

If you've spent any time using Linux, you'll be used to installing software like this:

The README says to download from this link. Huh, I'm not sure how to unarchive .tar.xz files - guess I'll search for that. Right, it says run setup.sh hmm, that doesn't work. Oh, I need to set the permissions. What was the chmod command again? OK, that's working. Wait, it needs sudo. Let me run that again. Hang on, am I in the right directory? Here it goes. What, it crapped out. I don't have some random library - how the hell am I meant to install that? My distro has v21 but this requires <=19. Ah, I also need to upgrade something which isn't supplied by repo. Nearly there, just need to compile this obscure project from SourceForge which was inexplicably installed on the original dev's machine and then I'll be good to go. Nope. Better raise an issue on GitHub. Oh, look, it is tomorrow.

As a developer, you probably don't want to answer dozens of tickets complaining that users are frustrated with your work. You thought you made the README really clear and - hey! - it works on your machine.

There are various solutions to this problem - developers can release AppImages, or Snaps, or FlatPaks, or Docker or whatever. But that's a bit of stretch for a solo dev who is slinging out a little tool that they coded in their spare time. And, even those don't always work as seamlessly as you'd hope.

There's an easier solution:

Follow the steps in your README
See if they work.
…
That's it.

OK, that's a bit reductive! There are a million variables which go into a test - so I'm going to introduce you to a secret zeroth step.

Spin up a fresh Virtual Machine with a recent-ish distro.

If you are a developer, your machine probably has a billion weird configurations and obscure libraries installed on it - things which definitely aren't on your users' machines. Having a box-fresh VM means than you are starting with a blank-slate. If, when following your README, you discover that the app doesn't install because of a missing dependency, you can adjust your README to include apt install whatever.

OK, but how?

Personally, I like Boxes as it gives you a simple choice of VMs - but there are plenty of other Virtual Machine managers out there.

Pick a standard OS that you like. I think the latest Ubuntu Server is pretty lightweight and is a good baseline for what people are likely to have. But feel free to pick something with a GUI or whatever suits your audience.

Once your VM is installed and set up for basic use, take a snapshot.

Every time you want to test or re-test a README, revert back to the original state of your box. That way you won't have odd half-installed packages laying about.

Your next step is to think about how much hand-holding do you want to do?

For example, the default Debian doesn't ship with git. Does your README need to tell people to sudo apt install git and then walk them through configuring it so that they can git clone your repo?

Possibly! Who is your audience? If you've created a tool which is likely to be used by newbies who are just getting started with their first Raspberry Pi then, yeah, you probably will need to include that. Why? Because it will save you from receiving a lot of repeated questions and frustrated emails.

OK, but most developers will have gcc installed, right? Maybe! But it doesn't do any harm to include it in a long list of apt get … anyway, does it? Similarly, does everyone know how to upgrade to the very latest npm?

If your software is designed for people who are experienced computer touchers, don't fall into the trap of thinking that they know everything you do. I find it best to assume people are intelligent but not experienced; it doesn't hurt to give slightly too much detail.

The best way to do this is to record everything you do after logging into the blank VM.

Restore the snapshot.
Log in.
Run all the commands you need to get your software working.
Once done, run history -w history.txt
- That will print out every command you ran.
Copy that text into your README.

Hey presto! You now have README instructions which have been tested to work. Even on the most bare-bones machine, you can say that your README will allow the user to get started with your software with the minimum amount of head-scratching.

Now, this isn't foolproof. Maybe the user has an ancient operating system running on obsolete hardware which is constantly bombarded by cosmic rays. But at least this way your issues won't be clogged up by people saying their install failed because lib-foobar wasn't available or that ./configure had fatal errors.

A great example is the Opus Codec README. I went into a fresh Ubuntu machine, followed the readme, ran the above history command, and got this:

sudo apt-get install git autoconf automake libtool gcc make
git clone https://gitlab.xiph.org/xiph/opus.git
cd opus
./autogen.sh
./configure
make
sudo make install

Everything worked! There was no missing step or having to dive into another README to figure out how to bind flarg 6.9 with schnorp-unstable.

So that's my plea to you, dear developer friend. Make sure your README contains both the necessary and sufficient information required to install your software. For your sake, as much as mine!

Wait! You didn't follow your own advice!

You're quite right. Feel free to send a pull request to correct this post - as I shall be doing with any unhelpful READMEs I find along the way.

I'm never going back to Matrix

@edent — Tue, 29 Jul 2025 11:34:22 +0000

I should love Matrix. It is a decentralised, privacy preserving, multi-platform chat tool. Goodbye Slack and your ridiculous free limits. Adiós Discord and your weird gamification. Suck it IRC with your obscure syntax and faint stench of BO. WhatsApp and Telegram can stick their heads in a bucket of lukewarm sick and sing sea shanties! Let's join the future!

The problem is - Matrix is shit. Not just on a protocol level, but on an organisational level as well.

I joined Matrix at FOSDEM - the largest gathering of open source nerds in Europe. We were all encouraged to use it - every talk had its own channel, all the official comms came from there, I was even invited to a top-secret private channel for speakers. This was going to be epic! Viva la rèvölūçïón, right? Wrong.

It was dead. Even among the most seasoned geeks on the planet, most people preferred to use other services like Signal, Telegram, and Slack. Why? Because those other tools actually work.

Matrix has two official Android apps - one of which is old and unsupported, the other is new and doesn't work with many of the basic chat features.

I want to be absolutely clear about this - the company behind Matrix have put out an app which doesn't work with their own product! Lest you think I'm exaggerating, here's a typical view of the official FOSDEM speaker room, using the official Matrix app:

It was embarrassing. People would pipe up in channels and say "this doesn't work" only to be told they were using the wrong app and should go back to the one marked unsupported. So they left, never to return. Even in the large talks, where people were encouraged to use the official Matrix chat, most of the conversation happened on other platforms. It was just too hard to use Matrix.

A few thousands geeks, all used to recompiling their own kernels and participating in the Fediverse, and most thought that Matrix was too much of a faff.

After FOSDEM, I kept the Matrix app on my phone. Occasionally receiving a ping from some long-forgotten channel.

And then, one day, I got hit with the most vile spam. A dozen notifications suddenly appeared on my phone with abuse, torture, and transphobic slurs in them.

You can view the screenshot - but, fair warning, it is grim.

This shouldn't be possible. It doesn't take an expensive team of moderators to add some keyword monitoring. It doesn't take a massive AI model to work out that a stranger shouldn't be able to bombard users with multiple notifications. You don't have to sacrifice your dream of a decentralised future - you just need to care about your users.

This stuff is basic.

I moaned about it on Mastdon and was surprised to receive a private reply from the official Matrix account.

Please do not encourage the spammer by giving them a platform and propagating their spam; you may want to consider deleting your post.

This is classic victim blaming. It is my fault for giving the spammer attention. I am the one who needs to take responsibility and delete the evidence. I shouldn't warn people that Matrix is actively dangerous to use.

Bullshit.

Here's what I expected them to say:

"We're sorry you had such a bad experience on Matrix. Rest assured we're working hard to block these spammers - here's a link to show what we're doing. You can protect your account further by doing x, y and z. Once again, sorry and we hope we can win back your trust."

I'm not saying scrappy open source projects have to hire anodyne corporate communications specialists; they just need to have a little empathy.

But, no, just constant whining about how it isn't their fault and how I am the one who needs to change my behaviour.

This is pretty typical behaviour from the team. Find any post complaining about some aspect of Matrix and you'll see their instant woe-is-me replies.

So I deleted the app. I would have liked to have nuked my account but apparently that's not possible.

I'm not the only one who feels like this. Here's an epic post by Marius, which concludes:

Between the slow performance, the increasing amount of spam, the miserable web client, and the unfinished state of Element X, the Matrix.org network is not something I am willing to continue to recommend, especially to non-technical users. Normal people are simply tolerating it to communicate with idealistic nerds like myself who insist(ed) on using it.

Matrix just isn't focussed on users. I'm not talking about user-experience tweaks like which shade of cornflower blue to use - I mean basic user needs like apps that work and a way to combat spam.

There's a long list of ways the protocol contributes to a poor user experience. It almost seems designed without regard for how it will actually be used.

While the protocol may be conceptually interesting and their intentions noble, I'm not prepared to suffer abuse in the name of technical purity.

Open Source and Open Standards nerds like me ought to know by now that the protocol is the least compelling thing about a service. Who cares if your home is built using only Stallman-blessed tools, when the walls are full of rats?

Towards a test-suite for TOTP codes

@edent — Sun, 02 Mar 2025 12:34:39 +0000

Because I'm a massive nerd, I actually try to read specification documents. As I've ranted ad nauseam before, the current TOTP⁰ spec is irresponsibly obsolete.

The three major implementations of the spec - Google, Apple, and Yubico - all subtly disagree on how it should be implemented. Every other MFA app has their own idiosyncratic variants. The official RFC is infuriatingly vague. That's no good for a security specification. Multiple implementations are great, multiple interpretations are not.

So I've built a nascent test suite - you can use it to see if your favourite app can correctly implement the TOTP standard.

Please do contribute tests and / or feedback.

Here's what the standard actually says - see if you can find apps which don't implement it correctly.

Background

Time-based One Time Passwords are based on HOTP - HMAC-Based One-Time Password.

HOTP uses counters; a new password is regularly generated. TOTP uses time as the counter. At the time of writing this post, there have been about 1,740,800,000 seconds since the UNIX Epoc. So a TOTP with an period of 30 seconds is on counter (1,740,800,000 ➗ 30) = 58,026,666. Every 30 seconds, that counter increments by one.

Number of digits

How many digits should your 2FA token have? Google says 6 or 8. YubiCo graciously allows 7. Why those limits? Who knows!?

The HOTP specification gives an example of 6 digits. The example generates a code of 0x50ef7f19 which, in decimal, is 1357872921. It then takes the last 6 digits to produce the code 872921.

The TOTP RFC says:

Basically, the output of the HMAC-SHA-1 calculation is truncated to obtain user-friendly values

1.2. Background

But doesn't say how far to truncate.

There's nothing I can see in the spec that prevents an implementer using all 10. The HOTP spec, however, does place a minimum requirement - but no maximum:

Implementations MUST extract a 6-digit code at a minimum and possibly 7 and 8-digit code. Depending on security requirements, Digit = 7 or more SHOULD be considered in order to extract a longer HOTP value. RFC 4226 - 5.3. Generating an HOTP Value

(As a minor point, the first digit is restricted to 0-2, so being 10 digits long isn't significantly stronger than 9 digits.)

Is a 4 digit code acceptable? The security might be weaker, but the usability is greater. Most apps will allow a one digit code to be returned. If no digits are specified, what should the default be?

Algorithm

The given algorithm in the HOTP spec is SHA-1.

In order to create the HOTP value, we will use the HMAC-SHA-1 algorithm RFC 4226 - 5.2. Description

As we now know, SHA-1 has some fundamental weaknesses. The spec comments (perhaps somewhat naïvely) about SHA-1:

The new attacks on SHA-1 have no impact on the security of HMAC-SHA-1. RFC 4226 - B.2. HMAC-SHA-1 Status

I daresay that's accurate. But the TOTP authors disagree and allow for some different algorithms to be used. The specification for HMAC says:

HMAC can be used with any iterative cryptographic hash function, e.g., MD5, SHA-1 [Emphasis added] RFC 2104 - HMAC: Keyed-Hashing for Message Authentication

So most TOTP implementation allow SHA-1, SHA-256, and SHA-512.

TOTP implementations MAY use HMAC-SHA-256 or HMAC-SHA-512 functions […] instead of the HMAC-SHA-1 function that has been specified for the HOTP computation RFC 6238 - TOTP: Time-Based One-Time Password Algorithm

But the HOTP spec goes on to say:

Current candidates for such hash functions include SHA-1, MD5, RIPEMD-128/160. These different realizations of HMAC will be denoted by HMAC-SHA1, HMAC-MD5, HMAC-RIPEMD RFC 2104 - Introduction

So, should your TOTP app be able to handle an MD5 HMAC, or even SHA3-384? Will it? If no algorithm is specified, what should the default be?

Period

As discussed, this is what increments the counter for HOTP. The Google Spec says:

The period parameter defines a period that a TOTP code will be valid for, in seconds. The default value is 30.

The TOTP RFC says:

We RECOMMEND a default time-step size of 30 seconds 5.2. Validation and Time-Step Size

It doesn't make sense to have a negative number of second. But what about one second? What about a thousand? Lots of apps artificially restrict TOTP codes to 15, 30, or 60 seconds. But there's no specification to define a maximum or minimum value.

A user with mobility difficulties or on a high-latency connection probably wants a 5 minute validity period. Conversely, machine-to-machine communication can probably be done with a single-second (or lower) time period.

Secret

Google says the secret is

an arbitrary key value encoded in Base32 according to RFC 3548. The padding specified in RFC 3548 section 2.2 is not required and should be omitted.

Whereas Apple says it is:

An arbitrary key value encoded in Base32. Secrets should be at least 160 bits.

Can a shared secret be a single character? What about a thousand? Will padding characters cause a secret to be rejected or can they be safely stripped?

Label

The label allows you to have multiple codes for the same service. For example Big Bank:Personal Account and Big Bank:Family Savings. The Google spec is slightly confusing:

The issuer prefix and account name should be separated by a literal or url-encoded colon, and optional spaces may precede the account name. Neither issuer nor account name may themselves contain a colon.

What happens if they are not URl encoded? What about Matrix accounts which use a colon in their account name? Why are spaces allowed to precede the account name? Is there any practical limit to the length of these strings?

If no label is specified, what should the default be?

Issuer

Google says this parameter is:

Strongly Recommended The issuer parameter is a string value indicating the provider or service this account is associated with, URL-encoded according to RFC 3986. If the issuer parameter is absent, issuer information may be taken from the issuer prefix of the label. If both issuer parameter and issuer label prefix are present, they should be equal.

Apple merely says:

The domain of the site or app. The password manager uses this field to suggest credentials when setting up a new code generator.

Yubico equivocates with

The issuer parameter is recommended, but it can be absent. Also, the issuer parameter and issuer string in label should be equal.

If it isn't a domain, will Apple reject it? What happens if the issuer and the label don't match?

Next Steps

If you're a user, please contribute tests or give feedback.
If you're a developer, please check your app conforms to the specification.
If you're from Google, Apple, Yubico, or another security company - wanna help me write up a proper RFC so this doesn't cause issues in the future?

Time-based One Time Passwords. Not the TV show you remember from your youth, grandad. ↩︎

A small contribution to curl

@edent — Tue, 11 Feb 2025 12:34:13 +0000

The venerable curl is one of the most fundamental pieces of code in the modern world. A seemingly simply utility - it enables other programs to interact with URls - it runs on millions of cars, is inside nearly every TV, used by billions of people, and is even in use on Mars.

And, as of last week, features a small contribution by me!

Look, I'm not an experienced bit-twiddler. I can't micro-optimise algorithms or spot intricate C-based memory leaks. What I can do is get annoyed at poor documentation!

You see, documentation and code comments are vitally important. Poor spelling might trip up non-native speakers, bad examples confuse learners, and ambiguous wording is a barrier to understanding.

As was written by the sages:

a computer language is not just a way of getting a computer to perform operations but rather that it is a novel formal medium for expressing ideas about methodology. Thus, programs must be written for people to read, and only incidentally for machines to execute. Abelson, Sussman, and Sussman Structure and Interpretation of Computer Programs

So, what did I fix? A few years ago, I noticed Google's documentation used example domains it didn't control. The same thing was happening in the curl source code.

One example domain used was "HTTPS://your.favourite.ssl.site" - when that code was written 23 years ago, the .site TLD didn't exist. Now it does.

Is there a serious risk that someone will register ssl.site and use it to take over the machine of anyone who unthinkingly follows that example? Probably not. But it also isn't terribly clear that it is an example. So I changed it to secure.site.example which uses the reserved .example TLD.

That should make it clear to everyone that it is a placeholder example and will prevent anyone from misusing that domain.

Similarly, there were a few comments which used domain.com as an example. However, that's a real website - so I updated those to example.com.

I was delighted to see the changes accepted.

And I was only slightly disappointed to have narrowly missed out on being contributor 1337, but being number 1342 ain't so bad 😄

You can see the full list of changes on GitHub.

Much like my patch to the Linux Kernel this might be considered a trivial matter - but I honestly believe that clear and accurate documentation can be as important as the code itself.

Huge thanks to Daniel for creating curl, for making such a welcoming environment for new contributors, and for handing out such brilliant stickers at FOSDEM!

Talking Contact Tracing at FOSDEM

@edent — Thu, 06 Feb 2025 12:34:19 +0000

I was delighted to be invited to speak at FOSDEM. And I was not at all intimidated to be speaking on the cavernous Janson stage. The audience were lovely⁰, asked interesting questions¹, and - most importantly - laughed in all the right places 😅.

Regular readers will recognise this as being an updated² version of the talk I gave at EMF 2024 - feel free to watch that one if you want to see if I've improved.

Huge thanks to the AV team and the video-wizards behind the FOSDEM infrastructure.

As I say in my introduction, these are my personal recollections. I no longer work for the Government, so feel free to send any complaints to the circular file.

Feedback

A few pieces of public feedback I got after the talk.

Except for one weird heckler who shouted out something incomprehensible. ↩︎
Well, one guy came up afterwards and asked "What exactly is Covid? Can you explain?" I politely suggested he speak to a medical professional. ↩︎
But, yes, still wearing the same t-shirt! ↩︎

Working around an old and buggy HTML Tidy in PHP

@edent — Sat, 17 Aug 2024 11:34:50 +0000

Dan Q very kindly shared his script to make WordPress do good HTML. But I couldn't get it working.

Looking at the HTML it was spitting out, the meta generator said it was HTML Tidy version 5.6.0. That's quite old! I confirmed this by running:

echo tidy_get_release();

Which spat out 2017/11/25. Aha!

There are a few bugs in this version of HTML Tidy, some of which are fixed in later versions.

Here's how to fix them.

Auto Indent doesn't work. This is fixed by manually specifying "indent" => 2

Indent with tabs doesn't work. So I told it to indent with 8 spaces using "indent-spaces" => 8,

Then I used a regex (naughty!) to replace 8 spaces with a tab.

$tidy = preg_replace( '/        /', "\t", $tidy );

Older versions of Tidy don't support newer HTML elements like

. This can be fixed with "new-blocklevel-tags" => "search",

The

element isn't closed properly. This was an annoying one. I had to manually rewrite my HTML to remove an

element from inside the summary.

Although not really a bug, I like to have HTML comments on a newline.

$tidy = preg_replace( '/>

Open Source – Terence Eden’s Blog

NHS Goes To War Against Open Source

Further Reading

Does Mythos mean you need to shut down your Open Source repositories?

How Can Governments Pay Open Source Maintainers?

Fixing The Plumbing

What Can You Offer?

Playing Well With Others

In Summary

Book Review: Diversifying Open Source - An Open Standards Playbook for Inclusive and Equitable Tech Projects by Paloma Oliveira ★★★★☆

The Peaceful Transfer of Power in Open Source Projects

How to actually test your readme

OK, but how?

Wait! You didn't follow your own advice!

I'm never going back to Matrix

Towards a test-suite for TOTP codes

Background

Number of digits

Algorithm

Period

Secret

Label

Issuer

Next Steps

A small contribution to curl

Talking Contact Tracing at FOSDEM

Feedback

Working around an old and buggy HTML Tidy in PHP

element from inside the summary.

Although not really a bug, I like to have HTML comments on a newline.

`$tidy = preg_replace( '/>`

Open Source – Terence Eden’s Blog

NHS Goes To War Against Open Source

Does Mythos mean you need to shut down your Open Source repositories?

How Can Governments Pay Open Source Maintainers?

Book Review: Diversifying Open Source - An Open Standards Playbook for Inclusive and Equitable Tech Projects by Paloma Oliveira ★★★★☆

The Peaceful Transfer of Power in Open Source Projects

How to *actually* test your readme

I'm never going back to Matrix

Towards a test-suite for TOTP codes

A small contribution to curl

Talking Contact Tracing at FOSDEM

Working around an old and buggy HTML Tidy in PHP

element from inside the summary. Although not really a bug, I like to have HTML comments on a newline. $tidy = preg_replace( '/>

How to actually test your readme

element from inside the summary.

Although not really a bug, I like to have HTML comments on a newline.

`$tidy = preg_replace( '/>`