Let's briefly step back into history.

The Enigma machine represented the most powerful form of convenient cryptography available in the early 20th century. There were only two practical ways to crack its encryption.

1. Capture a codebook with the encryption keys printed in it⁰.
2. Literally invent the computer¹.

The basis of Enigma is conceptually simple. You want to write a character. You follow a complex sequence of instructions and get the resulting encrypted character. You want to write the next character, so you follow the same instructions for the first which then changes the sequence of instructions for the next character. And so on. Every character you type changes the algorithm for the next character. Fiendish!

You could encrypt an Enigma message by hand. But it would be tiresome, error-prone, and take ages. So a machine was invented to do the hard work. A series of cogs and wheels and wires and lights.

One of the weaknesses of Enigma is that it used symmetric encryption. The password used to scramble the message was the same as the one used to descramble it. Each day the codes changed, so they were printed in a handy codebook which was distributed to each operator. If someone captured the codebook, they could decrypt all sent and received messages.

Decades after the war, asymmetric cryptography was invented². The magic of asymmetric encryption is that it allows you to have one password to scramble the message and a totally different one to unscramble it. This completely obliterates the risk of your codebooks being discovered; you can have a "public key" for encryption. Anyone with that key can encrypt a message, but not decrypt it. You have a private key for decryption which you guard with your life.

Asymmetric encryption powers the modern world. It is made possible by high-speed computer chips which can precisely perform mind-boggling calculations in microseconds.

Let us slip into an alternate timeline. The mathematics behind asymmetric encryption are conceptually simple - even if they are exceedingly difficult to execute without a computer. If the mathematicians of the day had made the necessary intellectual breakthroughs, could public-key encryption have worked in WW2?

I'm going to work through the following problem to prove that it was just about possible³.

0. Could a public / private keypair have been calculated in the 1930s?
1. Is it possible to use paper-and-pencil to encrypt a message using a very short public key?
2. Would it have been possible to build a machine to encrypt using longer public keys?
3. What key length would have prevented the private key being cracked by brute-force?

Let's take a look at the last question first.

Why brute force?

The original machines used to crack Enigma used brute-force; trying every possible combination until they discovered the right one. That's not strictly true - a large part of cryptanalysis was understanding the statistics behind the encryption algorithm, the likely content of messages, and common phrases that they contained.

Modern encryption algorithms are resistant to most of those statistical attacks. So the only feasible method of cracking a private key is by trying each combination sequentially.

Let's suppose there's a very short private key - for example just 4 bits long. There are 2⁴ possible combinations; 16 in total. It seems reasonable to suppose that, if the message can be easily decrypted by the intended recipient, it could easily be cracked by someone able to try 16 different key combinations.

For every bit of length added to the key, the number of combinations doubles. 2⁴=16. 2⁵=32. 2⁶=64. Once you get the 2³², you're at 4 billion combinations⁴. Trying one combination per second would take over 120 years to complete.

Of course, manually using a 32 bit key might be too complex for the technology of the day. So a shorter key might be easier to use while still retaining sufficient strength. How difficult is it to manually encipher and decipher messages with short keys?

Let's go back to question 0.

Generating a Keypair

How do we generate an asymmetric keypair? Remember, no computers allowed⁵!

What is a keypair?

Briefly and incorrectly put, keys are based on prime factors.

Multiply these two prime numbers: 29 and 113.

You can easily do that on paper or on a pocket calculator. It is trivial. But suppose I asked you to reverse the equation? Find out which two prime numbers are multiplied to give the number 40,133. That's much harder. For larger numbers, it is even harder than you think.

Calculating a keypair

Mathematically, it is relatively simple. You need to know the following concepts:

• What is a prime number? (A number divisible by nothing other than itself and 1. For example, 13.)
• What is a coprime? (Another number with no common factors to the first⁶. For example, the number 13 has a coprime of 9.)
• What is a modular inverse? (When the Prime is multiplied by Coprime, then divided by the modular inverse, the remainder is 1.)

You can follow along with this Python example, but the steps are simple enough to do by hand using sufficiently small numbers.

Here's the algorithm:

• Pick two prime numbers, 𝒑 and 𝒒
• Multiply them to get 𝒏
• Calculate (𝒑-1) × (𝒒-1) to get ϕ(𝒏)
• Pick a coprime of ϕ(𝒏) to get 𝒆
  • Any coprime can be randomly selected, although there are some choices which are bad.
• Calculate 𝒅 where 𝒅 × 𝒆 = 1 (mod ϕ(𝒏))
• The private key consists of 𝒏 and 𝒅
• The public key consists of 𝒏 and 𝒆

Let's do that with two small prime numbers 17 and 61. They are sufficiently small to be calculated by hand⁷.

• 𝒏 = 17 × 61 = 1037
• ϕ(𝒏) = 16 × 60 = 960
• 𝒆 = 77 (Chosen randomly)
• 𝒅 = 773 (Calculated using the Extended Euclidean Algorithm, which was first described in 1740)

Here's how calculating 𝒅 works. This isn't intended to be a complete explanation of how the algorithm works, but it is sufficient to show that generating keypairs would have been well within the grasp of mathematicians of the 1930s. Feel free to skip to the next section.

From Step 4, we have: 1 = 15 × 960 + (−187) × 77

We are interested in the coefficient of 77, which is −187. This means −187 × 77 ≡ 1 (mod 960)

We need a positive value for d, we add ϕ(𝒏) to −187.

𝒅 = −187 + 960 = 773

Let's Encrypt!

We have a plaintext message (𝒎), we calculate the encrypted version (𝒄) with the formula 𝒄 = 𝒎^𝒆 % 𝒏

Let's suppose our message starts "HELLO". We'll give every letter a number. Our message starts with H - the eighth number of the alphabet.

8⁷⁷ % 1037 = 638

You can do that today on any pocket calculator. But could a competent mathematician calculate that by hand?

Exponentials get very large very quickly. There are some shortcuts, like Modular Exponentiation, but it is a fairly manual process. Doable, but not pleasant.

With enough time, you could manually encrypt a message like HELLO from 8 5 12 12 15 to 638 768 388 388 835.

Let's Decrypt!

We have an encrypted version (𝒄), we calculate the plaintext message (𝒎) using the formula 𝒎 = 𝒄^𝒅 % 𝒏

638⁷⁷³ % 1037 = 8

Again, piece of cake today, but an almost insurmountable a manual grind for a pencil and paper computer.

The Land of Big Numbers

In the toy example above, we turned 8 5 12 12 15 into 638 768 388 388 835. That's a very bad way of encrypting text. Working on individual letters allows for fairly trivial attacks in the form of frequency analysis. You might not know what prime numbers were used, but you know what the most common letter in English is and which letters often come in pairs.

Let's pretend that a binary code like ASCII had been invented by the 1930s, giving each letter its own number⁸. So the text translates to 1001000 1000101 1001100 1001100 1001111 7 bits is sufficient to store 128 characters, which is good enough for text, numbers, and punctuation.

Smooshing those bits together gives 10010001000101100110010011001001111, which is 19,473,311,311 in decimal.

But here we hit a snag! And rather an important one. Our message 𝒎 must be less than the key 𝒏 otherwise the maths doesn't work. So each "chunk" that is encrypted must be less than, in this case, 1037. In binary, 1037 is an 11 bit number - 10000001101 so let's chop the long binary string into groups of ten bit numbers⁹.

1001000100 0101100110 0100110010 01111 which, in decimal is 580 358 306 15.

How easy is it to calculate 580⁷⁷ % 1037?

Well, 580⁷⁷ is this 213 digit number:

608072697981095702436950488113933187346914897948969284857902654293722732546509642998889930612690327222151182305275310987597838669368524800000000000000000000000000000000000000000000000000000000000000000000000000000

Erk!

But, using Modular Exponents as mentioned above, it is just about doable to calculate it modulo 1037, to give the answer of 287.

Manual decryption

The reverse starts with 287⁷⁷³ - which is a 1,900 digit monster:

8764076814703289747574882682976372218202744117677541506077211396661935894880088329104690573410169261407987339892509470735016770231680618285939631509696980819464283401576517028048058993738338212389624747625283501942530083096110122657663164247353946331224458655354563571410933387843518380823303070498533169970946532897148031815166130665099650412982697122333628263892989475993249398481489331587762699843745762158170438822856768199827373555952739238985122598013870248178442111493156638843557234189706340583484083198539928533412908164601212156510176835050241254357263891198022046581958723118373933025616238122851775785374806117735760339884871872459839891658484324244684568308566814363900160248669794871064158507228968139134265889106231940693454825286506694141354013548608249280312471711991110620182847512187270642477617113287609180070026599666866308043914633111575444534093596344978016090107810671208184558268833063049379828413374691670084694302906835473338305746126263384285289946318656379558672414106052389234651640556324925755102419977400705380530065200979454412655589210213499344464257545650144053951331232875248144510526902329651434948453811412299881757959472385612984529276884875992403776906120156323972516615974836750108651889727547635267691073017967972912857552981641062535868781437285961802657794299581952590634427029261618023579258415033039194531248890027241354305598494848058858007937053616885726584609180601099791624229397351704959975692231801353195625613188074479028803108341519961929058549691769420863241680253570449204941176209229012471375228095132443055306202854239923235089133210534407289759826457721708012298751583195590595588629155245907825766955419953439709562536473174022072079161000252263007473231270835574140865544015688184196904473193955950050289022786886850214941882872222159427413174019940977015728353647777297220099220515546638682392602897440872932518693189627668062600415788447

There is no sensible way for a human to calculate that without mechanical or algorithmic assistance.

So our 1939 cryptographers are ready to pack up and go home, right?

Wrong! Remember, the plaintext message 𝒎 = 𝒄^𝒅 % 𝒏

For this operation, that's 287⁷⁷³ % 1037

Let's turn 𝒅 into binary - decimal 773 is 1100000101 - we can now use the "square-and-multiply" algorithm to calculate the plaintext.

Again, you don't need to read this table and can skip to the next section - this is just to show that the calculations are possible (if somewhat convoluted).

Yeeeesh! Tedious, but absolutely doable by hand. As long as you don't make mistakes and don't fall asleep.

Could this be made easier? Perhaps - but let's consider whether this effort would be worth it.

The Benefits of Symmetric Encryption

As discussed earlier, Enigma's password let you encrypt and decrypt using the same password. That means if the password leaks, you have lost the secrecy of both your outgoing and incoming messages.

The key advantage of symmetric encryption is that it is much easier to use. You set today's secret, then you can send and receive with ease. You do not need to manage two separate codes.

Let's imagine that there is a theoretical machine which can mechanically or electronically code and decode messages. You have the public key for sending messages back to base - but what about if you want to receive a message?

You will need a private key. A key which has to be protected in exactly the same way as Enigma's codebooks. If your private key is captured, all the messages previously sent to you can be decrypted.

OK, perhaps the solution is to give every machine its own unique keypair? Well, to quote the sages, now you have two problems.

First is the complexity of managing all the public keys. You have to remember which one to use when sending information.

Secondly, it means that you cannot broadcast a general message to all recipients. If HQ wants to send a message, they need to encrypt it separately for each receiver and also broadcast it separately. That also means the receivers have to know which message is intended for them¹⁰. Similarly, if a single user wants to send an encrypted message to all nearby units, they need to know who they are and separately encrypt messages to them¹¹.

Simplicity is the main factor in making usable security as I've written about before. Regardless of whether a machine could have done the calculations, key management is a tough problem.

A brief look into key exchange

Diffie-Hellman key exchange is a cryptographic technique which allows two or more parties to use an insecure channel to exchange enough information to create a unique public/private keypair for themselves. As with all the other maths talked about, it is conceptually simple - but rather difficult to do by hand.

Given the limitations of the speed of 1930s technology, it might be easier just to broadcast in plaintext "Hello! I'm station 123 and my public key is ..." That would be a simple way of distributing your keys but has two disadvantages:

• The enemy can flood you with encrypted messages. You have no way to verify that they come from a legitimate source.
• There's no way to verify who the public key is from.

I'm not going to get into cryptographic signature verification because this blog post is already too long!

Would a machine have helped?

This, alas, is approaching the limits of my ignorance. I know it is possible to build a Difference Engine out of Lego. Similarly, in the late 1930s it was possible to build a mechanical calculator which was small, lightweight, and accurate.

There were massive analogue computers on battleships, able to solve "20-plus variable calculus problems in real-time". At around 1,400Kg these weren't as portable as the typewriter sized Enigma - but do go some way to showing it might have been possible to design a mechanical computer for these equations.

So it is possible?

The mathematics behind public key cryptography are simple and, in my estimation, could easily have been understood by people nearly a century ago.

The various algorithms for simplifying the necessary calculations were neither obscure nor difficult to implement.

With smaller keys, it is possible to hand-calculate encryption and decryption

But, could it work in practice? If you had suitably trained battlefield mathematicians, it would be just about feasible to encrypt a message for transmission and decrypt something you've received. You wouldn't want to do it while under fire or for any long messages or while using large prime numbers. But, technically it is possible to hand-calculate the encryption and decryption of public key cryptography!

Let's look through the steps again.

• Generate a public / private keypair.
  • Yes! Tedious for larger primes, but well within the abilities of skilled mathematicians.
• Converting a plaintext message to binary.
  • Yes! Baudot codes were well known, as were things like Morse code.
• Splitting a binary message into smaller chunks.
  • Yes! A trivial exercise on paper, but might be difficult mechanically.
• Encrypting a chunk.
  • Possible but difficult - especially with larger keys.
• Decrypting a chunk.
  • Possible but difficult - especially with larger keys.
• Creating a machine to do the difficult work.
  • A very cautious maybe. Large battlefield mechanical-computers existed and were precise. Given the effort that went on in Bletchley Park, I don't doubt something could have been created.
  • However, given the complexity of the calculations, I don't think a portable machine would have been possible.
• Key management.
  • A nightmare, as always.

Aside from the conceptual leaps required and the lack of computational power, the major problem with successfully deploying public key cryptography in 1939 is… usability!

The usability of security systems is often hidden from us. Managing a complex key infrastructure is a problem which still plagues the security industry. Despite decades of advances, we still regularly read stories about "secure" microchips getting hacked for their keys - I imagine it would be trivial to extract them from a mechanical computer.

Could public-key cryptography have been used in 1939? Possibly, but the complexity of mechanical computation would have made it impractical.

If you happen across a time machine with access to the mid-20th century, please pop back and let me know if I am right.

What should that next version be called? Modern software broadly bifurcates into two competing standards; SemVer and CalVer.

SemVer

Semantic Versioning is usually in the form 1.2.3, the last digit is usually for minor bug fixes, the second digit for new functionality, and the primary digit for big and/or breaking changes.

The semantics are pretty loose. There's no real consensus on when a new "primary" number should be issued. There are two main weaknesses:

1. The numbers might not be decimals. Is V1.29 newer or older than V1.3?
2. There's no semantic information about when the software was released.

Which leads us to…

CalVer

Calendar Versioning is, ironically, more semantic than SemVer. The version number is the date when the software was released. For example, Ubuntu releases are in the form of YY.MM - the latest stable release at the time of writing is 24.04 - so we can tell that it was released in April 2024.

There are three main problems with this approach.

1. ISO8601 or GTFO! Surely these should use YYYY-MM to make it obvious this is a date?
2. Minor bug fixes are often given a release number like 24.04.1 - is that still obvious it is date-based? Was it really released on the 1st of April?
3. No information about big and/or breaking changes. Software released several years apart may be functionally identical whereas software released days apart may be incompatible.

Alternatives

So, what other ways can we number software versions?

EffVer

Effort Versioning is, I think, a sensible way to standardise SemVer. It attempts to show how much effort it takes to move between versions.

PrideVer

How much Pride do you have in your software release?

This is SemVer for people with an ego and the coding chops to match.

RuffVer

Ruff is a sort of bastard child between SemVer and CalVer, but adds this delightful complication:

Stable releases use even numbers in minor version component: 2024.30.0, 2024.32.0, 2024.34.0, … Preview releases use odd numbers in minor version component: 2024.31.0, 2024.33.0, 2024.35.0, …

It's the versioning equivalent of setting up a fully scalable cloud database and hand-chiselling HTML out of stone for the cookery blog you update twice per year.

0Ver

Zero-based Versioning tells us that it is forbidden to ask and a sin to know when a piece of software will be completed.

Essentially, it is SemVer for cowards who are afraid to commit. The opposite of PrideVer.

PiVer

The venerable TeX uses Pi Versioning. The current version is 3.141592653, the next version will be 3.1415926535.

As the software gets refined, it gradually reaches a state of perfection. This is a charming versioning scheme which shouldn't be used by anyone other than Knuth lest hubris overtake you!

NameVer

Sometimes marketing takes the reins and insists that consumers need a Named Version to help prevent confusion.

Ubuntu uses things like Bionic Beaver, Distinct Dropbear, and Mantic Minotaur. By convention, names increase alphabetically, so you should know that Jaundiced Jackdaw is before Killer Kangaroo - until you've released 26 version and have to wrap around the alphabet again.

NameVer is helpful for distinct products which aren't related, but probably more confusing than necessary.

WinVer

Microsoft Windows uses this very logical scheme - 1, 2, 3, 3.11, 98, 2000, Me, XP, Vista, 7, 8, 10, 11.

It starts with more-or-less SemVer, then jumps to CalVer, then 4 digit CalVer, then to NameVer, then back to SemVer - skipping 9 because of alleged technical reasons.

Do not attempt to use this versioning unless you want to anger both gods and mortals.

KelVer

Absolute Zero is defined as 0K. And so, Kelvin Versioning counts down to stability.

Almost the opposite of PiVer - the closer this gets to zero, the closer the code is to being complete.

This versioning scheme is affront to most sane people. But here's to the crazy ones.

Non-Monotonic

You will notice that all of the above are monotonic. That is, they all proceed in one direction and never reverse. Any subsequent version was definitely released later than a previous version. So, in a sense, they all contain some level of semantics.

But they don't have to.

HashVer

Taking the Cryptographic Hash of the code, or a commit, allows one to create Hash Versioning. For example 43317b7 is a HashVer for something which would otherwise have the dull and unworthy name of v0.118.1

But, of course, a hash does have a modicum of semantic information - even if it is only loosely related to the content of the code. What if there were something with no semantics and no monotonic behaviour!?!?

RandVer

Embrace the weird with Random Versioning! It its heart, RandVer says pick any number that hasn't been used before.

Perhaps V7 is followed by V2.5, which is overtaken by V0xDEADBEEF

Absolutely guaranteed to have zero semantic content.

What have we learned today?

The square-root of bugger-all.

The TOTP algorithm uses a Hash-based Message Authentication Code (HMAC). The hash is calculated from a shared key and a time-based component.

The key is a short string of characters. The time-based component is calculated as the number of seconds between now and the Unix Epoch. When is the Unix Epoch? 00:00:00 UTC on Thursday, 1 January 1970⁰. It has been roughly 1.7 billion seconds since then. 64 bit computer systems can count up for another 290 billion years¹. So chrononauts journeying to the future should be fine.

But what about people travelling backwards? You and your friends want to go and see The Beatles perform in 1966. That's before 1970. So the time-based component will be a negative number.

I've tried a bunch of different TOTP generators and fed them a variety of negative numbers. They all crashed.

So, no. TOTP doesn't work for anyone travelling backwards in the 4th dimension. Pity.

Is there a serious point to this? Well, sort of.

Negative time is an unexpected input and leads to unusual behaviours. Could a crash in HMAC generation lead to an exploit?

Standards get used in all sorts of places - including retrospectively. Should standards writers specifically account for inputs which occur in the past?

How should computers deal with "preposterous" times?

What other common security tools fail if they're subjected to time-travel?

Which Beatles concert would you go to in 1966?

In her book about the divergence of the English and American languages, Lynne Murphy asks this question:

wouldn’t it be great if language were logical and maximally efficient? If sentences had only as many syllables as strictly needed? If each word had a single, unique meaning? If there were no homophones, so we’d not be able to mix up dear and deer or two and too?

That got me thinking about the creativity which can be expressed in code - and whether its a good thing.

Let's take an incredibly simple and common operation - incrementing an integer variable by one. How would you do that? You've probably see these variations:

$i = $i + 1;

or

$i = $i++;

or

$i = 1 + $i;

or

$i = int( float_adder( float($i), 1.00 ) );

or

i1, i2 = i1^i2, (i1&i2) << 1 

I'm sure you can come up with a few more esoteric methods.

The Python programming language has a list of aphorisms for good programming practice. One of which is:

There should be one-- and preferably only one --obvious way to do it.

Is that right? As described in What is Pythonic?, the Python language itself has multiple ways to accomplish one thing.

But, is it a good idea?

Back to Lynne Murphy again:

No, absolutely not. No way. Quit even thinking that. What are you, some kind of philistine? If Shakespeare hadn’t played with the number of syllables in his sentences, he would not have been able to communicate in iambic pentameter.

Shakespeare wasn't writing Python though, was he?

There are many different definitions of what "Open Source". We can have a lovely argument over a pint as to whether GPLv3 is too open or if a licence which hasn't been validated by the OSI counts. But, more fundamentally, I think Open Source roughly falls into seven levels.

These aren't in any particular order of importance. And feel free to argue in the comments if you think I've radically misunderstood something.

1. Look but don't touch

This is the bare minimum. The source is "open" in that you can look at it, examine it, and possibly even learn from it. But that' is it.

You can't redistribute it. You can't edit it. You can't build on it. But you can see it.

2. Do What Thou Wilt

The source is yours to do with as you please. You can distribute it, build on it, print it out, eat it, use it in a weapons system. There are no restriction.

Have fun!

3. Do As You Would Be Done By

There is ponderous legal language, but it all adds up to one thing - you have to comply with our requirements.

Perhaps they say "only redistribute with this licence" or maybe "you must make everything this touches open". Either way, you aren't quite as free to do what you want.

Have fun - but don't piss off anyone.

4. I'd rather you didn't

These are less often seen, but becoming more common. You are free to do anything you want with this code... unless you're someone we don't like.

Some code says you can't use it for military purposes, others restrict its usage if you're going to be racist with it, and some say it can only be used by a particular class of people.

These licences are controversial. Openness means this is for everybody. Sure, no one likes the thought of their code being in a bomb. But your agents of imperial oppression are my freedom fighters.

5. Contributors Welcome

We're on GitHub! We actively want you to participate! Not only is the code open - but so is the community! Anyone with an IDE and an idea is welcome to pitch in!

Come play!

6. Blessed Contributors

We're open! But only certain people are allowed to contribute. All others will be shunned.

This is the model Google takes with Android - fully open, but good luck getting even a comma changed. There's also a popular open source project which requires its contributors to be religious!

This is open; but only for the chosen few.

7. The Future

There is something coming that you and I cannot understand. Deep in the darkest trenches of the Internet comes a new breed of hacker. Their social norms diverge from ours. They aren't beholden to the old ways and care not for our pettifogging traditions.

The are building a new form of Open Source. Something that reflects the needs and concerns of their generation, rather than the tired problems of ours. Old farts will harrumph and grumble about how it isn't proper Open Source - and moan that the youngling don't fear their elders any more.

But, make no mistake, the future is coming and it doesn't need your old-fashioned opinions.

The text of Shakespeare's Romeo and Juliet is about 146,000 characters long. Thanks to the English language, each character can be represented by a single byte. So a plain Unicode text file of the play is about 142KB.

In Adventures With Compression, JamesG discusses a competition to compress text and poses an interesting thought:

Encoding the text as an image and compressing the image. I would need to use a lossless image compressor, and using RGB would increase the number of values associated with each word. Perhaps if I changed the image to greyscale? Or perhaps that is not worth exploring.

Image compression algorithms are, generally, pretty good at finding patterns in images and squashing them down. So if we convert text to an image, will image compression help?

The English language and its punctuation are not very complicated, so the play only contains 77 unique symbols. The ASCII value of each character spans from 0 - 127. So let's create a greyscale image which each pixel has the same greyness as the ASCII value of the character.

Here's what it looks like when losslessly compressed to a PNG:

That's down to 55KB! About 40% of the size of the original file. It is slightly smaller than ZIP, and about 9 bytes larger than Brotli compression.

The file can be read with the following Python:

from PIL import Image
image  = Image.open("ascii_grey.png")
pixels = list(image.getdata())
ascii  = "".join([chr(pixel) for pixel in pixels])
with open("rj.txt", "w") as file:
    file.write(ascii)

But, even with the latest image compression algorithms, it is unlikely to compress much further; the image looks like random noise. Yes, you and I know there is data in there. And a statistician looking for entropy would probably determine that the file contains readable data. But image compressors work in a different realm. They look for solid blocks, or predictable gradients, or other statistical features.

But there you go! A lossless image is a pretty efficient way to compress ASCII text.

If I set the compression to be lossy - the photo looks good but the UI looks bad. If I set the compression to be lossless - the UI looks good but the filesize is huge.

Is there a way to selectively compress different parts of an image? I know WebP and AVIF are pretty magical but, as I understand it, the whole image is compressed with the same algorithm and the same settings.

There are two ways to do this. The impossible way and the cheating way.

Selective Compression

In theory it should be possible to tell an image format to compress some chunks of an image with a different compression algorithm.

And yet... none of the documentation I've found shows that's possible.

GiMP's native XCF and Photoshop's PSD files work; they store different layers each of which can have a different filetype. I understand that TIFF and .djvu also have that capability.

But those sorts of files don't display in web browsers.

So...

Let's Cheat!

It's possible to use an SVG to embed multiple images of different formats. SVG is used as, effectively, a layout engine.

The syntax is relatively straightforward:

<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 1080 512">
   <image width="1080" height="512" x="0" y="0"
      xlink:href="data:image/jpeg;base64,..........."
   />
   <image width="1080" height="512" x="0" y="0"
      xlink:href="data:image/png;base64,..........."
   />
</svg>

That draws the JPG then draws the PNG on top of it. If the PNG has a transparent section, the JPG will show through. The JPG can be set to as low a quality as you like and the PNG remains lossless.

Here's what it looks like - click for full size:

Embedded images are Base 64 encoded, which does lose some of the compression advantages. But, overall, it's smaller than a full PNG and better quality than a full JPG.

Look, if it's stupid but it works it's not stupid.

But surely there must be a way of doing this natively?

It's a short and readable paper with some jaw-dropping anecdotes. Like the man who immediately got a pay rise after his transition, despite working in exactly the same job as before; women were on a lower pay scale...

At a basic level you can see why, when computer memory was measured in tens of kilobytes, it made sense to say male==0 and female==1. Why waste precious bits on something which could only ever be binary? Why create an option to change a data field which is immutable? Why design a schema which would allow a woman to be married to another woman?

And yet, even with those constraints, people were able to change their "official" gender within the database. Oh, sure, there were all sorts of cludges (both technical and political) - but it was possible.

The paper sparked four main thoughts for me.

There's No Such Thing As Immutable

For all the talk of Blockchain solving the world's issues (🤣) sometimes it is necessary to "rewrite history". People make mistakes. Assumptions change. Knowledge improves. Lots of facts, it turns out, are matters of perspective.

A really good example of this is time. I don't mean pesky things like timezones and leap seconds. I mean that, due to general relativity, one second on the moon is not equal to one second on Earth. How does your time-ordered database cope with that?

You might very well live in a culture where divorce is impossible, or where sexual consent cannot ever be revoked, or where a person can only be married to one other person at a time. But these are all societal conventions which are liable - and indeed likely - to change.

I'm almost tempted to say that the boolean type shouldn't exist in modern databases!

Diverse Teams Build Better Products

I don't know how many computer programmers in the 1960s were part of the LGBTQ+ community. And I don't know how accepting their colleagues would have been of them.

Perhaps you have read and memorised every single one of the Falsehoods Programmers Believe About... lessons. But surely it is more efficient to build a team who are empowered enough to confidently correct their colleagues' incorrect assumptions about how the world is arranged?

We bake rigid assumptions into our designs not out of malign intent (usually) but because we're ignorant. That's only shameful if we refuse to listen to other people's experiences.

Computers Serve Humans - not the other way around

Most of us have been forced to lie to a computer at one time or another. Perhaps it is a system which insists that you must have a US-style ZIP code. Or that your name must be longer that three characters. Or that you don't have an apostrophe in your email address. Or that your wife is Mrs, not Ms.

I know for sure that you've filled in a paper form where the boxes were too small and you've had to decide how to truncate your data.

Why? Because people have designed a schema which doesn't account for the variety in the world.

Today's constraints aren't tomorrow's

As I said at the start, it's understandable that designers designed around the constraints they faced. But these days, we have an awareness of the likely progress of technology.

It's said that the Apollo Moon landings were only possible because the designers skated to where the puck was going to be. They made reasonable assumptions about what technology was going to be developed in the future.

Yes, we should try and build things which perform well on existing and historic hardware. But we can't ignore the fact that tomorrow's computers will be smaller, faster, cheaper, and more efficient.

Does it make sense to store a human's name as:

CREATE TABLE people (
  name VARCHAR(32) CHARACTER SET latin1
);

Probably not. Disk space is cheap and getting cheaper. Perhaps people of the future will have names consisting of 500 emoji? Or perhaps people with "exotic" Unicode characters will want to use our services.

Oh, I'm sure there will be a performance hit if every column is essentially unlimited. But that's an argument to design better database engines - not to limit human expression.

Read More

You can read "Hacking the Cis-tem" in the IEEE or, if that's not available to you, read the pre-print.

There's only a smattering of actual code you need to read - most of it is constrained to gorgeous hand-drawn illustrations. Although I got a bit of a shock in the 2nd essay when I was confronted by ξ3 < exp(ΔE/τ)! Thankfully the rest of the chapter does a good job of explaining it and why it is important.

There are some amazing confessionals from people who, arguably, made the world a little worse with their creations. It is a pity the author of the Morris worm refuses to discuss his creation though.

A few of the essays are a little credulous when it comes to grand claims - the BitCoin essay is laughably naïve - but these are relatively short sketches which will hopefully drive you to read more about their subject.

It does do a good job of pointing to papers I'd never heard of, like "Hacking the Cis-team: Transgender Citizens and the Early Digital State" which describes how the binary of machines was used to rigidly enforce the gender binary. Sadly, it suffers a little from constantly using Wikipedia and Britannica for references - I'd much rather see source material.

There's an excellent range of information in here. It perfectly demonstrates that, of course, it isn't the lines of code which change the world - it's the people who dare to dream about a new paradigm. Not everyone emerges from the book a hero, but they all made an impact.

Well, that was the most bizarre day. A few days ago, Amanda Brock - the CEO of OpenUK - asked if I'd be on a podcast. I agreed, and offered up my office's media studio for the recording. Then she asked if it was OK if Bruce Perens came to record an episode.

Errr... OMG, yes!

So I got to spend 10 minutes showing Bruce around the GDS office, gushing about how much open source stuff we did - then got to watch him record a podcast.

Amanda - who has has a canny eye for when people are slightly star-struck - asked if I'd like to interview Bruce at the weekend. At the National Museum of Computing. For a new exhibition.

Errr... OMG, yes!

So me and Lowena Hull got to spend some time chatting to Bruce on camera about his involvement in open source, the early days of Debian, and why OSS is still relevant today.

Terence Eden is on Mastodon

@edent

Doing some filming with @BrucePerens & @LowenaHull for @openuk_uk. Thanks to @tnmoc for the great location! pic.x.com/nK4ihOhH0T

---

We had a good rambling chat - the majority of which never made it off the cutting room flaw. He was such an eloquent speaker - always taking a moment to marshal his thoughts and knowing exactly when to finish his soliloquy.

It was a fairly long and tiring afternoon. But we had just enough time to take a wander through TNMOC. Bruce - like lots of elder-statesgeeks - seemed to be in 7th heaven. I got a running commentary of "Wow! You know, I had an early one of these..." and "I think I've still got the original one in my attic..." and "When I worked at Pixar, I was chatting to Steve [Jobs] about this and we both thought it was pretty nifty..." along with the obligatory "Why is a slide-rule in a museum?"

I watched him go toe-to-toe with the geeks who maintain Colossus - discussing the intricacies of synchronising tape speeds, various Baudot codes, and where valves / tubes could still be acquired from.

I saw him in a spirited discussion about various flavours of ALGOL and the mechanics of repairing a filesystem on hard-drives the same height as a man.

We had an interesting chat about IBM and their complicity in the Nazi regime's use of Hollerith cards. And I got to give him a précis of the BBC Micro's impact on the UK.

He asked me to take a few photos of him standing by ancient monuments of technology

Bruce Perens K6BP - I Stand with Ukraine

@BrucePerens

1/2 Somehow I walked all of the way through @bletchleypark today without noticing any mention that the machines they talk about are actually operating next door at @tnmoc, due to the effort of incredibly skilled volunteers. And there is no flow between the two museums. Please ... pic.x.com/w11orsCof9

---

And, at the end of the day, he shook my hand and thanked me for showing him such a great time.

What a delightful person.

But computerists? Some of us have academic qualifications - which may or may not be as useful as a bootcamp qualification - and some of us learn from MineCraft mods. There aren't many external bodies which assess or validate our knowledge.

I'm not sure if I'm happy with that situation or not. I do, occasionally, need to prove to an employer that I'm as good as I say I am. Take-home-tests and whiteboard interviews only get you so far, I've found.

So I was curious to see that BCS - the Chartered Institute for IT⁰ - was offering a discount on its chartering process. This was a chance to have my skills and achievements accredited by someone external. I figured I'd give it a go.

The process and requirements were pretty simple. Mostly filling in some forms explaining what I'd done in my career, how I'd met certain criteria, and evidence of professional development. My self-promotion had to be signed off by someone who knew me professionally and belonged to an organisation which followed a code of conduct. Luckily my boss² was just such a person. Thanks Martin!

After a bit of waiting, I was delighted to find this in my inbox:

And, just like that, I was listed on their register¹

What does this mean?

For me, there are two distinct advantages. Primarily, when applying for jobs which list membership of a professional body, or want evidence of my competence, it will be a strong signifier that I have those things. It shows my commitment to Continuing Professional Development, and that I have achieved some level of distinction in my career.

A secondary benefit is getting to use the post nominals CITP MBCS. ✨Fancy!✨

What can I say? It appeals to my vanity.

Semantic Data

I think the correct way to represent this in Schema.org is using hasCredential

<script type="application/ld+json">
    {
        "@context": "https://schema.org",
        "@type": "Person",
        "name": "Terence Eden",
        "hasCredential": {
            "@type": "EducationalOccupationalCredential",
            "credentialCategory": "CITP",
            "recognizedBy": {
                "@type": "Organization",
                "url": "https://www.bcs.org/"
            }
        }
    }
</script>

Do correct me if I'm wrong.

And finally

The other day I had to fix a multi-line Regular Expression (RegEx). After a few hours of peering at it with a variety of tools, I finally understood the problem. Getting that deep into the esoteric mysteries made me feel like a powerful wizard with complete mastery of my domain. And I think that's dangerous.

I'm sure we've all read a story about a witch or wizard who distractedly substitutes eye-of-newt with iron-ute with disastrous consequences. Humans are easily confused. And confusion leads to unexpected mistakes.

Look, most humans are very bad at reading compiled code. Without any external tools - can you tell me what the following code does?

0000000 c031 d88e c08e 15be b47c ac0e 003c 0474
0000010 10cd f7eb 48f4 6c65 6f6c 202c 6f57 6c72
0000020 2164 0a0d 0000 0000 0000 0000 0000 0000
0000030 0000 0000 0000 0000 0000 0000 0000 0000
*
00001f0 0000 0000 0000 0000 0000 0000 0000 aa55
0000200

No. Of course not⁰. That's why we write code in a more human readable language and then compile it to computer readable instructions.

Regular Expressions are a sort-of halfway house. They're slightly readable by humans - but written in such a terse vocabulary as to be mostly unintelligible without concentration. There's no space for comments. Different engines have variable support for all their functions. They are a symbolic language with unhelpfully indecipherable and inconsistent symbols.

As a result, once the RegEx becomes more than trivially complex they're hard for most humans to understand. That makes them difficult to debug. It also makes it difficult to add or remove functionality.

I genuinely - and possibly misguidedly - believe that even something like ^\w+([-+.']\w+)*@\w+([-.]\w+)*\.\w+([-.]\w+)*$ might just as well be written in BrainFuck.

My contention is that almost all RegExs would be better served by more human readable code and that the very existence of RegEx101.com ought to bring shame on our industry.

Here are some positive use-cases for RegEx:

• You want to show off how smart you are.
• You need maximum efficiency when combing through a billion lines of text.
• You have a desire to build something hard to debug.
• You don't have lots of printer paper and need to make your code as terse as possible.
• You think if/else and switch/case statements are the mark of a diseased mind.
• You don't trust compilers.

I know what you're thinking: "This guy's too stupid to get regular expressions!" Yes. Yes I am. So are most people.

What I'm getting at is that source code is designed to be read and edited by busy and distracted humans. We should be writing intelligible code for each other and letting computers do the boring work of making it more efficient.

You don't have to agree with me. That's fine. But, perhaps you'll take note of the famous maxim from the "Wizard" book:

a computer language is not just a way of getting a computer to perform operations but rather that it is a novel formal medium for expressing ideas about methodology. Thus, programs must be written for people to read, and only incidentally for machines to execute.

Structure and Interpretation of Computer Programs

We are not wizards. Nor should we strive to be. The alchemists fell.

Charles Babbage - inventor of the first mechanical computer - found himself starring in a few articles. Here's a scan of one from 1844 (extracted text follows)

Returning to the new Houses of Parliament, we must again protest against the money lavished upon them.

For what business, we ask, has been effected in the present session that might not have been as efficiently transacted in a sentry - box? Wherefore, then, these costly buildings for men to do nothing in? Why, indeed, should we have Members of Parliament at all, when sure we are that MR. BABBAGE would construct a legislating machine, which, in the hands of SIR ROBERT PEEL, would fully answer all the purposes of living senators?

We are serious. We therefore propose that, at the next election, every gentleman desirous of affixing M.P. to his name should, on his election, send his proxy machine to the House of Commons, setting the instrument at "Aye" and "No," as MR. BABBAGE will instruct him, on certain questions, and suffering the Ministry to take their divisions accordingly. The result of the present session convinces us that the "business of the country" will be got through quite as well as by the present mode; with this advantage, that the machines may be put anywhere.

While it is mildly amusing that people have always been complaining about the cost of the Houses of Parliament, I think there's a deeper and more interesting story here.

Firstly is the nature of distributed work. As we saw during lockdown - it's perfectly possible for people to work remotely. MPs were able to debate and vote from home. You can argue whether that's a good thing. But it is undoubtedly a thing which happened. There were infrastructure costs - buying laptops, broadband, teaching people how to use equipment - but that is surely nothing compared to the cost of physically maintaining the crumbling building.

But, more interesting, is that over a hundred-and-fifty years ago people were taking the piss out of the idea of a Technocracy. The idea that one man could build a machine which would always give the correct answer to any political question was as laughable then as it is now.

Wouldn't it be amazing if we could compute the correct answers to complex policy questions?

Sadly, we have the small matter of the messy nature of reality to deal with.

There is bias in the way we collect data. There are biases in the way we store, process, and validate data. The algorithms we use are flawed. And even the questions we ask of data reveal the limitations of our imaginations.

Babbage himself had this to say on the subject:

On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.

Passages from the Life of a Philosopher (1864), ch. 5 "Difference Engine No. 1"

It doesn't matter how perfect our computer code is - it will always have to deal with imperfect inputs made by imperfect people.

Let's consider a set of 1 dimensional data. For example, the alphabet:

ABCDEFGHIJKLMNOPQRSTUVWXYZ

What do we mean by "1 dimensional"?

Consider the element "B". From there you can go left to "A" or right to "C". Left and right are the only directions you can move. You cannot go up or down from "B". There's no way to go in or out. You're trapped in a single dimension.

What if we want to write the data onto a 2 dimensional space - like a bit of paper or a website.

Well, if the space is big enough, we can write it as we have above. A single line. Job done, go home.

But what if the space is too small in the horizontal direction to contain it all? Well, we could write it in the vertical direction:

A
B
C
D
…

OK, but what if the 2D space is too small in both the horizontal and the vertical directions to contain the 1D data? Now we get into the "exciting" world of space-filling algorithms.

Here's the approach that you're probably most used to. It's certainly the one which springs to most people's minds:

ABCDEFG
HIJKLMN
OPQRSTU
VWXYX

The algorithm is simple. Write horizontally from left to right until you run out of space, then move down one line, return to the left-most side, and continue writing. Repeat.

Except! That's not necessarily the simplest or most obvious way of writing!

There is a different algorithm - Boustrophedon:

ABCDEFG
NMLKJIH
OPQRSTU
  ZYXWV

Start at the left, when you run out of space, move down a line and continue writing in the opposite direction!

→ABCDEFG
        ↓
 NMLKJIH←
↓
→OPQRSTU
        ↓
   ZYXWV←

If you haven't spent years learning to read "properly" this form of bi-direction writing may well be easier. You don't need to move your finger or eyes back in a perfectly straight line.

We can go a little further. What if we started at the centre of the page, and wrote out in a spiral?

Z
YJKLM
XIBCN
WHADO
VGFEP
UTSRQ

As pretty as this looks, there are some obvious drawbacks. Unless you know exactly how long the 1D data is, you won't know where to start on the page. It also might not fill the space in the most efficient manner.

Here's a different algorithm. It starts in the top left corner of the space and gradually spreads out. Growing to fit the data in the space:

 ABIJYZ 
 DCHKX  
 EFGLW  
 PONMV  
 QRSTU  

If you're having trouble following that, here it is in diagram form:

 →↓→↓→ 
 ↓←↑↓↑ 
 →→↑↓↑ 
 ↓←←←↑ 
 →→→→↑ 

That's quite a nice algorithm. It starts in a predictable place and it fills up both horizontal and vertical space in equal amounts.

Introducing Hilbert Curves!!

Let's start with a simple curve:

D__C
   |
A__B

Where do we go from here? The Hilbert Curve is a fractal algorithm which starts like this:

F__G  J__K
|  |  |  |
E  H__I  L
|        |
D__C  N__M
   |  |
A__B  O__P

And ends up doing this:

Lovely!

Non-rectangular curves

Not every 2D space is a rectangle. Some have the audacity to be hexagons!

3D!

And then...

I wonder what interesting things I can do with this new knowledge?

Most programming languages have the concept of relational operators. How does variable X compare to variable Y?

If we want to ask if X is less than or equal to Y, we write X <= Y. Which is a bit weird, because Unicode has a "less-than-or-equal-to" character: ≤. But because there's no button for that on a standard keyboard, we use a hacky solution of jamming two characters together.

In the R language, the assignment operator is <- this is because programming keyboards used to have a ← button on them. That button has gone from modern machines, so we make do and mend.

There are some programming fonts which contain ligatures to automatically convert these characters (although this makes some people very grumpy).

Unicode includes ⩶ (U+2A76) and ⩵ (U+2A75) as distinct glyphs. So why are we still hitting the = key multiple times like savages?

This leads to the deeply weird situation where "⩵" ≠ "==" and absolutely no programming language that I know of will let you use the correct Unicode characters!

(Except, of course, Raku - which made this change based on my suggestion.)

Ideally, we need programming languages which support Unicode symbols as first class characters, and keyboards with a hundred more buttons.

When you are a child, the whole world is complicated. By the time you're a teenager, the whole world is simple. Once you grow up, you realise just how complicated everything is. As you obtain mastery, you find a way to simplify everything.

I expect that by the time I'm old(er) and grey(er) I'll have gone through several more cycles of this process.

When we write code in a traditional programming language, we write strings of characters. In more sophisticated visual programming languages, like Scratch, we manipulate objects.

But, whatever the programming paradigm, all our data are stored and transmitted as bits.

No one wants to write binary. It is almost impossible for a human to read or write anything with a moderate level of complexity. That's why we have debuggers and decompilers to use when we're forced to deeply examine a truculent piece of code. And that's why we have moved away from the binary language of moisture vaporators and on to higher languages.

Humans are pattern matching machines. We love categorising things. So much so that occasionally wars break out over whether something is Type X or Type Y. Our love of categorising comes with other problems.

Terence Eden is on Mastodon

@edent

Oats + Water = Porridge.
Oats + Milk = Porridge.
Oats + Plant Milk = Porridge.

Oats + Water + Salt = Porridge.
Oats + Water + Sugar = Porridge.

Oats + Coca Cola = ...

---

Porridge: (90)
90
Not Porridge: (185)
185

---

Taxonomy is hard, as this video adequately explains:

The philosophy of numbers plagues programming. Integers, Floats, Fraction, Booleans, Strings - all valid ways to represent numbers, all with complex subtleties and peculiar interactions.

But, in the end, they're all just stored as bytes in memory.

Eventually, talented hackers found a way for it to do simplistic 3D graphics and even speech synthesis.

Recently, people have worked out a way to perform ray-tracing on it!

@bbcmicrobot@mastodon.me.uk

@bbcmicrobot
Automated

FEATURE ANNOUNCEMENT! @bbcmicrobot is at warp speed thanks to beebjit from @scarybeasts !

Include a 🚀 in your tweet run for 3 HOURS emulation time - 21 billion emulated 6502 cycles - and you'll get a screenshot reply in around a minute.

NB GXR ROM is disabled in this mode pic.x.com/XZut4o2RpO

---

The next computer our house got was the Sega Megadrive. The first game that console saw was, I think, Alex Kidd. A basic 2D platformer.

Sure, it was streets ahead of the Beeb, but the graphics weren't amazing.

But, over the years, they got better. By the time the MegaDrive stopped getting new games in 1997, the graphics and audio available were utterly transformed. In eight years, we'd gone from a limited pallet 2D screen to this:

Stunning music, and liquid smooth 2D graphics with parallax and complex transformations.

3D! Three-friken-Dee!

Some enterprising hackers managed to get Wolfenstein 3D running on hardware which was originally intended for cheap side-scrollers.

And nothing about the console had changed. The tools used to create games had improved. The maths and algorithms had leapt ahead. And the ingenuity of the designers had increased. But the physical hardware was identical.

Once you understand a system - deeply understand - it can do things that its designers never thought possible. You can push hardware beyond its apparent limits.

We're so spoiled today. Every week a newer, faster processor is released. Hardware gets cheaper and we can just throw more chips at the problem.

What would the world be like if that wasn't the case? What if our progress in computer speed suddenly came to a stop? I think history shows us that we would be able to work around the restrictions to do things which seem impossible.

Even when machines break down, they can still be made to perform unexpected miracles. A billion-and-a-half kilometres away, the Voyager 2 probe broke down. Software instructions were sent which told the craft to do something it wasn't intended to do. The craft was instructed to cycle its heating system in order to correct the flow of lubrication. All of a sudden its capabilities were upgraded and it was able to continue its mission for another half-century.

Mars is the only planet in the solar system which is entirely populated by robots from Planet Earth. Those robots aren't receiving any hardware upgrades any time soon. But their software gets upgraded to allow their hardware to perform new tasks.

We don't necessarily need faster, better hardware. We need more thoughtful, and more creative humans.

Houses are blessed, leaving for university requires a party, graduates get a silly gown and certificate. Finally, Death demands a wake.

Rituals are powerful and a seemingly ubiquitous part of human life. So why don't we have them for developers?

The first program you write will, inevitably, spew out "Hello, World!" - that seems to be the last time we consciously engage in ritual performance.

When I returned to my old secondary school, I saw a huge wooden board filled with names of former head-boys and -girls. Nestled amongst them was my old mate Richard. An unbroken line of upstanding young people, from before the war to the present day.

Head Boy and Head Girl - milestones so important that they were carved into wood and put on display.

The military awards medals, and there is a clear hierarchy of ranks through which one can progress.

Academics get recognition and prizes and tenure.

An apprentice becomes a journeyman becomes a master.

Part of the problem is that computer science is a young discipline. We don't have hundreds of years of august bodies and a good understanding of how people progress.

So, here are my questions to you. What are the milestones that every developer should pass? How should we mark them? Should we mark them? Is it helpful to show exactly where a computerist is on their journey?

Here are a few lazy suggestions. Feel free to add your own.

Medal Ceremonies

vkmc

@vkmc

Thanks @OpenStack community for the mentor of mentors award :) I feel honored! #WeAreOpenStack pic.x.com/lzy1prkgze

---

Feels a bit like it is appropriating the military's culture.

Community Awards

I love the idea of the OpenStack Community Awards. But deciding who gets an award is fraught with bias.

Immortalise

Projects like the Faces of Open Source seek to create a permanent memorial of those who do good. But it falls into the same problem as other awards - who chooses?

Formal Qualifications

I gave kids certificates, and they bloody loved them.

Terence Eden is on Mastodon

@edent

I was amazed by how happy the kids were to receive certificates.
Such a pleasant rite of passage. x.com/codeclubworld/…

Code Club Global

@CodeClubGlobal

Coming to the end of term with your Code Club? Celebrate success by giving out certificates! Sign in to download: codeclubworld.org/sign-in/ pic.x.com/s7uxzoyt88

---

---

The hacker mentality seems to eschew formal qualifications. And, to be fair, many are just pay-to-play, or exercises in gatekeeping.

Is this necessary?

It isn't necessary - but there's constant undercurrent of who is - and isn't - a "real" developer. People get beleaguered with imposter syndrome because there are no real metrics to judge themselves against. We have no way of celebrating achievement - and no way to welcome people into our tribe.

And a part of me finds that a little sad.

So I downloaded PuTTY and never looked back. It had a simple GUI. Well… simple enough. It was impossible to type an IP address into a box designed for a port number, it could tell that username was unlikely to contain a non-ASCII symbol, and that made it slightly easier to configure a connection.

The job of the computer is to do the hard work for us. Computers shouldn't recognise that we've made a mistake - they should stop us making mistakes in the first place!

Flash forward to today. I wanted to release an update to my NPM package of SuperTinyIcons - which are hosted on GitHub.

I figured that I'd have to click a link on GitHub, do the OAuth dance, click on a few GUI elements, and hit a big save button.

Nope!

I have to create YAML text files. Manually. Because it is the Year of Our Lord 2020 and computer engineering hasn't evolved since the earliest days of UNIX.

Don't get me wrong - I love the utility and versatility of the command line. I'll happily manually edit a JSON file in nano and use sed to update a bunch of documents. I rely on diff and grep daily. But when it comes to configuring things - I am undone. I don't want to learn whitespace rules just to get something working.

Let's take a simple example. Here's a snippet of YAML from the official guide to releasing NPM packages via GitHub:

jobs:
  build:
    runs-on: ubuntu-latest

What are valid options for "runs-on"? Where would I find them? What are the differences between them? What happens if I spell something wrong?

Luckily, the GUI was invented to save us from ourselves:

--Please choose an option-- Ubuntu Latest (This is the most common option) Grarch 3.5 (Useful if you need flargon support) Plan9 (Choose this if you require Python 1) Windows 3.11 (Only works on older, 16 bit projects)

M A G I C !

This goes for almost every element of the config file. If there are a set list of choices, present them to the user. If there is only one option, hide it. If there are two options, give them a checkbox.

The thing is, GitHub gets part of the way there. If you want to setup a webhook - you get a GUI. Why does one part of GitHub offer a user-friendly way to manage configuration, and another part only provide a user-hostile environment?

To make it worse… NPM IS OWNED BY GITHUB!

I'm not saying you can't have a text file. I'm not trying to take your toys away from you. Have a sudo mode which lets user hand-stitch a config together with butterflies and a binary keyboard. Go nuts! Want to store the config in git so you can see what changes have been made? Brilliant! To be explicitly clear - store the config in a text file. But have a GUI first, and expose the text file to any masochists who want to manually replace all the tabs with spaces.

I just can't be bothered reading 22 thousands words of YAML spec, a further two thousands words of GitHub instructions, and - no doubt - a hundred different StackOverflow questions, just to send data from one computer to another.

I heard someone say that part of Amazon's success was its relentless focus on convenience. How is manually creating fragile text files convenient for me?

We have to stop this fetishization of the text file as the primary method for configuring computers.

Dude... DUDE! What if... What if, like, we're all just brains in a jar and, like, a machine is projecting reality around us...? Whoa...!

I bet you've had that conversation with someone. Probably after you first watched The Matrix. As a philosophy, it is a lot less modern than you might think:

I shall suppose that some malicious, powerful, cunning demon has done all he can to deceive me [...] I shall think that the sky, the air, the earth, colours, shapes, sounds and all external things are merely dreams that the demon has contrived as traps for my judgment.

René Descartes in 1641

How far does your doubt extend? Hyperbolic doubt is, simplified, a philosophy which insists you should question everything - including your own senses.

Philosophy, naturally, leads us on to Computer Science.

How do you know your computer isn't lying to you? Open source code means that you can see the code your computer runs. Of course, the compiled code may be different from what's published.

Do we trust verifiable build? Perhaps not.

OK, so we compiler our own code.

But do we trust our compiler? Same problem as above.

Let's write our own compiler!

But what if there's a secret chip in our computer inserting malicious code? Argh!

It's malicious demons all the way down. But, at least, we can see the code and compare it against the binary code and compare that to the output of the computer. That's a fairly reasonable way to dispel doubt.

Hurrah for open source licences!

But what about code that runs on someone else's server?

How can you tell what a distant computer is doing with your data? There are some Open Source Licences which are specifically for servers:

The GNU Affero General Public License is a modified version of the ordinary GNU GPL version 3. It has one added requirement: if you run a modified program on a server and let other users communicate with it there, your server must also allow them to download the source code corresponding to the modified version running there.

Why the Affero GPL

So someone can give you the code. But that doesn't really help, does it?

How do you know they're running the exact same code on their server?

How do you know that their server hasn't been hacked?

How do you know if, halfway through your session, the software is replaced?

How do you know?

This isn't an argument against open source. More transparency is always good. But it's important to know its limits. There are some computational tasks which are too intense for a home machine. If you want to communicate with another person, you have to trust the code running on their machine.

Computo Ergo Sum.